Renting an AWS p5.48xlarge for 3 years reserved would cost you $1.13M, which is a lot lower than renting such capacity on demand, which would cost $2.58M. No one is going to do on demand for 3 years, but this shows the relative price per unit of compute
It is funny what courses were the most fun and most useful when we look back at college. Both microeconomics and macroeconomics stand out, as does poetry writing, philosophy, and religious studies despite the focus on engineering and American literature.
– is true of GPU-accelerated systems, which are creating the gravity at the core of the AI galaxy. The cost of AI hardware, whether you buy it or rent it, is the dominant expense of AI startups the world over, and at this point somewhere around 80 percent to 85 percent of the money in these systems is going to Nvidia for GPUs, system boards, and networking.
In its announcement, AWS said that the new P5 instances will reduce the training time for large language models by a factor of six and reduce the cost of training a model by 40 percent compared to the prior P4 instances. Part of the reason this is true is that AWS charged a premium for its A100 GPU, which is why we built a salient characteristics table that brings together the P2, P3, P4, and P5 instances, what it costs to rent them on demand or with one-year or three-year reserved instances.
To figure out an operating profit for the P-class GPU nodes in the table above, we assumed peak power usage, which we realize is a worst case scenario. We calculated the electricity use of the nodes, and then added on more power consumption to cover the cooling of the machines. We also calculated the 30 year cost of datacenter space , and added three years of cost on that. This is the AWS Facility cost in the chart above.
As you can see, assuming our math is in the right ballpark, even with all of these costs in there, AWS is going to make a pretty good operating profit on these P5 instances. Just as it has done with prior P-class generations. And assuming that there is a mix of on demand capacity mixed in with the reserved instances – 60 percent reserved versus 40 percent on demand – doubles the aggregate operating income over three years for a P5 instance.
Just for fun, given the cost and rental data we ginned up above, we put an AWS EC2 UltraCluster with 20,000 H100s up against some of the big AI systems that we know are being built either in the clouds or in HPC centers in the United States. This table just keeps getting more and more interesting as we add vendors. Take a gander:
Australia Latest News, Australia Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
Microsoft India snags former AWS India boss Puneet ChandokMicrosoft India poaches former AWS India boss Puneet Chandok
Read more »
Sony and Microsoft really are propping up AMD's beleaguered GPU business right nowIt's only the console chips keeping AMD's gaming market afloat while GPU sales continue to suffer.
Read more »
Gamers, brace yourselves: AI development could lead to another GPU shortageIs another shortage on the horizon? We hope not!
Read more »
Microsoft makes concession for Office in AWS virtual desktopInterest from antitrust regulators spurred action? Still a pricey option, says analyst, and change doesn't help Google or Alibaba customers
Read more »