# RAM Shortages

There's currently a RAM crunch driven by Google, Amazon, Microsoft, and other hyperscalers buying out massive amounts of RAM manufacturing capacity.

Last year, 64 GB of DDR5 RAM for a PC desktop cost around $240. Now it costs around $1000. That's a 400% increase.

In March you could buy a Mac Studio with 512GB of RAM. In April you could buy a Mac Studio with 256GB of RAM. Today, you can buy a Mac Studio with 96GB of RAM, and the latest (M4) has a maximum of 64GB. Lead time if you order one of these high RAM models is 10 weeks.

Samsung reportedly can't make money on their phones because the cost of RAM is too high.

# The Inference Market

Industry projections currently estimate that demand for AI inference is growing around 20% YoY. Hyperscalers' investments in infrastructure are increasing 100% YoY. That's 5x the investment spend compare to projected demand growth.

Additionally, while the price of inference (see Anthropic's recent price hikes for Claude) is going up, AI inference is getting cheaper and more efficient over time. Google recently announced research showing a massive drop in memory needs for AI. Nvidia claims that Mythos, Anthropic's state of the art, too-dangerous-to-be-released-to-the-public model, was trained on prior generation chips, with modest hardware needs.

Given the efficiency gains in AI inference, major competition between labs, and availability of open source models from China, we should expect inference to become commoditized, costs to plummet, and pricing to be a race to the bottom.

While the technical costs for inference have dropped dramatically over the past year, inference pricing has not. The reason is that Google, Amazon, Microsoft, and other hyperscalers are using their market positions to secure exclusive access to RAM chips, not because they need more capacity, but simply to deny access to their competitors.

Many of the announcements you see from hyperscalers are not actual expenditure, but rather, "expressions of intent" or "commitments to invest" over the next few years. This allows them to negotiate contracts to lock up RAM supply and shock the market without actually spending all the money up front, and possibly without even fielding actual infrastructure.

I imagine in a year or two we will learn that there are massive warehouses filled with RAM that no one is using. There is real capital outlay, but again, it's not proportional to market demands. There's a great deal of performative spending happening.

In the mean time, soaking up the RAM supply allows hyperscalers to artificially inflate the price of their services, inference in particular. You can't afford to buy at 4x the cost, so you have to rent from them.

Hyperscaler denial is the first monopoly factor driving up RAM prices.

# Manufacturing

Global RAM manufacturers are up next. In the past, RAM manufacturers increased production capacity in response to demand, but this resulted in a price crash. During the recent RAM shortages, RAM manufacturers initially announced they had no plans to invest in additional capacity.

Instead, they shifted production from consumer brands to server components, and raised prices, rather than increase capacity. Crucial, Micron's consumer line, is just gone, Samsung is losing money on phones, and Steam hardware is delayed due to RAM availability. There are only 3 major memory producers in the world — Micron, SKHynix, and Samsung — so they can sit by and rake in money without much fear of competition.

A Chinese RAM producer, CXMT has announced plans to develop direct-to-consumer RAM production lines. And more recently, possibly in response to CXMT, Micron announced up to $200 billion planned investment in manufacturing capacity. Any increase in supply output is years away, though.

RAM manufacturers are the second monopoly factor driving up RAM prices.

# References

This article was written by a human, without AI.