If you have followed the AI boom, you have heard about the chips that do the thinking, Nvidia's GPUs above all. Less discussed, but just as essential, is the memory that keeps them fed. That memory is called HBM, and it has quietly become one of the most sought-after products in the technology industry.
What HBM actually is
HBM stands for high-bandwidth memory. It is a form of DRAM, the working memory that temporarily holds the data a processor is using right now, as opposed to long-term storage. What makes HBM special is its shape. Ordinary memory chips sit flat on a circuit board. HBM instead stacks a dozen or more memory chips vertically, like floors in a tower.
Those stacked floors are joined by thousands of microscopic vertical wires called through-silicon vias, as reference material describes. Picture them as electrical elevator shafts running straight through the tower, letting data move between all the floors at once. The whole stack sits right next to the processor, so information travels the shortest possible distance. All of that engineering serves one goal, captured in a single word: bandwidth.
Bandwidth, and the "memory wall"
Bandwidth is simply how much data can move between memory and the processor each second, the width of the pipe rather than its raw speed. HBM uses an extremely wide data path, reported at 1,024 bits per stack, many times wider than ordinary memory. That width lets it shift huge volumes of data without running at punishing speeds, which also helps control power use.
This matters because of what engineers call the "memory wall," a term TrendForce and others use. Over the past two decades, raw processing power has grown far faster than the speed at which ordinary memory can supply data. The result is that a modern chip can often calculate faster than it can be fed, so it sits partly idle, waiting. That gap between a fast processor and slower memory is the wall.
Why AI needs it
Training and running large AI models means moving gigantic amounts of data, a model's billions of parameters, to the processor over and over. An AI accelerator that costs tens of thousands of dollars is only worth the money if it stays busy; starve it of data and you are paying for idle silicon.
HBM is what keeps those accelerators fed. Sitting beside the processor and moving data through its very wide path, it delivers the throughput AI work demands. That is why almost every high-end AI chip today ships with HBM built in, and why demand has soared alongside AI spending.
The generations
HBM has advanced through steady generations, each roughly doubling performance. The current volume product, HBM3E, moves on the order of 1.2 terabytes of data per second per stack. The next standard, HBM4, was adopted by the industry's standards body, JEDEC, in 2025 and targets around 2 terabytes per second, TrendForce reports. For scale, one terabyte per second is roughly a thousand gigabytes of data every second.
The market, and why investors care
HBM is hard to make, and only three companies do so at scale: South Korea's SK Hynix, which leads the market; Samsung; and America's Micron. Their exact shares are disputed and shift from quarter to quarter, but SK Hynix has been the clear front-runner, with Micron and Samsung competing behind it.
The business logic is stark. HBM is complex, capacity-constrained and highly profitable, and much of the coming supply is reportedly booked well in advance. That has helped swing memory makers from the losses of a cyclical downturn to record profits, tying their fortunes tightly to AI capital spending, the vast sums that cloud providers are pouring into data centers.
That same link is the main risk. HBM demand rides the AI investment cycle. If that spending cools, or if manufacturers add capacity faster than demand grows, today's shortage economics could reverse. None of this is investment advice; it is a map of why a small stack of memory chips has become one of the most closely watched products in technology.
