The culprit behind memory price hikes: What exactly is HBM memory?

The rapid advancement of AI technology and its application ecosystem has brought profound impacts on people's work and daily lives.

At the same time, people have noticed that memory prices have skyrocketed like a rocket, completely breaking the long-standing pattern of "overcapacity, iterative price drops, and increasing affordability year by year." Consequently, smartphones and laptops have either reduced specifications or increased prices, and desktop DIY has become instantly neglected. The culprit behind this is HBM memory.

The full English name of HBM is High Bandwidth Memory, a high-end storage product specifically designed for AI training and high-performance computing. From the perspective of underlying principles and application scenarios, it still falls within the category of DRAM dynamic random access memory.

It shares the same origin as the common DDR4 and DDR5 memory used in our computers and phones but differs in form. For example, the NVIDIA H200 is equipped with HBM3e memory, which consists of six square HBM die stacks surrounding the GPU core, integrated on the same interposer substrate alongside the CPU.

In terms of principles, traditional DDR memory uses a planar tiling packaging layout, with limited chip pin count. The data channel width and transmission speed are constrained by the physical structure, making it difficult to increase capacity and performance. It can only meet the needs of general computing and entry-level AI applications.

However, HBM revolutionizes the packaging architecture. It employs a 3D vertical stacking process combined with TSV through-silicon via technology, vertically stacking dozens of DRAM chips layer by layer (sounds familiar?). Through micron-level precision through-silicon vias, high-speed interconnection between each layer of chips is achieved, with shorter transmission distances and extremely low signal loss. This enables a leap in memory data throughput and prevents GPUs from being idle while waiting for data, making it a core supporting resource for AI chips.

From a performance perspective, each HBM stack has a bus width of up to 1024 bits or even 2048 bits, while ordinary DDR5 memory has only 64 bits. This is equivalent to upgrading from a "country lane" to a "highway," allowing the amount of data transmitted per unit time to increase by orders of magnitude.

In terms of bandwidth, GDDR6 memory on graphics cards has only about 0.064 TB/s of bandwidth, while a single HBM3e stack can achieve 1.2 TB/s, and HBM4 can exceed 3.3 TB/s. The "feeding speed" is nearly 20 times that of GDDR.

The root cause of HBM dominating the storage market pricing lies in the mismatch between supply and demand and the capacity siphon effect. Since the global AI industry fully erupted in 2023, the demand for HBM memory for deploying AI computing centers and building computing clusters has exploded.

At the same time, the production technology for HBM memory has extremely high barriers, and market capacity is almost entirely monopolized by storage giants like SK hynix and Micron. Even the capacity orders for 2026 and 2027 are already fully sold out, causing HBM memory prices to soar continuously. The profit margin per unit is several times or even dozens of times that of ordinary DDR memory.

To seize the high-end market and earn high profits, storage manufacturers have been adjusting their overall capacity structure, diverting core production lines, high-end equipment, and technical teams entirely to the R&D and production of HBM products. They have also proactively reduced the production capacity of traditional DDR memory, directly leading to a continuous tightening of DDR supply, and naturally, prices have risen. As for what we can do, besides working hard to earn money, we can only wait.