HBM is the DRAM industry’s attempt to short-circuit the slowing pace of Moore’s Law by using 3D chip-packaging technology. Each HBM chip is made up of as many as 12 thinned-down DRAM chips called dies. Each die contains a number of vertical connections called through-silicon vias. The dies are piled atop each other and connected by arrays of microscopic solder balls aligned to the TSVs. This DRAM tower—well, at about 750 micrometers thick, it’s more of a brutalist office block than a tower—is then stacked atop what’s called the base die, which shuttles bits between the memory dies and the proc
How Will the Tech Industry Handle the DRAM Shortage?… This article appears in the April 2026 print issue as the feature “ AI is a Memory Hog.” The sidebar from that article is here “ Low-Cost Computers Nearly Double in Price as RAM Shortage Hits. “ From Your Site Articles A Radical Proposal: Replace Hard Disks With DRAM › Hynix Makes No-Capacitor DRAM…
… High‑bandwidth memory HBM marks a major inflection along this path. By stacking DRAM dies — scaling to 16 layers and beyond — and placing memory much closer to the processor, HBM enables rapid access to ever‑larger working datasets. …
… Rabii says that most memory interfaces are designed to operate over a short physical distance—sometimes only a few millimeters. That limits how much memory can be placed. “You get this shoreline at the compute die where you can put your HBM. …
… Memory bandwidth and data flow Groq’s approach to accelerating inference relies on interleaving processing units with memory units on the chip. Instead of relying on high-bandwidth memory HBM situated next to GPUs , it leans on SRAM memory integrated within the processor itself. …
… Tensordyne’s Napier AI chip includes 144 gigabytes of HBM, but the real power comes from its unusual math. …