… Memory is already in short supply and HBM is among the scarcest. This is where being a South Korean company with close ties to both the SK chaebol and Samsung comes in handy. SK Hynix and Samsung are the largest suppliers of HBM in the world. …
… That compute is tied to a relatively large pool of SRAM memory, which is orders of magnitude faster than the high-bandwidth memory HBM found in GPUs today, but is also incredibly inefficient in terms of space required. Each LPU only has enough die space for 500 MB of on-chip memory. …
… Seagate and Supermicro say a smart AI stack separates short-term memory flash from long-term memory disk and uses each tier for what it does best: Real-time access tiers GPU HBM memory, CPU DRAM, local and network NVMe SSDs : handle the “right now” context — active tokens, hot embeddings, and frequ… …
… To recap, Rubin packs up to 288 GB of HBM4 memory good for 22 TB/s of bandwidth and 35-50 petaFLOPS of dense NVFP4 performance depending on the use case. …