Samsung says zHBM enhances energy efficiency threefold compared with HBM5 and reduces thermal resistance by more than half. It also cuts the distance data must travel by stacking memory above the AI accelerator, yielding higher bandwidth and better power efficiency. Answered
Samsung reveals zHBM: future RAM is stacked above AI accelerator, boasts 8x performance of HBM5 and 3x energy efficiency… The SoC also exceeds Nvidia's A100 INT8 energy efficiency by an order of magnitude, the joint paper claims. Yet, these claims are largely unsubstantiated. …
… Energy efficiency is another target. Running coolant at higher temperatures reduces the need for power-intensive chilling equipment. As AI systems continue increasing in density and power consumption, reducing the energy overhead associated with cooling becomes increasingly important. …
… Also, as Gizmodo points out, Nvidia’s blog post doesn’t mention the cost of building this style of data center vs. one using less efficient air cooling, but claims that “every cloud provider and data center operator building for Rubin is making the transition.” The efficiency gains are partly due t… …
… Performance gains for the AI era The nanostack architecture could pave the way for 50 percent higher computing performance or 70 percent greater energy efficiency than IBM’s previous generation of 2-nanometer node chips, according to projections from the company’s published technical reports. …
… Samsung says zHBM enhances energy efficiency threefold compared with HBM5 and reduces thermal resistance by more than half. It also cuts the distance data must travel by stacking memory above the AI accelerator, yielding higher bandwidth and better power efficiency. …
… Why this matters The industry's largest companies are now under pressure to demonstrate that future AI growth can occur without placing unsustainable demands on water and energy resources. NVIDIA is not alone here — Microsoft has made similar zero-water cooling claims for its newest data centers. …
… Amazon claims its data centers are seven times more water-efficient than the industry average, based on an adjusted number from a peer-reviewed research paper released last year.
AI + ML PrismML debuts energy-sipping 1-bit LLM in bid to free AI from the cloud Bonsai 8B model is competitive with other 8B models but 14x smaller and 5x more energy efficient PrismML, an AI venture out of Caltech, has released a 1-bit large language model that outperforms weightier models, with … …
… This 12-layer design works out to 48GB of capacity – that is per stack, of course, and most AI accelerator designs will use multiple stacks. SK hynix claims that its HBM4E memory is 20% more power efficient than the previous HBM4. …
… Chevron, according to Gustavson, will “use some of that compute” generated by the power from the power plant serving Microsoft “to actually power AI inside of our company.” “Chevron and Microsoft have worked together for years to accelerate digital transformation, leveraging the capabilities of a t…
