Nvidia has developed its new NVHBM memory for artificial intelligence chips. The company's new solution offers more bandwidth compared to the HBM4E standard while also reducing power consumption.
Significant Touch to HBM Architecture
NVHBM is positioned as an extension of Nvidia's NVLink Fusion platform. The fundamental change in the solution is the removal of the memory controller from the XPU and its direct placement onto the HBM's base die.
According to Nvidia, this approach increases both memory performance and energy efficiency compared to standard HBM solutions. The company states that it will achieve more than a 30 percent increase in memory bandwidth and a 15 percent reduction in power consumption compared to standard HBM4E.
Removing the memory controller from the XPU also provides a significant gain in processor die area. Nvidia states that this change can free up to 25 percent more space on the main chip, and the reclaimed area can be used for additional computational capacity.
According to the company's comparison with the JEDEC HBM4E standard, the area required for PHY and support circuits can be reduced by up to 67 percent. A narrower memory interface also simplifies connections on the interposer. Nvidia states that this could provide up to 80 percent more usable silicon area across the entire layout.
Preparing for a New Era
Nvidia aims to increase memory bandwidth, which has become critical due to models with trillions of parameters and growing AI workloads. NVHBM is expected to reduce bottlenecks, especially in agentic-based and physical AI applications.
On the other hand, similar efforts are underway in the industry. In addition to Qualcomm's HBC solution, Samsung, SK Hynix, and Micron are also working on similar HBM architectures. However, Nvidia plans to make NVHBM a standard solution that can be offered by multiple memory manufacturers.
Amazon's Annapurna Labs team will be one of the first NVHBM users. The two companies will work to combine the technology with the NVLink scaling architecture to enhance AI performance and energy efficiency in AWS infrastructure.
Amazon's next-generation AWS Trainium chips will also leverage Nvidia's NVLink Fusion ecosystem. According to the company's statement, the new generation of Trainium chips, starting with Trainium4, will use NVLink Fusion to connect Nvidia GPUs with Amazon's own chips under a common rack-scale architecture.
NVHBM technology is expected to be used for the first time in the Feynman generation of GPUs, which are anticipated to be released in 2028.
0 Comments: