Yes, AMD has a secret weapon to fight off Nvidia AI armada — no, it has absolutely nothing to do with GPUs and everything to do with HBM

Key Takeaways:

– AMD aims to surpass Nvidia as the leading provider of components for generative AI systems by leveraging advancements in high-bandwidth memory (HBM).
– Xilinx, owned by AMD, showcased its Virtex XCVU7P card at OCP Summit 2023, featuring eight accelerator-in-memory (AiM) modules, in collaboration with SK Hynix’s HBM3E memory unit.
– Performing compute operations directly in memory eliminates the need for data movement between components, resulting in improved performance and energy efficiency.
– PIM (processor-in-memory) technology, combined with SK Hynix’s AiM, demonstrated ten times shorter server latency, five times lower energy consumption, and reduced costs in AI inference workloads.
– Nvidia is also incorporating HBM technology into its GPUs, including the A100, H100, and GH200 models, through a partnership with Samsung.
– Several companies, including Samsung, have been exploring PIM technology, with Samsung showcasing its processing-near-memory (PNM) module and an HBM-PIM card prototype developed with AMD.
– The addition of an HBM-PIM card in collaboration with AMD increased performance by 2.6% and improved energy efficiency by 2.7% compared to existing GPU accelerators.

TechRadar:

AMD will rely on advancements in high-bandwidth memory (HBM) in its bid to unseat Nvidia as the industry leader for making the components that power generative AI systems.

Building on the theme of processor-in-memory (PIM), Xilinx, which is owned by AMD, showcased its Virtex XCVU7P card, in which each FPGA had eight accelerator-in-memory (AiM) modules. The firm showcased this at OCP Summit 2023, alongside SK Hynix’s HBM3E memory unit, according to Serve the Home.

Source link

AI Eclipse TLDR:

AMD is looking to surpass Nvidia as the leading provider of components for generative AI systems by leveraging advancements in high-bandwidth memory (HBM). Xilinx, a company owned by AMD, demonstrated its Virtex XCVU7P card at the OCP Summit 2023, featuring eight accelerator-in-memory (AiM) modules for each FPGA. SK Hynix’s HBM3E memory unit was also showcased alongside it. By performing compute operations directly in memory, data does not need to move between components, resulting in improved performance and energy efficiency. Using processor-in-memory (PIM) with SK Hynix’s AiM, server latency was reduced by ten times, energy consumption was five times lower, and costs were halved for AI inference workloads. Nvidia is also incorporating HBM technology into its GPUs, and both companies see significant advantages in power and efficiency by optimizing the relationship between compute and memory. Samsung has also pursued PIM, showcasing its processing-near-memory (PNM) module with a bandwidth of up to 1.1TB/s. AMD collaborated with Samsung on an HBM-PIM card prototype, which boosted performance by 2.6% and energy efficiency by 2.7% compared to existing GPU accelerators.