D-Matrix bets stacked DRAM can solve inference’s memory bandwidth problem

What happened
D-Matrix is working on Raptor, its upcoming AI inference chip, which uses a design that stacks 3D DRAM memory underneath the compute logic. This approach is intended to address memory bandwidth limitations that can slow down AI inference tasks. The company demonstrated the core technology for Raptor with a test chip called Pavehawk, and plans to release Raptor next year. D-Matrix is also integrating its accelerators with Nvidia's NVLink Fusion infrastructure to connect with existing AI systems.
Key details
- Raptor is D-Matrix's second-generation inference chip featuring stacked 3D DRAM.
- The Pavehawk test chip was used to validate the 3D DRAM stacking technology.
- Raptor aims to deliver 10 times higher energy efficiency and bandwidth compared to high-bandwidth memory (HBM).
- Pairing D-Matrix's Corsair-based servers with Nvidia Hopper GPUs can increase hardware utilization from 20% to 80%.
- D-Matrix is adopting Nvidia's NVLink Fusion to connect its accelerators to Nvidia AI factories.
Why it matters
Companies running large AI inference workloads could see faster processing and lower energy costs if they use hardware like Raptor. Data centers and cloud providers integrating D-Matrix chips with Nvidia systems may achieve higher utilization rates, which can reduce operational costs and improve performance. This could allow organizations to generate more results per dollar spent on AI infrastructure.