SK hynix’s StreamDQ: Breaking the Memory Wall for AI Inference
SK hynix has introduced StreamDQ, a near-memory dequantization architecture designed to accelerate LLM inference. By performing weight dequantization directly in HBM, the system reduces data transfer bottlenecks and improves energy efficiency.
As Large Language Models (LLMs) continue to scale, the hardware industry is facing a massive "memory wall" challenge. SK hynix is addressing this with a new technical paper titled "StreamDQ: Near-Memory Weight DeQuantization in Custom HBM." The proposed architecture shifts the task of dequantization—converting compressed low-precision weights back to higher precision during computation—closer to where the data lives: the High Bandwidth Memory (HBM).
Traditionally, dequantization happens in the GPU or NPU, which requires moving vast amounts of compressed data across the memory bus, creating bottlenecks and consuming significant power. StreamDQ implements a lightweight dequantization logic within the HBM stack itself. This allow the system to stream expanded data directly to the processors, significantly increasing the effective bandwidth and reducing the latency of AI inference.
This advancement is critical for the "agentic AI" era, where models must respond in real-time with low power consumption. By optimizing the semiconductor architecture for the specific mathematical requirements of modern AI models, SK hynix is showing how custom silicon can provide exponential gains over general-purpose designs. As AI workloads become more specialized, we expect to see more logic "seeping" into the memory layers of the chip stack.
Source: Semiconductor Engineering
