AI Inference Reshapes Storage Hierarchy as SSDs Move to Center of Operations

AI inference workloads are restructuring the data storage hierarchy. A new tier of SSD infrastructure is emerging between GPU memory and traditional data lakes.

AIC Inc. and Solidigm, a trademark of SK Hynix, are leading this industry shift. Their collaboration targets two critical challenges: moving data fast enough to feed GPUs and scaling capacity for exponentially growing AI-generated data.

CT Sun, vice president of engineering at AIC, highlighted the necessity of low-latency capacity during the Nvidia GTC AI Conference. He noted that AI-generated data volumes now exceed natural world data inputs. To close the memory gap, AIC is building non-volatile memory express platforms with data processing unit capabilities.

The upcoming BlueField-4 DPU, integrating Nvidia’s Vera Rubin architecture, will accelerate this transition. Solidigm is addressing thermal constraints with liquid-cooled E1.S SSD form factors. Pompey Nagra, product manager at Solidigm, warned that supply and demand gaps in flash storage may persist for three years.

Strategic allocation is essential as storage capacity demands grow exponentially within single systems. Liquid cooling has become a paramount requirement for next-generation servers.