Wendell Wenjen, director of storage market development at Supermicro, discusses The explosive growth of enterprise AI – from generative tools to agentic models – and how it is fundamentally reshaping data centre design. For years, storage was an afterthought, but as inference workloads become increasingly distributed and data-intensive, poorly optimised infrastructure now poses a significant threat to performance, costs, and even sustainability goals.
The surge in enterprise AI workloads, from generative AI to agentic models, is elevating storage as a critical priority in data centre design that could bottleneck performance significantly. For years, storage was treated as a back-end concern, isolated from AI strategy. That approach no longer holds. As AI inference becomes more distributed and data-heavy, poorly optimised storage infrastructure risks throttling performance, increasing costs, and complicating sustainability goals.
In response, many IT leaders are re-evaluating their infrastructure stack, weighing on-premises versus hybrid storage, and seeking more flexible, software-defined architectures. From cost and sovereignty concerns to high-throughput demands in AI pipelines, storage is now a key design priority — not an afterthought. This article explores four emerging trends that are reshaping storage strategies for AI-driven enterprise environments.
Challenging on-premise sovereignty and security concerns
Rising geopolitical instability and changing data protection laws are pushing more businesses, particularly in Europe and Asia, to keep sensitive AI training data on-premises. This infrastructure offers better control over performance and latency, which is crucial for inference workloads that rely on large local datasets and tight compute-storage integration. Data locality matters in AI, especially when training on sensitive or regulated datasets such as health records, financial transactions, or IP-protected industrial data.
Enterprises are increasingly turning to composable, rack-scale solutions where compute, networking, and storage can be configured and secured in line with internal compliance requirements. The AI use case is prompting organisations to invest in scalable, turnkey solutions and parallel file systems that can be owned and operated internally. Such a structure gives data centre designers more flexibility over long-term data retention, orchestration, and auditability.
How a hybrid cloud has shifted the balance
Public cloud remains critical for burst capacity and early-stage AI concepts, but many enterprises are finding cost and performance limitations as workloads scale. Hybrid strategies combine cloud economics with on-premise control. These offer a practical route for businesses looking to scale AI inference whilst maintaining control over their capital and operating expenses (CAPEX and OPEX). Object storage platforms are increasingly being deployed across both environments. These allow models to be trained in the cloud but inferenced closer to where the data is generated or needed.
Another limitation is outdated or obsolete technology. Enterprises are modernising their legacy storage systems with software-defined object or file systems that can bridge cloud and on-prem seamlessly. The right hybrid architecture can also mitigate vendor lock-in, enable data tiering, and help balance CAPEX and OPEX across the AI lifecycle – from experimentation to production rollout.
AI and its energy requirements
AI-ready storage systems – particularly those feeding high-throughput GPU clusters – need to be designed to balance compute density with cooling and energy efficiency. Traditional air-cooled systems struggle to meet the power density needed for large AI training or inference clusters; direct liquid cooling and immersion are now key design considerations. Storage-optimised servers using PCIe Gen 5, GPU Direct Storage, and RoCE are helping reduce latency and CPU load by streamlining direct access from storage to GPU memory.
Power efficiency must be engineered holistically, considering CPU design and integration (as different brands have different requirements), memory bandwidth, and the Input/Output connections between storage and AI accelerators. As PUE (Power Usage Effectiveness) becomes a board-level KPI, businesses are demanding more pre-tested, rack-scale storage solutions that combine energy savings with predictable performance.
Rethinking storage as a resilient foundation
Modern AI inference workloads, including agentic AI and RAG, rely on constant retrieval from large-scale vector databases and unstructured data lakes, making low-latency storage non-negotiable.
Software-defined storage platforms are replacing rigid appliances, enabling IT teams to customise file, object, and block storage for different AI pipelines. Furthermore, AI-native orchestration software is emerging to manage data lifecycle, automate tiering, and coordinate training data ingestion, model versioning, and real-time inference.
Enterprises are adopting tiered storage architectures, featuring an all-flash performance layer and a durable object layer for long-term training datasets. Testing and validation are critical to storage success, and like any technology, such platforms must be stress-tested under real-world AI workloads. This will ensure sustained throughput, fault tolerance, and consistent latency under load.
