[Shanghai, China, September 21, 2023] At HUAWEI CONNECT 2023, Huawei released a series of white papers titled Striding Towards the Intelligent World, which covers 10 major industries including data storage. The Data Storage white paper details 10 expectations for the evolution of data storage based on five perspectives: new apps, new data, new resilience, new technologies, and energy savings. It also recommends actions that enterprise IT infrastructure decision makers can take to stay ahead of the curve.
AI foundation models have already gone beyond our wildest imaginations. Intelligent enterprise applications, and more specifically, AI foundation models, have permeated a range of industries, heralding the dawn of an era powered by these models. AI foundation models require more efficient collection and preprocessing of massive amounts of raw data, higher-performance training data loading and model data storage, and more responsive and accurate industry inference knowledge repositories. A new AI data paradigm, represented by near-storage computing and vector storage, is rapidly gaining momentum.
The quantity and quality of the data determine the potential impact that AI can have. Huawei offers the following advice on building optimal data infrastructure for foundation models:
1、 Enterprises should build a reliable foundation model infrastructure that attaches equal importance to compute and storage. AI foundation models are widely used in various industries. Data quality and quantity are critical factors determining the potential of AI. In addition to computing power stacking, enterprises should consider employing a storage-centered data infrastructure that provides governance of mass unstructured data, optimal throughput performance, and robust data resilience.
2、 AI foundation models should adopt a data lake construction mode that shares the same data sources as high-performance computing (HPC) and big data, and upgrade the performance of the current data lake storage.
3、 Enterprises should build forward-looking data infrastructure that includes all-flash storage, data-centric architecture, data fabric, new data paradigms (vector storage and near-storage computing), and intrinsic resilience of storage.
4、 The hyper-converged infrastructure (HCI) is highly recommended for segmented industries due to its compact design. It integrates data storage nodes, compute (training/inference) nodes, switches, AI platform software, and management and O&M software, delivering one-stop services. HCI also reduces the costs of a large amount of adaptation, optimization, and system construction.
5、 Enterprises should also assemble a team of technical experts that have the skills and experience to work with AI foundation models, particularly in the storage domain.
Typically, every application transformation comes alongside a new class of data infrastructure architectures. In addition to discussing the trends associated with AI foundation models, the white paper outlines other nine trends, including big data, distributed databases, and cloud native, and provides guidance on data infrastructure construction to help enterprises build more resilient, reliable, and efficient data foundations and embrace the ever-changing and diverse data applications in the digital era.
For more information about the white paper, please visit: https://www.huawei.com/en/giv/industries/data-storage
