New Apps, New Data, and New Resilience Drive the Development of Data Storage

Dr. Peter Zhou

President of Huawei IT Product Line

The pace of digital and intelligent development is faster than ever. As the digital transformation of various industries deepens and reaches a critical point, the data storage market is rife with challenges and opportunities.

There is an explosion of new applications. More than 56% of enterprises are deploying AI applications in their production and decision-making systems.And more and more cloud-native applications are shifting from public clouds to on-premises data centers.

New data is also growing. 80% of new data is unstructured, with a compound annual growth rate (CAGR) of 38%.

And finally, new resilience is increasingly being put to the test. Ransomware variants are growing at a rapid rate of 98% year over year, and 14.1% of businesses have had their data permanently damaged after a ransomware attack.The data storage industry as a whole needs innovative technologies and products to address these rapid changes and challenges.

New Apps: Storage Will Need to Accelerate Value Mining Through Data Diagrams for New Applications.

Take the training of self-driving vehicles as an example. The data involved to train them includes traffic signals, road signs, pedestrians, weather conditions, speeds, and the distance between vehicles. All of that data needs to be stored and most of data needs to be analyzed in real time. Thanks to large-scale AI models, we can figure out basic and advanced data patterns. But the data timing, convolution, and pooling needed for these models all increasingly rely on the performance, reliability, and resilience of the data storage resources.

The collaboration model between data storage and applications is being reshaped. Storage systems will need to be able to provide near-data processing capabilities, for new data formats, high-performance shared caching, and network traffic aggregation. This will help AI-based big data models deliver easier data perception and quicker decision-making for deep neural networks, convolutional neural networks, and recurrent neural networks. All of these capabilities help facilitate the research and system training needed for autonomous driving.

In addition, data storage needs to adapt to various cloud-native applications, as more and more new applications are shifting from public clouds to on-premise data centers. The infrastructure of traditional data centers cannot accommodate these new kinds of applications as it lacks appropriate container technologies. Future data storage will need to be able to better support a container ecosystem.

There is an increasing demand for storage with integrated container storage interfaces (CSIs) and expanded container disaster recovery (CDR) interfaces. Moving upper-layer applications to the cloud will allow storage systems to seamlessly interconnect data between different data centers while building data persistence. Huawei Data Storage offers the industry's first container storage solution with active-active disaster recovery (DR), which can achieve zero RPO and less than 90 seconds RTO. Currently, more than 80 strategic organizations around the world have chosen Huawei's container storage solution.

New Data: Storage Needs to Resolve Problems from Data Gravity with Intelligent Data Fabric

Take China Mobile as an example. China Mobile has a large data system, with data being generated across the country and stored locally. Data fabric technology allows data to be stored across multiple locations in a similar way to how it would be on a single device, better supporting upper-layer applications to unlock the value of data.

Storage systems need to provide unified global data view and scheduling functions across systems, regions, and clouds. Leveraging a Global File System (GFS), Huawei's storage solutions allow for data to be easily accessed between different clouds (private or public), and even to heterogeneous storage. The global data view is always available and ready for customers, no matter which kind of clouds they use. This enables global data invocation and usage to help obtain "data-centric applications" and valued data.

Unstructured data facilitates decision-making and production processes, and the volumes of unstructured data that we deal with are growing every day and every year. This means we need higher performing scale-out storage, which is designed for unstructured data. In addition, unstructured data needs a higher-performance storage architecture than backup and archive data. Bandwidth and IOPS will need to be increased by 100 times and 1000 times, respectively.

Take Shanghai Ruijin Hospital as an example. A single pathologist can retrieve over 1000 slide images every second for comparison, analysis, and modeling. Huawei's scale-out storage is a perfect match for this kind of unstructured data, accommodating massive amounts of data without notably raising costs. The performance of Huawei’s storage solutions comes from their adoption of innovative software (advanced data coding and compression algorithms) and hardware technologies. This creates higher system density, increasing the storage capacity per unit by more than 30%.

New Resilience: Storage Needs Intrinsic Data Security Capability

Storage also needs to defend against both environmental and human damage, especially as we see more and more risks from ransomware. The embedded data security features enable the storage to proactively defend against damage. A typical IT infrastructure is layered (app, network, and premier) and so requires layered protection. Since it is a key part of IT infrastructure, there is no doubt that the storage must come equipped with embedded data security capabilities to act as a last line of defense for data resilience.

When you had a dragon's hoard of treasure, you would want to build a castle with heavy gates and have teams of soldiers patrolling the area to protect it. However, the most important part of your security system would be a safe. It would be the final line of defense. Simply putting your money on a table without a safe would make it easy for any crafty enough thief to trick their way in and out of your castle.

Many enterprises rely on tools like firewalls and intrusion detection systems at the edge, setting sandboxes and blocklists at the network layer, and implementing access controls at the application layer. However, these measures may fall short when it comes to actually safeguarding critical data. To address this concern, Huawei data storage serving as the last line of defense by leveraging advanced capabilities, including precise detection, Write Once Read Many (WORM), data encryption, secure snapshots, and air gaps, to ensure unbreakable systems, unmodifiable data, and recoverable services.

Since its inception in the 1990s, highly reliable and high-performance SAN storage has been the go-to choice for core database applications. Around the year 2000, with the Internet gaining widespread popularity, a need for reliable file storage and efficient file sharing increased. This gave birth to the NAS storage system. In 2010, with the rise of "cloud computing" and the need for resource pooling in data center construction, a unified storage system that integrated both SAN and NAS became an ideal choice, as it supports multiple VM applications simultaneously. Then, around 2015, all-flash storage systems became increasingly popular due to their high performance, high reliability, and energy efficiency, while HDD storage was widely being replaced. All-flash storage systems were a perfect fit for the mobile era, as they met the requirements for a tenfold increase in application performance.

New apps, new data, and new resilience will propel the development of the data storage industry. The total shipment of top 5 External Controller-Based (ECB) storage vendors in 2022 was 3 times higher than that in 2012. The shipments are expected to grow another 10 times over the next decade, to 100 EB of data in 2032, ushering in the YB era.

Disclaimer: The views and opinions expressed in this article are those of the author and do not necessarily reflect the official policy, position, products, and technologies of Huawei Technologies Co., Ltd. If you need to learn more about the products and technologies of Huawei Technologies Co., Ltd., please visit our website at e.huawei.com or contact us.