A Glance at Financial Data Storage Top-Level Architecture White Paper

Recently, Huawei unveiled the Financial Data Storage Top-Level Architecture White Paper, aimed at guiding financial institutions in building robust data storage foundations. Here is a snapshot of how Huawei's storage architecture design and planning address major challenges in data centers, which serve as the information hubs of financial institutions.

Trends and Challenges

1. Explosive data growth

As internet finance and mobile payments surge, financial data is increasing exponentially, with the overall data volume in the industry having reached the Exabyte (EB) level. In 2023, China's Financial Information and Technology Institute (FITI) reported that Chinese financial institutions' data volumes were at the Petabyte (PB) level, and such data is projected to grow by 24.33% annually over the next five years. All of this is prompting financial institutions to demand higher storage system capacity, performance, and scalability.

2. Business continuity assurance

Given the high value of financial data, storage systems must be highly reliable and stable. Interruptions can lead to transaction and capital losses, causing customer churn. For example, a medium-sized bank processes 1,000 to 3,000 transactions per second, meaning each second of downtime could result in over 1,000 lost transactions, user churn, and regulatory penalties.

3. Data security and ransomware protection

Ransomware attacks in the financial industry are on the rise. According to Veritas, mentions of cyber attacks in annual reports of finance enterprises increased by 55% over the past three years. In 2022, mentions of ransomware rose by 88% compared to 2020. A 2021 white paper on data disaster recovery in China in 2021 highlights a significant lack of ransomware protection measures, with only about 10% of backups being remote and local backups often incomplete. Thus, financial institutions urgently need a robust data security foundation.

4. Intelligent O&M

Financial institutions should adopt intelligent management and automated O&M for storage resources to cut costs and enhance efficiency in dynamic storage environments. Flexible scheduling, resource visualization, and efficient O&M are essential for financial services in the intelligent era.

Target Architecture Design and Planning

1. Production & Transactions — Resource Pools with Disaggregated Storage and Computing Architecture

Most banks report over 50% annual growth in emerging financial service transactions. To support this growth, data center infrastructure must ensure high throughput, reliability, and scalability. A decoupled storage-compute architecture is set to be a future-proof solution.

Characteristics of the reference architecture:

• Tiered service systems: Three or four tiers classified by service level, throughput, and reliability, each addressing distinct needs.

• Resource allocation: Compute and storage resources vary by tier. Systems in the same tier receive standard resources and implement application isolation and resource guarantees via quotas and QoS.

• Enhanced utilization: Improves resource utilization and system reliability by handling varied data read needs across physical servers, virtualization platforms, and container platforms.

• Unified resource pools: Layered and decoupled pools of the same tier support unified procurement, configuration, and O&M, forming a standard system.

2. Data Analytics — Decoupled Storage-Compute Architecture for Big Data

According to IDC PeerScape, the big data market expenditure of China's financial industry reached US$2.97 billion in 2023 and is projected to rise to US$6.46 billion by 2027, with a CAGR of 21.4%. The focus of big data platforms has shifted from data processing to data value mining, making cost reduction and efficiency improvement key requirements.

Characteristics of the reference architecture:

• Lower costs and higher efficiency: All-flash scale-out storage achieves 10 times higher device performance compared to general-purpose HDD servers. Decoupled storage and compute resources improve CPU utilization by over 50% and reduce device quantity by over 30%.

• Smooth evolution is ensured with metadata gateways for data migration, HDFS and S3 protocol support, and multi-vendor adaptability.

• Efficient management is achieved with a global file system, SmartQoS, and robust remote DR capabilities.

3. Finance AI Services — AI Data Lake Architecture

As AI models evolve from natural language processing (NLP) to multimodal capabilities, their parameters have surged from hundreds of billions to tens of trillions. AI has proven valuable in finance by optimizing product marketing, enhancing risk management, improving operational and development efficiency, and enabling new services. And as the foundation of intelligence, data is pivotal in the training and optimization of AI models.

An ideal AI data lake solution should have the following core capabilities: Global data management and efficient data circulation; high performance; large capacity and scalability; data resilience and reliability; real-time knowledge update and memorization; and cost-effectiveness.

4. High-Availability Data Center Base — Geo-Redundant 3DC Architecture

With the digital and intelligent transformation of global financial services, storage faults often disrupt services. To address this, high-availability disaster recovery (DR) systems are required for core services in the financial industry.

Characteristics of a geo-redundant 3DC architecture:

• Multi-level protection and higher reliability: Three data copies + local active-active + intra-city replication + remote replication; local RPO = 0, RTO ≈ 0, and intra-city RPO = 0

• Diversified DR solutions: For example, the "active-active +synchronous replication" solution supports conversion between synchronous and asynchronous replication, to ensure robust intra-city replication.

• Easy management: Easy operation and deployment and quick DR drills facilitate quick DR failover.

5. Data Protection

The financial industry is known for its high informatization and strict information security standards. Ransomware attacks occur frequently. As such, the financial industry urgently needs to build a reliable cyber security protection system to maintain the security and stability of the global financial system.

The financial industry's mainstream DR and backup solution centers on storage products, adhering to the following "32110" principle for data backup:

• Three physically separate copies

• Two different media to support redundancy of backup data

• At least one offline copy stored at the intra-city/remote site

• One copy at the isolation zone, which is protected by anti-tampering technology

• Zero error upon recovery to ensure backup data security and recoverability

Summary: Amidst these opportunities and challenges, Huawei keeps innovating and exploring high-reliability, high-availability data storage architectures to guarantee continuous and stable business operations.