Constructing the World's First SKA Prototype with Huawei OceanStor Mass Data Storage

Shanghai Astronomical Observatory has deployed Huawei OceanStor mass data storage to build prototype core service storage architecture for the Square Kilometer Array (SKA), an international scientific project that aims to build the world's largest radio telescope, with the aim of eventually having a collection area of over a square kilometer in total.

Conceived in the early 1990s and initiated by more than 10 countries, the telescopes that form SKA will be installed in areas with little radio interference, across sites located in eight countries including Australia and South Africa. Shanghai Astronomical Observatory released its SKA regional center prototype at the SKA Shanghai Meeting in November 2019 and it was recognized by the SKA Organization as the first to be developed, As such, it was selected as one of the top 10 national astronomical science and technology advances in China in 2019.

Data Challenges Lie Ahead

SKA will produce the largest data flow ever seen in astronomy's history. Just the first phase of SKA — which accounts for just 10% of the entire project — will create data at a growth rate of approximately 106 PB each year once operational. By 2027, the data volume generated by SKA is expected to reach 760 PB.

Clearly, the construction of supporting storage systems is critical for the project, given such data volumes as well as associated High-Performance Computing (HPC)clusters and a global SKA user base.

These storage systems are going to have to offer extremely high bandwidth and low latency, to support those HPC clusters and process literally astronomical big data. Dynamic capacity expansion and a modular design, to maximize Return On Investment (ROI), are also key, with the ability to smoothly expand critical to minimizing Total Cost of Ownership (TCO). And, of course, total reliability with zero tolerance for data loss is a given.

Handling Oceans of Data

With an innovative Erasure Code algorithm ensuring data reliability, fully distributed architecture allows for dynamic and flexible capacity expansion. Performance increases linearly as more nodes are added, meeting the data growth needs of the SKA regional center over the next few years.