Huawei Helps DTU Build a Supercomputing Cluster

Background

For almost two centuries DTU, Technical University of Denmark, has been dedicated to fulfilling the vision of H.C. Ørsted — the father of electromagnetism — who founded the university in 1829 to develop and create value using the natural sciences and the technical sciences to benefit society.Today, DTU is ranked as one of the foremost technical universities in Europe.

Challenges

DTU promotes promising fields of research within the technical and the natural sciences, especially based on usefulness to society, relevance to business and sustainability. DTU focuses on basic science that has significant challenges and clear application prospects, from atomic-scale materials analysis to quantum physics and renewable energy. As the material application environment becomes increasingly complex, laboratory research for materials performance analysis has become even more challenging.

DTU aims to understand the nature of materials by developing electron structural theory, and design new functional nanostructures through new-found insights. These studies require the analysis of the structure, strength, and characteristics of new materials, involving intensive, complex numerical computation and simulation tests on material and energy. This will produce a vast number of computational data. Therefore, High-Performance Computing (HPC) resources that can accelerate performance modeling and solving are particularly important to research in this field.

In order to speed up the process from discovery to application of new materials and maintain a leading edge in research, DTU needed to expand and upgrade its supercomputing cluster, Niflheim, which is deployed at the Computational Atomic-scale Materials Design (CAMD) Center.

Solution

The existing Niflheim cluster at DTU was built from 2009 to 2015, and was capable of a peak computing capability of only 73 TFLOPS. The cluster was equipped with previous generation and even earlier computing product hardware. The oldest products had limited processor performance, small memory capacity, with low-bandwidth but high-latency computing network. The old cluster was failing to meet the growing demands of computing-intensive simulation tests. As a result, the cluster became a bottleneck since the CAMD center needed research efficiency improvements.

DTU wanted to deploy a new supercomputing system to give the Niflheim cluster a boost in computing resources and performance, and meanwhile also prepare the cluster for future technology evolution as well as cluster-scale expansion. DTU carefully studied various solutions in terms of overall performance, product quality, and service capabilities, and finally selected Huawei as the vendors to help the university build a new-generation computing cluster with their innovative technologies and computing products. 

Figure 1.  Supercomputing cluster running at the CAMD center of DTU

Figure 2.  Huawei X6800 (XH620 V3) high-density server

Solution highlights:

Nodes up to 845 GFLOPS compute power per node

Nodes configured with 256 GB DIMMs and 240 GB SSDs, eliminates I/O bottlenecks, and improves data processing efficiency with high-speed data caching

Omni-Path Architecture (OPA) builds a two-layer fat-tree fabric, delivers bandwidth of up to 100 Gbit/s, and end-to-end latency as low as 910 ns

Power Supply Units (PSUs) and fan modules shared by multiple nodes, enhanced with Huawei’s Dynamic Energy Management Technology (DEMT), lower system energy consumption by over 10 percent

4U chassis configured with eight 2-socket compute nodes, delivers computing density twice that of traditional 1U rack servers, significantly improves rack space utilization

Supports aggregated management network port for unified management, meanwhile reduces cable connections

Uses a modular design, can hot swap all key components, greatly improves Operations and Maintenance (O&M) efficiency

Figure 3.  New-generation Niflheim supercomputing cluster

Benefits

The new-generation Niflheim cluster went live in December 2016. It helps more researchers carry out research and analysis on new materials and new energy, but also provides a great leap in feedback speeds of test results. It has enabled new levels of scientific research progress and strength, helping DTU generate new innovation capabilities in the field of material analysis.

The Niflheim cluster delivers a computing power of up to 225 TFLOPS, which is three times the level of the original system

Substantially shortens material analysis time, enabling researchers to discover and apply new materials more quickly

With flexible expandability, the cluster can seamlessly expand up to 112 nodes without requiring additional cabinets