Kevin Tolly2021-11-03 27
Traditional data centers may be built on multiple fabrics – Ethernet for server interconnection, Fibre Channel (FC) for storage networks, and InfiniBand for high performance compute clusters. With new development of Ethernet, such as lossless Ethernet technologies, high performance Ethernet NICs supporting RDMA and MPI, NVMe over RoCE all-flash storage systems, Ethernet-based data center network solutions have started to replace FC and InfiniBand solutions to provide equivalent or better performance. Furthermore, Ethernet solutions are typically more cost effective. They are supported by more vendors. And 400GE Ethernet switches are now widely available on the market, while current FC switches typically only support up to 64G FC ports. For all these reasons, modern data centers are moving from multi-protocol mode towards converged all-Ethernet.
Based on all of the above, Huawei launched the CloudFabric 3.0 hyper-converged DCN solution. It consists of two components: Huawei CloudEngine Ethernet data center switches and Huawei iMaster NCE-Fabric, an intelligent network management and control system. The Tolly Group focused on the storage and HPC scenarios. We'll see how the Huawei Ethernet-based solution performs against a FC solution in storage test cases, and against InfiniBand in HPC test cases.
In a centralized storage scenario, there are two major types of fabric, which are RoCE and FC. The two solutions undergoing testing are the NVMe over RoCE solutions from Huawei, and one NVMe over FC solution.
Overall, the NVMe over Fabric storage test results demonstrated three key advantages of the Huawei NoF+ solution. First of all, storage area networks must be lossless, which means the traffic rate may be pushed down. Meanwhile, the network must be able to fully use the available bandwidth for high throughput. Huawei NoF+ solution's intelligent lossless DCN ensures zero packet loss under high throughput. This is demonstrated in the high storage I/O performance results. Secondly, the Huawei NoF+ solution supports fast failover within a second for link and storage module failures. Thirdly, the Huawei NoF+ solution supports the IP-based SAN zoning feature. Once the zones are configured on the reflector switch, they are automatically synchronized to all client switches. Connections are automatically established after hosts and storage devices in the same zone are connected to switches.
In this area, RoCE and IB are the two most popular compute interconnection protocols. Tolly engineers tested Huawei's intelligent and lossless Ethernet network and IB network. The HPC tests include bandwidth and latency tests in a congestion condition, MPI benchmark tests, and HPC application tests. We'd like to introduce the in-network computing feature of the Huawei solution. This function transfers part of computing operations from computes to network switch devices to reduce communications in compute clusters, thus significantly reducing network latency in HPC scenarios and improving computing efficiency.
While Ethernet now also supports a lossless network, has higher port bandwidth, offers a cheaper price, has a more robust ecosystem, and many more future proof advantages, modern data centers are evolving into converged all-Ethernet. The Tolly Group test results show that Huawei's CloudFabric 3.0 Hyper-Converged DCN Solution can deliver up to 93% higher storage Input/Output Operations Per Second (IOPS), with 49% lower latency, compared to an FC network under the same test conditions. Huawei's lossless Ethernet solution also comprehensively outperforms an IB network, delivering up to 17% higher performance and 95% lower latency. Our test results demonstrate that in many test cases, Huawei's Ethernet-based data center network solution can provide better performance than other solutions.
Disclaimer: The views and opinions expressed in this article are those of the author and do not necessarily reflect the official policy, position, products, and technologies of Huawei Technologies Co., Ltd. If you need to learn more about the products and technologies of Huawei Technologies Co., Ltd., please visit our website at e.huawei.com or contact us.