This week's biggest shock: The race to 200Gbps has been won

Comment As we reported today, Mellanox announced today it has broken through the 200Gb/sec barrier. We're told it will ship an end-to-end 200Gb/s interconnect solution in the first half of 2017.

Mellanox will refresh its Quantum smart switches, ConnectX 6 adapters, along with the cables that connect the two together. What does this mean for you folks expected to test and deploy this tech?

Looking at the switch first, this is more than a speed bump upgrade. Their new high-end Quantum switch will sport 80 ports of 100Gb/s EDR or 40 ports of 200Gb/s of HDR. The entire switch has 16Tb/s capacity, which puts it at the top of the stack, competition-wise.

It can also handle more than 390 meeelion messages per second, per port, which is twice as high as anything we’ve seen before.

All of this speed pays off when you’re configuring the switch into your cluster. Using Quantum as a core switch at 100Gb/s, it can support up to 1,600 nodes with two layers of switches. Other high-speed switches would need three layers of switches, plus the associated cables and power, to get close to that number.

Quantum also allows customers to configure a larger number of nodes for the money. According to Mellanox, 128,000 nodes are possible in a three-level fat-tree topology using Quantum at 100Gb/s speed. The biz claims this is 4.6 times larger than the next competitive solution.

On the adapter side, Mellanox announced the next generation ConnectX 6 adapter, which claims to offer full 200Gb/s throughput, latency of 0.6 microseconds end to end, and can handle 200 million messages per second.

It's not all about raw speed, though. Mellanox utilizes offload technology in their network products. This means that both the switch and the adapters handle the entire network processing load – taking strain off the CPU.

But this new 200Gb/s gear goes quite a bit farther – it handles many of the MPI computing chores as well. The adapter alone can perform computational chores for many several MPI collectives (like broadcast, scatter/gather, matching, etc.). The Quantum switch is one of the first truly smart switches, and can handle tasks like aggregation and reduction within the switch, as the data is being streamed to the node or nodes.

For contrast, in on-load mechanisms, the CPU is responsible for handling all network and MPI processing tasks. This can cause them to get bogged down with network communications, causing application performance to suffer. Offload takes those tasks away from the CPU, allowing it to continue application processing, which can radically increase application throughput.

Mellanox believes that processing should occur wherever there is data, including the data paths connecting nodes to each other. It’s a philosophy that has obviously paid off, given that one of the next fastest publicly known computers in the world will its InfiniBand technology, along with 50 per cent of the petascale systems on the TOP500 list. ®