Arista goes modular with 10 Gig E switches

The High Performance Computing Linux Financial Markets trade show is on today in New York, and Andy Bechtolsheim, twice the chief technology officer at the Oracle-borged Sun Microsystems, is using the event to launch the first modular 10 Gigabit Ethernet switch from his latest company, Arista Networks. And it looks like Bechtolsheim will, in his quiet way, be shaking up the networking market enough that someone will have to buy his company again.

The Arista 7500 is now the flagship of the company's 10 Gigabit Ethernet lineup, with 384 wirespeed ports and an aggregate of 10 TBit/sec of capacity for Layer 2/3 switching. All of this is crammed into an 11U rack-mounted unit, and according to Anshul Sadana, vice president of systems engineering at Arista, the 7500 is unique among high-end modular switches in that it has very deep buffer memory for packets - 18 GB, to be precise.

With the deep buffers, the error correction on the 7500 has enough data that it doesn't have to go asking for packets again across the network if something gets scrambled. Also, unlike other modular switches that cram a large number of ports in their leaves and rely on oversubscription of a set amount of bandwidth in the switch's brains to service those switches, the 7500 has enough switching capacity to feed its ports at linespeed thanks to that 10 TB/sec of fabric capacity. And it does so at an average latency of around 4.5 microseconds on benchmarks using small packet sizes. That's about one third to one half the latency of competitors' 10 GE switches.

The Arista 7500 packs a lot of oomph into a relatively small space, and according to Sadana, has a "breakthrough price" that is set to shake up the high-end of the emerging 10 Gigabit Ethernet market. The Juniper EX 8216 has 128 ports, only eight GB of packet buffers, 2.56 Tb/sec of bandwidth and can deliver 1.92 billion packets per second (Bpps) of throughput at the Layer three level of the network. But by Arista's math, it takes 65 watts per port to power that switch, and it takes up 21U of space.

Cisco Systems' Nexus 7010 comes in a 21U cage as well, but only has 64 10 GE ports, only 5.75 GB of packet buffer memory, only 3.6 Tb/sec of fabric bandwidth, only 960 Mpps of L3 throughput, and burns an incredible 139 watts per port. Force10 and Brocade fare a little better in the comparisons - a few more ports, a little more bandwidth, or a little less energy, and they would have to so they can compete with Cisco and Juniper.

Arista Networks' 7500 Modular Switch: So many ports, worth about $1,200 a pop

But the Arista 7500 is roughly half the size and has enough bandwidth at 5.76 Bpps that it will be able to support 40 GE and 100 GE switching when it becomes available. And using 10 GE switches, the Arista 7500 only burns about 13.2 watts per port.

A base Arista 7500 costs $140,000, and a fully configured machine with all 384 ports and other bells and whistles runs to $460,800, or $1,200 per port. This machine will draw 5,072 watts of juice and take up a little more than quarter of a rack.

Compare this to a Cisco Nexus 7010 setup to get 384 wirespeed ports and deliver the same 5.76 Bpps of L3 throughput, and you need to get 18 of the units at a cost of $13.7m. Such a configuration will draw 160 kilowatts and take up 378 rack units of space - nine full racks. Arista can do the 384 ports in 1/34th the space and 1/30th the price.

And they offer scalability, too. If you daisy chain four of these Arista 7500s together, slap in a few spine switches and a bunch of leaf switches, that gives you 1,536 10 GE ports or 15,360 Gigabit ports. And if you do a 2:1 oversubscribe ratio, you can lash together 30,720 servers at Gigabit speeds. HPC shops with embarrassingly parallel workloads, but working in a tight space and thermal budgets, are going to take a hard look at the Arista 7500s.

This kind of performance and packaging difference is why Sun became a player in servers in the beginning of the 1990s, why Granite Systems (an earlier Bechtolsheim company) became a player in InfiniBand switching at the end of the 1990s and was eaten by Cisco, and why Arista Networks can be self-funded and will likely soon either be snapped up by someone (Oracle if they have any sense, IBM if they have more) or will go public.

The 7500 modular switch joins the 48-port Arista 7048 Gigabit switch that got the company going in 2008, as well as the 7100 T 24/48 1/10 GE data center switches and the 7100 S low latency models, which have latency as low as 600 nanoseconds. ®