Everyone Will Want Higher Bandwidth And More Ports – Eventually

The speed bumps with switch ASICs are coming fast and furious these days, and the datacenter is no longer dominated by the big switch incumbents such as Cisco Systems and Juniper Networks, which made their own chips, switches, and operating systems, and the networking divisions of server OEMs like Dell and Hewlett Packard Enterprise, which made some of their own gear – including ASICs – and acquired other companies to help build out their networking businesses – who sometimes also did their own ASICs.

But in the past decade, independent switch ASIC vendors have come on strong, creating an independent merchant switch chip market and putting tremendous pressure on the switch incumbents as well as on each other. This has been a boon for innovation and has, quite frankly, jump-started the networking stack so it is back on track with the Moore’s Law curve that it fell off of in the 2000s for a lot of complicated reasons, but the big ones were the difficulties in switching to faster SERDES and vendors over-managing the Ethernet roadmap in favor of their profits over boosting performance.

Those days are gone. Long gone. The technology roadmaps are aggressive but paced, the demand for bandwidth and higher port counts is through the datacenter roof, and the competition among those who make switch chips as well as the switches themselves is fierce.

Innovium is one of the switch chip makers that has been tearing up the Ethernet roadmap since it was founded in December 2014. The company has raised $165.3 million in four rounds of funding, and like Barefoot Networks (just acquired by Intel), is trying to take a bite out of the dominant market share that Broadcom has in datacenter switch ASICs. This is no easy task, of course, but as history has shown, upstarts can and do steal away business from incumbents when they build a better mouse trap.

As you know, we are hosting the Next I/O Platform event in San Jose on September 24 (which you can register for here) and Rajiv Khemani, founder and chief executive officer at Innovium, will be on hand to talk about what switch ASIC upstarts have to do to win business among the hyperscalers and cloud builders and how the networking needs of these companies today set the stage for the networking that other organizations will need in their future datacenters. Khemani started out his career at Sun Microsystems and NetApp, and eventually became the general manager of Intel’s networking business through its acquisition of Netboost; he then spent a decade as chief operating officer and as a general manager in various divisions at Cavium and was instrumental in taking that company public in 2007.

Innovium co-founders Puneet Agarwal (chief technology officer) and Mohammad Issa (vice president of engineering) were CTO for switching and vice president of engineering at Broadcom and significantly, and they brought several generations of the highly successful “Trident” and “Tomahawk” families of network ASICs from whiteboard to datacenters over the course of a decade.

We profiled Innovium back in March 2017 when it first dropped out of stealth mode, and threw down the gauntlet rather forcefully to be the first to bring out 200 Gb/sec and 400 Gb/sec Ethernet ports with its Teralynx ASICs. Only three companies have announced switch ASICs with 12.8 Tb/sec aggregate bandwidth, which is as good as it gets these days. These include Broadcom with its Tomahawk 3 last year and its Trident 4 this year, Innovium with its Teralynx 7, and Intel/Barefoot with its Tofino 2. Mellanox Technologies, soon to be part of Nvidia, is only driving 6.4 Tb/sec with its Spectrum-2 ASIC announced in July 2017, but presumably it has a way to double that up to draw level with its competitors in the merchant switch chip market. We were under the mistaken impression that Marvell’s Prestera (XPliant) switch ASICs are not pushing the bandwidth limits in the same way, but Marvell is fielding its own Prestera 12.8 Tb/sec switch chip, which we need to drill down into.

“What has become very clear to the industry – especially with the acquisition of Mellanox by Nvidia, and that was a highly contested acquisition as you know between Xilinx and Intel and Nvidia – is that compute and networking are converging at datacenter scale and for the very first time in our history,” Khemani tells The Next Platform. “This compute and network convergence is being driven by applications such as distributed storage as well as AI, where the network has to do more than just standard Internet transport, and we are at the forefront of that movement.”

At this point, Innovium is at the front of the 12.8 Tb/sec wave, with Cisco picking up the Teralynx 7 for its Nexus 3408 and Nexus 3432 switches and also being picked up for large proofs of concept by a number of the big hyperscalers and cloud builders. Significantly, Microsoft’s SONiC network operating system abstraction layer has been ported to the Teralynx chip, and Khemani says that volume deployments will happen with the Teralynx 7 chips by the end of the year with multiple hyperscaler and cloud customers.

But the business doesn’t just stop there, and that is because large enterprises, telcos, and service providrs are starting to build networks that are inspired by those created by Google, Facebook, Microsoft, and Amazon Web Services in conjunction with companies like Innovium and less like the three tier networks using modular switches from the likes of Cisco or Juniper at the core and the aggregation layers of the network and fixed port, top of rack switches in the server and storage rack.

“We started with switches that were optimized for these huge datacenters,” Khemani explains. “The minute we said datacenter switch, people would immediately think we meant the public cloud. But that has changed dramatically in the last two years. Cloud is no longer just public cloud, it’s private cloud, it’s hybrid cloud and now edge cloud – all comprised of this new datacenter architecture at various scales. Applications are changing. It’s a highly distributed environment, and people value the elastic scalability and the reusability of this infrastructure tremendously. This environment, with the right networking of course, lends itself to distributed microservices, distributed storage, all kinds of acceleration running in compute nodes. We think that spending for this new architecture for IT will go from 10 percent to 20 percent of IT infrastructure to maybe 70 percent to 80 percent the next few years. And we are thrilled about that.”