Cisco Cuts Network Costs By Welding Nexus Switch To AMD DPU

There are two ways to make a programmable switch that can run network applications and accelerate certain network functions.

The first and more obvious way to create a programmable switch is to add programmability – hopefully through the addition of engines that can run algorithms created in the P4 programming language – into the very fabric of that switch ASIC itself. This is what the now-defunct Barefoot Networks created in 2016 before it went into the gaping maw of Intel in 2019 never to be seen again.

There was nothing inherently wrong with what Barefoot did with its “Tofino 1” and “Tofino 2” switch ASICs, although they did run a little bit hot compared to non-programmable switch ASICs of comparable speed. Which stands to reason because nothing is free in this universe, and extra processing always costs extra power somewhere. Given enough time and support from Intel, programmable switches might have taken off at the hyperscalers and cloud builders. Intel got distracted and eventually Barefoot was shelved. In the longest of runs, the hyperscalers and cloud builders may create their own ASICs and they would certainly be programmable in some fashion.

The other way to create a programmable switch is to do what Cisco Systems is doing with its new Nexus N9300 line of enterprise switches. And that is to take a switch ASIC and wrap a bunch of P4-programmable DPUs around it, offloading the running of those network applications and the acceleration of network functions from the switch ASIC to the DPU.

And not surprisingly, Cisco has gone to AMD’s Pensando DPU division, which was created by former Cisco executives, for the DPU it is wrapping around its new Silicon One E100 switch ASIC to create what it calls a “Smart Switch,” an analog to the “SmartNIC” that does network offload functions for server nodes.

Pensando was founded by Prem Jain and Soni Jiandani, serial entrepreneurs who along with Mario Mazzola and Luca Cafiero created the Nexus family of switches and the UCS converged server-networking platforms under spinouts for Cisco, which were then spun back in once the development was successful and productized and sold by Cisco. Many expected that the same thing might have happened with Pensando, which was not funded by Cisco, but AMD beat Cisco to the punch when the former acquired Pensando in April 2022 for $1.9 billion. The benefit of the Pensando DPUs – there are four thus far on the roadmap – is that they are programmable in P4, a language created just for packet processing, unlike FPGAs that have to be tweaked in their native RTL/VHDL language.

The switch-DPU hybrid just launched by Cisco is a variant of the server-switch hybrids that we have seen a number of times come out of development but never really go mainstream. Pluribus Networks created one back in 2014, and was eaten by Arista Networks for its intellectual property in 2022. Juniper Networks added X86 cores and FPGA engines to its QFX switches (using Broadcom Trident-II ASICs) in 2015 to create a similar kind of switch, this one aimed at high speed traders who wanted to compute closer to the wire for a latency advantage. Perhaps this time, with its Hypershield AI-powered, automated network segmentation security software as a killer app, the idea will take off. And by putting the DPU in the switch instead of a much larger of server endpoints, Cisco can make this programmable switching more affordable and ubiquitous.

The Nexus Hybrid Switch-DPU Godboxes

The heart of the new Nexus N9300 is a variant of Cisco’s homegrown and merchant Silicon One switch ASIC called the E100, which is rated at a fairly modest 4.8 Tb/sec. Cisco has previously launched the P100, Q100, and Q200 variants for routing and the G100, G200, and Q200 variants for switching, and as the name suggests, the whole point for Silicon One is that a single architecture can span a wide range of use cases for switching and routing, from a top-of-rack box all the way put to a datacenter interconnect box.

The E100s are the first wave of a new line of Silicon One devices that are aimed at switches with relatively low bandwidth but much higher functionality, Kevin Wollenweber, general manager of Cisco’s datacenter and service provider business, tells The Next Platform. While the G100 at 25.6 Tb/sec and the G200 at 51.2 Tb/sec are aimed at spine switches and high capacity AI back-end networks, the E100 is focused more on the enterprise top of rack and network border connectivity use case.

We are not interested in the Nexus N9300 because it is a high capacity switch, but because it might be a harbinger of things to come.

Our case in point is a 12.8 Tb/sec variant of the Silicon One chip – Wollenweber didn’t mention the ASIC by name, but the Q200L is the 12.8 TB/sec version on the switching side – that had eight Pensando DPUs wrapped around it and was created for an unknown number (but multiple) hyperscalers and cloud builders. This was done in the past few years, when hyperscalers and cloud builders were deploying 100 Gb/sec Ethernet switches in the core networks, and by the way, they still are because this machine is in pilots at those hyperscalers and cloud builders and they are getting ready to deploy at initial volumes of thousands of units scaling up to tens of thousands of units, says Wollenweber.

The Nexus N9300 is a chip off that hyperscaler block, starting with a more modest Silicon One ASIC and a lower number of two different AMD Pensando DPUs. As a refresher, here is the Pensando DPU roadmap:

The Nexus 9324C switch has the Silicon One E100 paired with four “Elba” Pensando DPUs in the box. The Elba DPU launched in 2021 and is the second generation DPU from that company.

The Elba device was etched using 7 nanometer processes from Taiwan Semiconductor Manufacturing Co, and has 144 custom match processing units, or MPUs, running at 2 GHz for chewing through P4 algorithms as well as sixteen Arm A72 cores running at 3 GHz to assist in network application processing. It also has dedicated data encryption and storage offload engines, the latter of which may not be all that useful in this specific Cisco use case. Here is a block diagram for the Elba DPU:

The Elba device has a pair of DDR4 interfaces running at 3.2 GHz that support 8 GB to 64 GB of local main memory and 56 Gb/sec SerDes with PAM4 signaling to provide two Ethernet ports running at 200 Gb/sec, which can obviously be split successively to add more virtual ports. The Elba supports 2,000 virtual NICs and 16 million hardware queues and has ROCE v2 latency reduction to memory.

The Nexus N9324C is configured at 24 ports running at 100 Gb/sec, but the services throughput runs at 800 Gb/sec speeds. It is aimed at cloud and edge, zone segmentation, and datacenter interconnect use cases. It will be available in April.

The other hybrid Cisco switch is the Nexus N9348Y, which has the same Silicon One E100 switch ASIC, but it is surrounded by a pair of the newer generation “Giglio” DPUs from AMD, which were announced in 2023. We don’t have a block diagram for the Giglio DPU, but we know that it is a version of the 7 nanometer Elba chip that has been optimized for low power consumption and relatively high performance despite that. The Giglio chip handles stateful packet processing (but not stateless as well as stateful like the Elba device) and has DDR5 memory running at 5.6 GHz.

Not only are the DPU types and counts different with the Nexus N9348Y, but the ports on the E100 are carved up differently and matched to an actual top of rack configuration that is typical in the enterprise, where 25 Gb/sec is still a common speed for an Ethernet port out of a node. (It is still common in the clouds, too.) This switch comes with 48 ports running at 25 Gb/sec plus six uplinks running at 400 Gb/sec and two other ports running at 100 Gb/sec. This one will be available in August this year.

There is, of course, a software-defined angle to this, which Cisco says will save customers a bunch of money. Here are two total cost of ownership comparisons that the company has from two large banks, one deploying a firewall to replace an appliance from rival Juniper Networks and another replacing two firewalls interlinked with four switches with a pair of the Nexus N9300s:

These comparisons include the cost of Cisco’s Hypershield security software. It would have been useful to see the raw data behind these charts, so we would know precisely what was being compared and over what term.