Driving Down The AI System Roadmap With Nvidia

In the early years of the GPU acceleration of application performance – really from “Kepler” datacenter GPUs in May 2012 to “Volta” in May 2017 – Nvidia, the world’s most important technology company and still the overwhelmingly dominant supplier of hardware and systems software for the GenAI revolution, was very good about putting out roadmaps.

But for a few years until 2021, the company kept its roadmaps folded up in the front left inside pocket of co-founder and chief executive officer Jensen Huang’s leather jacket, but as the GenAI boom went from chemical to nuclear, the company correctly surmised that with everybody trying to synchronize money, land, power, cooling, and systems to all come together in the largest infrastructure buildout the IT market has ever seen, everybody needed a real roadmap, extending out a few years, so they could plan.The first such new era roadmap came out at the end of 2023 in a financial presentation, not in a GTC conference slide from Huang, and we edited the heck out of it to add missing components like the some of the GPUs and DPUs and putting the correct calendar years on the columns, but all along being grateful Nvidia was outlining where it was at and where it was going. We gathered up all of the roadmaps we could find between 2021 and 2023 and put them in this story so you would have them for reference.

That October 2023 roadmap reveal was also when we got the first wind of the annual cadence of updates that Nvidia was scheduling for its AI system components. On this late 2023 roadmap, the 2025 products were called the GX200, the GX200NVL, the X100, and the X40, which had me thinking they were going to recycle the “Xavier” codename from the gaming side of the house, but we conceded that X could also be a variable. The 2025 products turned out to be the “Blackwell” GPUs that were detailed by Huang at the Computex conference in June 2024 in the new-style roadmap we by now have seen a bunch of times with additions. (The font is pretty small for those of us of a certain age, so you might have to squint a little.)

Nvidia unfolded its datacenter roadmap out to 2027 in June 2024, when we learned about the “Vera” CV100 Arm server CPUs and the “Rubin” R200 GPU accelerators for the first time. And then Huang folded out another year and showed us the datacenter roadmap out to 2028 at the GTC conference last year.

At the GTC 2026 conference, Huang added some more details on the machinery between 2026 and 2028, but he did not talk about a future and likely “Feynman Ultra” GPU expected along with updated ConnectX-10 SmartNICs and maybe even an updated Groq LPU that could came that year, too.

Nvidia Mostly Owns Training, And Can Compete On Inference

These roadmaps are important to the OEMs and ODMs that convert Nvidia’s technology into the systems that run AI training and inference for the vast majority of the world. And also for customers, who as we all know invest in roadmaps but do not simply acquire point products. Despite all of the glorious competition from the Cambrian explosion in AI compute engines and networking, Nvidia by far still has dominant market share and will for many years to come. How much remains to be seen.

If you do some rough math, which as you know I love to do, the total server market in 2025 generated somewhere between $420 billion and $450 billion based on the limited data we see out of IDC and Gartner, and about $190 billion of the bill of materials for those systems passed through to Nvidia as revenue. Moreover, the machines sold by the OEMs and ODMs that had at least Nvidia GPUs (and very likely more components) installed in them probably represented somewhere between $275 billion and $325 billion in revenues in 2025. That gives machines based on Nvidia technologies somewhere around a low of 61 percent share to a high of 77 percent share of the overall systems market. We have to use a quantum probabilistic distribution to get any more accurate than that (you were supposed to laugh there), or see all of the financials of all the public and private server makers and add everything up.

I guess the point is that damned near all of the profits for AI systems are going to Nvidia, as its gross, operating, and net income clearly show.

Truly amazing.

Which brings us to the 2026 roadmap as presented at Huang’s GTC keynote address:

This time around, the evolution of the “Oberon” and “Kyber” racks is being explicitly called out alongside the evolution of the compute and networking engines.

You will also note that Quantum InfiniBand is not mentioned, and that is not because Nvidia is stopping development on InfiniBand but more that Nvidia does not expect for AI factories to deploy InfiniBand even if there are cases where HPC centers running smaller clusters or even some AI centers might opt for that.

Furthermore, as we pointed out in our prior coverage of Huang’s keynote, the “Rubin” CPX long context and attenuation processing engine, which was unveiled last September, is not on the roadmap. The Rubin CPX was expected to be delivered at the end of this year for AI context windows of 1 million tokens or more as well as helping with video generation for models that do pictures instead of words. It may be premature to count CPX out of the picture for such workloads. In fact, you might see a combination of Nvidia CPX and Groq LPU compute engines handling both kinds of inference – and Vera-Rubin compute complexes not involved. (Nvidia did not say this, but I am.)

The Vera-Rubin systems are locked and loaded for volume shipments in the second half of 2026, as planned. The Vera Arm server CPU has 88 custom Nvidia “Olympus” cores with two threads per core and a 1.8 TB/sec NVLink chip-to-chip interconnect that can be used as a high speed connection between one or more “Rubin” R200 GPU accelerators. Rubin is, as we know from this time last year, a pair of reticle-sized GPU chips linked by NVLink C2C ports inside a single socket that has 288 GB of HBM4 memory and delivers 50 petaflops of FP4 performance on its tensor cores compared to 10 petaflops for the “Blackwell” B200 and 15 petaflops for the B300. These B200 and B300 GPUs have 288 GB of HBM3E stacked memory. Rubin is expected to be etched using the 3 nanometer N3E or N3P process from Taiwan Semiconductor Manufacturing Co. As far as we know, the Oberon racks will have the same 72 GPU sockets and the same 36 CPU sockets in NVL72 rackscale systems as were done in the Blackwell generation with the B200 and B300. (Nvidia was calling these NVL144 by counting GPU chips, not sockets for a while, confusing itself and more than a few customers.)

Alongside Vera and Rubin, the Groq LP30 will ship inside of dedicated racks with a regular Spectrum Ethernet spine (sometimes called a backplane). As far as we know, this Ethernet spine not use the Spectrum-6 ASICs with co-packaged optics, but it could us optics in the spine and copper in the chip-to-chip spine connectors coming off the Groq chips.

Nvidia calls this an Oberon ETL256 configuration, which means that either 256 Vera CPUs or 256 Groq LPUs can be linked to this backplane.

The Groq sleds coming this year have eight LP30s in four sockets per sled, and they look like this:

A rack of the LP30s is called the Groq 3 LPX system, and it has 32 sleds with a total of 315 petaflops of FP8 inference computing, 128 GB of SRAM on the 256 chips, 40 PB/sec of aggregate SRAM bandwidth, and 640 TB/sec of aggregate scale up bandwidth across the Spectrum ETL backplane. (Again, it is not clear if this is Spectrum-5 or Spectrum-6 with the CPO removed. We suspect it is Spectrum-5, which is simpler.)

Later this year, it will also be possible to get whole racks of Vera server CPUs in the Oberon racks with the ETL spine. (Meta Platforms is going to be an early customer for this.) If you do the math, that is eight Vera CPUs (possibly four two-way Vera-Vera nodes) in each sled, with 32 sleds in a Vera ETL racks. That is 256 CPUs with a total of 22,528 cores and 512 TB of main memory and 300 TB/sec of bandwidth across that memory. 

Presumably this will be called a Vera CPX rack, with CPX being short for Compute Processing Rack (not to be confused with the Rubin CPX processor). The storage racks based on the BlueField-4 DPUs running various distributed storage software stacks from a dozen or so partners is called a BlueField STX rack, and similarly a rack of Spectrum-6 switchery is called the Spectrum-5 SPX rack.

Perhaps the X was not a good idea in the naming. Perhaps, just perhaps, these should have been called CPR, STX, and SPR? Naming matters. The are all based on the MGX modular server architecture, and MGX is not to be confused with the private equity firm in the Middle East that is underwriting a lot of AI facilities around the globe these days.

As we roll forward into 2027, the “Rubin Ultra” GPU, presumably called the R300, is just a doubling up of GPU chips inside of the Rubin socket, from two to four chips and supplying 100 petaflops of FP4 performance. Nvidia is going to double up the number of sockets to 144 inside the new “Kyber” rack, which will have a copper midplane instead of huge spaghettis of thousands of copper cables cross-coupling the GPU sockets in the rack. Nvidia will have sixteen banks of HBM4E memory against those four Rubin GPU chips, for 1 TB of capacity and 32 TB/sec. (In theory, that HBM4E memory could run at 64 TB/sec, and we are wondering why Nvidia is gearing it down – perhaps for power consumption and heat dissipation reasons.)

Let’s talk about NVLink ports and NVSwitch memory fabric interconnects for a second. The names were out of phase a bit because the initial NVLink 1.0 that debuted with the “Pascal” P100 GPUs back in 2016 did not have a switch but rather used a mesh interconnect to share memory across the Pascal GPUs. The names were of the ports and switches were lockstepped with the Blackwell B300 GPUs (I think) and going forward the chip and port generations are names in synch. Like this:

There are many ways the NVSwitch memory fabric ASICs can be enhanced, but I think it is safe to say that the radix – the number of ports per ASIC – is getting to be too low and I think there is a more than even chance that Nvidia will start thinking about not chiplets but waferscale designs for ASICs. (And might even do that for future Groq LPUs, now that I think on it.) These would not have to be full wafer scale, but it would mean doing away with the C2C interconnects everywhere as well as all of the buffering that needs to be done as data moves from one chip through the C2C interconnects and to the adjacent chips. (This what we think secretive networking chip startup Eridu is already working on, and Cerebras has shown how it works well for parallel compute.)

Suffice it to say, NVLink 6 ports on the Rubin GPUs will double to their bandwidth over NVLink 5 ports, to 3,600 GB/sec, and they will double yet again with the Rubin Ultra GPUs, which stands to reason given the performance doubling and the HBM4 memory bandwidth almost tripling between Rubin and Rubin Ultra.

In the Rubin generation, the Spectrum-6 Ethernet ASICs will have co-packaged optics, and that generation of 102.4 Tb/sec switches will also support the scale out network needs of the Rubin Ultra systems. The 2027 Rubin Ultra product line will see the Groq LP35 chip get FP4 floating point processing in the NVFP4 format so it matches the precision of the Blackwell and Rubin GPUs. And with the Groq LP40 compute engine coming in the Rosa-Feynman systems in 2028, NVLink ports will be added so the Groq engines so they can have memory coherency with the Rosa Arm server CPUs (named after Rosalyn Sussman Yalow, a Nobel Prize winning medical physicist who developed the radioimmunoassay method of detecting tiny amounts of chemicals in blood or tissue) and with the Feynman GPUs (named after famous physicist and bongo player Richard Feynman).

You will see in the roadmap that in 2028 Nvidia will add CPO to NVLink 8 ports and presumably for the other end at the NVSwitch ASIC. While we are always pushing for compute engine makers to do CPO on their devices, they can be copper on that end with a multitier network of switch ASICs using CPO on the other side. You do not have to have CPO on both sides. (Nvidia seems to be using NVSwitch and NVLink loosely in this chart, so be careful.) We think that CPO for NVSwitch is interesting because it will allow for fast, high-bandwidth, two-tier NVSwitch networks to create larger GPU compute memory domains for models to play in.

With Hopper GPUs, the official scalability was eight GPUs with linked memory, but the unofficial scalability was 256 GPUs using a two tier network. With Blackwell, the official GPU memory domain size is 72 GPUs, but with multiple tiers of NVSwitch, it can, in theory, be boosted to 576 GPUs. The Kyber racks, which cram twice as many GPUs on vertical sleds and have a copper backplane, the rackscale domain will be 144 GPUs. Eventually, with the advent of NVSwitch 8 CPO (I know the chart says NVLink 8 CPO), the single rack will remain at 144 GPUs, but across a multitier network (we think two tier, but we can’t know that without knowing the radix of the NVSwitch 8 device) Nvidia will have a domain size of 1,152 GPUs.

Decades ago, Cray supercomputers had copper backplanes in the rack and optical links coming off their routers to interlink the racks. We suspect that Nvidia will do much the same. The rule is always: Copper when you can, optics when you must, and that is an economic rule as much as a technical one. But, with Nvidia accounting for such a large share of AI systems investment, if any workload can get CPO to ramp up in volumes and therefore push down unit prices, it is GenAI inference, and if any company can drive that effort and coordinate it across a supply chain, it is Nvidia. One might argue that only Nvidia can make this happen, and if it does, all systems will benefit.

The factor of 16X more GPU sockets combined with the performance increases expected with the Feynman GPUs – all Nvidia is saying is that it will have die stacking and custom HBM memory with this generation of chips – will result in a massive throughput boost for CPU-GPU hybrid systems.

If the die stacking is just for SRAM cache (which is relatively easy to do), that still allows for more 2D GPU cores to be added to a socket. Nvidia will probably move to 2 nanometer processes or smaller for Feynman, which also means a move to gate all around (GAA) transistors and the High NA EUV process, which also means the maximum reticle size moves from 858 mm2 to 429 mm2 because the chips can only be half as tall. So whatever Feynman is or isn’t, it will have at least eight GPU chips in a socket compared to the four chips in a Rubin Ultra socket and use the process shrink to add more circuitry.

There is always a chance that Nvidia might be stacking both SRAM and compute with the Feynman devices, of course. That would be very interesting, indeed.