The Datacenter Impact Of The GlobalFoundries 7 Nanometer Spike

Making money from sand is not as easy as making money from oil, and the Emirate of Abu Dhabi, which has been assembling the GlobalFoundries chip making giant for the past decade is finding that out the hard way and as a consequence, the company is backing off on its development of 7 nanometer manufacturing techniques, which included a double whammy of traditional immersion lithography techniques as well as a move towards bleeding-edge extreme ultraviolet (EUV) technology.

We visited the GlobalFoundries Fab 8 plant in Malta, New York back in February and actually had a tour of the facility and saw the behemoth 7 nanometer EUV equipment from ASML being installed in a section of the massive, completely automated facility. The chip industry knows it has to move to EUV eventually because the wavelengths of ultraviolet light currently used to etch chips, even with double and quadruple patterning tricks, are not short enough to make transistors much smaller than this 7 nanometer pitch. GlobalFoundries wanted to spread out its risk a little, developing two versions of its 7LP process, so that in the event that it had trouble with EUV it could fall back on conventional immersion lithography.

This made a certain amount of sense, but it also meant trying to develop two different but related technologies at the same time. On the systems front, AMD and IBM are the two biggest chip customers for GlobalFoundries, and their respective “Naples” Epyc 7000 and “Nimbus” and “Cumulus” Power9 chips are etched using variants of the company’s 14 nanometer processes. The writing was on the wall last month, when Lisa Su, AMD president and chief executive officer, who has been showing off the “Rome” kicker to the Epyc 7000s, told Wall Street that the future chip was being made by Taiwan Semiconductor Manufacturing Corp – not by GlobalFoundries.

At the time, AMD was saying that it had hedged its bets on foundries from the beginning and was able to make Epyc chips at either foundry. As it turns out, most of AMD’s production has already been shifted away to TSMC for both future Epyc server chips and future “Navi” Radeon GPU accelerators – the two lines that go into the datacenter and therefore what we directly care about – so this was a moot point. Neither GlobalFoundries nor AMD are talking about what happened, but clearly TSMC, which has pushed down from 14 nanometers to 12 nanometers with production processes (the 12 nanometer process is used to make Nvidia’s “Volta” GPU) and which has shipped 7 nanometer Rome samples back to AMD, has hit its milestones. We have to infer that GlobalFoundries did not hit its 7 nanometer milestones or it would have kept the Epyc wafer baking job because you have to believe that AMD wants to give its former foundry the business if it can.

This is a kind of self fulfilling prophecy, then. As AMD and IBM lost confidence in the 7 nanometer efforts at GlobalFoundries, then GlobalFoundries lost confidence that it could get a decent return on the enormous investment that was required to do traditional ultraviolet immersion lithography and then push it to extreme wavelengths. And so Tom Caufield, a former IBMer who has been the general manager of Fab 8, the flagship factory of GlobalFoundries since it was opened and who was named chief executive officer of the company in March, made the hard call to back away from 7 nanometer and push the envelope on its 14 nanometer and derivative 12 nanometer processes and try to actually make some money rather than lose so much.

As of a year ago, when the 7 nanometer roadmap was put out, GlobalFoundries has invested $12 billion in the Malta plant and was expanding its capacity by 20 percent through the first half of this year while at the same time investing heavily in EUV to be a “fast follower” behind Intel, TSMC, and Samsung, the four remaining big foundries in the world. Intel doesn’t really count as a merchant foundry because it really doesn’t have much business outside of its own chips, and Samsung has a very high proportion of its chips being made for itself, too. So really, as far as independents go, it has come down to GlobalFoundries and TSMC for quite some time.

As for AMD, the shift from GlobalFoundries to TSMC for its future Epyc and Radeon chips could very well turn out to be a boon, not just a lifesaver. Having been burned by delays at its own foundries many times in the past, AMD hedged its bets and found the right horse – or so it seems right now – at the 7 nanometer node.

AMD’s top brass is not speaking in any detail about what happened at GlobalFoundries, but Mark Papermaster, another former IBM server chip architect who is chief technology officer at AMD, put out a statement reaffirming AMD’s commitment to 7 nanometer processes and its expectation that it can leapfrog Intel and gain advantage in the datacenter next year.

“The industry is at a significant inflection point as the pace of Moore’s Law slows while the demand for computing and graphics performance continues to grow,” Papermaster said. “This trend is fueling significant shifts throughout the industry and creating new opportunities for companies that can successfully bring together architectural, packaging, system and software innovations with leading-edge process technologies. That is why at AMD we have invested heavily in our architecture and product roadmaps, while also making the strategic decision to bet big on the 7 nanometer process node.”

The Rome kicker to the current AMD Epyc X86 server chips, based on the Zen2 core, is sampling and is on track to ship next year, and the Navi GPU has taped out and will launch later this year. Going forward, TSMC will be AMD’s sole foundry for 7 nanometer parts, so the hedge it had with two chip makers is now gone. It is hard to quantify what value has been lost by not having two fab partners looking ahead to the 5 nanometer or 4 nanometer nodes, but clearly AMD has dodged a very big bullet here and, if all works as planned, actually has a chance for Rome Epyc chips to eat some market share away from the future “Cascade Lake” and “Cooper Lake” Xeon SP processors that Intel will field in 2018 and 2019. As we have previously revealed, Intel will not get “Ice Lake” Xeon SPs based on 10 nanometer processes to market until early 2020. While the features in Cascade Lake, which we learned about last week at the Hot Chips conference, will be welcome, they are not anything like the architectural jump that happened with the Skylake Xeon SPs last year and we don’t expect much from Cooper Lake, also a 14 nanometer part, either. There is a huge process gap that TSMC can help AMD exploit.

As for IBM, the company has been quietly hedging its own bets on foundry partners for its Power and mainframe processors.

“Our agreement with GlobalFoundries was not into perpetuity for all technology nodes,” Bob Picciano, senior vice president of Cognitive Systems, tells The Next Platform. “It was focused on 22 nanometers and then 14 nanometers and one more node – not a couple of nodes – beyond that. We were evaluating the field anyway.”

To be precise, IBM has kept investing in semiconductor research and development in conjunction with GlobalFoundries, Samsung, TSMC, ASML, and Applied Materials to the north of Fab 8 at the Albany NanoTech Complex at the State University of New York since selling off IBM Microelectronics to GlobalFoundries back in 2014. IBM, GlobalFoundries, and Samsung demonstrated the first 7 nanometer test chip in the world back in July 2015, and two years later in the summer of 2017 the trio pushed the research fabs down to 5 nanometers. Samsung said back in May that it would have its 7 nanometer process, called 7 Low Power Plus or 7LPP, ready for early production in the second half of this year and for volume production in the first half of next year. What is interesting here is that Samsung is going all-in on EUV and is not trying to hedge with conventional ultraviolet immersion lithography.

A gambler looking at this situation would bet that IBM, knowing the pressure TSMC would be under to fulfill orders for 7 nanometer AMD and Arm CPUs, Nvidia and AMD GPUs, and Xilinx FPGAs would do a deal with Samsung and not go to the back of the TSMC line. The Samsung alliance with IBM makes sense also because the company has a roadmap that shrinks down to 5 nanometers and then to 4 nanometers and finally – and we really do mean finally here – down to 3 nanometers. After that, we think all bets are off on Moore’s Law shrinking of transistor sizes and chips will start getting bigger rather than smaller, with all kinds of effects on the server, switching, and networking industries.

In the meantime, it is full steam ahead for Big Blue with the Power9 ramp and the Power9’ (that is a prime not a plus) kicker expected sometime in the near future and that we will tell you all about separately today.

There are two major long-term effects that the spiking of 7 nanometer processes at GlobalFoundries will have on the datacenter. The first is that a lot of datacenter eggs are going to end up in the TSMC basket, and if something goes wrong, everyone is going to all have to pivot at the same time. There has been a 10 nanometer shock to Intel’s system, opening up market share opportunities for rivals who make CPUs, GPUs, FPGAs, and switch ASICs, but the same thing can happen to TSMC and Samsung at a future process node and then the gaps all close or shift back to Intel, which we presume will have GPU accelerators in the coming years.

There is no indication that this will happen, but in a sense, it is already happening across the industry. The transistor shrinks, in full nodes with important half nodes that have been so vital in recent years included – in nanometers: 250, 180, 130, 90, 65, 45, 32 or 28, 22, 14 or 12, 10 or 7, 5, 4, 3 – are getting harder to come by and not yielding performance gains as in the past. We could sacrifice clock speed and get a lot more cores on the die in the same thermal envelope, but how many beefy cores will be on a die at 3 nanometers? Will it be 64 cores, 72 cores, or 96 cores? Could it ever be more than that? Probably not.

Kudos to TSMC for pulling off 7 nanometers, because the industry outside of Intel and IBM is counting on it. It is hard to count on anything else looking ahead, given the technological difficulties and immense investments necessary.

The other big affect on systems will be that architectures are going to have to necessarily shift, and IBM has been at the forefront of not just embracing accelerated computing with an offload model that is on steroids, but in creating the kinds of processors that by design assume diverse memory and accelerator types will be needed to hook tightly and coherently into processors to create systems that are highly tuned to specific workloads. As Brad McCredie, who is an IBM Fellow in charge of Power chip development and a co-founder of the OpenPower consortium, put it three years ago, accelerated computing will be the new normal.

This is what is necessary when you hit the Moore’s Law Wall, and when the chips can’t get smaller and faster, you have to use high speed networking and clever software to make it all look and act like a single system. All vendors of compute, storage, and networking chips that go into the datacenter will help shape that much more heterogeneous and necessarily more efficient future. There will not be a single transistor or watt of electricity or BTU of heat or dollar to pay for it all to waste.