Hyperscalers Are Pushing Big Banking Infrastructure Around

Everyone who can afford it wants to emulate how the hyperscalers operate their infrastructure. High performance, efficiency, and cost maximization are worthy goals but for big banks, the need to play in hyperscaler territory goes beyond utilization or datacenters: these companies are becoming the competition.

The largest consumer banks see how Google, Amazon, Facebook, and others are coming after their customers. Several have taken out banking licenses and all have working on building wallet and payment systems. These companies often have advantages that even the most trusted financial institutions don’t: access to the diverse picture of a potential customer’s life via their browsing and buying habits. Banks can see some of that, but only a fraction of what a Google or Apple might. And besides, even if the big banking entities could see this data, they are not well equipped to act on it.

While the hyperscale web company’s world revolves around massive scale with workload-optimized efficiency, most big banks still rely on mainframes to handle at least some of their transactional processing. These companies have been slow to change, and for good reason, but there are now two big business drivers that will force them into a rethink of their IT infrastructure—and it will all have to happen soon.

The first driver is the hyperscale encroachment into traditional banking territory. The second is more near-term but with long-lasting impacts. That is COVID, which has pushed most consumer banks to close branches and get better than ever at handling record online transactions. If we want to get a bit more nuanced, we could also suggest that AI/ML is a fourth driver, although it intersects with the others in interesting ways. More on that in a moment.

Big banks look to hyperscale web companies for IT inspiration but the irony is, these infrastructure role models have morphed into their own most formidable competitors and are flexibility-native versus building unending workarounds for legacy systems.

It’s one thing to see all of this from the outside, but John Ashley know what hardware and cloud scenarios the big banks are buying into and why. He’s spent over twenty years straddling the big infrastructure, big banking lines. For the last decade-plus he’s been at NVIDIA and now is GM for the financial services and technology vertical for the GPU maker. While he’s obviously pushing the AI/ML angle since that’s NVIDIA’s datacenter bread and butter these days, he makes some excellent points about what is changing in big banking workloads and the hardware they need—not to mention how that hardware is accessed (cloud versus on-prem).

The big consumer banks, Ashley says, have started to see an “existential threat coming from these hugely profitable, well-funded, and data-savvy companies, all taking aim at some of their traditional business. At the same time, there’s a round of startups that are extending bravely here too, although the regulators haven’t given their financial pronouncements.”

What we’re seeing, he explains, are “innovators with challenge business models and the business pressures are pushing the big banks and insurance companies to move quickly with the first wave of this happening a couple of years ago with the JP Morgan, Goldman Sachs level companies hiring people out of industry and academia to keep pace.”

While hyperscale companies think about IT infrastructure as the all-important backbone of their operations, we have to remember they started this way. Big banking institutions are where legacy infrastructure lives. Making big changes means the business drivers have to be imminent. But they’re starting to see the light. Ashley says one eye-opening case has been around fraud detection where implementing the right solution takes longer to do than the amount of time for that solution to pay for itself—a matter of weeks.

This is all to say that banks have less interest in being in the infrastructure business than they do responding to competitive pressures. And so, one might think anyway, that should mean the logical step is to the cloud.

Not so much. And indeed, consider banks closing branches—needing the ability to scale down and spin back up quickly. That’s just one use case that might indicate growth in cloud adoption. This is actually all over the place, depending on the bank, Ashley says.

“If banks are managing on-prem infrastructure it’s because they feel like they have to because of data gravity. They’re being forced to maintain their own infrastructure. But others view it as a source of competitive advantage.” But the important change is that as they keep building capabilities to stay competitive and some banks have hundreds of AI use cases, “they’re working on the cloud but that bill is adding up. We are starting to see the pendulum swing even at big companies toward a hybrid model. They’ll hold their noses and have some on-prem because it saves millions of dollars. But there are still others who, even with the savings potential and IT expertise in house, don’t want the political fight over who gets servers when.” Not that the cloud guarantees anything of course, Ashley tells us there’s a bank where the data scientists have to get up at 2 a.m. if they need GPUs before a big NeuroIPS or other conference because there aren’t enough GPUs left in their region.

With all of this in mind, the question is, how will banks jump to and adopt a hyperscaler vision—and what parts of that IT philosophy will they pull first? And also we have to keep in mind that being flexible and nimble is easier for hyperscale web companies that aren’t offering the kind of tightly regulated services, of course.

Just how aggressive the change is depends on how high up and active the sponsorship of that transformation is, Ashley says. “We’ve seen some firms where transformation is starting from the top, as with SPERBANK in Russia and Ping An in China (insurance but similar services/drivers). In these cases they’re saying they’re going to be so good they can offer their own secure financial cloud to customers and partners and are rearchitecting from the ground up for a new way of doing things with AI, among other initiatives, in the foreground.”

For SPERBANK and Ping An, these systems are broadly useful across the business but are the foundation for ambitious AI programs, “almost like a factory. They don’t want to produce a model, they want many and are refreshing those and continually improving them. They look at this in a very industrial way.” On the other hand, he adds there are others who have internal sponsorship driven by the business, versus technology, side. These are the companies taking it all to the cloud or increasingly to specialized alternative locations with high-speed networking and cheap power (datacenters in Canada or Iceland, for example).

Aside from the Ping An or SPERBANK cases at the bleeding edge, a bulk of banks are trying their best to blend the best of all worlds via the hybrid cloud approach. Unfortunately, unlike with the tech-first companies, they are carrying around a lot of legacy infrastructure. And it’s not easy to shut off. Ashley tells us about one big bank that can’t kick its mainframe habit but wants to integrate AI. They have found a workaround via a GPU payments model that’s pre-integrated with a mainframe workflow. “That’s an early indicator—they’re being forced to do this because the business use cases around AI are getting more compelling.”

When it comes to where these companies emulating the “big tech” companies that began with modern infrastructure, the workload that keeps coming up is, generally put, ML/AI. Here is where the insidious decision-making happens: it’s damn hard to be “half-in” when it comes to implementing broad AI initiatives. Models need to be built in droves and retrained consistently to be useful. There is no way to make a small investment unless it’s just a research project. While Ashley thinks rollouts will be happening soon, consumer banks need to think about who they are and who they want to be. It’s a bit of an identity crisis and hardware choices are only a sliver of that inner turmoil.

As a final related note about identity, a more nebulous driver for big banks as they consider what to do next: Customers aren’t as loyal as they used to be. Competition is coming from all quarters and at the heart of it, especially without in-person experiences to make up the difference, the next several years will likely only make this worse for traditional banks.

With so much at stake, big banks need to take a lesson from the broader financial services community: invest big and boldly, all at considerable risk, and the rewards too may be great.