European Bank Sees Path to ‘Fastest AI Supercomputer’

While Hungarian banks with collective names like OTP Group might not have household recognition here in the U.S., they do represent a notable portion of central Europe’s banking base with 17 million customers.

OTP Group has ambitious plans to have Europe’s “fastest AI supercomputer” when it is fully deployed in 2022 but the architecture is different from other systems that have claimed the “AI super” title. Instead of a mix of robust CPUs and high-end GPUs from Nvidia or to an increasing extent AMD, the banking entity has gone the SambaNova route. The systems will be multi-rack and spread across more than two sites, the company’s product head, Marshall Choy, tells The Next Platform.

On the surface it might sound unorthodox to claim the largest supercomputer when it’s not tightly interconnected in the same datacenter. It might also sound off that the Group wants to deploy massive GPT-3 models but across multiple systems (since much of the emphasis on GPT-3 is focused on single-system scaling). The “fastest” designation will come next year when Choy says OTP Group will be able to show the fastest training times for GPT-3 for its model parameter size.

The first point is hard to argue — a supercomputer’s performance is based on single-machine scaling. But Choy says what OTP Group is doing is representative of an emerging trend it’s seeing with customers. Instead of looking to deploy a monolithic machine to handle full-scale, OpenAI-class 175+ billion parameter training, real enterprises want to run smaller, domain-specific models they can tune to their business requirements.

“If you look at some of the OpenAI GPT-3 results for over 175 billion parameters, the accuracy for tasks requiring specificity have been lackluster. While OPT Group will build large models, it will be a multi-model approach with smaller GPT-3 based models ranging from 1.5 to 13 billion parameters that are pre-trained, in this case, with both Hungarian and a financial industry-specific corpus that will give them a domain specific language model that can better address specific tasks,” Choy explains. As a result, OTP Group should get much higher accuracy than it would with something like the big showcase 175+ parameter GPT-3 models.

SambaNova has been aggressive in going after the wealth of enterprise use cases around GPT-3. For those who don’t follow language models, this is not your grandmother’s NLP. Language models promise extensive understanding based on text and other data sources, serving as a potentially ever-learning, vast scale source of real intelligence over standard relational, graph analytics, sentiment, and other forms of traditional analysis. It was the first AI systems maker to offer its systems with a GPT-3 overlay — a big deal in the era of long waitlists to get access to the model — and has provided a mix of remote and on-prem racks with its custom accelerator.

SambaNova’s flagship offering, Dataflow-as-a-Service, is an extensible AI services platform, and enables  organizations to jump-start AI initiatives by augmenting existing capabilities and staffing with a simple  subscription. The platform is powered by DataScale, an integrated software and hardware platform use built on SambaNova’s Systems Reconfigurable Dataflow Architecture.

In terms of how banks might roll out similar systems equipped with GPT-3 models, Choy says next-level sentiment analysis is one of the major drivers for financial services in particular. “Specific use cases there include combing through different media types around earnings reports, for example. This can include social feeds, articles, and questions from financial analysts. Those are fed in and trained to analyze specific company or competitor-specific reporting. He adds that another use case for banking in addition to a wide range of other segments is text extraction and entity or document recognition. “The use case there would be combing through multiple document submissions for a claims department, generating the claim document in the proper legal format, and automating how a lot of manual, labor-intensive processes operate today.”

“OTP Group has an impressive industry-leading plan to make AI its key technology driver for the next decade. With the support of ITM, the AI environment that it’s creating will provide accessible, cutting edge infrastructure for  universities, research centers, public and private organizations across Hungary,” said Prof. Dr László Palkovics,  Minister for Innovation and Technology in Hungary.

“This is a unique collaboration between OTP Group, ITM, and SambaNova Systems to provide an incredible resource  to the country and the Central and Eastern European region,” said Péter Csányi, deputy CEO, head of Digital Division  at OTP Group. “We are pleased to announce that this groundbreaking supercomputer will represent a unique AI capability to build GPT-3 level language models for languages across CEE.”