Although our appetite for a vast range of AI accelerators appears to be waning, or at least condensing down to a few options, there might be methods on the horizon to let accelerator designers explore new concepts in an interesting way.
Such accelerators are generally human designed but there have been big efforts underway from EDA and chip companies alike to add AI to the design pipeline.
Cadence has its Cerebrus Intelligent Chip Explorer, which automates the optimization of power, performance, and area (PPA) in chip design flows, allowing engineers to work on multiple blocks concurrently, which they say is a significant advantage for complex system-on-chip (SoC) designs.
Similarly, Synopsys recently launched Synopsys.ai, a comprehensive AI-powered electronic design automation (EDA) toolset aimed at accelerating the chip development process at several points along the design route. Google’s DeepMind has also been making strides in this domain, exploring methods to improve TPU, claiming their AI algorithms can bring chips to the field faster and more cost-effectively, fueling competition against specialized chipmakers like Nvidia and AMD.
Nvidia itself has also shown research on how AI can determine the optimal placement of transistors on a silicon wafer, thereby affecting a chip’s cost, speed, and power consumption. By employing AI in these innovative ways, these companies are significantly speeding up the traditionally time-consuming and complex process of chip design.
What these efforts share are highly-tuned, specific pipeline flow speedups but we haven’t seen much to date about how large language models, a slightly different approach to the AI/EDA problem, actually work.
We got some insight via researchers at Georgia Tech, who have started exploring the use of Large Language Models (LLMs) including GPT-4 to automate the design process for AI accelerators in particular. The goal of their recent work was to see if AI can design its own “brains,” so to speak, in a way that’s more efficient and effective than the meatbag approach.
The Georgia Tech team has introduced a framework called GPT4AIGChip to investigate how LLMs stack up in accelerator design. The framework uses GPT-4 and a High-Level Synthesis (HLS) language with results via a unique performance metric called “Pass@k” to evaluate how well the AI’s designs compile successfully.
Training the model to take on this specialized task involved a two-stage process. First, it was finetuned using 7,000 publicly available HLS code snippets to enhance the base hardware knowledge. It was further trained using custom HLS templates to equip it with the specifics of AI accelerator design.
The team found that LLMs like GPT-4 could indeed design AI accelerators, but there were (surprise) some critical limitations. These models struggled with complex tasks and couldn’t quite match the performance of specialized, closed-source models. However, they did find that these limitations could be mitigated by using a modular approach to design and by giving the AI “demonstration” code snippets for context.
One of the noteworthy aspects of the experiment is the introduction of a modular and decoupled hardware template, which the team had to come through due to the above challenges. This template let them address the limitations of LLMs by breaking down the complex design space into smaller, more manageable modules, giving focus on each part individually, thus simplifying the overall complexity.
Another notable aspect is the “Demo-Augmented Prompt Generator,” which helps guide the model in creating new designs by selecting relevant demonstrations from a curated library of past designs. The team explains why this is particularly useful given the limitations on how much data the AI can consider at one time in the full paper.
Despite the challenges, the Georgia Tech crew thinks the AI-designed accelerators were competitive with those created by human experts and gives some examples of how they outperformed accelerators generated by existing automated design tools.
They argue that this kind of work shows that AI can be a viable tool for designing complex hardware, potentially speeding up the development cycle as well as lowering the barriers to entry. In short, they think the need for specialized human expertise is reduced, making it easier for more people to engage in the development of AI accelerators.
Time will tell.