Why Julia is Turning Heads in 2021

After toiling away in relative obscurity for over a decade, open source Julia is right on time for the big enterprise AI/ML code modernization party.

There is plenty to be said for legacy codes—the ones that still do the job even if they lack flexibility and elegance. But with more science and engineering codes in need of a refresh to make AI/ML a possibility, starting with a cleaner slate with something that provides the best of C or C++ and the usability of Python, for example, is becoming more attractive, especially when all the differential programming and other tools to make machine learning in engineering are built in specifically for those science and engineering domains.

All of these capabilities are at the heart of what the open source Julia language project set out to do over a decade ago. The question now is will its stability, notable commercial adoption, and monetization via Julia Computing support it through its own code and business transitions as the Fortune 500 and large research centers keep dead-lifting code modernization?

When Julia was conceived in 2009 at MIT the goal was to solve a problem that still exists: the need to use two (or more) languages, one for high performance (C or C++) and another that made programming complex systems a more pleasant experience (the Python example). While using both could get the job done, there is inherent friction between those interfaces and processes. In addition to this basic mismatch, many of the codes in high-value science and engineering are the product of decades of building. They are inherently messy and rooted in codes that were state of the art in the 1980s, particularly in modeling and simulation.

Despite the clear call to arms and solid support from MIT, it wasn’t until 2012 that Julia became an open source language project and even then, it was a still a relatively small effort, not producing a 1.0 release until late 2018. There were some bumps in the road, learning to live as an open source effort with big fixes every months requiring early users to continually adjust, but things have been stable since then—and at exactly the right time.

With the language as a springboard, one of the long-time committers to the Julia language, Keno Fischer, started to look around at the real-world problems Julia could begin solving—not just as a standalone language but as a supported platform, a self-contained ecosystem. After nearly a decade working on Julia’s low-level compiler and other nitty gritty features, Fischer, along with two other long-term Julia creators, co-founded Julia Computing. The goal was to put Julia to the test, not just as a language but as a more streamlined way to code for pharmaceutical, financial, HPC, energy, and other segments.

In the last year or two, these efforts have paid off. Julia Computing has aided Pfizer in simulating new pharmaceuticals, AstraZeneca with AI-based toxicity prediction, European insurance giant Aviva with its compliance issues, energy provider Fugro Roames with an AI-based grid network failure prediction system, the FAA with its airborne collision prevent program, Cisco with ML-based network security, and a number of national labs and academic institutions with various research programs. Julia Computing made waves this month as well with a DARPA grant to bring semiconductor codes up to date for more efficient, modern simulation codes. In fact, that DARPA work highlights why we’ll be hearing more about Julia—the language and the company that spun off.

“What makes any area ripe for us, including in the semiconductor industry, is that the standard tooling is based on marginal improvements made starting from the dawn of computing, the 70s and 80s. Someone started writing software commercially or in academia and all that has been strapped onto this ancient base of software,” Fischer says. “In the semi space, if you look at something like the SPICE circuit simulator, everyone now has proprietary versions of this cobbled together thing with various versions. Julia solves problems like this. Here is the two-language problem where the simulator is written in C but the scripting is all Python but people want to do advanced things like parameterizations and measurements to integrate ML but all those tools from the 80s keep getting in the way.” It’s not exactly a click of a button but with a small effort, Julia can provide functionality on top of a much more modern stack, Fischer argues.

When it comes to industry adoption, there’s a gap between open source technologies that are cool but bringing it into the market to the point where run of the mill data scientists at Fortune 50 companies want to use the tech is a big leap. For Julia, the leap has been in slow motion—but it is quickening now.

As a company, Julia Computing, formed in 2015, took $4.6 million at the beginning but has brushed off cash since that time, existing well, Fischer says, on early consulting, especially for financial services users.

“Since 1.0 we’ve been on a journey to figure out how this is a real sustainable business. Consulting and support is good but it’s also tied to the size of the community and we are at the level where it’s enough to keep the language going while we imagine the big problems in pharma and other areas we can solve,” Fischer tells us. “The real focus is leveraging this technology in specific industries now. We have the differential equation solvers and modern compiler technology that can replace 30-40 year old grad student-written Fortran code, we think we can help industries with this tooling and made a good business without impinging on the open source effort.”

And this is the crux of the matter for folks who spent years toiling away in open source obscurity on a project that might only finally have its day in the sun commercially. But how to lifeboat it?

“There’s always a tension if you try to monetize open source. Early in our journey some VCs said it was easy to monetize by holding back and selling the last 2X performance. We didn’t want to do that. We developed it to give people the tooling to solve really hard problems and it didn’t make sense to take that away to make money. The strategy of leveraging the technology instead of trying to impede the tech to extract money from it is key,” Fisher says.

When asked if they’re still profitable with their 40 full time employees, nearly all of whom are long-time Julia committees, Fischer says they are but everything goes directly back into the business to hit R&D goals. These include putting greater effort into differential programming, especially as a lot of the domain-specific work they’re doing is aimed at incorporating ML into traditional scientific and engineering applications. This will let Julia Computing carve out an important niche in these areas of being able to fill in missing knowledge with data.

On the company development side, Fischer says they’re putting heavy effort into their JuliaHub cloud platform, which is onboarding early users who are doing big compute. The other emphasis is on specific domains. The semiconductor work is one example of domain-specific targeting but Julia Computing’s real successes have actually be in pharma via work with Pumas.AI, a partner company, that has done notable COVID work for some of the largest vaccine and drug makers.

Fischer says that large companies with decades-old codes that have gotten them by thus far are finally waking up to what’s needed to keep evolving. At the same time, the open source traction Julia took so long to develop is finally paying off with developers choosing Julia for new projects. While it’s a long shot to say the world will be working strictly in the Julia language, for certain key scientific and industrial users with a bad, long-hauler case of dueling languages and interfaces though, the AI/ML impetus might be enough to push Julia into a much brighter spotlight in the year ahead—and probably beyond.