This ex-Googler helped launch the Gen AI boom. Now he wants to reinvent vaccines

Former Google AI researcher Jakob Uszkoreit was one of the eight co-authors of the seminal 2017 paper “Attention is All You Need,” which introduced the Transformers architecture that went on to underpin ChatGPT and most other large language models (LLMs). The fact that he is the only one of the cohort that transitioned into biotech — co-founding Inceptive, which recently raised $100 million from investors like Nvidia and Andreessen Horowitz — is no surprise, Uszkoreit told VentureBeat in a recent interview. 

“I believe it’s actually a testament to the fact that while our interests overlap a lot, we also are a very diverse group,” he said of the former Google Brain pack (all have since left Google) that includes Aidan Gomez, now CEO of Cohere; Noam Shazeer, now CEO of Character AI; and Llion Jones of Sakana. 

“It would have been kind of surprising to see everybody go off in the same direction,” he added. “The fact that this didn’t happen is, in my book, the specific reason that the group is still incredibly effective.” 

The Palo Alto-based Inceptive, which was founded in 2021 by Uszkoreit and Stanford University’s Rhiju Das to create “biological software” using Transformers, has built an AI software platform that designs unique molecules made of mRNA, which Pfizer and BioNTech used to make their Covid-19 vaccines. Essentially, the company designs mRNAs with neural networks, tests the molecules, and licenses them to pharmaceutical companies that put them through clinical trials. 

From Google to biological software

For Uszkoreit, biology was a long-time interest, but three things happened in quick succession in late 2020 that moved him to launch Inceptive.

There was the mRNA Covid vaccine efficacy results that came out in 2020, Uszkoreit said — vaccines that quickly went on to save millions of lives. Then there was DeepMind’s unveiling of its AlphaFold 2 results, where it became clear that the AlphaFold team had truly solved the problem of protein folding — thanks to the use of Transformer-inspired models. 

“It made it absolutely crystal-clear that large-scale Transformers are totally ready for primetime in molecular biology and technology in particular,” he said. 

The third thing that happened was more personal — during the same period, Uszkoreit’s daughter was born. “I guess anybody with kids can relate to looking at certain things quite differently, very suddenly,” he said. 

Those three elements created what Uszkoreit saw as a “moral obligation” to use Transformers to develop new vaccines and drug treatments. “There were very few people looking at building models for RNA, because there was very little data,” he explained. “For protein structure prediction, there’s at least a few hundred-thousand instances, but there were less than 2,000 known RNA structures validated.” 

Competition in the space

Lately, of course, AI and drug development have gone together like peas and carrots, thanks to a race by investors and pharmaceutical companies to capitalize on a $50 billion market opportunity for AI in the sector, according to a Morgan Stanley report.

But Uszkoreit is unconcerned with the competition. “It’s such a green field situation that we’re in right now,” he said. “I’d be hard-pressed to see where there isn’t enough room for even more companies, even more amazing teams to go after potentially really world-changing opportunities here.” 

Still, he does think two important elements set the Inceptive team apart. One is that what Inceptive is working on is not really a biology problem or a deep learning challenge — it touches on both, requiring a level of experience and expertise that goes beyond interdisciplinary. 

“We’ve really had to approach this with a beginner’s mindset,” he said. “The company recognizes that this is a discipline that quite possibly will have a name a few years down the road but doesn’t have one yet.” Many on the team are world-class experts, he explained, but will have to adapt and be pushed in new directions. 

In addition, there are the highly-complex scientific challenges: “It’s not the case that we can just go and apply RNA biochemistry methods and then do novel deep learning on it,” he said. “Nor is it the case that we can just apply standard deep learning and basically push the envelope in the realms of biochemistry. That’s not enough. You have to really have to go beyond and really push the boundaries of both of those things simultaneously.” 

That includes coming up with a new method for gathering data — in Inceptive’s case, it means running experiments with robots, people, models and neural networks, to generate novel, synthetic mRNA molecules. 

Besides science, it all takes a bit of magic, Uszkoreit added. “One interesting tagline that’s crystallized over the years is that the magic, the magical work — most of our work, I would say — happens on the ‘beach’… where the wet [lab, for manipulating liquids, biological matter, and chemicals] and the dry [lab, focused on computation, physics, and engineering] meet in harmony.” 

Generative AI attention has been focused on LLM labs

While most of the generative AI hype has been focused on the large AI research labs developing LLMs — OpenAI, Anthropic, Cohere, and others — Uszkoreit doesn’t feel the need to raise his hand to get attention for Inceptive’s work. 

“Let’s put it this way: If we’re successful, no advertising will be necessary,” he said, adding that the generative AI hype will eventually cool off. But ultimately, he said that it is important to recognize that “there is a ton of value that is being created across a really broad range of different applications,” whether it’s weather forecasting, climate modeling, language understanding, predicting the structure of proteins, or creating mRNA molecules. 

“Many of the foundational findings from any of these endeavors will actually have a pretty high probability of improving or helping with many of the others,” he said. “This is really a tide that lifts all the boats.” 

Keeping in touch with the Transformers cohort

Since it is only six years since the Transformers paper was published, Uszkoreit thinks “It’s too early for nostalgia to happen.” But the group of eight co-authors definitely keep in touch. 

“We have a little group chat,” he said. “We share notes, advise each other quite often and I’ve received amazing advice from some of the other coauthors, basically ever since we’ve left the Google mothership.” 

It was the diverse interests of each one of the researchers, he reiterated, that helped the Transformers paper develop with such a “level of polish.” 

“The results required many, many things to basically do right,” he explained. “And that included a very careful implementation done by people who really, really know how accelerators work, [and have] maybe some hazier, intuitive ideas, [with] extremely careful experimental design, all the deep learning alchemy bag of tricks.” 

All of those things had to come together and would only work in combination, he said, adding: “That kind of goes hand in hand with the fact that it’s a very interesting set of people to remain in touch with.”