What you need to know about Sakana AI, the new startup from a Transformer paper coauthor

The landscape of generative artificial intelligence (gen AI) is in a rapid state of evolution these days, not only with OpenAI, Meta, and Google competing directly on foundation models, but doing so while some of their biggest in-house talent leaves to launch their own gen AI startups.

Last month saw one of the biggest defections: Llion Jones, one of the co-authors on the seminal 2017 research paper “Attention Is All You Need,” which kickstarted the generative AI revolution by developing the architecture of the transformers used in leading large language models (LLMs), announced he’d left Google to found a new startup.

This week, we learned just what he is working on: Sakana AI, a new AI company based out of Tokyo.

Joining him as a co-founder of Sakana is David Ha, previously the head of research at Stability AI and a former Google Brain researcher.

Together at Sakana, they are pursuing a strikingly different model for AI: one of biomimicry, specifically looking to the collective intelligence in systems found in nature such as schools of fish and beehives, to design AI models that are flexible, reactive and economically efficient.

At Sakana AI, Jones’ vision is to capitalize on this technology to craft a trail-blazing generative AI model proficient in generating text, images, code and multimedia content.

So far, Sakana is keeping its backers close to the vest, declining to comment to Reuters on who is investing in the company.

New tech inspired by nature

Sakana AI distinguishes itself with a novel approach that involves developing numerous smaller AI models that collaborate, much like a swarm, to deliver complex results.

This methodology challenges the dominant trend of constructing extensive AI systems.

Sakana AI is confident that its swarm-based approach, inspired by collective intelligence, can provide results on par with larger systems while being more economical and flexible.

“Ants move around and dynamically form a bridge by themselves, which might not be the strongest bridge, but they can do it right away and adapt to the environments,” Ha told Bloomberg in an interview. “I think this sort of adaptation is one of the very powerful concepts that we see in natural algorithms.”

The name Sakana was derived from a Japanese word for fish (さかな) meant to elicit “a school of fish coming together and forming a coherent entity from simple rules,” the two founders told The Financial Times.

Sakana AI’s decision to establish its base in Tokyo is a calculated move. With its advanced technical infrastructure and highly educated talent pool, Tokyo is primed to nurture the growth of AI startups and attract international expertise. Sakana plans to exploit these benefits to bolster its research and development activities, although the company has yet to set up office space for its new headquarters.

Will nimble startups be able to out-maneuver AI industry titans?

The vision of Sakana AI transcends the creation of AI models, as Ha and Jones underscore the shortcomings of contemporary AI systems that often end up inflexible. Instead, they propose AI models that embody principles of evolutionary computing, inspired by the adaptability of nature. This approach could potentially resolve issues related to cost and security in AI systems.

The cofounders told CNBC they believed Google’s focus on a single type of generative AI technology, large language models, was a mistake “because that’s quite a restrictive framework,” Jones said.

The cofounders said they’ve been in talks around using LLMs, but haven’t made a final determination. However, Ha did not write them off: “I would be surprised if language models were not part of the future,” he told CNBC.

The transition of both Jones and Ha from Google to Sakana AI is a noteworthy development in the AI sector, especially in the realm of generative AI.

“It’s just a side effect of big company-itis,” Jones said in the CNBC interview. “I think the bureaucracy had built to the point where I just felt like I couldn’t get anything done.”

Similarly, Ha sees challenges facing the current gen AI leader, OpenAI. He said he views the use of the research he was part of at Google as less than innovative, and expressed concern over the lack of development-sharing with the wider community.

By harnessing the swarm concept and evolutionary computing principles, Sakana AI aims to deliver flexible and cost-effective AI solutions. With Tokyo shaping up as a nucleus for AI innovation, Sakana AI’s presence in the city will contribute significantly to generative AI’s future evolution, with the AI industry awaiting the transformation this collaboration proposes to usher in.