Even OpenAI’s Ilya Sutskever calls deep learning ‘alchemy’ 

A VentureBeat conversation with machine ethicist Thomas Krendl Gilbert, in which he called today’s AI a form of ‘alchemy,’ not science, raised many eyebrows in this week’s AI Beat. 

“The people building it actually think that what they’re doing is magical,” he said in the piece. “And that’s rooted in a lot of metaphors, ideas that have now filtered into public discourse over the past several months, like AGI and super intelligence.”

Many on social media agreed with this assessment, or agreed to disagree. But while it was unclear whether he was specifically referring to VentureBeat’s article, Meta chief AI scientist Yann LeCun simply disagreed, posting on social media that it is  “funny how some folks who think theory has some magical properties readily dismiss bona fide engineering and empirical science as alchemy.” He linked to a talk posted on YouTube on The Epistemology of Deep Learning, about “why deep learning belongs to engineering science, not alchemy.” 

VentureBeat reached out to LeCun for comment, but has not yet heard back. 

But it turns out that even Ilya Sutskever, co-founder and chief scientist of OpenAI, which developed ChatGPT and GPT-4 – and was also a coauthor on the seminal 2012 AlexNet paper that jump-started the deep learning revolution — has called deep learning “alchemy.”’

‘We did not build the thing, what we build is a process which builds the thing’

In a transcript from a May 2023 talk in Palo Alto provided to VentureBeat by Nirit Weiss-Blatt, a communications researcher who recently posted quotes from the transcript online, Sutskever said that “You can think of training a neural network as a process of maybe alchemy or transmutation, or maybe like refining the crude material, which is the data.” 

And when asked by the event host whether he was ever surprised by how ChatGPT worked better than expected, even though he had ‘built the thing,’ Sutskever replied: 

“Yeah, I mean, of course. Of course. Because we did not build the thing, what we build is a process which builds the thing. And that’s a very important distinction. We built the refinery, the alchemy, which takes the data and extracts its secrets into the neural network, the Philosopher’s Stones, maybe the alchemy process. But then the result is so mysterious, and you can study it for years.” 

VentureBeat reached out to a spokesperson affiliated with OpenAI to see if Ilya and the company stood by the comments from May, or had anything additional to add, and will update this piece if and when we receive a response.

Alchemic reactions

VentureBeat reached out to Gilbert to respond to the strong reactions to his comments on AI as “alchemy.”’ He said he found the response “not entirely surprising.” 

A lot of the criticism, he continued, “is coming from an older generation of researchers like LeCun who had to fight very hard for particular methods in machine learning – which they relabeled ‘deep’ learning – to be seen as scientifically defensible.” 

What this older generation struggles to understand, he added, “is that the ground has shifted beneath them. Much of the intellectual energy and funding today comes from people who are not motivated by science, and on the contrary sincerely believe they are inaugurating a new era of consciousness facilitated by ‘superintelligent’ machines. That younger generation–many of whom work at LLM-focused companies like OpenAI or Anthropic, and a growing number of other startups–is far less motivated by theory and is not hung up on publicly defending its work as scientific.” 

Gilbert pointed out that deep learning gave engineers permission to embrace “depth” — more layers, bigger networks, more data — to provide “more interesting results, which furnishes new hypotheses, which more depth will enable you to investigate, and that investigation breeds yet more interesting results, and so on.” 

The problem is that this “runaway exploration” only makes sense, he explained, “when it remains grounded in the key metaphor that inspired it, i.e. the particular role that neurons play in the human brain.” But he said the “uncomfortable reality is that deep learning was motivated more by this metaphor than a clear understanding of what intelligence amounts to, and right now we are facing the consequences of that.” 

Gilbert pointed to the talk LeCun linked to, in which he frames deep learning as an example of engineering science, like the telescope, the steam engine, or airplane. “But the problem with this comparison is that historically, that type of engineering science was built atop natural science, so its underlying mechanisms were well understood. You can engineer a dam to either block a stream, a river, or even impact oceanic currents, and we still know what it will take to engineer it well because the basic dynamics are captured by theory. The size of the dam (or the telescope, etc.) doesn’t matter.” 

Modern large language models, he maintained, are not like this: “They are bigger than we know how to scientifically investigate,” he explained. “Their builders have supercharged computational architectures to the point where the empirical results are unmoored from the metaphors that underpinned the deep learning revolution. They display properties whose parallels to cognitive science–if they do exist–are not well understood. My sense is that older researchers like LeCun do care about these parallels, but much of the younger generation simply doesn’t. LLMs are now openly talked about as “foundational” even though no one has a clear understanding of what those foundations are, or if they even exist. Simply put, the claimants to science are no longer in control of how LLMs are designed, deployed, or talked about. The alchemists are now in charge.” 

What do we want intelligence to be?

Gilbert concluded by saying the overall discussion should be an invitation to think more deeply about what we want intelligence to be. 

“How can we reimagine the economy or society or the self, rather than restrict our imaginations to what cognitive science says is or isn’t possible?” he said. “These are human questions, not scientific ones. LLMs are already starting to challenge scientific assumptions, and are likely to keep doing so. We should all embrace that challenge and its underlying mystery as a field of open cultural, political, and spiritual problems, not keep framing the mystery as strictly scientific.”