Why the OECD wants to calculate the AI compute needs of national governments

The Organization for Economic Co-operation and Development (OECD) wants to help national governments understand their AI compute demand needs. As part of the work, the multinational economic policy group is creating a task force that draws together data from a range of sources to make it easy for policymakers to understand how their investment strategy compares to that of other nations. Alongside datasets and algorithms, compute or computing power is an essential part of training predictive models.

Former OpenAI policy director and AI Index co-chair Jack Clark will be a member of the OECD task force. He told VentureBeat that calculation of AI compute may seem like a wonky pursuit, but understanding capacity will be important for policymakers.

“Think of it this way — if no one measured resources like electricity or oil, it’d be difficult to build national and international policy around these things,” he said. “Compute is one of the key inputs to the production of AI, so if we can measure how much compute exists within a country or set of countries, we can quantify one of the factors for the AI capacity of that country.”

OECD AI Policy Observatory administrator Karine Perset said “There’s nothing that helps our member countries assess what they need and what they have, and so some of them are making large but not necessarily well-informed investments.”

The task force intends to develop an initial framework by this fall, and then begin gathering data. Perset said if the group succeeds in making a single metric for nations to measure compute resources, economists can then consider correlations between compute investments and other economic indicators, like income inequality or per capita income.

According to a database the OECD is compiling, Perset said approximately 80 countries have something like a national AI strategy. Initially, such efforts came from primarily Asian and European countries, but more policy is now coming from Africa and Latin America. Some countries focus AI policy on particular areas of interest. Egypt may focus on farming, she said, while France focuses on defense and transportation.

Establishing needs and means

The task force will be led by Nvidia VP of worldwide AI initiatives Keith Strier, who has worked with dozens of national governments during his five years as Global AI leader at consulting firm Ernst and Young. The AI compute demand task force will include up to 30 people and is being actively assembled now through conversations with some of the largest private AI hardware providers, like AMD, Intel, Microsoft, and TSMC.

There’s a remarkable gap in understanding AI despite it being a publicly identified policy priority for many governments, Strier said. “If you’re a prime minister or the president of a country, you want to know three things: How much AI infrastructure do I have? How does it compare to other countries? And is it enough? Those sound like simple enough questions, but if you can’t answer them, how could you possibly know you’re making the right investments?”

The establishment of the task force is the OECD’s latest effort to bring together officials representing national governments around the world to carry out AI policies. In May 2019, the OECD became the first organization to bring more than 40 nations together to agree to a set of AI principles. That was one of the first multinational agreements on the societal benefits nations want AI to have, but some see the principles as vague to the point of being meaningless to a machine learning engineer. In order to help nations put such principles into action, the OECD established its AI Policy Observatory roughly one year ago.

Helping nations understand what they need is almost an esoteric or philosophical question that speaks to the priorities of a nation-state or its elected officials. It also involves considering trade-offs between size, scale, and access. For example, distributing compute resources can be better for the environment and spread access to compute power to more people than building a single, giant supercomputer.

Last fall, an analysis of AI research found a growing compute divide between elite universities (and the Big Tech companies they often work with) and lower-tier schools. That same dynamic, Strier said, will occur among nation-states. “It’s not just about elite universities in the United States. This is all true on a national basis across the world,” he said.

Supercomputers and sovereign clouds

The OECD’s compute count will begin with establishing the levels of compute in datacenters or supercomputers owned and operated by government agencies. From there, the task force will assess the national AI clouds owned by sovereign governments, which Strier called a growing trend among nations in Asia, Europe, and the Middle East to support small to medium-size business adoption of AI.

As part of the National Defense and Authorization Act (NDAA) Congress passed earlier this month, the U.S. introduced a national AI cloud for researchers to power their experiments. That research cloud previously received support from members of Congress from both major political parties, as well as businesses like AWS, Google, Mozilla, and Nvidia.

In addition to the recently launched task force, the OECD AI Policy Observatory has three working groups. The first group is developing a framework for policymakers and procurements officers, as well as government agencies working with contractors to determine the level of risk associated with deploying any AI model. The second group is working on tools and educational resources for computer scientists and the public, and the third comprises national AI strategy leaders of member nations who share policy worth emulating and mistakes to avoid.

What to count and what to leave out

Pulling together a single metric presents a lot of potential obstacles. Chief among them: Private businesses could agree to share information, but they aren’t obligated to share anything. And two major uses of compute resources won’t be included: military AI usage and edge devices. The group also won’t consider public cloud offerings from companies like AWS and Azure.

Then there are ventures that mix state-backed business with public cloud offerings. For example, in December 2020 state-owned Saudi Arabian oil company Saudi Aramco partnered with Google Cloud, a deal that may make AI cloud services available to Saudi businesses. The task force will need to decide whether and how to count such efforts.

The current attempt to calculate nations’ AI compute needs is not the first effort to create a metric that will help nations understand the impact AI is having on business and society. In testimony before Congress last fall about the role AI will play in U.S. economic recovery, MIT professor and economist Daron Acemoglu warned against the potential impact of excessive automation, sharing analysis that found every robot replaces about 3.3 human jobs. And in 2019, economist Erik Brynjolfsson and colleagues from MIT created a model to measure investments in emerging technology like AI.