IBM will use AI to pipe in simulated crowd noise during the U.S. Open

As it has every year since around 2015, longtime U.S. Open sponsor and media partner IBM detailed the AI technologies it will use to support the tournament beginning on August 31. In a typical year, these technologies aren’t anything out of the ordinary. In 2019, IBM demoed AI that measures player performance and selects highlights, for example. But the pandemic introduced challenges that have required the company — along with countless internet service providers and utilities — to develop new solutions.

The first is AI Sounds, which aims to recreate the ambient noise normally emanating from the stadium. IBM says it leveraged its AI Highlights platform to digest video from last year’s U.S. Open and rank the “excitement level” of various clips, which it compiled into a reel and classified to give each a crowd reaction score. IBM used hundreds of hours of footage to extract crowd sounds, which it plans to make available to ESPN production teams that will serve it dynamically based on play.

How natural these AI-generated sounds will be remains a question. Some fans have taken issue with the artificiality of noises produced by platforms like Electronic Arts’ Sounds of the Stands, which simulates crowd sounds using technology borrowed from the publishers’ FIFA game series. The NBA has reportedly considered mixing in audio from NBA 2K during its broadcasts, and the NFL is expected to use artificial fan noise for its live games this year if they’re played in empty stadiums.

But even if AI Sounds is a dud, IBM hopes to engage fans with Watson Discovery, its text analytics and natural language processing service. The company says it will facilitate tennis debates between viewers on USOpen.org, starting with questions like “Is Billie Jean King the most influential tennis player in history?” As these debates occur, Watson Discovery will analyze “millions” of news and sports sources for insights to deliver pro and con arguments.

Watson Discovery will also provide “AI-powered” insights through USOpen.org and US Open smartphone apps ahead of each tennis match, courtesy of a feature called Match Insights. As its name suggests, Match Insights will search for and analyze articles, blogs, statistics, and more to gather relevant information about opponents and players and translate that information into narrative form.

IBM says AI Sounds and the Watson Discovery features were developed by IBM iX, its in-house digital design agency, in 75 days. (The United States Tennis Association made the decision to hold the U.S. Open without fans on June 17.) A team of IBM iX employees collaborated virtually with the United States Tennis Association digital team, cocreating the fan experiences to fast-track their deployment.