Rovio’s Angry Birds, a title that tasks players with slingshotting colorful fowl toward ill-intentioned, egg-stealing swine, has been characterized by one reviewer as “one of the most mainstream games out right now.” It’s spawned dozens of graphic novels and books (including cookbooks), two movies, and four animated series, not to mention countless spinoff games on mobile and other platforms. But it wasn’t until recently that Angry Birds inspired AI designed to beat the game’s top players — or at the very least, achieve performance at par.
In a preprint paper published this week on Arxiv.org, researchers at Charles University in Prague detail an AI system — DQ-Birds — trained using Deep Q-learning, a technique pioneered by Alphabet’s DeepMind that instructs an agent which action to take under what circumstances using a random sample of prior actions. In the Deep Q-learning flavor the researchers chose to implement, Double Q-learning, a policy distinct from the one used to select the next action is used to evaluate the first policy’s decision.
“Angry Birds appears to be a difficult task to solve for artificially intelligent agents due to the sequential decision-making, non-deterministic game environment, enormous state and action spaces and requirement to differentiate between multiple birds, their abilities and optimum tapping times,” wrote the researchers. “[T]o successfully solve the task, a game agent should be able to predict or simulate the outcome of it is own actions a few steps ahead.”
To this end, the researchers’ system captures game screenshots that it then crops to hide UI elements like the menu and restart level buttons. (It waits 5 seconds before snapping images in order to let the game physics settle.) Post-cropping, it resizes and normalizes them, after which it passes them to the machine learning algorithm trained using Deep Q-learning.
To learn their model, the team compiled a data set of 21 levels of Angry Birds Classic’s Poached Eggs episode, consisting of over 115,000 screenshots. Next, they set the AI system loose on a validation set comprising 10 levels of Poached Eggs that the model hadn’t seen before. The researchers report that their agent was able to surpass a group of four expert humans players’ scores on some levels, but that it fell short in terms of the total sum of obtained scores from 21 levels. (It particularly struggled with level 18.)
Separately, the team submitted their model for consideration in the Angry Birds AI Competition, an annual competition held during the IJCAI conference that has agents solve eight previously unseen Angry Birds levels in three rounds. DQ-Birds didn’t win, but it managed to solve three out of eight levels, surpassing the 2017 semifinalist.
“One of the goals that we did not quite achieve in this work, is to outperform humans in Angry Birds,” wrote the coauthors, who chalk the system’s shortcomings up to a training data set that lacked sufficient level diversity. “[Still], our agent was [often] able to learn to complete the levels [in] only one try. Another interesting point is that most of the time, it [used] only one precise shot [at] some weak point which lead to the completion of the level.”
Author: Kyle Wiggers
Source: Venturebeat
