Square Enix fans aren’t pleased with machine learning test

Square Enix recently released a free game on Steam called “Square Enix Test Preview: The Portopia Serial Murder Case.” This remake of a 1983 text-based adventure game used Natural Language Processing (NLP) to fix what the publishers saw as limitations in the game’s original input-based gameplay. However, users aren’t happy with this new feature, to put it mildly.

NLP is a form of artificial intelligence that helps machines understand natural or conversational language. Reviews of the game show that Portopia, on the other hand, is not a good demonstration of this tech, if only because it doesn’t seem to understand natural language. This would seem to indicate that, even if AI and NLP can be used in games, it’s not yet at the point where it should be — at least with regards to how much agency its creator is willing to give the player.

What is The Portopia Serial Murder Case?

The Portopia Serial Murder Case is a murder mystery adventure, one of the most influential games in the visual novel and 2D adventure game genres. It was originally developed by Yuji Horii and released by Enix, one of the antecedents of modern Square Enix. In it, the player takes on the role of a detective trying to solve a murder in 1980s Japan with the help of their partner, and most of the gameplay involves bossing him around. This tech preview is the first release of Portopia in English.

As Square Enix explains, the original version of the game relied on “command inputs” that required players to type commands to the game. The page acknowledges the limitations of this: “However, they did come with one common source of frustration: players knowing what action they wanted to perform but being unable to do so because they could not find the right wording. This problem was caused by the limitations of PC performance and NLP technology of the time.”

The updated port of Portopia does not do away with command inputs entirely. Instead, it attempts to use NLP to give players an easier time delivering the commands. The company says the in-game Natural Language Understanding (NLU), “is used to help the junior detective understand the player’s instructions.” In theory, this would allow users to get their desired response even if their command was not precisely what the game expected.

It appears that NLP didn’t fix the problem as well as the publisher hoped it would. At the time of this writing, the game’s rating sits at “Very Negative” on Steam. The general temperature from the reviews is that the tech preview and the Natural Language Processing doesn’t seem to understand natural language. Often players had to simplify their language as much as possible to get the game to understand what they wanted it to do.

So what went wrong?

Reviews of the game on Steam show that the game frequently misunderstands or ignores commands despite them being only marginally different from the desired one. The AI appears to be splitting hairs. If it doesn’t understand what you say, your partner will respond in-game by telling you, the detective, to focus on the task at hand.

One of the reasons this seems to have failed is that Portopia does not offer much text-based freedom. The player must follow the story the game sets out. Any inputs are always a matter of guessing until the player hits upon the correct one, and the game will not progress until they do. As such, the developers cannot give the players much agency with regards to their own speech — they “must” say a particular thing in order to advance the plot.

As an example of NLP used successfully in another kind of game, a 2005 game called Facade used AI to tell a story in which the player was an active agent. They could say whatever they wanted, and the in-game characters would react accordingly. Even if the player didn’t type a specific sentence, the game’s two characters are able to get the gist thanks to the game’s NLP. This gives the player freedom to explore their options, and the story continues even if the game doesn’t fully understand the player.

If nothing else, this seems to show that AI, or at least natural language processing, will not work in all games all the time. Visual novels don’t give the players much room to maneuver. Unlike Facade, Portopia can’t allow the player to say and do whatever they want. As such, the in-game AI can only respond to specific phrases. However, even within those limitations, Portopia’s NLP comes across as unsophisticated and unpolished.