Why does AI like goblins and Japan so much? | Technology

Source Domain: english.elpais.com

“But here is the annoying little cave goblin,” and ”brutal little goblin of a dynamic” are two responses ChatGPT gave to a Reddit user in February. “Since 5.3 and 5.4, it’s started comparing anything negative to being a goblin,” said the Reddit user.

And they were not the only person to experience this. “After the 5.4 update, ChatGPT uses ‘goblin’ in almost every conversation. Sometimes it’s ‘gremlin.’ A recent chat of mine used goblin three times in four messages,” said another user on the popular tech forum Hacker News.

Indeed, the appearances of all these goblins led OpenAI — the creators of ChatGPT — to investigate and publish an article on their blog, titled “Where the goblins came from.”

The short answer is: it was an accident. Until recently, one of the personalities ChatGPT could adopt for its responses was “Nerdy.” In training this personality, they encouraged the model to use metaphors of fantastical creatures. “We unknowingly gave particularly high rewards for metaphors with creatures. From there, the goblins spread,” says the OpenAI article.

These unusual or unexpected reactions from AI models are more common than they seem. A group of Spanish researchers recently published a scientific article with another surprising finding: AI chatbots love to talk about Japan. “It was a surprise to see how Japan began to stand out in the models’ responses,” says Carla Pérez Almendros, a professor at Cardiff University and co-author of the study.

It’s already known that the models are biased towards Western values, but the passion for Japan went even further: “In English, Japan is the most frequently mentioned country, because we exclude the U.S. and the U.K., but even more interesting was seeing that the same thing happened in Spanish and Chinese, because that’s where we would have expected the U.S., for example, to be the preferred choice. But no, there was Japan,” explains Pérez Almendros.

OpenAI employees…

Source