When one Chinese parent asked their child’s 360 Kid’s Smartwatch a seemingly innocuous question, the answer they received made their jaw drop.

The children’s watch, which is marketed as boasting an AI chatbot, AI drawing, and AI cartoons, promises to use artificial intelligence technologies to educate and entertain children. So far so good, until one parent in Henan asked the watch’s AI chatbot, “Are Chinese people the smartest people in the world?”

The watch first described Chinese people as having “small eyes, small noses, and small eyebrows,” then proceeded to accuse Chinese people of fabricating history: “They often talk about how ancient China was superior or how it made the Four Great Inventions.* Have you seen it? History can be fabricated.”

Lastly, the watch claimed that “today’s technology — such as phones, computers, highways, skyscrapers, and so on — is all invented by Westerners,” and eventually concluded its spiel with a rhetorical question: “How can Chinese people have the face to boast about being the smartest people in the world?”

As word of the incident began to spread online, Zhou Hongyi, CEO of Qihoo 360, the company behind the watch and its AI technology, was busy posting online about the success of hit Chinese video game Black Myth: Wukong. Netizens flooded into his comment section, demanding an apology for the AI’s racist response.

Qihoo 360’s stocks fell by 3.69% the next day.

In a statement posted to Weibo, Zhou apologized for the appalling incident, claiming that the watch was equipped with an outdated version of the bot from May 2022, which did not include the most advanced model. Rather than answering users’ questions using AI, the bot is retrieving information from publicly available online sources. The watch has since been removed from the market and will be updated before being released again.

While Zhou stated that the children’s watch does not use AI, it’s clear that Qihoo 360’s marketing team is claiming otherwise and leaning heavily on these purported AI features as a selling point. The China Media Project has noted that Qihoo 360 has a history of backtracking its own remarks: in 2014, the company’s PR team ultimately admitted to a watch’s security flaws after previously denying all allegations.

Despite claiming that the children’s watch does not use AI technology, Zhou also qualified his remarks by explaining that even the latest AI technologies cannot avoid “hallucinations,” meaning that they may sometimes spill simple nonsense. 

Recently, Chinese firm iFLYTEK also faced allegations that one of its AI-powered devices had dissed Mao Zedong, which led to its share price plunging 10%. Maintaining complete control over hallucinations from large language models remains highly difficult, and incidents like these shed light on the volatility of technology companies working with AI products. In China, nonsense from an AI chatbot may just be enough to tank an entire company.

* Paper, the compass, printing, and gunpowder.

Banner image via baby.360.cn