On Wednesday, Mark Zuckerberg announced that Meta, formerly known as Facebook, had successfully developed a real-time translation system for the primarily oral language Hokkien.

In the short clip, Zuckerberg demonstrated how the system works. He started a conversation with one of the program’s researchers in English, and the AI immediately read out the translated sentence in Hokkien. The translator successfully works both ways.

Hokkien, or Minnan, as they call it on the Chinese mainland, originates from the coastal area of Fujian province in southeastern China. The language has around 46 million speakers worldwide, with 28 million on the Chinese mainland and 13.5 million in Taiwan. It is also widely spoken by Chinese diasporas in Singapore, Malaysia, and the Philippines.

Instead of being taught in schools, Hokkien is usually passed down from generation to generation via conversation.

According to Meta’s press release, building an AI-powered translation system around an unwritten language like Hokkien proved extremely challenging, as the current technique for programming an AI model requires large amounts of written text.

Therefore, Meta developed a new model using innovative approaches, like transforming speech into a sequence of acoustic sounds, generating waveforms from them, and relying on text from related languages, like Mandarin.

The new model is still a work in progress, as it can only translate one complete sentence at a time. Meta is now open-sourcing the model, hoping that others can build on their work and help to break down more barriers for the remaining 3,500 living languages that are also primarily verbal.

“We believe spoken communication can bring people together wherever they are located — even in the metaverse,” reads the press release.

China has had similar success in using AI technology to break down language barriers and preserve local dialects, but those programs have mainly been text-based. For instance, Flytech developed an AI transcription program that carries out translations and transcriptions to and from Mandarin for Cantonese, Uyghur, and Tibetan, as well as more than 10 dialects spoken in Taiyuan, Guizhou, and Yunnan.

Cover image via Wikimedia