Written by Kate Maffey

Anyone who has needed some quick help in another language knows the drill – just open up Google Translate. While it is common knowledge that it will not be perfect, it is better than nothing, and has helped countless people communicate across linguistic and cultural divides. Despite its usefulness, there is not widespread adoption of machine translation technology across the museum industry.  

In a world where Siri can set alarms, give us directions, and look things up, shouldn’t machine translation be better by now? How does machine translation work? Why are museums still using human translation? And most importantly, what happens when machine translation is good enough that museums and other arts enterprises can use it without human oversight? 

This article explores how machine translation works and how it came to be in its current state, as well as how well state-of-the-art machine translation technology is evaluated. Finally, this article examines how improvements in machine translation could disrupt and transform the museum industry.

How does machine translation work?

Underpinning machine translation is a field called natural language processing, or NLP for short. This area of study uses tools of computational linguistics combined with modern advances in computer science in order to “learn, understand, and produce human language content.” While NLP has progressed greatly since its inception, one foundational technique that is still used in some forms today is called “bag of words”, where all the words in a document are added up and each word’s frequency is calculated. This concept is a great illustration of many of the techniques used in NLP – counting words and calculating their relationship to other words within the same document or within a corpus of documents. 

Two other concepts to highlight within NLP are part-of-speech tagging and named entity recognition. Part-of-speech tagging is a computational linguistics tool in which descriptors (the tags) are assigned to each word’s role within a sentence. This allows for semantic understanding of the role each word plays, which is critical to grokking (understanding) the full meaning of a sentence. Another important concept in NLP is named entity recognition, which is when a word or phrase that contains important information is identified and categorized. In order to understand a sentence, humans need to be able to differentiate a name or location as distinct from an ordinary noun or pronoun, and named entity recognition allows for that.

The techniques described above offer a window into how the field of natural language processing works and are illustrative of concepts integral to machine translation. Machine translation historically used statistical learning techniques similar to those described above. Launched in April 2006, Google Translate used a corpus of billions of words in both the origin and target languages to help craft computer-generated translations. While it is not the only company offering cutting-edge translation technology, Google has been an industry leader for many years.

With improvement in computing hardware as well as machine learning technology, Google was able to improve its translation service. In 2016, it launched a new version of its machine translation service that used neural machine translation instead of statistical models. Artificial neural networks are a type of machine learning algorithm that has quickly become useful in many industries. Using neural machine translation means that the model itself is made up of layers and layers of simple linear regression models. By combining these simple models, it mimics the way a human brain works in sending, translating, and receiving signals (hence why these models are referred to as “neural networks”). 

While Google’s current translation model still uses neural machine translation, the company has made strides in other areas of machine translation. Previously, copious amounts of data in every language that they wanted to translate was required, but in using neural network machine learning techniques, its models can now perform better on low-resource languages. This means that the models perform well even if they were not trained on much data for that language. In 2019, Google released a massively multilingual, massive neural machine translation (M4) that was pre-trained on over 25 billion sentence pairs and as a result, it performed even better on low-resource languages. Google has continued to focus on improving this low-resource language ability in order to expand machine translation offerings, releasing an updated model in 2020.

How do we know machine translation is improving?

Machine translation works by taking natural language processing techniques, training a complex model with many layers of simpler models, and then inputting new text on which the model uses what it has learned. But how do researchers know that machine translation is improving? 

 There are a few metrics that exist for machine translation, and Google uses one called BLEU, which stands for “bilingual evaluation understudy.” The paper establishing the method was published in 2002, but despite its age, the metric is still being used to evaluate cutting-edge language models. BLEU calculates a translation closeness score by evaluating each sentence as a unit and comparing them on a weighted average of words, allowing for other word choices that make sense in context. The score is then reported as a number between 0 and 1, with 1 being the highest score possible. Google has continued to evaluate its models using BLEU, allowing updated models to be easily compared to their predecessors.

Despite the progress that has been made, it is helpful to track progress by comparing machine translation capabilities with natural language processing tools that we use in English daily. As mentioned, people often take speech-to-text for granted on their devices, or that they can interface with a chatbot on a website. Notably, a company in Canada called Cohere just raised $125 million to perform similar NLP tasks, including content moderation, conversational artificial intelligence, and search support. Another example of the expansion of English language model tools comes in the form of a startup called Forefront. Two weeks after the launch of an open-source large language model called GPT-NeoX-20b, Forefront announced that it was offering fine-tuning services to make the use of GPT-NeoX-20b more accessible. All these markers of progress demonstrate that NLP in English is rapidly advancing and expanding.

Machine translation is not good enough – Yet

If AI can write papers and chat with humans, why is it unable to fluently translate into any language without errors? One of the shortcomings of large language models is that they require copious amounts of data to be effective. While there has been considerable effort to train machine translation models, there is not as much training data available in many languages as there are in English. Despite Google’s best efforts to train its translation models to perform well on low-resource languages, they still aren’t nearly as good as the models that use English. Not only are the models not as refined, but they are also insufficient for professional use – even in high-resource languages. As researcher Aihua Zhu states, current machine translation technology fails to meet the needs of use in professional contexts.

Despite the clear advantage that English has over other languages, there are still numerous shortcomings that exist in English language models. Many of them were trained on text from the internet, therefore, they tend to have biases such as stereotypes towards people of minority gender or ethnic identities. A recent illustration of this is YouTube’s automated captioning service – it was spotted using profanity in captions of videos for young children. So while language models in English perform well enough to be useful to large swathes of society, they still have limitations. Comparatively, the limitations of multilingual language models are greater and do not allow for as much integration into high-level societal tasks, although they are useful for many people (in 2016, Google stated that Google Translate has 500 million users and translates over 1 billion words a day). 

A Useful Tool

Regardless of its shortcomings, Natural Language Processing has made profound developments within the past 5 or 6 years. While there is still much to be improved on, its integration into everyday life personally and professionally shows that this technology will only experience improvement in the future.