Baloneydar for Translation Consumers: Learning to Doubt the Extraordinary

The plethora of information in cyberspace has placed extra burdens on people seeking information and services. At a distance from consumers, it is easy for a translation seller to exaggerate or simply lie about capabilities and expect to get away with it a reasonable percentage of the time.

When the subject matter is widely understood, people who take liberties with the truth can be quickly discovered. When a non-specialist encounters narratives spun in totally unfamiliar fields, however, things get more difficult. This often happens with translation services.

Carl Sagan warned people to doubt extraordinary claims and to demand extraordinary proof for extraordinary claims. He proposed what he called a baloney-detection kit. Something of that should be useful for translation consumers as well. Here are some claims made by translation brokers that a well-tuned, sensitive translation baloneydar should pick up as being suspicious or simply deceitful.

We have 10,000 vetted translators! This is clearly nonsense. No translation company “has” anywhere near that number of translators. In fact, most translation is sold by translation brokers that “have” no translators. One translation broker, however, recently claims to have 194,000 vetted translators. Vetting even a tiny fraction of that number (or a tiny fraction of even 10,000 translators) is simply nonsense.

To start with, translation companies rarely have any more than a few translators themselves as employees, and even those translators might not work in the languages you need. As noted above, many have no translators, since they purchase and resell translations as opposed to executing translations.

The above claim is almost certain to be wrong in another way, because agencies claiming to have that many translators don’t even have the capability of vetting translators and evaluating translations themselves, these also being services that they must outsource.

We translate 150 languages! A translation seller making this claim might only rightly lay claim to undertaking to find translators of all those languages after they receive an order. Why after? Because translation sellers are not going to have or even know many translators working in that many languages.

The reality is that translation between about ten languages accounts for almost all commercial translation activity. The remaining languages are not commercially important enough to invest in human resources—or even outside contractors—to handle them.

Our linguists can handle any field of translation! This one is wrong in two ways. A claim of having linguists do your translations is simply wrong. It shows either a serious misunderstanding of what a linguist is, or a desire to inflate the title translator, which needs no inflation. Just as many translators might have taken courses in linguistics (in addition to language courses, which are totally different) but are not linguists, very few linguists are capable of meeting the needs of commercial translation. These are two completely different knowledge domains and skill sets.

In addition, claims of having translators at bay ready to translate subject matter in any field are not believable. If you doubt that claim—as you should—call the translation company and ask to speak to the translator specializing in your field. You will almost certainly not be put in touch with a subject-matter expert translator and, unless you are dealing with a small, specialized translation provider, you very likely will not be able to interact with anyone who is even a translator in the language pair you require.

We provide translations in 24 hours! There might be a grain of truth here. In fact, there might be more than a grain, but what kind of translation are we talking about? Achieving any two of the three goals of speed, quality, and low price is not that difficult or astounding; adding a third is extremely difficult and usually quite expensive.

The above are just a few examples of unbelievable claims. Similarly incredible claims are not at all rare. The key is to be vigilant and, perhaps more importantly, to actively engage with the entity you intend to purchase translations from, and remember that the low-tech telephone is an excellent tool to use in qualifying a translation vendor making extraordinary claims.

Would you hire this translator?

Imagine you have some important Japanese documents to translate into English. Further imagine that you have searched for a translation provider, and you have found one that boasts of having a translator who can handle your documents, who we will call Translator X.

Translator X is claimed to translate from Japanese to English with high quality and absolute consistency. But in the vetting process, several reasons for concern were identified.

One is that Translator X’s native language is neither Japanese nor English. In fact, because of X’s upbringing, X can claim no native language at all.

Another is that Translator X, as amazing as it might seem, has never lived in a country in which either Japanese or English is spoken as the native language and has never communicated with the natives using that language.

Yet another cause for concern is that Translator X has never held a real-world job other than translation of received documents.

And, as if the above-noted concerns were not enough, there is reliable evidence that Translator X has absolutely no understanding of the subject matter of the documents you need translated and is totally incapable of catching errors in a document being translated.

Perhaps, as the ultimate deal-breaker for mission-critical translations, Translator X is not legally competent to attest by a declaration to the accuracy of the translation they produce.

Would you hire this translator? Probably not. But if a translator such as this is acceptable, look to machine translation, because all the characteristics noted above are those of machine translation.

You get what you pay for, and sometimes other people also get to keep what you pay for.

Law firms ordering translations from large translation brokers in the US should remember that they have essentially no control over who does their translations and in what country the translations are done.

Yes, the Internet is convenient and brings together supply and demand separated by great distances. But do you want your sensitive litigation or patent prosecution documents to be sent to China, to be translated by entities and people you (and often your US-based translation broker) can never know? I would think not, but many law firms using US companies purporting to do translations probably have no idea where their documents are being sent to be translated.

You might be receiving letters of accuracy certification attached to translations, but these are very often not signed by a translator and, when you see a translator’s name, we have discovered in numerous cases that it is a translator who had nothing to do with the translation or is a totally fictional name. In general, the person signing the certification letter is often a non-translator who has no ability to know or attest to the accuracy of the translation you were sold. It’s best you know what is going on after you place a translation order.

Avoiding the above-noted risk is not that difficult. One way is to engage with a translation company that actually does translations; most translations are done by a few large translation brokers that don’t do any translations. Your translation provider should also commit never to sending your documents to high-risk locations or to places that use translators that they cannot identify.

Why Translation is not a Commodity

Some translation clients and many translation brokers, those companies that sell the bulk of Japanese-to-English translations outside Japan, appear to mistakenly treat translation as a commodity; step up and order 100 pages, much as you would order 100 barrels of crude petrol or 100 tons of wheat. Numerous translation brokers say that they have thousands of translators; one even amazingly boasts of having 194,000 “vetted” translators, whatever that might mean. Not much, I am afraid, beyond their expectation of a high level of credulity on the part of their target client demographic.

If the world of translation were to be like that, things would be much simpler. Alas, they are not, and things are not at all simple. The reasons are various, but let us focus on the ways in which translation is not the commodity it is too-often treated as.

There are no generally applicable metrics to judge translation quality. Unlike commodities, translation quality cannot be judged by standard evaluation methods. Whereas the chemical properties of petrol are definable and measurable, translation requires highly skilled translators, not only to execute the translations, but also to evaluate the quality of already-executed translations, be they the products of other human translators or the output spewed from a machine translation system.

Lack of ability to stockpile reserves. You can stockpile petrol. With translation, however, when highly skilled translators are not needed for one particular translation demand, they will migrate to other assignments. And companies positioning themselves as translation companies and claiming to “have” translators at the ready are guilty of more than just stretching the truth. In almost all cases those companies are merely purchasing translations from translators not under their control and will usually need to scramble to find a translator when they receive an order, because they do not and could not “have” a reserve of translators; and, of course, most have no translators of their own at all.

The translators producing translations for you are not interchangeable.  It takes many years to become a skilled Japanese-to-English translator; studying Japanese language in a university can be very valuable, but is rarely sufficient.

The translator not only must acquire familiarity with the source language far exceeding textbook learning, but also must acquire field-specific knowledge. That process usually takes years, and the above-noted 194,000 “vetted” translators have surely not embarked on journeys that would lead to such knowledge.

One translator cannot be dropped into a position of another to translate something outside of their field of expertise without risking serious quality problems. Translators are simply not interchangeable components in the translation process. They, like translation, are not commodities.

There is no manual defining the process of producing a high-quality translation.  People aiming at being translators can go to a university to learn a foreign language and even participate in a translation program and graduate with honors, but still be quite unable to master the skills required to produce high-quality translation.

As impressive as fluency in two languages might be, it does not make someone a translator. Translation is an essential skill separate from language fluency and must be acquired by translators, who are certainly not replaceable with translators who only know the two languages they purport to work between.

There is no assurance that a particular individual aiming at becoming a translator will have what it takes to succeed. Some people acquiring a foreign language will never succeed at translation. With due respect to professionals in fields such as law and medicine, the risk of failing is surely greater with translation than in those fields, particularly since, as noted above, there is no manual to describe definitively how to produce good translations. And, of course, there is a good amount of nature mixed in with nurture in the development of a skilled translator.

Translation is people.  It is as simple as that. There are myriad paths into the field of translation, but none of the ones followed by skilled professionals lead to—or should lead to—the position of being a commodity or producing commodity translations. Translation is much more complex and fraught with uncertainties than the translation brokers boasting of owning all the translators in the world would like you to believe. In short, translation is not a commodity.

Some Misconceptions about the Japanese-to-English Translation Process

The other day we introduced a blog post about myths some translation users believe—and are led to believe by translation providers—about the translation business. Today we will focus on the translation process, particularly as it affects translation quality, pointing out some misunderstandings about the process. Some of these misconceptions are related to the highly vaunted MT (machine translation) capability sold by some companies, but others apply as well to translation done in the conventional way, using human translators.

Misunderstanding One: Machine Translation Has Taken over from Humans.

I recall a day in the mid-1980s when a company developing MT here in Japan (Bravice International) apparently convinced Mainichi, one of the three largest daily newspapers here, to run a substantial article announcing that automatic translation had finally been developed. Well, translation of a sort.

The article was very similar to a press release from the company, and at least people in the translation business realized that MT was not even close to being poised to take over from humans just yet. But shortly thereafter I did have some people comment to me that I might be out of work shortly, what with machine translation taking over. That was 35 years ago. Not much has changed, MT has not taken over, and we don’t yet feel threatened. We are amused, however, by some misconceptions about MT harbored by a small portion of potential clients, but those are easily overcome. Our existing clients are a bit wiser.

Misunderstanding Two:  Well, at least machine translation has eliminated the need for human translators to do the actual translation; humans can just fix what the MT got wrong later.

The prospect of eliminating expensive and slow human translators has long been implied by people selling MT. Alas, that has not yet been achieved. Humans (and more importantly, human translators) are still required to rescue the low-quality output of machine translation systems (and to rescue the people who thought they could rely on MT). More on this below.

Misunderstanding Three:  A translation that is 95% accurate, be it from a human translator or a machine translation system, only requires a small additional investment to make it a 100% accurate translation.

People who believe this should try machine translation and tell their human editing/rescue team that, to find and correct the translation problems, they have been allotted only 5% of the time that they would have needed to translate the Japanese-language source text from scratch, without the “aid” of machine translation. That clearly will not work. In fact, it probably will fail even if you give the human rescue team 50% of the from-scratch translation time to fix the MT output.

Often the rescue team (made up of people who are optimistically referred to as post-editors) will need to spend nearly as much time fixing a poor translation from an MT system (or a low-skilled human translator) as they would have spent in producing a translation from scratch. And even then, because of the desire on the part of the human repair team to give the benefit of the doubt to the machine translation (or poor human translator), the finished product is still likely to fall short of the quality of a translation produced from the ground up by a qualified human translator. There are several problems at work here that might not be obvious to a first-time user of MT.

The above misunderstanding has three aspects, related, respectively, to translation quality metrics (or, more precisely, the lack thereof), the ability of a human translator to find errors, and the questionable advisability of using a translator—human or machine—capable of only 95% accuracy.

Metrics.  How does one quantitatively assess the accuracy of a translation to start out with? And what does 95% accuracy mean when even five percent of the words mistranslated can make a translation totally wrong and unusable.

What’s more, finding even small parts of a translation that make it a bad or unusable translation is a decidedly non-trivial task. There are no automatic methods that obviate human intervention in either assessing a percentage of mistranslation or evaluating the usability of a translation, regardless of whether the translation was done by software or by a human, or to find the N% that was incorrectly translated, let along correct the errors.

The most commonly cited standard that has been established specifically for translation, ISO 17100, addresses only the translation process, dealing with the administrative processes of executing and providing translation. It lacks guidance regarding either how to assess the quality of a translation or the definition of a good translation. Addressing those tasks continues to require a human translator. To be sure, the above-noted process-directed standard apparently calls for a translator’s work to be checked by another translator, but the issue of the abilities of those translators remains unaddressed, as does the definition of a good translation.

Finding errors.  It is highly unlikely that a non-translator (i.e., inexpensive) “post-editor” (a distinct misnomer in most cases) is going to easily find the “mistranslated 5%.” That is a task for a translator. And even a human translator will have to go through both the source document and the translated document carefully to find errors. This is certainly the case when using machine translation, but it is just as true when checking and correcting a translation done by a poor human translator.

The result of that effort might very well be that the errors found would be of no consequence to some particular purpose of the translation. But learning even that ultimate outcome will require a full check through the source text and the translated document by a qualified human translator. Ultimately, if you need a high-quality translation, you are very unlikely to gain much in speed or economy by having a poor translation done first, whether the translator is made of flesh and bones or of software code.

Should you even consider using a 95% accurate translator?  Is a translator, human or machine, only capable of producing 95% accuracy usable? People in the translation field know the answer is a resounding no, and I know no human translators who boast of only 95% accuracy or who think that 95% accuracy (whatever that might be taken to mean) would be sufficient. Regardless of the oohs and aahs that an MT system gets when it reaches that level, being impressed with an MT system reaching 95% accuracy merely demonstrates that the bar has been lowered to allow or promote use of technology. The lowering of standards to make use of wonderful modern technology is not unique to machine translation, of course, but that is a topic for another day and another article.

Because the next two misunderstandings are somewhat interrelated, I will discuss them together.

Misunderstanding Three:  If a document is only something like an email, it can be translated satisfactorily by an MT system.

Misunderstanding Four:  MT systems cannot translate literature, but can handle non-creative documents such as found in industry.

Over decades of observing the marketing of MT, we have repeatedly heard that MT can’t, of course, translate a novel, but (sometimes left as an unspoken implication) it can translate non-literary texts.

It is certainly correct that you should not attempt to use MT to translate a novel. But that in no way should lead you to conclude that MT can be generally used without ill effects to translate texts that are not literary or creative in nature, and it is just such non-literary texts that make up the overwhelming portion of commercial Japanese-to-English translation.

The notion that Japanese-language texts produced in the business and technical/industrial domains are not informed and influenced by Japanese culture is far from being correct. The very act of writing a message to someone, even in a business or technical/industrial domain, is influenced by the context and the culture shared by the writer and the reader. Neither a machine translation system nor a human translator who has not lived in one of the source- and target-language cultures (commonly the case these days with discovery document translation) is up to the task.

Take, for example, the case of email exchanges that need to be translated in an antitrust matter. Some common expressions used in everyday business and normal life in Japan, when placed in the context of people accused of antitrust behavior, can take on (or be purported to take on) a significance far beyond their common use as boilerplate formalities in everyday email exchanges. The translation of such communications without taking context into consideration could be misleading at best.

In such cases, even humans have difficulty translating the source texts. Machine translation systems, totally unable to understand culture and not caring about context, are breathtakingly bad at handling such translation tasks. I have seen this time and time again, including with documents presented to witnesses in depositions. And a non-translator human rescue squad is also in serious danger of failing if they do not both know the languages and understand and apply the appropriate contextual and cultural clues. The bottom line is that these texts should be translated from the start by qualified human translators aware of and able to understand the context of such communications. The need to know and consider context has been with us since people started translating foreign languages; the appearance of machine translation has not changed that. The blinding speed of MT and ostensibly low cost of MT should not blind translation consumers to the pitfalls.

As noted in a recent article, a human translator with neither the source language nor the target language as their native language is almost as limited as a machine translation system when confronted by texts written in the real world.

Misunderstanding Five: Translators are Linguists.

Translators are sometimes characterized as linguists. Some translators are indeed linguists. But many good translators are not linguists, and many linguists are not translators and would be poor translators, even if they wanted to be translators, and many do not. Additionally, merely being fluent in two languages does not give one the ability to translate well. Translation skills need to be acquired separately from (and can be acquired without) formal training in the field of linguistics, although linguistic knowledge is often acquired in the process of learning a language and learning translation, these being two processes that are essentially separate from the process of learning the formal discipline of linguistics.

Misunderstanding Six: Machine Translation is Very Fast.

Here, finally, is a misunderstanding with a bit of truth in it. Fast, yes. The questions remain as to how much quality you need, how tolerant you are of serious errors, and whether you need accountability for the accuracy and appropriateness of the translation.

Quality.  If the level of quality produced by machine translation is “good enough” for your application (for example, a quick gist translation, followed by a proper translation done from scratch by a responsible and accountable human translator afterward if the content is judged to be important), then it is perhaps “good enough.” But it is foolishly optimistic to hope that quickly produced MT output can be brought in a short amount of time up to the quality needed to trust its accuracy. And who could attest to even 95% accuracy of a machine-produced translation, when we don’t even have a definition of 95% accuracy?

Risks.  Anything worth spending money to have translated warrants translation to some expected level of accuracy and suitability.

The largest Japanese-to-English translation demand in the US is for civil litigation involving Japanese entities. Litigators need to assess just how bad a translation could be and still be acceptable as a translation of a Japanese discovery document, and should remember that both machine translation systems and low-quality human translators are quite capable of producing unbelievably bad translations. If we extend the discussion to translations in the medical and pharmaceutical fields, the risks extend to the health and wellbeing of many persons not involved in the translation process.

Accountability.  Placing a machine translation system in the translation loop, even with an after-the-fact human damage control team, blurs the chain of accountability. And, of course, it is folly to think that anybody would be foolish enough to declare that a raw machine-produced translation is reliable and usable to N% (where N is even a low number) accuracy.

Although a human translator that is identified can be deposed and examined about a translation they did, you are not going to be able to depose a collection of computer code, and the people using the MT system cannot but trust as an act of faith that the translation it produces is usable. That is a misplaced belief in almost all cases, based on observing a large amount of machine-translated texts.

Summing up, unless you have a high tolerance for serious errors and the damage they can cause, a human translator is still very much an essential part of the process of producing the translations you need. And the humans executing translations need to be highly qualified, with subject expertise and real-world knowledge, two areas in which machine translation has yet to make a sizable dent. In a future article, I will discuss the making of a translator, something about which there are also some misunderstandings, including misunderstandings by some people aiming to be translators or already working in the field.