FAQ FAQ | Contact Us Contact Us | Client Portal Client Portal
Professional Language Services

Blog

Blog
Feb 21

Assessing the Quality of Your Translation Quality Assessment: A Guide to Bad QA

Among all the factors that go into a translation project, quality and accuracy have almost become a given. No matter how cheap, quick, or specialized you want your translation to be, you assume it’ll be good quality. But you don’t speak Russian—so how do you actually know if it’s good?

Let’s step back from translation for a second and talk about cars.

If you’re one of the 86% of Americans of age who drive, you probably have an elementary understanding of how a car works. You can tell when something is really wrong, like a flat tire or an engine that won’t start in the cold. A couple times a year you take this expensive hunk of machinery—which you rely on to get you to work and go on road trips and keep you alive while hurtling down the highway—to someone who’s most likely a complete stranger, and you let them take it apart and put it back together again, then you drive away feeling relatively assured nothing is going to explode as you pull out of the lot.

But how do you know that mechanic got all the screws back in place? That the wiper fluid didn’t get switched with the antifreeze? What does a spark plug even do?

Those probably aren’t the questions you’re asking, and you probably don’t pop the hood to give things a once-over before you drive off the lot. Why? Because you went to a mechanic you trust, who has a system in place to make sure that they avoid exploding cars and wobbly tires.

Back to translation. You speak a language, maybe a couple if you’re lucky. You know how grammar works and you might even be able to diagram a sentence. You can tell the difference between Arabic and Chinese, and you know that if you see “□□□□□□□” anywhere in your survey, there is probably something wrong. But how do you know if everything is spelled correctly? If the translators used all the right words? That they didn’t leave out an entire question or actually type out their favorite haiku instead of a demographic question?

This is where Translation Quality Assurance comes in. But, like everything, there are good and bad ways to make sure you’re getting what you paid for. To top it all off, most of the bad ways are expensive and time consuming processes with very little payoff.

Bad QA #1: Back-translation gone wrong

After translation, some clients request back-translation services to measure the objective quality of the translation. While back-translation tells you what the translation says in a language you can’t read, it doesn’t tell you much about the actual translation quality.

Back-translation seems to make a lot of sense: if you can translate a sentence in one direction, you should be able to translate it back and get the exact same result. If the result isn’t the same as the original, the sentence could be wrong:

English source text Spanish target text Possible back-translation
Their boss is very nice. Su jefe no es muy amable. Their boss is not very nice.

In the example above, the back-translation caught an error. But errors like this are extremely rare when the standard translation and edit quality steps are performed by qualified and attentive linguists.

This logic also denies the subjective variations that exist in each language, as well as the reality that any one word can have several correct translations. Consider the example below, where the English sentence was translated into Spanish and back:

English source text Spanish target text Possible back-translation
Their boss is very nice. Su jefe es muy amable. His manager is very friendly.

There are considerable differences in the two English sentences, even though the Spanish translation for each of those English sentences is equivalent. In this case the back-translation result seems to flag an error where there isn’t one, and the process has not added any value to the translation.

In order to objectively evaluate translation quality, it’s best to have another linguist (who didn’t work on the translation or edit) check over the translated materials to make sure they’re ready to go. This extra quality assurance step is called a third-party review.

What you think you’ll get from back-translation:

Two equivalent English files if the translated file is 100% correct; an objective quality measure

What you actually get:

Another translation that probably doesn’t match your English; probably a headache

Car metaphor equivalent:

“You took my car apart and put it back together again. To make sure you got it right, why don’t we just take it apart one more time?”

Bad QA #2: Google Translate

As a translator, watching Google Translate evolve over the years has been kind of like watching a puppy grow up. It used to be really, really bad (ate your shoes, bit your fingers), but it learns and gets better (housetrained, sits and stays) as new human-translated content is added. Nowadays you can put something into Google Translate and be reasonably confident that the output will be understandable, but no matter how much you train your puppy, he probably shouldn’t be trusted to drive your car alone. (Even these pups had close supervision)

We often get questions from clients along the lines of, “I took this word from your translation and plugged it into Google and that word doesn’t match the English. Can you check it?”

Besides my concern that they just spent several dreadful hours copying and pasting a file into Google Translate, this is another example of bad QA. Just like back-translation, Google can only ever give you another translation of the content, and likely an imperfect one at that. It’s good for picking out huge errors (“This is a haiku, not a demographic question!”), but those aren’t the kind of “errors” you’re going to find in a professional translation. In other words, Google Translate can’t tell you if the translation will actually sound good to a native speaker of that language.

English source text Spanish Translation English Google Translate
Their boss is very nice. Su jefe es muy amable. Your boss is very kind.

If you used Google Translate alone to assess my translation, you might think that I translated the whole text in the second person instead of the third. When sentences get more complicated than this, or if you try a less machine-translatable languages like Russian, you’ll start to see differences in word order, terminology, and maybe even tenses or subjects.

Google Translate is a strong and useful tool, but it’s not a foolproof solution to understanding a foreign language, and it certainly can’t tell you if the subtleties of a given translation are on par with your source text.

This is another place you might consider third party review.

What you think you’ll get from Google Translate:

A word-by-word assurance that the translator didn’t get anything wrong

What you actually get:

Non-helpful machine translation, copy/paste hand cramps

Car metaphor equivalent:

“You took my car apart and put it back together again. My dog and I are just going to take those tires back off and check things out for ourselves to be sure.”

Bad QA #3: My co-worker’s cousin’s best friend took French in high school

It might be tempting to run those translations past someone—anyone—who speaks the language. I get this one! I just called my dad yesterday to ask about my tires, because he might not know as much as a mechanic but he definitely knows more than I do.

The problem with this is that a non-native speaker, or someone who isn’t trained as a linguist, isn’t likely to pick out subtle things in the translation that could be improved. They’re also likely to introduce errors into a translation that might have been perfectly correct before.

However, if you do want another opinion on a translation, you have two good options:

The first is—you guessed it!—a third-party review.

The second, which will actually add even more value to your translations, is an in-country review. This is a process where someone from your client’s organization reviews the translations to see that they line up with the specific language and terminology that they use to talk about their company.

What you think you’ll get from your co-worker’s cousin’s best friend who took French in high school:

confirmation that this is the real deal

What you’ll actually get:

Maybe some changes, and probably some errors introduced into the text

Car metaphor equivalent:

“The mechanic just finished up my car, I think I’ll have my co-worker’s cousin’s best friend who owned a car once take a look at it.”

Conclusion

You might have noticed a pattern here. If you really want to do quality assurance that adds value to your finished translations, third party or in-country reviews are great options. But one of the best ways to ensure quality is to start out with a quality process. If you know that your translation provider is rigorous about translator qualifications, uses good technology, and is experienced with your subject matter and languages, then you’re much more likely to get a quality product the first time. No copy and pasting into Google Translate required.

If you liked this post by Katherine, you might also like her post where she uses a cooking metaphor to illustrate the process for translating software strings

Katherine Rucker

Katherine combines her experience as a translator with great technical proficiency and a proactive attitude, providing high-caliber translation solutions for her clients. She holds an MA in Literary Translation Studies from the University of Rochester and a BA in Spanish from Centre College. She studied abroad at the University of Reading in the United Kingdom and has taught English in Spain and Spanish in the United States.

Leave a Reply


← Back to Blog