Translation memory and TMX, explained

A translation memory (TM) is a database of previously translated material. When a translator works on a new sentence, the tool looks for a similar sentence already translated and retrieves the stored translation (Bloodgood and Strauss, 2014). The authors date the main ideas to the late 1970s and early 1980s, and wide use to the late 1990s. A TM pays off on repetitive work, such as updated product manuals or several drafts of legislation (Koehn and Senellart, 2010). Two things govern its use in practice: TMX, the format that moves a memory between tools, and the fuzzy match percentage that decides which stored translation a translator sees.

The TMX standard and who maintains it

TMX, Translation Memory eXchange, began as a standard of OSCAR (Open Standards for Container/Content Allowing Re-use), a special interest group of the Localization Industry Standards Association (LISA). The version published by a standards body is ETSI GS LIS 002 V1.4.2, issued in February 2013 by the European Telecommunications Standards Institute (ETSI). It recreates TMX version 1.4b, whose change notes cite revisions dated October 7, 2004, and April 26, 2005. The stated purpose is a standard way to describe translation memory data exchanged among tools or translation vendors while losing little or no essential data.

The document records the handover. LISA was declared insolvent in March 2011. LISA's standards were authorized for posting under a Creative Commons Attribution 3.0 license, and LISA designated ETSI's Localisation Industry Standards (LIS) Industry Specification Group as successor for its standards portfolio. An ETSI release of July 13, 2011, announcing the group, said its members planned to extend TMX and the Segmentation Rules eXchange (SRX) format. The 2013 document points to a Globalization and Localization Association (GALA) web address for past LISA standards, which returned a not-found page on October 22, 2026.

How a TMX file is built

TMX is defined in two parts. The first is the container. A file has a tmx root holding a header, with metadata about the file, and a body with the translation units in no specific order. A translation unit, tu, is an entry of aligned segments in two or more languages. Each unit holds at least one variant, tuv, for a given language, and a complete memory has at least two per unit. The text itself sits in a seg element. The second part is inline markup: elements such as bpt, ept, and ph wrap the formatting codes of the original document.

The two levels of support matter. Level 1 carries plain text only, which the specification calls enough for data without inline codes, such as software messages, and not sufficient for documentation-type formats. Level 2 adds the content markup, so a tool that supports it can re-create the translated document from the TMX file alone. How the text was split into segments travels separately, in an SRX file. TMX files are always Unicode: UTF-16, UTF-8, or 7-bit ISO-646 with numeric character references. Header attributes record the source language, segment type, creation date, and creator ID.

How fuzzy match percentages are computed

Vendors rarely say. Bloodgood and Strauss, of the University of Maryland Center for Advanced Study of Language, write that commercial retrieval algorithms are typically not disclosed but are widely believed to rest on edit distance: the number of word insertions, deletions, and substitutions that turn one sentence into the other. Koehn and Senellart give one formula: one minus the edit distance between the new sentence and the stored source sentence, divided by the length of the longer of the two. Their example compares "The second paragraph of Article 21 is deleted ." with a stored "The second paragraph of Article 5 is deleted ." Counted in words, one substitution in nine words gives 89 percent. Counted in letters, the score is 95 percent. The same pair carries two percentages, depending on what the tool counts.

Bloodgood and Strauss call a 70 percent threshold a widely used setting whose basis remains something of a black box. They tested retrieval methods against helpfulness ratings from five crowd workers per pair, using a 10,000-pair memory. On 400 Chinese-English software documentation sentences, edit distance retrieved the most helpful segment 193 times and their weighted n-gram method 282 times, and the two agreed on the best segment only 35.0 percent of the time. On 300 French-English medical sentences, the counts were 148 and 201.

Translation memory and machine translation together

In a 2009 study, Ana Guerberof had eight professional translators work from English into Spanish on 791 words without knowing where each segment came from: 265 words of new text, 264 of TM matches in the 80 to 90 percent range, and 262 of statistical machine translation (MT). Against translating from scratch, the mean productivity gain was 25 percent with MT and 11 percent with TM matches. The medians were 13 and 10 percent, and the spread was very high. The TM segments also carried 5 errors left by the human translators who built the memory.

Koehn and Senellart kept the stored translation and had a statistical MT system translate only the mismatched words. That method beat both plain MT and plain TM for fuzzy matches of 80 percent and above; below that, MT alone did best. A second method of theirs led over 70 percent. Bulté and Tezcan carried the idea into neural MT in 2019 by appending the translations of retrieved fuzzy matches to the source sentence. On 3,207 English-Dutch test sentences from the DGT-TM translation memory, BLEU, a measure of overlap with reference translations, rose from 51.45 for the neural baseline to 58.91. In the 90 to 99 percent match range, the gain over that baseline was 21.95 points for Dutch and 22.76 for Hungarian. The method paid off from fuzzy scores of about 50 to 60 percent upward.

What to check

• Does the TMX export support Level 2, or will it drop the inline codes that documentation needs?
• Does the tool count fuzzy matches in words or letters, and divided by which sentence length?
• Was the match threshold set with data from these documents, or left at a habitual 70 percent?
• Have the stored translations been checked against the current glossary, given that legacy TM segments carried human errors?
• When TM and MT are combined, in which fuzzy match range was the gain measured?
• Does the TMX file keep creation dates and creator IDs, so a reviewer can trace each stored translation?

Azcuna's BABEL reads, translates, and searches foreign-language documents entirely on-device, with no cloud and no connection, and keeps every claim tied to the passage it came from.

Sources

1. 1ETSI GS LIS 002 V1.4.2 (2013-02), Localisation Industry Standards (LIS); Translation Memory eXchange (TMX) · European Telecommunications Standards Institute, Industry Specification Group LIS (February 2013)
2. 2New Industry Specification Group on Localisation Industry Standards · European Telecommunications Standards Institute news release (July 13, 2011)
3. 3Translation Memory Retrieval Methods · Bloodgood and Strauss, EACL 2014 (ACL Anthology)
4. 4Convergence of Translation Memory and Statistical Machine Translation · Koehn and Senellart, JEC 2010 workshop, Denver (ACL Anthology)
5. 5Productivity and Quality in MT Post-editing · Ana Guerberof, Beyond Translation Memories workshop, MT Summit XII, Ottawa, 2009 (ACL Anthology)
6. 6Neural Fuzzy Repair: Integrating Fuzzy Matches into Neural Machine Translation · Bulté and Tezcan, ACL 2019 (ACL Anthology)

Where this shows up in our work

• BABEL, offline multilingual translation and search translates and searches foreign-language documents on-device, every fact tied to its source passage.