TER translation edit rate: formula and examples

Translation edit rate (TER) measures the edits needed to make a machine translation match a reference, divided by average reference length.

Lower scores mean fewer edits per reference word. The metric counts word insertions, deletions, substitutions, and shifts of word sequences. Snover and colleagues introduced TER and its human-targeted variant in their 2006 paper. (A Study of Translation Edit Rate with Targeted Human Annotation)

What is the translation edit rate formula?

The translation edit rate formula is the number of edits divided by the average number of reference words. Multiply the ratio by 100 when reporting a percentage.

TER = edits / average reference words
TER percentage = 100 × edits / average reference words

With multiple references, take the lowest edit count found against any reference. The denominator remains the average length of those references. That is how sacreBLEU's TER implementation calculates each segment's statistics. (sacreBLEU TER metric)

Suppose 2 references contain 8 and 12 words. Their average length is 10 words. If the best match needs 3 edits, TER is 3 / 10, or 30%.

Do not divide by the machine output's length. Also, do not automatically use the length of whichever reference supplied the best edit count.

For a whole test set, sacreBLEU adds the edit counts and reference-length denominators before dividing. It does not simply average the sentence percentages. (sacreBLEU corpus aggregation)

For example, consider segments requiring 1 edit over 2 reference words and 2 edits over 8 words. Their individual scores are 50% and 25%. The corpus result is 100 × 3 / 10 = 30%, rather than 37.5%.

What counts as a TER edit?

A standard TER edit costs 1, whether it inserts, deletes, substitutes, or shifts words. A matching word costs 0. The original TERCOM software documents these costs. (TERCOM documentation)

The shift operation accounts for word order without charging separately for deleting and reinserting every moved word. A phrase can move as a block while retaining its internal order.

The scoring algorithm searches for useful shifts, then calculates the remaining word edits. sacreBLEU follows TERCOM-style heuristics and limits that search. A computed score therefore should not be described as a guaranteed global optimum over every possible rearrangement. (sacreBLEU edit and shift algorithm)

What does a worked TER example look like?

This example needs 1 shift and 1 substitution, producing a TER of about 28.6%.

The reference contains 7 words. Treat spaces as word boundaries and use the text exactly as shown.

1. Move “today” from the beginning to the end. That costs 1 shift.
2. Replace “checks” with “checked.” That costs 1 substitution.
3. Divide 2 edits by 7 reference words.
4. Multiply by 100: TER = 28.57%, rounded to 28.6%.

No insertion or deletion is needed. The word “today” was already present, and the shift gives it the reference position.

This is an illustrative text comparison. The score describes the changes needed to match this reference; it does not determine whether another wording is acceptable.

HTER vs TER: what changes?

Human-targeted translation edit rate (HTER) uses a reference tailored to the particular machine output. An editor makes the output fluent and faithful while preserving acceptable wording.

Ordinary TER compares against independently prepared references. HTER adds human judgment when creating the target text. The National Institute of Standards and Technology (NIST) describes that process in its 2012 evaluation plan. (NIST evaluation plan, section 6.1)

The denominator needs an explicit definition. Snover's original experiment kept the original references for length normalization, excluding the human-targeted reference from that calculation. NIST's plan likewise uses the independent gold-standard reference length. (Snover et al., section 4, NIST, section 6.1.2)

Other studies use the post-edited text's length. Sanchez-Torron and Koehn's 2016 study states that convention directly. Consequently, “HTER” alone does not fully specify the calculation. (Machine Translation Quality and Post-Editor Productivity)

HTER measures differences between the initial and final texts. It does not count every keystroke, abandoned revision, or minute of thought. The 2016 study distinguishes this final-text measure from actual editing activity. (Sanchez-Torron and Koehn, section 4.3)

The machine translation post-editing guide explains the human task surrounding the metric.

How does sacreBLEU compute TER?

sacreBLEU prepares the text, calculates edits against each reference, and aggregates the resulting counts. It reports TER on a percentage scale. (sacreBLEU TER implementation)

The defaults are case insensitive, retain punctuation, and leave optional normalization and Asian-language processing disabled. The dedicated TER tokenizer applies these choices; the tool name does not mean TER uses another metric's tokenizer. (sacreBLEU TER tokenizer)

Asian-language support affects processing in conjunction with normalization or punctuation removal. Record the combination rather than assuming that setting works independently. (sacreBLEU tokenizer options)

To score aligned reference and machine-output files:

1. Put corresponding segments on corresponding lines in both files.
2. Use detokenized text and choose the required TER settings.
3. Run the command below.
4. Save the score, full metric signature, and software version.

sacrebleu reference.txt -i hypothesis.txt -m ter

The repository documents reference-file input, metric selection, and signatures. The sacreBLEU guide covers the wider tool. (sacreBLEU documentation)

Passing a post-edit file as the sole reference makes sacreBLEU use that file's length. That does not reproduce an HTER protocol requiring a separate denominator. TERCOM provides an alternative-reference option specifically for length calculation. (sacreBLEU reference statistics, TERCOM alternative references)

How should you read a TER score?

Read TER as an edit rate: lower is better under the same evaluation conditions. A percentage score of 20 means 20 counted edits per 100 reference words.

It does not mean the translation is 80% accurate. A shift can move several words for 1 edit. A substitution can change either a minor grammatical detail or a central fact.

A score of 0 means no edits remain after the chosen preprocessing. Scores can exceed 100 when the edit count exceeds the reference-length denominator. For example, 12 deletions against a 5-word reference would produce 240%.

Compare systems using the same test set, references, and settings. Keep a record of changed reference files as well as changed software options.

The machine translation metrics guide covers other measures. Use text review to understand which errors produced the score, especially when a small wording change alters the meaning.

What to do next

• Declare whether the evaluation uses independent references or human-targeted post-edits.
• State the denominator explicitly, especially for HTER.
• Save sacreBLEU's full signature and the exact reference files.
• Check sentence examples alongside the corpus score.
• Measure editing time separately if productivity is the question.

Azcuna's BABEL reads, translates, and searches foreign-language documents entirely on-device, with no cloud and no connection, and keeps every claim tied to the passage it came from.

Questions people ask

What is the translation edit rate formula?

Translation edit rate is the number of edits divided by the average number of reference words. Multiply by 100 for a percentage. The sacreBLEU implementation chooses the lowest edit count across references for each segment, while averaging their lengths for the denominator.

Is a lower TER score better?

Yes. Lower TER means fewer counted edits relative to reference length under the same settings. A score of 0 means no edits remain after preprocessing. The TERCOM documentation defines an edit-distance measure, so a score of 20 should not be interpreted as 80% translation accuracy.

What is the shift operation in TER?

A shift moves a contiguous sequence of words to another position and costs 1 edit. TERCOM assigns the same unit cost to a shift as to a word insertion, deletion, or substitution. The moved words retain their internal order, which lets the metric account for reordered phrases.

What is the difference between HTER and TER?

Human-targeted translation edit rate compares machine output with a human-targeted correction rather than an independent reference alone. NIST's 2012 evaluation plan describes editors preserving meaning while making minimal changes. The denominator must also be stated because published HTER protocols use different length conventions.

How does sacreBLEU calculate TER?

sacreBLEU applies its TER tokenizer, finds edit counts against the references, and aggregates edits and average reference lengths across segments. It reports a percentage score. The documentation specifies case-insensitive scoring by default, with punctuation retained and optional normalization disabled; the metric signature records the settings.

Does HTER measure post-editing time?

HTER measures the difference between the machine output and the selected final text. It does not record time spent thinking, typing, or undoing changes. Sanchez-Torron and Koehn's 2016 study distinguishes this final-text measure from actual editing activity, so productivity requires separate time measurements.

Sources

1. 1A Study of Translation Edit Rate with Targeted Human Annotation · Association for Machine Translation in the Americas
2. 2TERCOM documentation · TERCOM authors
3. 3Broad Operational Language Translation phase 1 machine translation evaluation plan, May 24, 2012 · National Institute of Standards and Technology
4. 4Machine Translation Quality and Post-Editor Productivity · Association for Machine Translation in the Americas
5. 5sacreBLEU documentation · sacreBLEU maintainers
6. 6sacreBLEU TER metric implementation · sacreBLEU maintainers
7. 7sacreBLEU edit and shift algorithm · sacreBLEU maintainers
8. 8sacreBLEU TER tokenizer · sacreBLEU maintainers

Where this shows up in our work

• BABEL, offline multilingual translation and search translates and searches foreign-language documents on-device, every fact tied to its source passage.