How offline machine translation works

Most translation software you have used is a thin client. The text goes to a data center, a very large model translates it, and the answer comes back. That design is fine for a restaurant menu. It is disqualifying for a captured phone, a seized ledger, an intercepted message, or any document that must not leave a secure perimeter.

Offline machine translation inverts the design. The model lives on the analyst's machine. Nothing leaves. This page explains how that works in practice, what it costs in quality, and what BABEL, the Azcuna Dynamics platform, does at each step. It is written for the people who have to evaluate offline translation software, not sell it.

What "offline" has to mean

"Offline" is a claim about the whole pipeline, not one component. A product that translates locally but phones home for language detection, dictionary lookups, telemetry, license checks, or model updates is not offline. It has an outbound dependency that will fail in a SCIF and leak in the field.

The standard BABEL holds itself to is zero outbound bytes. Language detection, tokenization, translation, entity extraction, search indexing, and the user interface all run from the local disk. Updates arrive as signed packages carried in by hand. The software does not know whether a network exists and does not care.

That standard has consequences for the engineering. Every model must be small enough to ship on removable media and fast enough to run on hardware an analyst already has. Every language resource must be bundled. Every step must degrade gracefully when the input is ugly, because there is no fallback service to call.

The model: what actually translates the text

Modern translation runs on transformer neural networks. Two families matter for offline use.

Dedicated translation models are trained only to translate. They are compact, fast, and predictable. Their weakness is rigidity: they translate sentence by sentence and have little sense of document context or domain vocabulary.

General language models used as translators are larger and slower, but they carry world knowledge, handle mixed-language documents, follow instructions such as "preserve the numbers and unit designations," and can be steered with terminology. The current generation of open-weight models in the 2 to 9 billion parameter range translates high-resource languages well enough to be the default choice for document work.

BABEL runs open-weight models from the general family through a local inference engine and picks the model per deployment from a small approved set. The model file is a single quantized artifact on disk. Swapping it is a file copy, which matters when a program's origin policy changes or a better model is released.

Quantization: fitting a model on a laptop

A 9 billion parameter model at full precision needs about 36 GB of memory. That is a server. Quantization stores each weight in 4 to 6 bits instead of 16 or 32, cutting the model to 5 or 6 GB with a loss in translation quality that is measurable but small. In our testing the difference between a 6-bit and a full-precision model on Arabic-to-English document translation is a fraction of a point on standard metrics, well inside the day-to-day variance of the metric itself.

The practical result: the whole model fits in the memory of a 16 GB laptop with room left for the operating system and the document. On Apple Silicon the model runs on the GPU through Metal. On a workstation it runs on an NVIDIA card. On a bare CPU it still runs, slower, which is the fallback that keeps the capability alive on whatever hardware is in the room.

Tokenization, and why Arabic and Chinese are harder offline

Models do not read characters. They read tokens, fragments of text chosen when the model was trained. English is cheap: a word is usually one token. Arabic, Farsi, Urdu, Chinese, Japanese, and Korean are expensive: a single character can cost three or four tokens.

This matters offline because a local model has a fixed context window, the amount of text it can hold at once. A window that comfortably fits a two-page English document may overflow on half a page of Arabic. When the window overflows, the model silently loses the beginning of the passage and the translation degrades or stops.

BABEL handles this by chunking documents by script. Latin-script text is chunked in larger pieces; Arabic and CJK text is chunked at about 1,200 characters, with sentence boundaries respected so a chunk never ends mid-thought. A quick mode uses a smaller window for speed on short texts; a full mode uses a 16,000-token window for long documents where context matters more than seconds.

Chunking, context, and consistency

Chunking solves the memory problem and creates a consistency problem. A name transliterated one way in chunk three may come out differently in chunk nine. A unit designation abbreviated in the source may be expanded in one place and left alone in another.

Three mechanisms keep a long translation coherent.

Overlap. Each chunk carries the last sentence or two of the previous chunk as context, so the model sees what came before.

Terminology steering. An analyst can supply a glossary: this name is spelled this way, this acronym stays untranslated, this unit is rendered this way. The glossary is injected into the model's instructions for every chunk.

Translation memory. Approved sentence pairs are stored in a local database. When the same sentence appears again, in the same document or a later one, the stored translation is reused. This makes the second pass over a document set faster and makes repeated boilerplate identical every time. The memory is a file on the analyst's disk. It is not shared unless someone copies it.

Quality controls that do not need the cloud

Cloud translation services quietly run several models and pick the best output. An offline system can do the same on its own hardware, at the cost of time.

Minimum Bayes risk decoding generates several candidate translations and selects the one most similar to the others, which reliably filters out the occasional bizarre output a single sample can produce.

Self-refinement asks the model to review its own translation against the source and correct errors, a second pass that catches dropped negations and mistranslated numbers.

Caching stores every translated passage locally so that re-opening a document or re-running a search does not re-translate anything. The first translation of a long document is the slow one. Every subsequent view is instant.

All three are switches. An analyst under time pressure turns them off and takes the fast single-pass translation. An analyst preparing a document for a report turns them on.

What quality to expect, honestly

Offline translation on a laptop is not as good as the best cloud service on every language. Any vendor who tells you otherwise is selling. What the evidence supports is narrower and more useful.

For the languages that dominate defense and intelligence work, Arabic, Farsi, Russian, Urdu, and Chinese, a tuned local model produces translations an analyst can triage from, search across, and quote from with source-checking. The gap to cloud services on standard benchmarks is a few points, not a category difference, and it closes further with domain adaptation on the deployment's own approved material.

For very low-resource languages the gap is larger, and a local system should say so rather than guess. BABEL reports a confidence estimate per passage and flags low-confidence output so the analyst knows where to look twice.

The point of offline translation is not to beat the cloud on a benchmark. It is to make foreign-language material usable at all in places where the cloud is not permitted, and to do it without a single byte leaving the room.

Where translation fits in the pipeline

Translation is the first step, not the product. Once a document set is in English, BABEL indexes it for semantic search, extracts people, organizations, places, dates, and values, and links them into a graph an analyst can pivot through. Each of those steps runs on the translated text and each carries the original passage alongside so a claim can always be checked against the source.

Search across a translated corpus is where the value compounds. A question asked in English is answered from documents that were written in four languages, with the source passage and its translation shown together. That is the workflow the rest of the capabilities pages describe, and it is only possible because translation happened first, locally, on every document.

Evaluating offline translation software: a checklist

If you are comparing products, ask each vendor these questions and require demonstrations, not slides.

• Run it with the network cable unplugged and the Wi-Fi radio off. Does every feature still work?
• Feed it a 40-page Arabic PDF. Does the translation stay consistent from page 1 to page 40?
• Supply a ten-term glossary. Are the terms honored in every chunk?
• Translate the same document twice. Is the second pass instant?
• Ask what model is inside, who trained it, and where the weights came from. Read the model provenance page on this site for why that matters.
• Ask what is logged, where, and who can read it.

BABEL passes each of these in a live session on a laptop. That is the demonstration we offer, and the recorded ELLMENT session on this site shows the same engine answering questions from a document set with page-level citations, fully disconnected.