Tesseract, explained

Tesseract is an open-source optical character recognition (OCR) engine: a program that takes an image of a page and returns the text printed on it. The project README describes an engine developed at HP between 1985 and 1994, open-sourced in 2005, developed by Google from 2006 to August 2017, and now maintained under the Apache License 2.0, with Stefan Weil as lead developer. The Hugging Face processors for LayoutLMv3 and LayoutLMv2 call it by default as their OCR step. This page covers where it came from, how the two recognition engines work, which model files to use, how to tell it what kind of page it is looking at, where it fails, and how to measure the result.

Where Tesseract came from

Ray Smith, who led the engine's development at HP and later at Google, gave the history in An Overview of the Tesseract OCR Engine, published in the proceedings of the Ninth International Conference on Document Analysis and Recognition (ICDAR) in 2007 and listed on Google Research. Tesseract began as a PhD research project in HP Labs, Bristol, and grew as a possible add-on for HP's flatbed scanners at a time when, in Smith's account, commercial engines failed on anything but the best-quality print. A joint project with HP's scanner division in Colorado gave it a lead in accuracy over commercial engines, but it never became a product. Development ceased at the end of 1994. The engine was then sent to the University of Nevada, Las Vegas (UNLV) for the 1995 Annual Test of OCR Accuracy, where it appeared as "HP Labs OCR" and did well against the commercial engines of the day. In late 2005 HP released it as open source.

The release notes carry the story forward. Version 3.00, dated September 30, 2010, added a page layout analysis module and hOCR output. Version 3.02.02, dated October 23, 2012, added simultaneous multi-language recognition, right-to-left and bidirectional output for Hebrew and Arabic, and paragraph detection. Version 4.0.0, dated October 29, 2018, added the LSTM neural-network engine. Version 5.0.0, dated November 30, 2021, made 32-bit float the default in place of 64-bit double, cutting memory use and increasing speed, and broke API compatibility with 4.x. The most recent release listed on the page is 5.5.3, dated July 24, 2026.

How the original engine works

Smith's 2007 paper is the only full description of the classic engine, now called the legacy engine. Two design decisions shape how it behaves on a scanned manual.

First, Tesseract did not do its own layout analysis. HP had a separate, proprietary page layout technology, so, in Smith's words, Tesseract "never needed its own page layout analysis" and assumed its input was a binary image with text regions optionally marked. Layout analysis was added in 3.00, but the recognizer was designed to be handed lines of text.

Second, the pipeline is a traditional step-by-step one with some unusual stages. A connected component analysis stores the outlines of every ink blob, which made inverse (white-on-black) text as easy to detect as normal text. Blobs are grouped into text lines, and lines are checked for fixed-pitch or proportional spacing. The line finder was designed so that a skewed page could be recognized without de-skewing, and baselines are fitted with a quadratic spline so that curved baselines, which the paper calls a common artifact in scanning and not just at book bindings, still work. Recognition runs in two passes: words recognized confidently on the first pass train an adaptive classifier, and a second pass re-reads the words that scored poorly.

The paper's results table compares Tesseract 2.0 with the 1995 UNLV results on the same four 300 DPI test sets. The total was 59,119 character errors, 7.31 percent fewer than in 1995, and 12,503 non-stopword errors, 5.39 percent fewer; the character error rate fell on the DOE and magazine sets and rose on the business-letter and newspaper sets. Smith's conclusion was frank: after lying dormant for more than 10 years, Tesseract was behind the leading commercial engines in accuracy.

The LSTM engine in Tesseract 4 and 5

The engine that closed much of that gap arrived in version 4.0.0. The release notes describe a new LSTM (long short-term memory) neural-network OCR engine with major accuracy gains and LSTM models for 123 languages, and state that the pattern-matching engine of earlier versions remained available. The Cube engine, an auxiliary recognizer that 3.x used for scripts such as Arabic and Hindi, was removed.

The project's page on neural nets in Tesseract 4.00 explains what changed. The neural subsystem is configured as a textline recognizer: it takes an image of a whole line and emits the characters, rather than segmenting and classifying one character at a time. It originated in the OCRopus Python LSTM implementation and was redesigned for Tesseract in C++. The LSTM engine can run on a single line with page segmentation mode 13 or on a full page through the existing layout analysis, and it added vertical text for Chinese, Japanese, and Korean. The cost is compute: the page describes the subsystem as heavily compute-intensive, using roughly ten times the CPU of the base engine, offset by OpenMP parallelism across four cores and SSE/AVX acceleration of the matrix multiplications.

The project's accuracy and performance page gives one comparison, from a large test in a Google data center that the page labels, with a question mark, as Hindi. The character-error column reads 13.9 for Tesseract 3.04, 11.0 for 3.04 with Cube, and 7.6 for the LSTM engine; word-recall errors were 30 for 3.04 and 20.9 for LSTM. The page does not state its units, and notes that the LSTM engine was faster than 3.04 in both wall time and CPU time on that test.

Version 5 kept both engines. The README describes 5 as the current stable version, starting with 5.0.0 on November 30, 2021, and the release notes record that 4.0.0 added an option to compile Tesseract without the legacy engine's code and that 5.3.1 (April 1, 2023) re-enabled code blocks wrongly disabled in such builds, so a packaged binary may ship without the legacy engine.

Tesseract languages, scripts, and the traineddata sets

A Tesseract installation recognizes nothing until a .traineddata model file for the language is present. The Tesseract User Manual says official model data is available for 100+ languages and 35+ scripts, for the LSTM engine. The tessdoc page on traineddata files for version 4.00 and later lists 102 language codes in its 4.0 table, from afr (Afrikaans) and amh (Amharic) through ara (Arabic), fas (Persian), heb (Hebrew), rus (Russian), urd (Urdu), chi_sim and chi_tra (Chinese), jpn, kor, and on to yid (Yiddish). Some codes are script variants of one language, such as aze_cyrl, srp_latn, and uzb_cyrl. The companion page on data files in different versions lists 125 languages across all versions, plus the osd and equ modules, and 37 script files: Arabic, Armenian, Bengali, Canadian Aboriginal, Cherokee, Cyrillic, Devanagari, Ethiopic, Fraktur, Georgian, Greek, Gujarati, Gurmukhi, simplified and traditional Han (each with a vertical variant), Hangul and Hangul vertical, Hebrew, Japanese and Japanese vertical, Kannada, Khmer, Lao, Latin, Malayalam, Myanmar, Oriya, Sinhala, Syriac, Tamil, Telugu, Thaana, Thai, Tibetan, and Vietnamese. A script file is one model for every language written in that script; the page points to the langs.txt files for what each covers.

The same models come in three sets, held in three repositories, and the choice matters more than most users realize. The Data Files page describes them:

The tessdata_fast README explains the trade: the models are a speed-accuracy compromise, picked for the best value in speed against accuracy and then integerized to 8 bits for further speed, and "fine tuning/incremental training will NOT be possible from these fast models". The same README says most users will want these files and that they are what Linux distributions ship, with Debian and Ubuntu package names of the form tesseract-ocr-LANG for languages and tesseract-ocr-script-SCRIPT for scripts. The tessdata README adds that the legacy models for Indic and Arabic-script languages were removed from that set, so those scripts run on the LSTM engine only. Most of the script models in tessdata_fast include English training data as well as the script, but not Cyrillic. Two consequences follow. A pipeline that installed Tesseract's language data from Debian or Ubuntu packages is running tessdata_fast, the least accurate set, since the tessdata README says those are the files packaged there. And anyone planning to fine-tune on a manual's own fonts needs tessdata_best, which is a separate download.

Tesseract page segmentation modes, output formats, and the license

Before recognizing anything, Tesseract decides how the page is organized. The --psm option sets that, and the tesseract manual page lists the 14 modes:

The default, mode 3, runs the automatic layout analysis but does not detect orientation or script, so a page scanned sideways is not corrected unless mode 1 is used or the image is rotated first. Mode 6 is the usual choice when a region has already been cut out by a separate layout step, and mode 11 for a drawing with scattered callouts. The command line usage page adds that -l names the language, defaulting to English, and that several languages can be combined with a plus sign, as in -l eng+deu.

A second option, --oem, picks the engine: 0 for the original Tesseract only, 1 for the LSTM engine only, 2 for both, and 3 for the default based on what is available. The Data Files page notes that tessdata_best and tessdata_fast only work with --oem 1, so modes 0 and 2 fail unless the tessdata set is installed. The manual page also documents --dpi, with 300 given as a typical value, for images that lack resolution metadata.

Output formats and the license

The README lists the outputs: plain text, hOCR (an HTML microformat), PDF with the image and an invisible text layer, an invisible-text-only PDF, TSV, ALTO, and PAGE XML. The manual page names the configuration files that select them, such as hocr, tsv, alto, page, and pdf. For a pipeline, hOCR and TSV matter most, because they carry each word's bounding box and confidence rather than text alone; the OCR on scanned manuals page covers why those two fields are worth keeping.

The license is simple. The repository's LICENSE file is the Apache License, Version 2.0, January 2004. All three model repositories state that all data in them is licensed under Apache-2.0. The README notes that Leptonica, the library Tesseract uses for opening input images, essentially uses a BSD 2-clause license.

Right-to-left scripts

Arabic, Persian, Urdu, and Hebrew run right to left, and Tesseract has handled them since the 3.x series, with known rough edges. The training guide for versions 3.00 to 3.02 records that 3.01 added top-to-bottom languages and 3.02 added Hebrew, a right-to-left language, and that building a dictionary for an RTL language requires the -r 1 flag to wordlist2dawg. The release notes for 3.02.02 describe right-to-left and bidirectional capability in the output iterators for Hebrew and Arabic. In the LSTM training tools, the combine_lang_model manual page documents a --lang_is_rtl flag, true if the language is written right to left, with Arabic and Hebrew as the examples.

The issue tracker shows where this still breaks. Issue 4428, opened June 16, 2025 against version 5.5.1, reported that Arabic text in the searchable PDF output looked right but was stored in reverse logical order, so searching the PDF for a word found nothing and copying it produced the letters backwards; a maintainer closed it the next day as a duplicate of a much older issue, number 238. Pull request 4629, opened September 26, 2026 and still open at the time of writing, addresses reading order: Tesseract decides whether a model is right-to-left by counting the left-to-right and right-to-left characters in its character set, and a model for Persian, Arabic, Urdu, or Hebrew that also includes Latin letters can be classed as left-to-right, so a two-column page is read in the wrong column order. The proposed fix is a tessedit_right_to_left parameter that overrides the count. The Urdu Nastaliq page covers the separate problem of a script whose letter shapes defeat Arabic-trained models.

Known limits on scanned manuals and tables

The project's own page on improving the quality of the output is the honest list. Tesseract works best on images of at least 300 DPI. It converts the page to black and white internally with Otsu's method, which the page says can be suboptimal when the background is unevenly dark; 5.0.0 added Adaptive Otsu and Sauvola thresholding as alternatives. Line segmentation degrades significantly if a page is too skewed, so pages should be rotated until the text lines are horizontal. Dark borders left by a scanner can be read as characters. If the text is not mostly dictionary words, the page suggests disabling the built-in dictionaries by setting load_system_dawg and load_freq_dawg to false, and restricting the character set with tessedit_char_whitelist; a parts list full of alphanumeric codes is the case this describes.

On tables the page is blunt: "It is known tesseract has a problem to recognize text/data from tables." The page ties the problem to the absence of custom segmentation or layout analysis and points to two outside workarounds. The cause is the design history above. Tesseract reads text lines, and a table is a grid in which the reading order is down a column as often as across a row; without cell boundaries, values and headings are interleaved. The FAQ adds that handwriting recognition is possible but will not work very well, since the engine is designed for printed text. A manual with torque tables, stamped revision marks, and a technician's pencil notes hits all three. Document layout analysis is the step that finds the table before OCR runs; Tesseract does not supply it.

How to measure Tesseract accuracy

The standard tools are the ones UNLV wrote for the annual tests Tesseract once competed in. The ISRI Analytic Tools user guide, Technical Report TR-96-02 by Stephen V. Rice and Thomas A. Nartker, dated August 1996, states that errors are the edit operations, character insertions, substitutions, and deletions, needed to correct the OCR-generated text. The accuracy program compares a correct text file with the OCR output and writes a report with the ground-truth character count, the error count, the accuracy, the errors by edit operation, accuracy by character class, and a ranked list of confusions. In the guide's example, 756 characters and 39 errors gave 94.84 percent character accuracy. The wordacc program scores words, using by default a list of 110 stopwords so that non-stopword accuracy can be reported separately; in the guide's example, run with a 200-word stopword file, word accuracy was 84.87 percent and non-stopword accuracy 80.52 percent.

The tools live on as ocreval, a modern port of 17 ISRI tools with UTF-8 support, under the Apache License 2.0. The method is: take a sample of pages from the manuals in question, transcribe them by hand, run Tesseract with the model set and segmentation mode the pipeline uses, and score the output. Report the character-class breakdown, because digit errors are the ones that change a part number, and report the result for each model set tried, since tessdata_fast and tessdata_best on the same page give different numbers.

What to check

• Which model set is installed, tessdata, tessdata_best, or tessdata_fast, and was it chosen or did it arrive with the package manager?
• Which engine mode and page segmentation mode does the pipeline use, and was the segmentation mode chosen for the page type, such as mode 6 for cut-out regions or mode 11 for drawings?
• Were the scans at 300 DPI or higher, de-skewed, and cropped before OCR, and does --dpi get set when the image lacks resolution metadata?
• For a bilingual or right-to-left manual, has the searchable PDF layer been tested by searching and copying a word, and has the reading order been checked on a two-column page?
• Were tables located by a separate layout analysis step before Tesseract ran, or were they fed to the engine as plain pages?
• Was accuracy measured with the ISRI tools on hand-transcribed pages from these manuals, with digits and identifiers reported separately from running text?

Azcuna's TRACE is one offline viewer for S1000D, MIL-STD-3001 XML, and PDF technical manuals. It keeps the source location and the markings of everything it shows and never modifies the source. It was proposed to NAVAIR under Navy SBIR Phase I topic DON26BZ05-NV078, and the proposal was submitted on September 23, 2026.

Questions people ask

What is Tesseract OCR?

Tesseract is an open-source optical character recognition engine. The project README says it was developed at HP Labs in Bristol and at Hewlett-Packard in Greeley, Colorado, between 1985 and 1994, open-sourced by HP in 2005, and developed by Google from 2006 until August 2017. It reads images such as PNG, JPEG, and TIFF and writes plain text, hOCR, PDF, TSV, ALTO, or PAGE output. The code is licensed under the Apache License, Version 2.0.

Is Tesseract OCR free?

Yes. The LICENSE file in the Tesseract repository is the Apache License, Version 2.0, January 2004, and the tessdata, tessdata_best, and tessdata_fast model repositories each state that all data in the repository is licensed under Apache-2.0. The Tesseract User Manual says the engine can be used in your own projects under the terms of that license. Leptonica, an image library Tesseract depends on, uses a BSD 2-clause license.

What is the Tesseract LSTM engine?

A neural network recognizer added in Tesseract 4.0.0, released October 29, 2018. The release notes describe it as a new LSTM neural-network OCR engine with major accuracy gains and models for 123 languages. The project's Tesseract 4 documentation says the subsystem is configured as a textline recognizer, originated in the OCRopus Python LSTM implementation, and was redesigned for Tesseract in C++. The older pattern-matching engine stayed available as the legacy engine, selected with --oem 0.

How many languages does Tesseract support?

The Tesseract User Manual says official model data is available for 100+ languages and 35+ scripts, and the README says the engine recognizes more than 100 languages out of the box. The tessdoc language-file table for version 4.0 lists 102 language codes, from afr (Afrikaans) to yid (Yiddish), including ara, fas, heb, rus, urd, chi_sim, chi_tra, jpn, and kor. A separate table lists 37 script files, such as Arabic, Cyrillic, Devanagari, Hebrew, and Latin.

What is the difference between tessdata, tessdata_best, and tessdata_fast?

The tessdoc Data Files page describes three sets for Tesseract 4.0 and later. tessdata holds legacy models plus integerized LSTM models and is the only set that supports the legacy recognizer. tessdata_best holds float LSTM models, is the slowest and most accurate, and is the only set that can be used for retraining. tessdata_fast holds integer models from a smaller network, is the fastest and least accurate, and is what the page says most users will want.

What are Tesseract page segmentation modes?

The --psm option tells Tesseract how to divide the page before recognition. The Tesseract manual page lists 14 modes, numbered 0 to 13. Mode 3, fully automatic page segmentation without orientation and script detection, is the default. Mode 1 adds orientation and script detection, mode 4 assumes a single column of text of variable sizes, mode 6 a single uniform block of text, mode 7 a single text line, mode 8 a single word, mode 11 sparse text in no particular order, and mode 13 a raw line.

How accurate is Tesseract?

There is no single figure, because accuracy depends on the scan, the language model, and the page. Ray Smith's 2007 ICDAR paper compared Tesseract 2.0 with the 1995 UNLV test results and reported 59,119 character errors across the four 300 DPI test sets, 7.31 percent fewer than in 1995. The project's own Tesseract 4 page compares the LSTM engine with 3.04 on a test it labels, with a question mark, as Hindi. For a specific collection, the ISRI Analytic Tools score the engine's output against hand-transcribed ground truth.

Does Tesseract handle Arabic and Hebrew?

Yes, with limits. The tessdoc release notes say version 3.02.02, released October 23, 2012, added right-to-left and bidirectional capability in the output iterators for Hebrew and Arabic, and the 4.0 language table includes ara, heb, fas, and urd. The training tools carry a --lang_is_rtl flag. The project's issue tracker records open problems, including a 2025 report that Arabic text in a searchable PDF was stored in reverse logical order and a 2026 pull request on reading order for right-to-left models that include Latin letters.

Why does Tesseract struggle with tables?

The Tesseract documentation on improving output quality states that the engine is known to have a problem recognizing text and data from tables, and suggests custom segmentation or layout analysis as a workaround. Ray Smith's 2007 paper explains the origin: HP's page layout analysis stayed proprietary, so Tesseract never needed its own and assumed its input was a binary image with optional text regions already defined. A layout analysis module was added in version 3.00, but table cells are still read as lines of text.

Sources

1. 1LayoutLMv3 (Transformers documentation) · Hugging Face
2. 2tesseract-ocr/tesseract README · Tesseract OCR project (GitHub)
3. 3An Overview of the Tesseract OCR Engine (ICDAR 2007) · Ray Smith, Google Inc., Proc. Ninth Int. Conference on Document Analysis and Recognition
4. 4An Overview of the Tesseract OCR Engine (publication record) · Google Research
5. 5Release Notes · Tesseract OCR project (tessdoc)
6. 6Neural nets in Tesseract 4.00 · Tesseract OCR project (tessdoc)
7. 74.0 Accuracy and Performance · Tesseract OCR project (tessdoc)
8. 8Traineddata Files for Version 4.00 + · Tesseract OCR project (tessdoc)
9. 9Data Files in different versions · Tesseract OCR project (tessdoc)
10. 10Tesseract User Manual · Tesseract OCR project (tessdoc)
11. 11tesseract-ocr/tessdata README · Tesseract OCR project (GitHub)
12. 12tesseract-ocr/tessdata_best README · Tesseract OCR project (GitHub)
13. 13tesseract-ocr/tessdata_fast README · Tesseract OCR project (GitHub)
14. 14tesseract(1) manual page · Tesseract OCR project (GitHub)
15. 15Command Line Usage · Tesseract OCR project (tessdoc)
16. 16Improving the quality of the output · Tesseract OCR project (tessdoc)
17. 17FAQ · Tesseract OCR project (tessdoc)
18. 18LICENSE · Tesseract OCR project (GitHub)
19. 19Training Tesseract 3.00-3.02 · Tesseract OCR project (tessdoc)
20. 20combine_lang_model(1) manual page · Tesseract OCR project (GitHub)
21. 21Arabic text in searchable PDF is stored in reverse logical order (issue 4428) · Tesseract OCR project issue tracker (GitHub)
22. 22Add tessedit_right_to_left to override the unicharset direction heuristic (pull request 4629) · Tesseract OCR project (GitHub)
23. 23The ISRI Analytic Tools for OCR Evaluation, Version 5.1 (TR-96-02) · Information Science Research Institute, University of Nevada, Las Vegas (via the ocreval repository)
24. 24ocreval: ISRI Analytic Tools for OCR Evaluation · GitHub (eddieantonio/ocreval)

Where this shows up in our work

• TRACE, the offline S1000D and IETM viewer opens S1000D, MIL-STD-3001 and PDF technical manuals on a tablet with every marking intact.
• BABEL, offline multilingual translation and search translates and searches foreign-language documents on-device, every fact tied to its source passage.