Table extraction from PDFs, explained

Table extraction is the task of turning a table printed on a PDF page back into rows, columns, and cells that software can read. A PDF does not store a table as a table; it stores glyphs at coordinates and, sometimes, the lines drawn between them, so the grid has to be rebuilt from the page. The research community splits the work into table detection, finding where the table is, and table structure recognition, recovering its cells, a division set out in the PubTables-1M paper by Smock, Pesala, and Abraham. On a maintenance manual the stakes are specific: torque values, part numbers, clearances, and fluid capacities live in tables, and a value shifted one column left is a wrong number that reads as a right one. This page explains why the problem is hard, what the common open-source tools do according to their own documentation, how a published model approaches it, how results are scored, and what happens to a table once it enters a retrieval pipeline.

Why tables in PDFs are hard: born-digital versus scanned

The first question about any PDF table is whether the file contains text at all. A born-digital PDF, exported from a word processor or a publishing system, carries each character with its position and font, so a tool can read the words and their coordinates directly. A scanned PDF is a picture of a page; until optical character recognition (OCR) runs, there are no words to arrange. The page on why a PDF manual is hard to read by machine covers the general problem; tables are its sharpest case.

The three open-source libraries covered here are explicit about which kind they handle. The pdfplumber README says the library "Works best on machine-generated, rather than scanned, PDFs." The Camelot documentation states that Camelot only works with text-based PDFs and not scanned documents, and offers a test: if text can be selected by clicking and dragging in a viewer, the PDF is text-based. The same documentation, for version 2.0.0 (released June 4, 2026), also describes an optional ml parser that, with an OCR extra, reads scanned and image-only PDFs, so the text-only statement now holds for Camelot's built-in parsers. The Tabula site carries the same note, that Tabula only works on text-based PDFs, not scanned documents. A legacy manual that exists only as a scan therefore needs a different route: OCR to recover the words with their boxes, and an image-based model to recover the grid.

Even born-digital PDFs are not uniform. Some draw the table's ruling lines as vector graphics; some draw filled rectangles whose edges look like lines; some draw nothing and rely on alignment. Cells span rows or columns. Headers stack two or three deep. Text wraps inside a cell, so one logical row occupies several text lines. A table runs across a page break with its header repeated or not. Each of these is a separate way for a tool to produce a grid that is plausible and wrong.

Ruling lines versus whitespace

The second question is whether the table has ruling lines. Every tool in this field has two modes for the two answers, under different names.

Camelot calls them flavors. The Camelot how-it-works page describes Lattice as the method for tables with demarcated lines and Stream as the method for tables that use whitespace between cells to simulate a structure; the same page also documents Network and Hybrid parsers that work from text alignment and the optional ML parser built on the Table Transformer. The tabula-java README exposes the same pair as command-line flags: -l forces lattice-mode extraction for a PDF with ruling lines separating each cell, and -t forces stream-mode extraction when there are none; older -r spreadsheet and -n no-spreadsheet flags are deprecated in favor of them, and -g asks the engine to guess the portion of the page to analyze. pdfplumber exposes the choice as strategies: lines uses the page's graphical lines, including the sides of rectangles, as cell borders; lines_strict uses graphical lines but not rectangle sides; text deduces imaginary lines from the left, right, or center of words for columns and the tops of words for rows; and explicit uses only lines the caller supplies.

The distinction matters because the two cases fail differently. With ruling lines, the grid is on the page and the risk is in reading it: a broken or faint line, a line that is a rectangle edge, or two lines a few points apart that should be one. Without ruling lines, the grid is inferred from alignment, and the risk is in the inference: a wrapped cell looks like two rows, a right-aligned number column and a left-aligned text column disagree about where the boundary is, and a wide gap inside a heading looks like a column break. A manual typeset in the 1980s often has ruled tables; one produced from a modern authoring system often does not.

Two problems: table detection and table structure recognition

PubTables-1M, posted September 30, 2021, defines the vocabulary the field now uses. Table detection (TD) locates each table on the page. Table structure recognition (TSR) recognizes its rows, columns, and cells. Functional analysis (FA) recognizes which cells are headers, the keys, and which are values. The dataset holds 947,642 tables for structure recognition, drawn from scientific articles in the PubMed Central Open Access subset, split 80, 10, and 10 at the document level: 758,849 training, 94,959 validation, and 93,834 test tables. The detection split covers 460,589 training pages, 57,591 validation, and 57,125 test.

The paper's main contribution beyond scale is a diagnosis of ground-truth inconsistency. Oversegmentation, in the paper's words, "occurs in a structure annotation when a spanning cell in a header is split into multiple grid cells": the human or automatic annotator has drawn the header "Torque (Nm)" as two cells over two columns instead of one cell spanning both. A model trained on inconsistent annotations learns to be inconsistent. The authors' canonicalization procedure merges adjacent cells under stated conditions and changed the annotations of 34.7 percent of all tables and 65.8 percent of complex tables, where complex means at least one spanning cell. Before canonicalization, 40.1 percent of tables had been marked complex.

The paper's results also set a reference point. Training the Detection Transformer (DETR), an object detector with no table-specific customization, gave detection average precision (AP) of 0.966 against 0.825 for Faster R-CNN. On structure recognition plus functional analysis, DETR reached 0.912 AP against 0.722. The number that matters most for a reader is exact content match, the share of test tables in which every cell's text came out right: 0.8138 overall, 0.9468 on simple tables, and 0.6944 on complex ones. Faster R-CNN managed 0.1039. Nearly one in three complex tables still had at least one cell wrong under the best model, which is the figure to keep in mind when a vendor describes table extraction as solved. The page on document layout analysis covers the detection side in more depth.

pdfplumber, Camelot, and Tabula: what their docs say they do

pdfplumber is a Python library built on pdfminer.six, under the MIT license, whose README describes it as a way to plumb a PDF for detailed information about each text character, rectangle, and line. The page.extract_tables(table_settings={}) method returns the text of every table found on the page. The README gives the algorithm in five steps: find lines explicitly defined or implied by the alignment of words; merge overlapping or nearly overlapping lines; find the intersections of those lines; find the most granular set of rectangles, the cells, with those intersections as vertices; group contiguous cells into tables. The default settings use the lines strategy in both directions with snap_tolerance, join_tolerance, intersection_tolerance, and text_tolerance each set to 3 points, and each can be split into an x and a y value. Those tolerances are the dials: raise snap_tolerance and two nearly coincident lines become one; raise text_tolerance and slightly misaligned words fall into the same column.

Camelot is a Python library under the MIT license that, per its README, returns each table as a pandas DataFrame and lets the user discard bad tables based on metrics like accuracy and whitespace without looking at each one. The how-it-works page gives the two methods in detail. Lattice converts the page to an image with pdfium by default, uses OpenCV erosion and dilation to isolate horizontal and vertical line segments, combines them with a logical AND to find intersections and a logical OR to find table boundaries, scales everything back to PDF coordinates, detects spanning cells from the segments and intersections, and assigns words to cells by their coordinates. Stream groups characters into words and words into text rows by overlap on the y axis, estimates the number of columns as the mode of the number of words per row, derives column x-ranges from those rows, extends them with words that fall outside, and assigns words to cells. Every table carries a parsing_report with accuracy, whitespace, order, and page; the documentation's example shows accuracy 99.02 and whitespace 12.24. The README lists five parsers in version 2.0.0: lattice, stream, the text-alignment network and hybrid, and an optional neural ml parser that uses the Table Transformer. The web interface Excalibur sits on top of the library.

Tabula began as a journalist's tool. The Tabula site says it was created by Manuel Aristarán, Mike Tigas, and Jeremy B. Merrill with the support of ProPublica, La Nación DATA, Knight-Mozilla OpenNews, and The New York Times, and that it will always be free and open source. The engine is tabula-java, under the MIT license, which the README calls the table extraction engine that powers Tabula, with bindings for Python (tabula-py) and other languages.

pdfplumber and Tabula perform no OCR and use no learned model, and neither do Camelot's lattice, stream, network, and hybrid parsers; they are geometric procedures over the text and graphics a PDF already contains. Camelot's optional ml parser is the exception: its documentation says the model supplies the structure while cell text comes from the PDF's text layer or, for scans, from OCR. The geometric procedures are predictable and tunable, which a learned model's behavior is not.

The Table Transformer and PubTables-1M

The Table Transformer, abbreviated TATR, is the model code released with PubTables-1M under the MIT license. The README describes it as a deep learning model based on object detection for extracting tables from PDFs and images, and it is the official home of both the dataset and the GriTS metric. Because it works on the page image, it applies to scans as well as born-digital files, which is the main thing it offers that the geometric tools do not.

The released weights are a DETR detection model with a ResNet-18 backbone trained on PubTables-1M and several structure-recognition models: TATR-v1.0 and TATR-v1.1-Pub trained on PubTables-1M, TATR-v1.1-Fin trained on FinTabNet.c, a canonicalized version of the FinTabNet financial-report dataset, and TATR-v1.1-All trained on both. On the PubTables-1M test set the README reports the detection model at 0.995 AP50, 0.989 AP75, 0.970 AP, and 0.985 average recall, and the v1.0 structure model at 0.902 AP, 0.9849 GriTS Top, 0.9850 GriTS Con, 0.9786 GriTS Loc, and 0.8243 content accuracy. Two cautions from the README itself: "The inference code built on TATR needs text extraction (from OCR or directly from PDF) as a separate input," so the model draws the grid and something else must supply the words; and the published figures are on scientific-article tables, with the financial model the only sign of another domain. A maintenance manual is neither.

How table extraction is scored: TEDS and GriTS

A table score has to answer two questions at once: did the grid come out right, and did the text in each cell come out right. Two metrics are in general use.

TEDS

TEDS, Tree-Edit-Distance-based Similarity, comes from Image-based table recognition: data, model, and evaluation by Zhong, ShafieiBavani, and Jimeno Yepes, posted November 25, 2019, the paper that also introduced the PubTabNet dataset of 568,000 table images with HTML ground truth from PubMed Central. A table is written as an HTML tree, and TEDS between trees Ta and Tb is 1 minus their tree-edit distance divided by the larger node count. Inserting or deleting a node costs 1. Substituting one node for another costs 1 if either is not a cell or if the two cells differ in row or column span; otherwise the cost is the normalized Levenshtein distance between the cell texts, which lies between 0 and 1. The authors argued that the older adjacency-relation metric, which lists pairs of neighboring non-empty cells and checks them by exact match, misses two things: errors involving empty cells or misalignment beyond immediate neighbors, and partial errors inside a cell's text. In their perturbation test, shifting 90 percent of the cells in the first row left the adjacency F1 near 80 percent while TEDS fell by about 60 points. Their results table gives a view of the tools above on PubTabNet, as average TEDS in percent over simple and complex tables: the paper's EDD model 88.3, Camelot 73.0, Tabula 67.9, Adobe Acrobat Pro from PDF 65.3, and pdfplumber 40.4, with every tool scoring lower on complex tables than on simple ones.

GriTS

GriTS, Grid Table Similarity, comes from the GriTS paper by the PubTables-1M authors, posted March 23, 2022. Rather than converting a table to a tree, it keeps the table in its natural form as a matrix of cells, finds the two most similar two-dimensional substructures of the predicted and reference matrices, and scores their entry-wise similarity with an F-score-style formula. Three variants use three cell properties: GriTS Top compares each cell's row and column span, GriTS Con compares cell text with normalized longest common subsequence, and GriTS Loc compares cell bounding boxes by intersection over union. The paper's complaints about earlier metrics are concrete. A single cell-segmentation mistake can offset every later cell by one position, which absolute-position metrics punish out of proportion. The adjacency metric mostly ignores the insertion of contiguous blank cells, while TEDS does not, and in the authors' tests TEDS reacted differently to removing rows than to removing columns, against the aim of treating the two alike. The exact 2-D problem is expensive, so the authors use a heuristic that yields an upper and a lower bound and report the lower one; on PubTables-1M the two bounds differed by less than 0.0001 after a few epochs of training and were equal on at least 93 percent of instances.

For an evaluator the lesson is that a single number hides the failure mode. A table with the right grid and one misread digit scores well on topology and location and loses a little on content; a table with one missed column boundary scores well on content and badly on topology. Ask for all three, and ask on which tables.

What happens to a table in a chunking pipeline

Once a document is prepared for retrieval, the text is cut into chunks for indexing, and a table that survived extraction can still be destroyed here. The page on chunking documents for retrieval covers the public tests; the table-specific points are these.

A chunker that splits on character count or sentence boundaries sees a table as a run of short lines. The header row lands in one chunk and the values in the next. A row's label is separated from its numbers. A cell's meaning, which is fixed by the intersection of its row label and column heading, is gone, because the two no longer sit in the same chunk. An embedding of a chunk that reads "12 14 16 18 22" carries almost nothing a query can match. The remedy is structural: treat each table as its own chunk, repeat the header row on every slice if the table is too long for one, and carry the section heading and caption with it, since a table titled only "Table 3" means nothing once it is separated from the paragraph that names the assembly.

This is why conversion toolkits built for retrieval now carry a table model inside them. The Docling technical report from IBM Research, posted August 19, 2024, describes an MIT-licensed package for PDF conversion that runs layout analysis with a model trained on DocLayNet and table structure recognition with a model called TableFormer, on commodity hardware. Whether a given pipeline does this is a question to ask rather than assume; many do not, and the symptom is a search that cannot find a value that is plainly in the manual.

One further point for scanned manuals. The words inside a table come from OCR, so every caution on the OCR on scanned manuals page applies with more force, because digits dominate tables and digits have no dictionary to correct them. Tesseract, an open-source engine, states in its own documentation that it has trouble with tables, which is why the grid should be found first and the cells read individually.

What to check

• Are the manuals born-digital PDFs with selectable text, or scans? If scans, which OCR engine supplies the words, and which image-based model supplies the grid?
• Do the tables have ruling lines, and was the tool run in the matching mode, lattice or stream, lines or text, with the tolerances tuned on these pages rather than the defaults?
• How are spanning header cells, wrapped cells, and tables that cross a page break handled, and were those cases in the test sample?
• Was extraction scored on a hand-checked sample of this manual's tables, and were topology, content, and location reported separately, or only one combined figure?
• In the retrieval index, is each table its own chunk with its header row and section heading attached, and can a value found by search be traced back to its row, column, and page?
• Can a reader open the page image for any extracted cell in one step?

Azcuna's TRACE is one offline viewer for S1000D, MIL-STD-3001 XML, and PDF technical manuals. It keeps the source location and the markings of everything it shows and never modifies the source. It was proposed to NAVAIR under Navy SBIR Phase I topic DON26BZ05-NV078, and the proposal was submitted on September 23, 2026.

Questions people ask

How do I extract tables from a PDF in Python?

Three open-source libraries do it for PDFs that contain text. pdfplumber, built on pdfminer.six, has a page.extract_tables method that finds cells from drawn lines or from the alignment of words. Camelot has camelot.read_pdf with a lattice flavor for ruled tables and a stream flavor for whitespace-separated ones, and returns each table as a pandas DataFrame. tabula-py wraps tabula-java, the engine behind the Tabula desktop tool. All three need text-based PDFs, except Camelot 2.0.0's optional ml parser, which reads scans with OCR.

What is the difference between Camelot and pdfplumber?

Both find tables in text-based PDFs, but by different routes. Camelot's lattice mode rasterizes the page and uses OpenCV morphological operations to find ruling lines and their intersections, then assigns words to cells; its stream mode groups words into rows by vertical overlap and estimates columns from the words per row. pdfplumber works from the PDF's own line and rectangle objects, or from imaginary lines through the edges of words, merges them, finds intersections, and builds cells. Both are MIT-licensed.

What is Camelot lattice vs stream?

Lattice and stream are Camelot's original parsing methods, called flavors. The Camelot documentation says lattice is for tables with demarcated lines and is more deterministic because it does not rely on guesses: it converts the page to an image, isolates horizontal and vertical line segments, finds their intersections, and detects spanning cells. Stream is for tables that use whitespace between cells to simulate a structure: it groups words into text rows and estimates the number of columns as the mode of words per row. Lattice is the default.

Can Camelot or Tabula extract tables from scanned PDFs?

Tabula cannot: its site says Tabula only works on text-based PDFs, not scanned documents. Camelot's built-in parsers also need a text-based PDF, and its documentation gives the test of whether text can be selected by clicking and dragging in a viewer. Camelot 2.0.0, released June 4, 2026, added an optional ml parser built on the Table Transformer that, with its OCR extra installed, reads scanned and image-only PDFs. pdfplumber's README says it works best on machine-generated rather than scanned PDFs.

What is table structure recognition?

Table structure recognition is the step after a table has been located: recognizing its rows, columns, and cells, including cells that span several rows or columns. The PubTables-1M paper separates it from table detection, which finds the table's position on the page, and from functional analysis, which identifies which cells are headers and which are values. The GriTS paper notes that a single mistake in cell segmentation can offset all subsequent cells by one position, so one missed column boundary can misplace every value to its right.

What is the TEDS metric?

Tree-Edit-Distance-based Similarity, introduced in the PubTabNet paper by Zhong, ShafieiBavani, and Jimeno Yepes (2019). A table is written as an HTML tree, and TEDS equals 1 minus the tree-edit distance between the predicted and reference trees divided by the larger tree's node count. Insertions and deletions cost 1; substituting one cell for another costs 1 if the spans differ and otherwise the normalized Levenshtein distance between the cell texts. The paper reports it as a percentage, with its EDD model at 88.3 on PubTabNet.

What is GriTS?

Grid Table Similarity, proposed by Smock, Pesala, and Abraham (2022) and used by the Table Transformer. It treats a table as a matrix of cells, finds the most similar two-dimensional substructures of the predicted and reference matrices, and scores them with an F-score-style formula. Three variants compare different cell properties: GriTS Top compares row and column spans, GriTS Con compares cell text, and GriTS Loc compares cell bounding boxes. The paper found the heuristic's upper and lower bounds nearly identical on PubTables-1M.

What is the Table Transformer?

The Table Transformer, or TATR, is Microsoft's object-detection model for table extraction, released under the MIT license with the PubTables-1M dataset. The repository provides a DETR detection model trained on PubTables-1M and structure-recognition models trained on PubTables-1M, on FinTabNet.c, and on both. On the PubTables-1M test set the detection model scores 0.970 AP and the v1.0 structure model scores 0.9849 GriTS Top and 0.9850 GriTS Con. The repository states that text extraction, from OCR or the PDF, must be supplied as a separate input.

Why are tables a problem for RAG and chunking?

A chunker that splits text by character count treats a table as prose. The rows become lines, the column headings end up in one chunk and the values in another, and a cell's meaning, which depends on its row label and column heading, is lost. The chunking page on this site covers the public tests; the practical rule is that a table should be its own chunk with its header row repeated on every slice, and the heading that names the component should travel with it.

Sources

1. 1jsvine/pdfplumber README · GitHub (jsvine/pdfplumber)
2. 2camelot-dev/camelot README · GitHub (camelot-dev/camelot)
3. 3Camelot v2.0.0 release · GitHub (camelot-dev/camelot)
4. 4Camelot: PDF Table Extraction for Humans · Camelot documentation (Read the Docs)
5. 5How It Works · Camelot documentation (Read the Docs)
6. 6tabulapdf/tabula-java README · GitHub (tabulapdf/tabula-java)
7. 7Tabula: Extract Tables from PDFs · Tabula project
8. 8microsoft/table-transformer README · Microsoft (GitHub)
9. 9PubTables-1M: Towards comprehensive table extraction from unstructured documents · arXiv (Smock, Pesala, and Abraham)
10. 10Image-based table recognition: data, model, and evaluation · arXiv (Zhong, ShafieiBavani, and Jimeno Yepes)
11. 11GriTS: Grid table similarity metric for table structure recognition · arXiv (Smock, Pesala, and Abraham)
12. 12Docling Technical Report · arXiv (Auer et al., IBM Research)

Where this shows up in our work

• TRACE, the offline S1000D and IETM viewer opens S1000D, MIL-STD-3001 and PDF technical manuals on a tablet with every marking intact.
• BABEL, offline multilingual translation and search translates and searches foreign-language documents on-device, every fact tied to its source passage.