What is retrieval-augmented generation?

Retrieval-augmented generation (RAG) is a way of building a language system that first looks up passages in a document collection and then writes its answer from them. The term comes from a 2020 paper by Lewis and colleagues, accepted at the Neural Information Processing Systems (NeurIPS) conference, which describes models that combine "pre-trained parametric and non-parametric memory" for language generation. Because the passages are real text, the system can show which ones it used. Later evaluations found that retrieval reduces made-up answers without ending them.

What the 2020 paper combined

The paper joined two parts. The first, which the authors call parametric memory, is BART-large, a pretrained sequence-to-sequence transformer with 400 million parameters. The second, the non-parametric memory, is a dense vector index of Wikipedia searched by the Dense Passage Retriever (DPR), a bi-encoder that turns the question and each passage into vectors.

For each question, the retriever returns the top K passages by maximum inner product search. The generator receives the question concatenated with a passage and writes the output. Of the two variants tested, RAG-Sequence uses the same retrieved passage for the whole answer. RAG-Token can draw on a different passage for each token it generates, so one answer can combine several sources.

The authors kept the document encoder and its index fixed and fine-tuned only the query encoder and the generator. They report that updating the index during training was not necessary for strong performance.

What it reported

The paper reports state-of-the-art results on three open-domain question answering tasks: Natural Questions, WebQuestions, and CuratedTrec. On Natural Questions, RAG-Sequence reached an exact match score of 44.5. T5-11B, a closed-book model with 11 billion parameters that answers from its parameters alone, scored 34.5. The DPR question answering system scored 41.5.

In a human comparison of 452 pairs of generated Jeopardy questions, judges rated BART alone more factual in 7.1% of cases and RAG-Token more factual in 42.7%.

The most practical result concerns updating. The authors built an index from a December 2016 Wikipedia dump and asked about 82 world leaders who changed between the 2016 and 2018 dumps. RAG answered 70% correctly with the 2016 index for 2016 leaders and 68% with the 2018 index for 2018 leaders. With the index and year mismatched, accuracy fell to 12% and 4%. No retraining was involved.

How retrieval grounds an answer and allows citation

In a RAG system the generator writes from retrieved passages, so the answer is tied to text a person can open. The Lewis paper names providing provenance for a model's decisions as an open research problem for models that hold knowledge only in their parameters. The authors note that their memory is raw text, which makes it human-readable and human-writable: editing the index changes what the model can draw on.

That is what makes citation possible: the system knows which passages it retrieved and can show them. A shown passage records what the model was given. It does not prove that the answer follows from it.

The National Institute of Standards and Technology (NIST) draws the same line. The Generative Artificial Intelligence (AI) Profile, NIST AI 600-1, published July 2024, defines confabulation as confidently presented erroneous or false content, and warns that outputs may include confabulated logic or citations. The profile's suggested actions include reviewing and verifying sources and citations in system outputs before deployment and during monitoring, and re-assessing risks after a RAG implementation.

What retrieval does not fix

Retrieval lowers made-up content without removing it. In human evaluations on the Wizard of Wikipedia dialogue task, Shuster and colleagues found hallucination in 68.2% of responses from BART-Large alone and in 7.9% from FiD-RAG, a retrieval model built on the Fusion-in-Decoder (FiD) method, given five documents. Given 25 documents, the same design rose to 19.8%.

Position matters too. Liu and colleagues, in a study published in Transactions of the Association for Computational Linguistics in 2024, found that models used relevant information best at the start or end of their input. With the answer passage in the middle of 20 or 30 documents, GPT-3.5-Turbo did worse in the worst case than its 56.1% closed-book score without documents.

A preregistered evaluation by Magesh and colleagues, posted May 30, 2024, found that legal research tools whose providers touted methods such as RAG as eliminating or avoiding hallucinations still hallucinated between 17% and 33% of the time, though less than a general-purpose chatbot.

An experience report by Barnett and colleagues, from the 3rd International Conference on AI Engineering in April 2024, lists seven failure points. They include content missing from the documents, an answer that does not rank high enough to be returned, and an answer present in the context but not extracted.

Why it suits a closed document set

The index-swap result is the reason RAG fits a fixed library. Knowledge sits in the index, so a new revision of a manual enters by re-indexing, not retraining, and a superseded page leaves the same way. An answer can be traced to a passage in a set the owner controls. Nothing in the method needs the collection to be online: the 2020 system searched a fixed index built in advance.

A closed set also makes the missing-content case testable. Barnett and colleagues describe the right response to a question the documents cannot answer as something like "Sorry, I don't know," and warn that questions related to the content can fool a system into answering anyway. With a known collection, an evaluator can write questions the documents do not answer and check that the system declines. The report's first takeaway is that validating a RAG system is only feasible during operation, so the checks continue after delivery.

What to check

• Which documents are in the index, and which revision of each? In the 2020 paper, swapping the index changed the answers.
• Does each answer show the passages it was given, and does someone verify that the cited passage supports it, as NIST AI 600-1 suggests?
• How does the system respond to a question the documents do not answer?
• How many passages reach the generator, and was accuracy tested with the relevant passage in the middle of the list?
• Was a hallucination rate measured on these documents with human judges, or assumed from the design?

Azcuna's ELLMENT, the E-2D Large Language Model Entity, was proposed to NAVAIR PMA-231 under Navy SBIR Phase I topic DON26BZ01-NV010. The prototype, built at private expense on publicly available doctrine, answers with a page-level citation and refuses when the documents do not support an answer. The proposal has been submitted.

Sources

1. 1Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks · arXiv (Lewis et al., NeurIPS 2020)
2. 2Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST AI 600-1) · National Institute of Standards and Technology, July 2024
3. 3Retrieval Augmentation Reduces Hallucination in Conversation · ACL Anthology (Shuster et al., Findings of EMNLP 2021)
4. 4Lost in the Middle: How Language Models Use Long Contexts · arXiv (Liu et al., TACL volume 12, 2024)
5. 5Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools · arXiv (Magesh et al., May 30, 2024 version)
6. 6Seven Failure Points When Engineering a Retrieval Augmented Generation System · arXiv (Barnett et al., 3rd International Conference on AI Engineering, April 2024)

Where this shows up in our work

• ELLMENT, the offline LLM for E-2D crews answers a crew's question from the aircraft's own manuals and cites the page, with no network.
• BABEL, offline multilingual translation and search translates and searches foreign-language documents on-device, every fact tied to its source passage.
• TRACE, the offline S1000D and IETM viewer opens S1000D, MIL-STD-3001 and PDF technical manuals on a tablet with every marking intact.