RAG hallucination: what a citation check actually catches
We built the cheap guardrail, checking a cited chunk holds the value the answer asserts, then measured it against the answer failures of four configs.
The cheap guardrail against RAG hallucination is well known: make the model cite the chunk it used, then check that the chunk actually contains the value the answer asserts. If it does not, the answer is ungrounded and you catch it.
We built it and measured it against the answer failures recorded under each of the four configurations that generated answers. Those are failures where the system produced an answer and the answer was wrong, which is the population a guardrail can act on; it is not every failure in the audit, because a retrieval failure that never reached the generator has nothing for a citation check to inspect. The figures come from our RAG failure audit and our confidence-routed extraction study.
The result
Across 4 configurations and 230 recorded answer failures, the citation check caught 1/230. Those configurations were measured separately and the totals below are their sum, so read the per-configuration rows rather than the total where the two differ.
| configuration | failures | caught by the check | already abstained | slipped through |
|---|---|---|---|---|
A | 54 | 0 | 12 | 42 |
B | 53 | 0 | 17 | 36 |
C | 63 | 0 | 14 | 37 |
M | 60 | 1 | 18 | 34 |
Before reading that as “guardrails do not work”, read the next two sections, because the interesting part is why.
Most of what looks like a catch is an abstention
61 of the 230 failures were on questions where the system had already declined to answer.
It is very tempting to book those as the guardrail working. They are not. There was no assertion to check. The guardrail had nothing to do, and counting them as catches reports protection that is not there.
This is the single most common way a guardrail evaluation flatters itself, and it is easy to avoid: count abstentions in their own column.
The failures that slipped were grounded
Of the 149 that slipped through, 102 satisfied the check. The answer was wrong, and the value it asserted was genuinely present in the chunk it cited.
That is the whole limitation in one sentence. A citation check tests presence, not correctness. If the model retrieves the right document and reads the wrong number out of it, or reads a superseded value that is really there, the check passes and the answer is wrong.
Another 47 had no checkable value at all: a summary, a qualitative statement, nothing a string comparison can verify.
Where the same check does work
On a different task the same idea contained the worst error class completely.
In our receipt extraction study, 34 factor-of-1000 magnitude errors were recorded, and 34 of 34 were routed to review with 0 shipped.
The difference is what is being checked. A magnitude error produces a value that is not in the document, so a grounding check fires. A superseded-but-present value is in the document, so it does not.
The same check, opposite results, and the thing that decides it is whether the failure mode produces an ungrounded value.
What this does not catch either
Omissions. In the extraction study, 96/254 auto-approved nulls were wrong: the model returned nothing where the document had a value. There is no asserted value, so grounding cannot fault it.
What to do instead
- Ask which failure your guardrail is shaped for. Grounding checks catch invention. They do not catch selection errors, staleness or omission.
- Count abstentions separately from catches. Always.
- Do not consult ground truth in a guardrail. Anything that reads the answer key is an evaluation, not a guardrail, and it will not exist in production.
- Report the check’s result per configuration. Ours differs at every one, and a number rolled up at a single config would not transfer.
The full recovery analysis is on the RAG failure audit page. For the metric-level version of the same trap, see four ways a good score misleads.