Hallucination Detection & Mitigation

The fairness lesson blocked a soft evaluator when it could route equivalent customer requests differently. Factual reliability needs an equally strict boundary: a fluent large language model (LLM) answer can't invent a carrier event just because the customer would like certainty.

ShopFlow's next ticket asks, "Where is order #A10234?" The admitted FastShip record says the parcel departed a regional hub on May 26 at 08:14 UTC. It contains no delivery estimate. A draft answer that adds "It will arrive on May 28" may sound helpful, but the system has no source for that promise.

This lesson builds tracking-answerer-v1, a small serving gate. It turns an answer into atomic claims, checks each claim against versioned evidence, routes unsupported details to abstention, and records why release remains blocked. For a customer-facing grounded answer, "not present in admitted evidence" is enough reason not to serve a factual claim. It doesn't prove the claim is false in the wider world.

Three verdicts are enough to start

Hallucination taxonomies can become abstract quickly. For a retrieved delivery record, begin with the source relationship:

Surveys of LLM hallucination distinguish statements that conflict with supplied context from statements that add information not established by it.^[1] In an open-world setting, an unsupported statement might later turn out true. In this answer path it's still unsafe, because the product promised evidence-grounded delivery updates.

Figure 1: A grounded delivery answer is served only when every factual claim has an admitted supporting source.

Put the evidence in code

The carrier record isn't prose decoration. It's the authority the answer must obey. Our first cell records its source ID and version alongside the candidate claims.

An LLM can write the ETA sentence because it has seen delivery-date patterns. That doesn't make the sentence admissible. The admitted evidence has no delivery_eta field.

Verify each atomic claim

FActScore popularized evaluating long-form generations as atomic facts rather than one answer-level impression.^[2] The verifier below uses the same reasoning on a small operational record: a claim is either supported by its cited source, missing from that source, contradicted by a different value, or missing an admitted source entirely.

The distinction matters. The ETA isn't proven false; it's simply absent from this source. The delivered statement is worse: it conflicts with status=in transit.

Route the response, not only the score

A detector that writes a warning beside an unsafe response has not protected the customer. Once any factual claim fails, tracking-answerer-v1 keeps supported information and replaces the invented promise with a bounded statement.

Measure failure before choosing mitigation

A single blocked ETA gives a regression case, not a release metric. Build a small suite that separates four failure modes: a clean scan summary, an unsupported ETA, a contradiction, and a missing source.

For a grounded status product, two simple metrics are useful:

Claim support can improve while unsafe serves remain unacceptable. A single contradicted delivery status shown to a customer is still a serious failure.

Consistency is an alarm, not evidence

Black-box methods such as SelfCheckGPT compare an answer to additional sampled generations; disagreement can flag factual content that the model is inventing without an external database.^[3] Semantic entropy similarly groups sampled answers by meaning and detects high uncertainty when those meanings vary.^[4] These are useful when no admitted source is available or when you need to prioritize expensive checks.

They don't authorize a delivery claim. A model can repeat the same unsupported ETA every time.

This fixes a common misconception: low uncertainty means "the model repeats itself," not "the fact is true."

Combine signals with the right authority

Evidence failure blocks a fact even when samples agree. When factual evidence passes but generation is unstable, consistency can route the case to review rather than presenting an answer that changes from run to run.

Citations must be checked, not decorated

A response that prints [fastship-A10234] isn't necessarily grounded. The citation must resolve to the admitted version and support the nearby claim. Otherwise a model can attach a real-looking source marker to an invented delivery promise.

This is a narrower version of claim-level factual evaluation: decompose, retrieve or admit evidence, verify, and retain the provenance for failed and passed claims. It also extends the RAG evaluation lesson: citation presence is no longer confused with citation support.

Attribute failures before adding complexity

An unsupported answer may begin with retrieval or generation:

Techniques such as Chain-of-Verification ask a model to draft verification questions, answer them independently, and revise the response; the original paper reports reduced hallucination across list questions, closed-book question answering, and long-form generation.^[5] That's a possible additional generator control. It doesn't replace an authoritative carrier record or the final claim gate in this product.

Evaluate a candidate gate

Research benchmarks help compare methods, but they don't execute this carrier feed, prompt, citation format, or customer route. Use external datasets for breadth and product regressions for release:

This is a useful repair on four deliberately small cases. It isn't enough to claim production factuality. The candidate abstains on three of four requests here because the test suite is failure-heavy by design.

Keep the mitigation stack small and testable

For this workflow, extra decoding tricks aren't the next priority. The current failure is already localized: the draft adds a fact absent from a live source. Start with controls whose effect can be tested against that source:

Hand the next lesson a trace

The next chapter can't monitor a correctness property that this chapter never records. Store enough information to reconstruct the answer decision without retaining unnecessary customer text: source versions, verdict counts, route, and failure stage.

The candidate gate fixes its known fabricated-ETA regressions, but promotion remains blocked. It still needs a representative labeled holdout and an operational owner for alerts. The code has now produced exactly the facts an observability system should aggregate.

Mastery check

Key concepts

• Supported, not-supported, and contradicted factual claims
• Atomic claim verification against admitted evidence
• Evidence-versioned citations
• Claim support, unsafe-serve, and abstention rates
• Consistency sampling as triage rather than truth
• Safe answer shortening and abstention
• Failure-stage attribution
• Regression release gates and monitoring events

Evaluation rubric

• Treats absence of source support as a serving failure without claiming outside-world falsity
• Decomposes a draft answer into testable factual claims
• Blocks both invented ETAs and claims contradicted by admitted state
• Demonstrates why stable repeated output isn't proof of truth
• Keeps citations coupled to verified source versions
• Locates first failure before proposing new mitigation layers
• Evaluates safe routing with both unsafe-serve rate and supported coverage
• Produces a trace that the monitoring lesson can aggregate

Follow-up questions

Common pitfalls

Repeated answers are mistaken for true answers

Symptom: A stable sampled ETA is served despite no source field establishing it. Cause: Consistency was treated as evidence. Fix: Use consistency to prioritize verification or review; require admitted source support for customer-facing facts.

Citations are generated but not verified

Symptom: The answer has plausible source tags beside an invented delivery promise. Cause: Citation formatting was checked, but claim-to-source support was not. Fix: Verify each cited factual claim against its source version before serving it.

The retriever is blamed for generator overreach

Symptom: More documents are indexed even though correct tracking evidence was already retrieved. Cause: The team didn't identify the first failed stage. Fix: Separate evidence-admission failure from claim-generation failure in every trace.

Abstention is declared a release success on tiny fixtures

Symptom: Four regression cases are used to claim production factual reliability. Cause: A focused regression suite was confused with a representative holdout. Fix: Keep the regressions, then collect labeled workflow slices and assign monitoring ownership before promotion.