Reranking and Cross-Encoders for RAG

Hybrid retrieval can find the necessary evidence inside a retrieval-augmented generation (RAG) system and still leave it outside the generator's context. Consider this fictional enterprise policy lab:

Today is day 10 after deprecation. Can a service account use the legacy token endpoint with audit logging enabled?

The retriever fires across the policy catalog. BM25 catches exact keyword overlap on an operational note (api-token-troubleshooting-v1) that mentions legacy token errors but explicitly warns that it doesn't authorize access. Dense vector search matches semantic proximity on a password-reset document (api-password-reset-v1). The actual governing rule, api-token-legacy-v2-rule (Rule AUTH-14), allows service accounts to use the endpoint within 14 days of deprecation if audit logging is active. That rule is retrieved at rank 3.

The generator prompt budget accepts only two passages. The troubleshooting note and the password-reset policy occupy the prompt while Rule AUTH-14 stays outside it. An assistant might consequently produce an unsupported refusal: "No, legacy token access is strictly forbidden." We haven't run a generator here; that response illustrates the downstream risk.

For this question's one labeled relevant chunk, candidate recall is 1: the rule is in the candidate set. The four candidates have trusted, current permission decisions in the fixture. This demonstrates an ordering failure at cutoff two, not corpus-wide retrieval quality or a production privacy guarantee.

This continues the hybrid-search pipeline: policy-answerer-v2 filters for current, permitted chunks, then fuses sparse and dense ranks. policy-answerer-v3 adds a reranker to score each query-chunk pair before context selection. The executable labs use a hand-written scorer to test ordering and trace contracts. They don't run a learned model or establish its accuracy.

The boundary: retrieve candidates, then improve their order

A two-stage retriever separates two problems. Retrieval decides whether evidence gets a chance to compete; reranking decides which admitted evidence reaches the top. Generation should support its claims with selected evidence. Merely restricting the prompt doesn't prevent a model from inventing claims or using its parametric memory.

The target is in the candidate set but below a two-chunk cutoff. Reranking may change its rank and selected context, not its source identity or access permissions.

The reranker can't restore a missing document. It must stay inside the same permission boundary: never search a restricted or superseded document as a shortcut. Permitted hybrid candidates enter scoring; admin-only and superseded records never do. That skip is the boundary, not a later filter.

This pipeline accepts permitted candidates, scores their query-chunk pairs, selects context, and records the decision. A freshness or permission revocation may remove a candidate between stages; it must never add a forbidden one. The fixed-membership examples assume an unchanged authorized snapshot.

Start with the first-stage receipt, before any model score. The small fixture shows api-token-legacy-v2-rule at rank 3. Its presence confirms recall; its position predicts a miss at a two-chunk cutoff. Predict the next output: the answer stays out until some stage changes the order.

The candidate record carries document_id and parent_id from the earlier RAG pipeline. Reranking may move evidence, but citation packing still points to the same source identity.

Keep the evidence boundary executable

Test the first-stage allowlist against a source store containing an admin-only rule and an expired revision. Neither should enter reranking. Here, permitted and current are trusted fixture decisions, not fields supplied by the question or source text.

The snapshot compares complete records, including text and parent identity. Matching an ID and version alone would admit changed content under an old receipt. In a service, fetch the recorded revisions and recheck the caller's current permissions before remote scoring and context packing. A changed record needs a fresh retrieval decision. Sending text to a hosted reranker is itself a disclosure: user access doesn't automatically authorize an external processor. Scope caches, logs, and provider use to the same data-handling policy.

Now replay context packing without reranking. The generator receives a troubleshooting note that grants no temporary access and a password-reset policy. The legacy-token rule is retrieved but absent from context.

The current hybrid search ordering leaves out the right chunk. Reranking is one possible repair; improving first-stage ranking or context selection is another. What additional signal comes from reading the query and chunk together?

Read the scoring contract, not just the model's family name

Which tensor becomes the score? Does it depend on one passage or the whole list? Those questions explain both serving behavior and cache validity. A decoder backbone doesn't imply that a reranker generates a ranked list or even uses next-token probabilities.

BERT-style encoder classification heads

BERT-style cross-encoders concatenate the query and candidate chunk: [CLS] Query [SEP] Chunk [SEP]. Bidirectional self-attention lets the two sides interact before pooling.^{[1]Reference 1Passage Re-ranking with BERT.https://arxiv.org/abs/1901.04085[2]Reference 2BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.https://arxiv.org/abs/1810.04805} This is one architecture, not a description of every checkpoint sold under the BGE or Jina name.

Let $h_{\mathrm{CLS}}(q, d) \in \mathbb{R}^d$ be the final hidden state of the [CLS] token. A linear projection head maps this pooled vector to a scalar logit:

$$z = \mathbf{w}^\top h_{\mathrm{CLS}}(q, d) + b$$

Applying a sigmoid function $\sigma(z) = 1 / (1 + e^{-z})$ bounds the logit into $(0, 1)$. During pointwise training, each query-passage example carries a binary relevance label $y \in \{0, 1\}$ and contributes binary cross-entropy:

$$\ell_{\mathrm{pointwise}}(z, y) = -y \log \sigma(z) - (1 - y) \log \bigl(1 - \sigma(z)\bigr).$$

At inference time, there are no gold labels or gradient updates. The model computes $z$ (or $\sigma(z)$) for each candidate and sorts descending. Mathematically, sigmoid is strictly increasing, so it preserves order. Finite-precision saturation can introduce ties near 0 or 1. A sigmoid score isn't automatically a calibrated relevance probability. Sentence Transformers documents checkpoints that return raw logits and an optional sigmoid transformation.^{[3]Reference 3Cross Encoder Usagehttps://github.com/huggingface/sentence-transformers/blob/main/docs/cross_encoder/usage/usage.rst}

Pointwise scoring evaluates each pair in isolation, but training may use a pairwise preference loss. Given a relevant passage $d^+$ with logit $z^+$ and an irrelevant passage $d^-$ with logit $z^-$, the pairwise logistic preference loss penalizes inverted score differences:^{[4]Reference 4Learning to Rank using Gradient Descent.https://www.microsoft.com/en-us/research/publication/learning-to-rank-using-gradient-descent/}

$$\ell_{\mathrm{pairwise}}(z^+, z^-) = \log\bigl(1 + e^{-(z^+ - z^-)}\bigr) = \operatorname{softplus}\bigl(-(z^+ - z^-)\bigr).$$

A model trained with pairwise preference losses still scores individual query-passage pairs independently at serving time. It doesn't need to evaluate all $O(k^2)$ passage pairs during inference.

Token-based scoring and decoder classification heads

monoT5 is an encoder-decoder model trained to predict true or false; its published scoring method renormalizes probabilities over those two target tokens.^{[5]Reference 5Document Ranking with a Pretrained Sequence-to-Sequence Model.https://arxiv.org/abs/2003.06713} Qwen3-Reranker uses a causal decoder and lowercase yes/no logits at the final prompt position.^{[6]Reference 6Qwen3-Reranker-0.6B model card and scoring examplehttps://huggingface.co/Qwen/Qwen3-Reranker-0.6B} An illustrative prompt might be:

Use the checkpoint's actual template and tokenizer, not this sketch. For two single-token labels at the same prediction position, log-odds equal the difference between their logits:

$$s(q, d) = \log \frac{P(\text{"Yes"} \mid q, d)}{P(\text{"No"} \mid q, d)} = z_{\text{Yes}} - z_{\text{No}}$$

Renormalizing over just these two labels gives:

$$p_{\mathrm{binary}}(\text{"Yes"} \mid q, d) = \frac{e^{z_{\text{Yes}}}}{e^{z_{\text{Yes}}} + e^{z_{\text{No}}}} = \sigma(z_{\text{Yes}} - z_{\text{No}}).$$

This is not the full-vocabulary probability $P(\text{"Yes"})$: other possible tokens were removed from the denominator. Binary-label methods need the relevant forward computation, not a long generated explanation. An encoder-decoder also runs its encoder; don't equate its execution path with a causal decoder.

Other decoder-based scorers have different contracts. BGE-reranker-v2-gemma's example takes the single Yes logit, without subtracting No. RankLLaMA's published model example loads a sequence-classification model with one scalar output.^{[7]Reference 7BGE-reranker-v2-gemma model card and scoring examplehttps://huggingface.co/BAAI/bge-reranker-v2-gemma[8]Reference 8RankLLaMA v1 passage reranker model cardhttps://huggingface.co/castorini/rankllama-v1-7b-lora-passage} Neither should be described as the binary formula above.

As of September 21, 2026, Qwen3-Reranker provides 0.6B, 4B, and 8B checkpoints under Apache 2.0. Its card documents 32K context, while the example wrapper chooses max_length=8192; a wrapper can impose a smaller effective limit.^{[6]Reference 6Qwen3-Reranker-0.6B model card and scoring examplehttps://huggingface.co/Qwen/Qwen3-Reranker-0.6B} Jina Reranker v3.5 is a 0.6B listwise model with causal attention and cosine-based scoring, not the BERT-style head above. Its published weights use CC BY-NC 4.0; check deployment licensing before commercial use.^{[9]Reference 9Jina Reranker v3.5 model cardhttps://huggingface.co/jinaai/jina-reranker-v3.5}

Hosted Cohere Rerank accepts a query and candidate list and returns ranked items with relevance scores.^{[10]Reference 10Rerank API.https://docs.cohere.com/v2/reference/rerank} API shape alone doesn't establish that each score is independent of the other candidates. Benchmark the actual checkpoint or service on your evidence and workload; family names don't supply universal latency, VRAM, or instruction-following guarantees.

Here are the losses for invented logits $z^+=1.2$ and $z^-=0.2$. This computes the objective, not a trained model's performance.

Training data must represent the ranking task. Hard negatives from the same first-stage retriever expose plausible near matches; random unrelated negatives can make the task too easy. Keep query and document leakage out of held-out evaluation, and check negation, dates, and no-answer questions separately from an overall average.

Preserve the text that was actually scored

Joint interaction also introduces a pair-length ceiling. Original BERT uses a maximum sequence length of 512 tokens, including special tokens.^{[2]Reference 2BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.https://arxiv.org/abs/1810.04805} Other checkpoints expose different limits, so read the tokenizer and model configuration instead of assuming 512.

If a wrapper silently truncates an overlong pair, the model may score only a prefix while the generator later receives the full chunk. An omitted clause could determine relevance or qualify a rule. This unrecorded mismatch hides the scope of the score; it doesn't prove that the generated answer is wrong. Log tokenized pair length, define a truncation policy, and reject unexpected truncation under that policy.

Truncation isn't only a local-model concern. Check a local wrapper's tokenizer configuration and truncation strategy. Cohere's Rerank API truncates documents to max_tokens_per_doc, default 4096.^{[10]Reference 10Rerank API.https://docs.cohere.com/v2/reference/rerank} Either way, ranked text can disagree with packed text. A sliding-window strategy can retain later clauses, but max-over-window aggregation can favor long documents and omit qualifications from other windows. Record the aggregation policy and keep necessary surrounding context.

The next lab uses whitespace tokens only to expose the accounting. It reserves three illustrative BERT special tokens. A real implementation must use the checkpoint's tokenizer and exact special-token count.

That failure is intentional: this lab refuses any scored/packed token mismatch. The checksum compares normalized toy token lists, not exact source bytes; lowercasing and whitespace splitting erase some differences. Real rerankers and generators can use different tokenizers, so record the scored source span and exact text separately from each model's token accounting. A parent expansion or explicit window score can be valid if its scope and aggregation are recorded.

With the text boundary explicit, test ranking independently of a model endpoint. The next fixture uses transparent phrase rules, not a trained transformer. Its integer scores exercise ordering and packing code; they aren't semantic entailment or authorization decisions.

The rule should rank first for this question. The scorer awards phrase-match points and subtracts six for the troubleshooting note's disclaimer. This deliberately narrow heuristic can't distinguish arbitrary paraphrases or resolve conflicting policies. The audit-export chunk gets timing points even though its 14-day window applies to a different action, a useful reminder of the heuristic's limits.

Context selection now changes for the right reason. The candidate set stays fixed while the supported rule moves to rank 1. A context budget is a maximum, not a quota, so low-scoring near matches are not admitted merely to fill space.

Two chunks is only this lab's simplified budget. Production packing counts generator tokens after reserving instructions and output space, limits redundant passages from one parent document, and preserves clauses needed to interpret a rule. If packing expands a scored passage into a larger parent section, record that expansion explicitly rather than claiming every added sentence received the passage's score.

The fixture's threshold of 5 separates this rule from these three distractors. It isn't a permission rule: with audit disabled, the rule still scores 5 and remains relevant because it explains why access isn't allowed. A generator must apply the actual policy conditions, not interpret a high relevance score as an automatic affirmative. Scores that rank a denial highly can be completely correct.

Score calibration and safe admission thresholds

Learned scores aren't automatically calibrated probabilities. A sigmoid score of 0.85 doesn't establish an 85% chance that the document answers the question, and 0.80 doesn't establish twice the relevance of 0.40. A calibration procedure must define and test the event its probability represents.

Scores can vary with query type, candidate distribution, and checkpoint. Imagine a hypothetical no-answer query whose best distractor scores 0.90 and an answerable query whose best evidence scores 0.60. The shared threshold alone can't separate them. These numbers illustrate the problem; they aren't measured score ranges for broad or technical queries.

An unvalidated static threshold (such as score >= 0.70) can produce two failure modes:

1. Evidence starvation: Useful evidence below the threshold never reaches generation.
2. Noise admission: Irrelevant candidates above the threshold consume context.

Compare these admission strategies on labeled development cases, including no-answer cases:

• Rank-first with context budget: Take the highest-ranked items that fit. Strictly increasing score transformations preserve mathematical order. This avoids a fixed score scale but still selects distractors when every candidate is irrelevant; it isn't an abstention rule.
• Score margin falloff gating: Rather than filtering on an absolute score, filter candidates whose score falls significantly below the top candidate's score: $$\text{Keep chunk } i \iff s(q, d_i) \ge s(q, d_1) - \Delta_{\text{margin}}$$ A common additive shift leaves this rule unchanged; changing score scale generally doesn't. Nearby scores don't prove that sources agree or that any is relevant. Tune $\Delta_{\text{margin}}$ for the checkpoint and inspect admitted evidence.
• Calibrated admission or abstention: Platt scaling fits $P(y=1\mid z)=1/(1+e^{Az+B})$; isotonic regression fits a monotone mapping.^{[11]Reference 11Probability calibrationhttps://scikit-learn.org/stable/modules/calibration.html} Define $y$ carefully: passage relevance isn't the same event as having enough evidence for a complete answer. Fit calibration and choose thresholds on development data independent of scorer training, then evaluate reliability and abstention on an untouched test set. Revalidate when the checkpoint, tokenizer, scoring policy, or workload changes.

Measure candidate recall and top-of-list relevance

With pair scoring in place, ask how to prove it improves the stage that owns the failure. Keep evaluation narrow, and let each metric answer a different question:

1. Recall@candidate_k checks whether first-stage retrieval gave the reranker a chance.
2. Mean reciprocal rank (MRR) averages how early the first relevant chunk appears across a release suite.
3. Normalized discounted cumulative gain (NDCG) at the context cutoff checks whether relevant chunks fit near the top of the context budget.^{[12]Reference 12Introduction to Information Retrieval.https://nlp.stanford.edu/IR-book/}

For this one request, reciprocal rank (RR) is easy to read: rank 3 gives $1/3$; rank 1 gives $1$. MRR is the mean of those per-request values. NDCG supports graded relevance.

This lab uses binary relevance, where a relevant chunk has $rel_i = 1$ and an irrelevant chunk has $rel_i = 0$. With binary labels, linear-gain and exponential-gain DCG formulas agree, so the numbers below don't depend on which variant you pick:

$$\operatorname{DCG}_k = \sum_{i=1}^{k} \frac{2^{rel_i}-1}{\log_2(i+1)} \qquad \operatorname{NDCG}_k = \frac{\operatorname{DCG}_k}{\operatorname{IDCG}_k}$$

Predict the fixture result before running it: moving the only relevant chunk from rank 3 to rank 1 should change RR from $1/3$ to $1$, and put the chunk inside NDCG@2.

These numbers check this hand-labeled fixture, not model quality or generated-answer correctness. Deduplicate IDs before scoring: repeating a relevant chunk mustn't earn gain twice. This implementation defines NDCG as zero when there are no relevant labels; report that convention and evaluate no-answer behavior separately. Treat unjudged documents carefully rather than assuming every missing label means irrelevance.

For a held-out query suite, compare per-query scores on identical authorized candidates, then aggregate MRR/NDCG with uncertainty and query-type breakdowns. Also score the final packed context: filtering, deduplication, and token budgets can remove a highly ranked passage after the reranker.

Candidate count sets the ceiling and the bill

An ordering metric can't rescue an item that never reaches the reranker. If the reranker receives only the first two candidates, api-token-legacy-v2-rule is cut before pair scoring. Use a candidate-recall gate before comparing reranker models.

For relevant-set $R$ and candidate-set $C_k$, candidate recall is $|R\cap C_k|/|R|$. Any context selected solely from $C_k$ has recall no higher than that value. With two required passages but only one retrieved, even an oracle reranker can't exceed 50% passage recall. For multi-hop questions, measure whether the whole required evidence set is present, not just whether one relevant chunk appears.

Predict the budget tradeoff: top 2 is too small to score the target; top 3 gives it a chance. The lab keeps every other variable fixed so that ceiling is visible.

Compute and latency budgeting in the two-stage funnel

A pairwise cross-encoder scores each query-candidate pair online. Increasing candidate depth increases total work, but wall-clock latency also depends on batch scheduling and hardware. For dense attention with fixed layer count $L$ and hidden width, the attention component scales as $O(LT^2)$ for pair length $T$. Doubling $T$ multiplies that component by roughly four, not necessarily the complete request latency.

Choose a retrieval shortlist by measuring the recall/latency frontier. There is no universal GPU millisecond range, CPU depth limit, or safe candidate count. Include actual query and passage lengths, queueing, concurrency, precision, model loading, and endpoint overhead in the measurement.

Batching also needs a concrete contract:

• Padding a batch to its longest sequence can waste work. Length bucketing reduces that waste, while waiting to form batches can add latency.
• Triton's default dynamic batching requires compatible input shapes. Ragged batching is optional configuration; the backend must handle concatenated inputs and their boundaries.^{[13]Reference 13Triton Inference Server: Ragged Batchinghttps://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/user_guide/ragged_batching.html} Selecting a runtime doesn't automatically remove padding.
• FlashAttention improves attention memory access; it doesn't remove dense attention's quadratic arithmetic. Whether a particular reranker supports variable-length execution must be checked in its implementation.
• Compare smaller checkpoints, quantization, and different batching strategies on both relevance and serving measurements. Faster execution alone doesn't establish preserved ranking quality.

This fixture uses a sequential cost model to make the candidate-depth trade-off visible. It isn't a p95 benchmark: production serving batches pairs, and percentiles must be measured end to end. Cohere recommends against sending more than 1,000 documents in one Rerank request; that's a hosted-API cousin of the same $k$ knob.^{[10]Reference 10Rerank API.https://docs.cohere.com/v2/reference/rerank}

For this fixture, three candidates recover the rule and fit the synthetic budget. The context cap is two, but only one passage clears the fixture threshold. A real service chooses these limits using held-out relevance labels and measured end-to-end p95 latency by candidate count, input length, concurrency, and batch strategy. With dense attention, a pair's attention work grows roughly quadratically in its token length; equal candidate counts can therefore have very different costs. Measure queueing, network, tokenization, model execution, and packing together.

Define failure behavior before serving: on a timeout, either fall back to the original authorized order and mark the trace as degraded, or abstain if policy requires reranking. Never fill the gap with unscored restricted evidence. On equal scores, this lab preserves first-stage rank and then uses chunk ID as a deterministic tie-breaker.

Choose an interaction design deliberately

Once candidate depth $k$ is chosen, choose when query and chunk are allowed to interact. Predict the storage tradeoff first: anything computed without the query can be indexed; a query-specific score must wait for request time.

Three ranking designs make different choices across the interaction spectrum:

Bi-encoders compress a passage into one vector before comparing it with the query.^{[14]Reference 14Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks.https://arxiv.org/abs/1908.10084} Encoding every condition into that representation can be challenging, but this doesn't mean a bi-encoder necessarily loses negation or entity roles. Those behaviors need evaluation on the actual checkpoint.

BERT-style cross-encoders let query and document tokens interact before the pooled score is computed. That creates an opportunity to model fine distinctions, not a guarantee that the trained model resolves them correctly. Its output is still a compressed scalar.

ColBERT bridges the gap through late interaction.^{[15]Reference 15ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT.https://arxiv.org/abs/2004.12832} It keeps document representations indexable offline as individual contextual token vectors, while preserving token-level comparison at query time. For each query token vector, the MaxSim operator finds its maximum dot product across all document token vectors, then sums those per-token maxima:

$$S(q, d) = \sum_{i=1}^{N_q} \max_{j=1}^{N_d} \operatorname{sim}\bigl(\mathbf{E}_q[i], \mathbf{E}_d[j]\bigr)$$

Original ColBERT L2-normalizes token embeddings, making token dot products equal cosine similarities.^{[15]Reference 15ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT.https://arxiv.org/abs/2004.12832} Token dimension $m$ is a configuration choice, not always 128. PLAID is a search engine for ColBERTv2 representations that uses centroid interaction and pruning; its reported speedups depend on the configuration and workload.^{[16]Reference 16PLAID: An Efficient Engine for Late Interaction Retrievalhttps://arxiv.org/abs/2205.09707} Don't infer a universal sub-10-ms result.

Two query vectors find their best matches independently: $q_1$ matches $d_1$ at $0.9$, $q_2$ matches $d_2$ at $0.8$, and the score is their sum. These invented 2-D vectors all have unit length; $\sqrt{0.19}\approx0.436$ is chosen so $0.9^2+0.19=1$. The lab illustrates MaxSim, not embeddings from a trained ColBERT checkpoint.

Cache invalidation and multi-tenant security

Cross-encoder scores are computationally expensive, which makes response caching attractive. Caching scores simply by chunk ID or (query, chunk_id) is a severe correctness and security vulnerability.

In dynamic enterprise environments, three events corrupt simplistic caches:

1. Document updates: If a policy document changes but keeps its chunk ID, an un-invalidated cache entry continues serving scores calculated against the obsolete version.
2. Model upgrades: A checkpoint or scoring-method change can shift score scales and ordering. Mixing incompatible cached scores can corrupt ranking.
3. Cross-tenant data leakage: In multi-tenant environments, Tenant A and Tenant B might issue identical queries. If Tenant A has access to a confidential contract and caches its high relevance score, Tenant B's retrieval pipeline must not see that score, receive that document ID, or learn of its existence through side-channel timing probes.

Fingerprint the exact scoring input, immutable model and tokenizer revisions, scoring settings, and trusted tenant/access namespace. Include instructions, templates, truncation or windows, aggregation, adapters, and relevant precision settings. A mutable model name isn't an immutable revision. Avoid normalizing away input distinctions unless the scorer deliberately uses that same normalization.

For an independently scored pair, the input contains one query and passage. For a listwise model such as Jina Reranker v3.5, it contains the complete ordered candidate list: changing a neighbor or its position can change the score.^{[9]Reference 9Jina Reranker v3.5 model cardhttps://huggingface.co/jinaai/jina-reranker-v3.5} Pairwise cache entries aren't interchangeable with whole-list results.

Use structured serialization rather than ambiguous string concatenation. This small fixture hashes already-rendered model input and explicit configuration. The revision strings below are invented teaching values, not real checkpoint pins.

The key identifies inputs; it doesn't authenticate a caller, encrypt sensitive material, or prove that text is correct. Derive access fields from trusted authorization, and recheck current access before exposing cached IDs, scores, or text, not only before generation. Cache isolation alone doesn't establish protection against timing side channels. Keep source revisions in cached result records for traceability as well.

Emit the trace that evaluation needs

Once the ranking looks right, make the decision reproducible. Without evidence-level traces, a bad answer could come from retrieval order, pair scoring, context packing, or source freshness.

Record first-stage IDs, actual reranker input, candidate source identity, policy versions, pair scores, model versions, selected context IDs, and release gates. A document version is part of correctness: a well-ranked stale rule is still unsafe context.

The trace records the actual three-candidate input and compares ordering on that subset. Its authority labels describe a trusted teaching snapshot, not implemented authentication. Gold-target and ordering gates belong to offline evaluation, where labels exist. Live traces record IDs, versions, latency, and selection, but normally don't know the correct answer. The substring check only detects complete fixture text; it misses fragments or encoded content and isn't a privacy audit. IDs, scores, and traces still require access controls.

Try to break the receipt

What if a scorer returns a different candidate with a very high score? Or someone edits text while preserving the ID and version? These tests exercise the actual boundary, not just the expected order. The final case also shows why a relevant rule can support a denial.

The fictional rule treats days 0 through 14 as inclusive and assumes the caller is a service account. These are this lab's interpretation rules, not general API policy. At day 15 or with auditing disabled, the chunk remains relevant and clears the +5 admission gate, while access is denied. The score selects evidence; interpreting that evidence is a separate operation.

Release checks for policy-answerer-v3

Use the gates as a diagnosis path. A missing target is a recall failure; a target below the context cutoff is an ordering failure; a score-text mismatch is a truncation failure; correct context with an unsupported answer belongs to the next generation and evaluation stage.

Before this reranker serves another legacy-token question:

Caching only by chunk ID is unsafe. Include the exact scoring input's checksum, immutable model/tokenizer revisions, scoring settings, and trusted access scope. For listwise scoring, input identity covers the full ordered list. Recheck access before disclosing a cache hit. Replay labeled cases after source changes; a high score for obsolete policy is still obsolete evidence.