LearnCore LLM FoundationsChunking Strategies

🔍MediumRAG & Retrieval

Chunking Strategies

Turn clean documents into retrieval units that preserve answers, citations, and measurable search quality.

12 min read

Learning path

Step 50 of 155 in the full curriculum

File Ingestion for AI LLM Benchmarks & Limitations

The previous lesson produced a clean record from the ShopFlow returns policy:

returns-policy-v3.pdf, page 7: Damaged electronics: report within 48 hours with photos.

That sentence is trustworthy evidence only while its pieces stay together. Split it into one chunk containing "Damaged electronics" and another containing "48 hours with photos," and a search result can retrieve the condition without the deadline or the deadline without its condition.

Chunking turns normalized records into searchable evidence units. In a retrieval-augmented generation (RAG) system, a retriever selects passages from an external index and the generator answers from those passages.^[1] Your chunk boundaries decide what a retrieved passage can prove.

A clean policy record split into answerable chunks with locator metadata, then indexed and checked against labeled customer questions. — Ingestion made text faithful. Chunking now decides which complete evidence units can be searched and cited.

What a useful chunk must preserve

Suppose a customer asks, "My headphones arrived damaged. When must I report it, and what do I send?"

Candidate retrieval unit	Searchable?	Answerable?	Problem
`Damaged electronics: report within`	yes	no	deadline and proof are missing
`48 hours with photos. Unopened electronics: return within 14 days.`	yes	no	condition is missing and a competing rule is present
`Damaged electronics: report within 48 hours with photos.`	yes	yes	preserves condition, deadline, and proof

The first engineering requirement isn't "make chunks small." It's "make each retrieved chunk a defensible piece of evidence."

check-answerable-chunks.py

from dataclasses import dataclass

@dataclass(frozen=True)
class Chunk:
    chunk_id: str
    text: str
    source_id: str
    locator: str

def is_answerable(chunk: Chunk) -> bool:
    required = ["damaged electronics", "48 hours", "photos"]
    lowered = chunk.text.lower()
    return all(phrase in lowered for phrase in required)

chunks = [
    Chunk("broken-condition", "Damaged electronics: report within", "returns-policy-v3.pdf", "page=7"),
    Chunk("broken-window", "48 hours with photos. Unopened electronics return within 14 days.", "returns-policy-v3.pdf", "page=7"),
    Chunk("complete", "Damaged electronics: report within 48 hours with photos.", "returns-policy-v3.pdf", "page=7"),
]

for chunk in chunks:
    print(f"{chunk.chunk_id}: answerable={is_answerable(chunk)}")

Output

broken-condition: answerable=False
broken-window: answerable=False
complete: answerable=True

Fixed windows show the boundary failure

The simplest splitter takes windows of tokens. If a window has size $N$ and overlap $O$ , the next window starts after $N - O$ tokens. Overlap repeats boundary text, which can help continuity, but it can't guarantee that a complete policy rule survives.

The lab uses whitespace-separated words as visible token stand-ins. A production pipeline should measure with the tokenizer used by its embedding model.

split-fixed-windows.py

def fixed_windows(text: str, size: int, overlap: int) -> list[str]:
    if size <= 0 or overlap < 0 or overlap >= size:
        raise ValueError("require size > overlap >= 0")
    words = text.split()
    step = size - overlap
    return [
        " ".join(words[start : start + size])
        for start in range(0, len(words), step)
        if words[start : start + size]
    ]

policy = (
    "Damaged electronics report within 48 hours with photos. "
    "Unopened electronics return within 14 days."
)

for index, chunk in enumerate(fixed_windows(policy, size=6, overlap=2)):
    print(f"{index}: {chunk}")

Output

Damaged electronics report within 48 hours
48 hours with photos. Unopened electronics
Unopened electronics return within 14 days.
14 days.

Now measure what overlap bought you. More repeated words increase indexed text, but the rule becomes useful only when a window carries all three required pieces.

measure-overlap-cost-and-coverage.py

def fixed_windows(text: str, size: int, overlap: int) -> list[str]:
    words = text.split()
    step = size - overlap
    return [
        " ".join(words[start : start + size])
        for start in range(0, len(words), step)
        if words[start : start + size]
    ]

def carries_damage_rule(text: str) -> bool:
    lowered = text.lower()
    return all(term in lowered for term in ["damaged electronics", "48 hours", "photos"])

policy = "Damaged electronics report within 48 hours with photos. Unopened electronics return within 14 days."
for overlap in [0, 2, 4]:
    chunks = fixed_windows(policy, size=7, overlap=overlap)
    complete = sum(carries_damage_rule(chunk) for chunk in chunks)
    indexed_words = sum(len(chunk.split()) for chunk in chunks)
    print(f"overlap={overlap}: chunks={len(chunks)} indexed_words={indexed_words} complete={complete}")

Output

overlap=0: chunks=2 indexed_words=14 complete=0
overlap=2: chunks=3 indexed_words=18 complete=0
overlap=4: chunks=5 indexed_words=28 complete=0

This is the right posture for overlap: a configuration to evaluate, not a ritual to apply to every document.

Prefer policy structure when you have it

File ingestion preserved headings and source locations. Use them. A heading boundary is more meaningful than an arbitrary word count when the source already says which rule belongs together.

LangChain's RecursiveCharacterTextSplitter is a common generic-text baseline. Its documentation describes trying separators in order, with the default sequence ["\n\n", "\n", " ", ""], so paragraphs are preferred before smaller cuts.^[2] Start with explicit heading boundaries when the source provides them. Add the smaller-cut fallback only for sections that still exceed your size limit.

split-markdown-policy-sections.py

from dataclasses import dataclass

@dataclass(frozen=True)
class PolicyChunk:
    heading: str
    body: str
    indexed_text: str
    locator: str

def headed_chunks(markdown: str, source_locator: str) -> list[PolicyChunk]:
    chunks: list[PolicyChunk] = []
    heading = "Document"
    body: list[str] = []

    def flush() -> None:
        if body:
            body_text = " ".join(body)
            chunks.append(
                PolicyChunk(
                    heading,
                    body_text,
                    f"{heading}\n{body_text}",
                    f"{source_locator}#{heading.lower().replace(' ', '-')}",
                )
            )
            body.clear()

    for line in markdown.strip().splitlines():
        if line.startswith("## "):
            flush()
            heading = line[3:]
        elif line.strip():
            body.append(line.strip())
    flush()
    return chunks

policy = """## Damaged electronics
Report within 48 hours with photos.
## Unopened electronics
Return within 14 days in original packaging."""

for chunk in headed_chunks(policy, "page=7"):
    print(chunk.indexed_text.replace("\n", " | "), "|", chunk.locator)

Output

Damaged electronics | Report within 48 hours with photos. | page=7#damaged-electronics
Unopened electronics | Return within 14 days in original packaging. | page=7#unopened-electronics

Each chunk's indexed_text includes its heading, so the searchable text keeps the condition attached to the deadline. The locator separately preserves the path back to original evidence.

A policy table needs an equally deliberate rule. Splitting a row away from its column names turns exact information into ambiguous fragments.

keep-policy-table-with-header.py

table = [
    "| Condition | Window | Evidence |",
    "| --- | --- | --- |",
    "| Damaged electronics | 48 hours | Photos |",
    "| Unopened electronics | 14 days | Original packaging |",
]

def table_chunk(lines: list[str], heading: str) -> dict[str, str]:
    return {
        "heading": heading,
        "text": "\n".join(lines),
        "quality_check": "header_present" if lines[0].startswith("| Condition |") else "review",
    }

chunk = table_chunk(table, "Return windows")
print(chunk["heading"], chunk["quality_check"], f"rows={len(table) - 2}")
print("48 hours" in chunk["text"] and "Photos" in chunk["text"])

Output

Return windows header_present rows=2
True

For a long section, split inside that section while carrying its heading and original locator into every child chunk.

split-long-section-with-provenance.py

def section_windows(text: str, heading: str, source: str, size: int) -> list[dict[str, str]]:
    words = text.split()
    return [
        {
            "text": " ".join(words[start : start + size]),
            "heading": heading,
            "source": source,
            "chunk_id": f"{source}#{heading.lower().replace(' ', '-')}-{start // size}",
        }
        for start in range(0, len(words), size)
    ]

children = section_windows(
    "Report within 48 hours with photos. Include order number and a clear image of damage.",
    heading="Damaged electronics",
    source="returns-policy-v3.pdf:page=7",
    size=7,
)

for child in children:
    print(child["chunk_id"], "|", child["heading"], "|", child["text"])

Output

returns-policy-v3.pdf:page=7#damaged-electronics-0 | Damaged electronics | Report within 48 hours with photos. Include
returns-policy-v3.pdf:page=7#damaged-electronics-1 | Damaged electronics | order number and a clear image of
returns-policy-v3.pdf:page=7#damaged-electronics-2 | Damaged electronics | damage.

Comparison of broken fixed windows, a complete section-aware policy chunk, and parent expansion for the damaged-electronics answer. — Only the complete retrieval unit preserves condition, deadline, and evidence requirement together.

Evaluate boundaries with labeled questions

A chunking strategy isn't good because it sounds sophisticated. It works when labeled questions retrieve complete supporting evidence.

Start with a tiny, transparent score before involving a vector database. For the query terms {damaged, electronics, photos, hours}, the complete damage-policy chunk matches all four. A fragment that contains only {damaged, electronics} may rank, but it can't answer the question.

Chunk	Matching query terms	Contains answer?
complete damage rule	4 / 4	yes
damaged-condition fragment	2 / 4	no
unopened-item rule	1 / 4	no

The following lexical scorer is deliberately simple. It isolates the effect of boundaries; replace its score with real embeddings after the fixture and expected evidence are stable.

retrieve-complete-evidence.py

import re

def terms(text: str) -> set[str]:
    return set(re.findall(r"[a-z0-9]+", text.lower()))

def retrieve(query: str, chunks: list[dict[str, str]]) -> dict[str, str]:
    query_terms = terms(query)
    return max(chunks, key=lambda chunk: len(query_terms & terms(chunk["text"])))

query = "damaged electronics photos hours"
chunks = [
    {"id": "damage-rule", "text": "Damaged electronics: report within 48 hours with photos."},
    {"id": "general-rule", "text": "Unopened electronics: return within 14 days."},
]

hit = retrieve(query, chunks)
print(hit["id"], hit["text"])

Output

damage-rule Damaged electronics: report within 48 hours with photos.

Now compare a broken fixed-window configuration against a section-aware configuration using the same question and the same expected evidence phrase.

compare-chunking-configurations.py

import re

def terms(text: str) -> set[str]:
    return set(re.findall(r"[a-z0-9]+", text.lower()))

def top_chunk(query: str, chunks: list[str]) -> str:
    query_terms = terms(query)
    return max(chunks, key=lambda text: len(query_terms & terms(text)))

query = "damaged electronics photos"
expected = "Damaged electronics: report within 48 hours with photos."
configs = {
    "broken-fixed": [
        "Damaged electronics: report within",
        "48 hours with photos. Unopened electronics: return within 14 days.",
    ],
    "section-aware": [
        "Damaged electronics: report within 48 hours with photos.",
        "Unopened electronics: return within 14 days.",
    ],
}

for name, chunks in configs.items():
    hit = top_chunk(query, chunks)
    print(f"{name}: complete={expected in hit}")

Output

broken-fixed: complete=False
section-aware: complete=True

One question proves little. Ship a small labeled set containing policy exceptions, tables, and boundary failures, then measure each candidate splitter on exactly that set.

run-chunk-regression-suite.py

import re

def terms(text: str) -> set[str]:
    return set(re.findall(r"[a-z0-9]+", text.lower()))

def retrieve(query: str, chunks: list[dict[str, str]]) -> dict[str, str]:
    query_terms = terms(query)
    return max(chunks, key=lambda chunk: len(query_terms & terms(chunk["text"])))

chunks = [
    {"id": "damage", "text": "Damaged electronics: report within 48 hours with photos."},
    {"id": "unopened", "text": "Unopened electronics: return within 14 days."},
    {"id": "refund", "text": "Approved refunds return to the original payment method."},
]
cases = [
    ("damaged electronics photos", "damage", "48 hours"),
    ("unopened electronics return", "unopened", "14 days"),
    ("refund payment method", "refund", "original payment"),
]

passed = 0
for query, expected_id, evidence in cases:
    hit = retrieve(query, chunks)
    ok = hit["id"] == expected_id and evidence in hit["text"]
    passed += int(ok)
    print(query, "PASS" if ok else "FAIL")
print(f"summary={passed}/{len(cases)}")

Output

damaged electronics photos PASS
unopened electronics return PASS
refund payment method PASS
summary=3/3

When you replace this lexical baseline with embeddings, the assertions stay useful: retrieve the correct source and retain text sufficient to answer.

Search small, answer with enough context

Some questions match a narrow sentence, while a faithful answer needs its surrounding section. A parent-child design indexes small children for search and stores a pointer to the larger source section returned for generation.

Diagram showing Clean page record, Damage policy section, Small indexed child, and Customer query. — Clean page record, Damage policy section, Small indexed child, and Customer query.

expand-child-match-to-parent-context.py

parents = {
    "damage": "Damaged electronics: report within 48 hours with photos. Agents must attach the order number.",
    "unopened": "Unopened electronics: return within 14 days in original packaging.",
}
children = [
    {"text": "48 hours with photos", "parent_id": "damage"},
    {"text": "14 days original packaging", "parent_id": "unopened"},
]

match = next(child for child in children if "photos" in child["text"])
print(match["text"])
print(parents[match["parent_id"]])

Output

48 hours with photos
Damaged electronics: report within 48 hours with photos. Agents must attach the order number.

Child windows inside a longer section also benefit from their section label. Embedding a child with a contextual header is a cheap hypothesis to test against your labeled set, not a promise of improvement.

prepend-section-context.py

def indexed_text(heading: str, text: str, source: str) -> str:
    return f"Source: {source}\nSection: {heading}\n{text}"

child = indexed_text(
    heading="Damaged electronics",
    text="Report within 48 hours with photos.",
    source="ShopFlow Returns Policy",
)
print(child)

Output

Source: ShopFlow Returns Policy
Section: Damaged electronics
Report within 48 hours with photos.

Escalate only when the baseline exposes a gap

Structure-aware chunks handle many handbooks and policy pages. Some corpora force different choices:

Failure after measuring baseline	Candidate experiment	What must still be checked
one paragraph shifts between multiple topics	semantic boundary detection	hard size cap and labeled-query score
tiny match lacks surrounding explanation	parent-child expansion	deduplication and generation budget
short child is ambiguous without its section	contextual header	retrieval comparison against no-header baseline
meaning depends on far-away document context	late chunking	model support, latency, and measured retrieval gain

Semantic chunking places candidate boundaries where neighboring sentence representations change sharply. The next small lab uses transparent topic vectors, so you can see the boundary without trusting an external embedding service.

find-semantic-boundary-candidates.py

import math

def vector(sentence: str) -> list[float]:
    lowered = sentence.lower()
    return [
        float(sum(word in lowered for word in ["damaged", "return", "photos", "hours"])),
        float(sum(word in lowered for word in ["refund", "payment", "approved"])),
    ]

def cosine(left: list[float], right: list[float]) -> float:
    dot = sum(a * b for a, b in zip(left, right))
    left_norm = math.sqrt(sum(a * a for a in left))
    right_norm = math.sqrt(sum(b * b for b in right))
    return dot / (left_norm * right_norm) if left_norm and right_norm else 0.0

sentences = [
    "Damaged electronics require photos.",
    "Report damaged items within 48 hours.",
    "Approved refunds return to the original payment method.",
]

for left, right in zip(sentences, sentences[1:]):
    similarity = cosine(vector(left), vector(right))
    print(f"{similarity:.2f}", "boundary" if similarity < 0.50 else "keep together")

Output

1.00 keep together
0.32 boundary

Late chunking is a different escalation. Günther et al. encode the longer text first and apply chunk pooling after the transformer's contextual token representations have been computed.^[3] That means a child representation can carry information from surrounding document text, but it requires a compatible long-context embedding stack and must earn its additional cost in evaluation.

Comparison of early chunking versus late chunking: early splits the document before embedding, while late chunking embeds the full document first and pools token ranges afterward. — Late chunking targets a specific gap: local chunks whose meaning depends on broader document context.

Ship a chunking decision, not a guess

For the ShopFlow policy corpus, a credible first release is straightforward:

Design choice	Initial decision	Evidence to collect
Default boundary	heading-aware sections, recursive fallback	answerable-chunk rate and retrieval regression suite
Tables	keep header plus rows together	exact-value questions preserve correct row meaning
Overlap	off for complete policy sections; test for fallback prose	index size, duplicate hits, and labeled-query results
Metadata	source, locator, heading, checksum	cited answer can return to original record
Escalation	parent expansion before semantic or late chunking	failure examples that justify extra complexity

Your release gate can be encoded as an executable manifest check.

gate-indexable-policy-chunks.py

from hashlib import sha256

chunks = [
    {
        "id": "damage",
        "text": "Damaged electronics: report within 48 hours with photos.",
        "source": "returns-policy-v3.pdf",
        "locator": "page=7#damaged-electronics",
        "heading": "Damaged electronics",
        "quality": "ready",
    },
    {
        "id": "broken",
        "text": "48 hours with photos.",
        "source": "returns-policy-v3.pdf",
        "locator": "page=7#fragment",
        "heading": "Fragment",
        "quality": "review",
    },
]

for chunk in chunks:
    chunk["checksum"] = sha256(chunk["text"].encode()).hexdigest()

required_metadata = ("source", "locator", "heading", "checksum")
indexable = [
    chunk for chunk in chunks
    if chunk["quality"] == "ready"
    and all(chunk[field] for field in required_metadata)
]
print(f"indexable={[chunk['id'] for chunk in indexable]}")
print(f"blocked={len(chunks) - len(indexable)}")

Output

indexable=['damage']
blocked=1

Mastery check

You now know how to take the clean evidence records from ingestion and turn them into retrieval units that can be evaluated.

Key concepts

A chunk is a searchable evidence unit, not an arbitrary slice of text.
Fixed windows expose boundary and overlap costs clearly.
Structure-aware splitting is a strong baseline when headings or tables carry meaning.
Parent-child expansion separates precise search from sufficient answer context.
Semantic or late chunking should be escalations justified by measured failures.
A labeled retrieval set must check both correct selection and answerable evidence.

Evaluation rubric

Foundational: Identifies why a broken chunk can't answer the damaged-electronics question
Foundational: Implements fixed windows and explains what overlap repeats
Intermediate: Preserves headings, tables, source IDs, and locators in chunk records
Intermediate: Compares candidate boundaries on labeled retrieval cases
Intermediate: Uses parent-child expansion when narrow retrieval lacks sufficient context
Advanced: Chooses semantic or late chunking only after measuring a baseline failure

Follow-up questions

Common pitfalls

A deadline loses its condition: A window retrieves 48 hours without Damaged electronics. Fix: split at section boundaries and assert answerability.
A table row loses its header: A price or return window becomes ambiguous. Fix: keep column names with rows and test exact-value questions.
Overlap creates duplicates without fixing answers: Repeated chunks dominate top results. Fix: measure overlap against both complete evidence and index cost.
Chunks can't be cited: Search looks plausible but support can't defend the answer. Fix: retain source, locator, heading, and checksum metadata.
An advanced splitter is adopted by reputation: Complexity rises without better results. Fix: preserve a simple baseline and evaluate every escalation on the same labeled set.

Next Step

Continue to LLM Benchmarks & Limitations

You can now build evidence-level retrieval tests for a RAG system. Next you will learn why evaluation sets and scores need the same care before they can support a model-quality claim.

PreviousFile Ingestion for AI

Share this article

X Facebook LinkedIn Bluesky Reddit Hacker News Email

References

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks.

Lewis, P., et al. · 2020 · NeurIPS 2020

RecursiveCharacterTextSplitter

LangChain · 2023

Late Chunking: Contextual Chunk Embeddings Using Long-Context Embedding Models.

Günther, M., et al. · 2024 · arXiv preprint

Back to Topics

LearnCore LLM FoundationsChunking Strategies

🔍MediumRAG & Retrieval

Chunking Strategies

Turn clean documents into retrieval units that preserve answers, citations, and measurable search quality.

12 min read

Learning path

Step 50 of 155 in the full curriculum

File Ingestion for AI LLM Benchmarks & Limitations

The previous lesson produced a clean record from the ShopFlow returns policy:

returns-policy-v3.pdf, page 7: Damaged electronics: report within 48 hours with photos.

What a useful chunk must preserve

Suppose a customer asks, "My headphones arrived damaged. When must I report it, and what do I send?"

Candidate retrieval unit	Searchable?	Answerable?	Problem
`Damaged electronics: report within`	yes	no	deadline and proof are missing
`48 hours with photos. Unopened electronics: return within 14 days.`	yes	no	condition is missing and a competing rule is present
`Damaged electronics: report within 48 hours with photos.`	yes	yes	preserves condition, deadline, and proof

The first engineering requirement isn't "make chunks small." It's "make each retrieved chunk a defensible piece of evidence."

check-answerable-chunks.py

from dataclasses import dataclass

@dataclass(frozen=True)
class Chunk:
    chunk_id: str
    text: str
    source_id: str
    locator: str

def is_answerable(chunk: Chunk) -> bool:
    required = ["damaged electronics", "48 hours", "photos"]
    lowered = chunk.text.lower()
    return all(phrase in lowered for phrase in required)

chunks = [
    Chunk("broken-condition", "Damaged electronics: report within", "returns-policy-v3.pdf", "page=7"),
    Chunk("broken-window", "48 hours with photos. Unopened electronics return within 14 days.", "returns-policy-v3.pdf", "page=7"),
    Chunk("complete", "Damaged electronics: report within 48 hours with photos.", "returns-policy-v3.pdf", "page=7"),
]

for chunk in chunks:
    print(f"{chunk.chunk_id}: answerable={is_answerable(chunk)}")

Output

broken-condition: answerable=False
broken-window: answerable=False
complete: answerable=True

Fixed windows show the boundary failure

The lab uses whitespace-separated words as visible token stand-ins. A production pipeline should measure with the tokenizer used by its embedding model.

split-fixed-windows.py

def fixed_windows(text: str, size: int, overlap: int) -> list[str]:
    if size <= 0 or overlap < 0 or overlap >= size:
        raise ValueError("require size > overlap >= 0")
    words = text.split()
    step = size - overlap
    return [
        " ".join(words[start : start + size])
        for start in range(0, len(words), step)
        if words[start : start + size]
    ]

policy = (
    "Damaged electronics report within 48 hours with photos. "
    "Unopened electronics return within 14 days."
)

for index, chunk in enumerate(fixed_windows(policy, size=6, overlap=2)):
    print(f"{index}: {chunk}")

Output

Damaged electronics report within 48 hours
48 hours with photos. Unopened electronics
Unopened electronics return within 14 days.
14 days.

Now measure what overlap bought you. More repeated words increase indexed text, but the rule becomes useful only when a window carries all three required pieces.

measure-overlap-cost-and-coverage.py

def fixed_windows(text: str, size: int, overlap: int) -> list[str]:
    words = text.split()
    step = size - overlap
    return [
        " ".join(words[start : start + size])
        for start in range(0, len(words), step)
        if words[start : start + size]
    ]

def carries_damage_rule(text: str) -> bool:
    lowered = text.lower()
    return all(term in lowered for term in ["damaged electronics", "48 hours", "photos"])

policy = "Damaged electronics report within 48 hours with photos. Unopened electronics return within 14 days."
for overlap in [0, 2, 4]:
    chunks = fixed_windows(policy, size=7, overlap=overlap)
    complete = sum(carries_damage_rule(chunk) for chunk in chunks)
    indexed_words = sum(len(chunk.split()) for chunk in chunks)
    print(f"overlap={overlap}: chunks={len(chunks)} indexed_words={indexed_words} complete={complete}")

Output

overlap=0: chunks=2 indexed_words=14 complete=0
overlap=2: chunks=3 indexed_words=18 complete=0
overlap=4: chunks=5 indexed_words=28 complete=0

This is the right posture for overlap: a configuration to evaluate, not a ritual to apply to every document.

Prefer policy structure when you have it

File ingestion preserved headings and source locations. Use them. A heading boundary is more meaningful than an arbitrary word count when the source already says which rule belongs together.

split-markdown-policy-sections.py

from dataclasses import dataclass

@dataclass(frozen=True)
class PolicyChunk:
    heading: str
    body: str
    indexed_text: str
    locator: str

def headed_chunks(markdown: str, source_locator: str) -> list[PolicyChunk]:
    chunks: list[PolicyChunk] = []
    heading = "Document"
    body: list[str] = []

    def flush() -> None:
        if body:
            body_text = " ".join(body)
            chunks.append(
                PolicyChunk(
                    heading,
                    body_text,
                    f"{heading}\n{body_text}",
                    f"{source_locator}#{heading.lower().replace(' ', '-')}",
                )
            )
            body.clear()

    for line in markdown.strip().splitlines():
        if line.startswith("## "):
            flush()
            heading = line[3:]
        elif line.strip():
            body.append(line.strip())
    flush()
    return chunks

policy = """## Damaged electronics
Report within 48 hours with photos.
## Unopened electronics
Return within 14 days in original packaging."""

for chunk in headed_chunks(policy, "page=7"):
    print(chunk.indexed_text.replace("\n", " | "), "|", chunk.locator)

Output

Damaged electronics | Report within 48 hours with photos. | page=7#damaged-electronics
Unopened electronics | Return within 14 days in original packaging. | page=7#unopened-electronics

Each chunk's indexed_text includes its heading, so the searchable text keeps the condition attached to the deadline. The locator separately preserves the path back to original evidence.

A policy table needs an equally deliberate rule. Splitting a row away from its column names turns exact information into ambiguous fragments.

keep-policy-table-with-header.py

table = [
    "| Condition | Window | Evidence |",
    "| --- | --- | --- |",
    "| Damaged electronics | 48 hours | Photos |",
    "| Unopened electronics | 14 days | Original packaging |",
]

def table_chunk(lines: list[str], heading: str) -> dict[str, str]:
    return {
        "heading": heading,
        "text": "\n".join(lines),
        "quality_check": "header_present" if lines[0].startswith("| Condition |") else "review",
    }

chunk = table_chunk(table, "Return windows")
print(chunk["heading"], chunk["quality_check"], f"rows={len(table) - 2}")
print("48 hours" in chunk["text"] and "Photos" in chunk["text"])

Output

Return windows header_present rows=2
True

For a long section, split inside that section while carrying its heading and original locator into every child chunk.

split-long-section-with-provenance.py

def section_windows(text: str, heading: str, source: str, size: int) -> list[dict[str, str]]:
    words = text.split()
    return [
        {
            "text": " ".join(words[start : start + size]),
            "heading": heading,
            "source": source,
            "chunk_id": f"{source}#{heading.lower().replace(' ', '-')}-{start // size}",
        }
        for start in range(0, len(words), size)
    ]

children = section_windows(
    "Report within 48 hours with photos. Include order number and a clear image of damage.",
    heading="Damaged electronics",
    source="returns-policy-v3.pdf:page=7",
    size=7,
)

for child in children:
    print(child["chunk_id"], "|", child["heading"], "|", child["text"])

Output

returns-policy-v3.pdf:page=7#damaged-electronics-0 | Damaged electronics | Report within 48 hours with photos. Include
returns-policy-v3.pdf:page=7#damaged-electronics-1 | Damaged electronics | order number and a clear image of
returns-policy-v3.pdf:page=7#damaged-electronics-2 | Damaged electronics | damage.

Evaluate boundaries with labeled questions

A chunking strategy isn't good because it sounds sophisticated. It works when labeled questions retrieve complete supporting evidence.

Chunk	Matching query terms	Contains answer?
complete damage rule	4 / 4	yes
damaged-condition fragment	2 / 4	no
unopened-item rule	1 / 4	no

The following lexical scorer is deliberately simple. It isolates the effect of boundaries; replace its score with real embeddings after the fixture and expected evidence are stable.

retrieve-complete-evidence.py

import re

def terms(text: str) -> set[str]:
    return set(re.findall(r"[a-z0-9]+", text.lower()))

def retrieve(query: str, chunks: list[dict[str, str]]) -> dict[str, str]:
    query_terms = terms(query)
    return max(chunks, key=lambda chunk: len(query_terms & terms(chunk["text"])))

query = "damaged electronics photos hours"
chunks = [
    {"id": "damage-rule", "text": "Damaged electronics: report within 48 hours with photos."},
    {"id": "general-rule", "text": "Unopened electronics: return within 14 days."},
]

hit = retrieve(query, chunks)
print(hit["id"], hit["text"])

Output

damage-rule Damaged electronics: report within 48 hours with photos.

Now compare a broken fixed-window configuration against a section-aware configuration using the same question and the same expected evidence phrase.

compare-chunking-configurations.py

import re

def terms(text: str) -> set[str]:
    return set(re.findall(r"[a-z0-9]+", text.lower()))

def top_chunk(query: str, chunks: list[str]) -> str:
    query_terms = terms(query)
    return max(chunks, key=lambda text: len(query_terms & terms(text)))

query = "damaged electronics photos"
expected = "Damaged electronics: report within 48 hours with photos."
configs = {
    "broken-fixed": [
        "Damaged electronics: report within",
        "48 hours with photos. Unopened electronics: return within 14 days.",
    ],
    "section-aware": [
        "Damaged electronics: report within 48 hours with photos.",
        "Unopened electronics: return within 14 days.",
    ],
}

for name, chunks in configs.items():
    hit = top_chunk(query, chunks)
    print(f"{name}: complete={expected in hit}")

Output

broken-fixed: complete=False
section-aware: complete=True

One question proves little. Ship a small labeled set containing policy exceptions, tables, and boundary failures, then measure each candidate splitter on exactly that set.

run-chunk-regression-suite.py

import re

def terms(text: str) -> set[str]:
    return set(re.findall(r"[a-z0-9]+", text.lower()))

def retrieve(query: str, chunks: list[dict[str, str]]) -> dict[str, str]:
    query_terms = terms(query)
    return max(chunks, key=lambda chunk: len(query_terms & terms(chunk["text"])))

chunks = [
    {"id": "damage", "text": "Damaged electronics: report within 48 hours with photos."},
    {"id": "unopened", "text": "Unopened electronics: return within 14 days."},
    {"id": "refund", "text": "Approved refunds return to the original payment method."},
]
cases = [
    ("damaged electronics photos", "damage", "48 hours"),
    ("unopened electronics return", "unopened", "14 days"),
    ("refund payment method", "refund", "original payment"),
]

passed = 0
for query, expected_id, evidence in cases:
    hit = retrieve(query, chunks)
    ok = hit["id"] == expected_id and evidence in hit["text"]
    passed += int(ok)
    print(query, "PASS" if ok else "FAIL")
print(f"summary={passed}/{len(cases)}")

Output

damaged electronics photos PASS
unopened electronics return PASS
refund payment method PASS
summary=3/3

When you replace this lexical baseline with embeddings, the assertions stay useful: retrieve the correct source and retain text sufficient to answer.

Search small, answer with enough context

expand-child-match-to-parent-context.py

parents = {
    "damage": "Damaged electronics: report within 48 hours with photos. Agents must attach the order number.",
    "unopened": "Unopened electronics: return within 14 days in original packaging.",
}
children = [
    {"text": "48 hours with photos", "parent_id": "damage"},
    {"text": "14 days original packaging", "parent_id": "unopened"},
]

match = next(child for child in children if "photos" in child["text"])
print(match["text"])
print(parents[match["parent_id"]])

Output

48 hours with photos
Damaged electronics: report within 48 hours with photos. Agents must attach the order number.

prepend-section-context.py

def indexed_text(heading: str, text: str, source: str) -> str:
    return f"Source: {source}\nSection: {heading}\n{text}"

child = indexed_text(
    heading="Damaged electronics",
    text="Report within 48 hours with photos.",
    source="ShopFlow Returns Policy",
)
print(child)

Output

Source: ShopFlow Returns Policy
Section: Damaged electronics
Report within 48 hours with photos.

Escalate only when the baseline exposes a gap

Structure-aware chunks handle many handbooks and policy pages. Some corpora force different choices:

Failure after measuring baseline	Candidate experiment	What must still be checked
one paragraph shifts between multiple topics	semantic boundary detection	hard size cap and labeled-query score
tiny match lacks surrounding explanation	parent-child expansion	deduplication and generation budget
short child is ambiguous without its section	contextual header	retrieval comparison against no-header baseline
meaning depends on far-away document context	late chunking	model support, latency, and measured retrieval gain

find-semantic-boundary-candidates.py

import math

def vector(sentence: str) -> list[float]:
    lowered = sentence.lower()
    return [
        float(sum(word in lowered for word in ["damaged", "return", "photos", "hours"])),
        float(sum(word in lowered for word in ["refund", "payment", "approved"])),
    ]

def cosine(left: list[float], right: list[float]) -> float:
    dot = sum(a * b for a, b in zip(left, right))
    left_norm = math.sqrt(sum(a * a for a in left))
    right_norm = math.sqrt(sum(b * b for b in right))
    return dot / (left_norm * right_norm) if left_norm and right_norm else 0.0

sentences = [
    "Damaged electronics require photos.",
    "Report damaged items within 48 hours.",
    "Approved refunds return to the original payment method.",
]

for left, right in zip(sentences, sentences[1:]):
    similarity = cosine(vector(left), vector(right))
    print(f"{similarity:.2f}", "boundary" if similarity < 0.50 else "keep together")

Output

1.00 keep together
0.32 boundary

Ship a chunking decision, not a guess

For the ShopFlow policy corpus, a credible first release is straightforward:

Design choice	Initial decision	Evidence to collect
Default boundary	heading-aware sections, recursive fallback	answerable-chunk rate and retrieval regression suite
Tables	keep header plus rows together	exact-value questions preserve correct row meaning
Overlap	off for complete policy sections; test for fallback prose	index size, duplicate hits, and labeled-query results
Metadata	source, locator, heading, checksum	cited answer can return to original record
Escalation	parent expansion before semantic or late chunking	failure examples that justify extra complexity

Your release gate can be encoded as an executable manifest check.

gate-indexable-policy-chunks.py

from hashlib import sha256

chunks = [
    {
        "id": "damage",
        "text": "Damaged electronics: report within 48 hours with photos.",
        "source": "returns-policy-v3.pdf",
        "locator": "page=7#damaged-electronics",
        "heading": "Damaged electronics",
        "quality": "ready",
    },
    {
        "id": "broken",
        "text": "48 hours with photos.",
        "source": "returns-policy-v3.pdf",
        "locator": "page=7#fragment",
        "heading": "Fragment",
        "quality": "review",
    },
]

for chunk in chunks:
    chunk["checksum"] = sha256(chunk["text"].encode()).hexdigest()

required_metadata = ("source", "locator", "heading", "checksum")
indexable = [
    chunk for chunk in chunks
    if chunk["quality"] == "ready"
    and all(chunk[field] for field in required_metadata)
]
print(f"indexable={[chunk['id'] for chunk in indexable]}")
print(f"blocked={len(chunks) - len(indexable)}")

Output

indexable=['damage']
blocked=1

Mastery check

You now know how to take the clean evidence records from ingestion and turn them into retrieval units that can be evaluated.

Key concepts

A chunk is a searchable evidence unit, not an arbitrary slice of text.
Fixed windows expose boundary and overlap costs clearly.
Structure-aware splitting is a strong baseline when headings or tables carry meaning.
Parent-child expansion separates precise search from sufficient answer context.
Semantic or late chunking should be escalations justified by measured failures.
A labeled retrieval set must check both correct selection and answerable evidence.

Evaluation rubric

Foundational: Identifies why a broken chunk can't answer the damaged-electronics question
Foundational: Implements fixed windows and explains what overlap repeats
Intermediate: Preserves headings, tables, source IDs, and locators in chunk records
Intermediate: Compares candidate boundaries on labeled retrieval cases
Intermediate: Uses parent-child expansion when narrow retrieval lacks sufficient context
Advanced: Chooses semantic or late chunking only after measuring a baseline failure

Follow-up questions

Common pitfalls

A deadline loses its condition: A window retrieves 48 hours without Damaged electronics. Fix: split at section boundaries and assert answerability.
A table row loses its header: A price or return window becomes ambiguous. Fix: keep column names with rows and test exact-value questions.
Overlap creates duplicates without fixing answers: Repeated chunks dominate top results. Fix: measure overlap against both complete evidence and index cost.
Chunks can't be cited: Search looks plausible but support can't defend the answer. Fix: retain source, locator, heading, and checksum metadata.
An advanced splitter is adopted by reputation: Complexity rises without better results. Fix: preserve a simple baseline and evaluate every escalation on the same labeled set.

Next Step

Continue to LLM Benchmarks & Limitations

You can now build evidence-level retrieval tests for a RAG system. Next you will learn why evaluation sets and scores need the same care before they can support a model-quality claim.

PreviousFile Ingestion for AI

Share this article

X Facebook LinkedIn Bluesky Reddit Hacker News Email

References

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks.

Lewis, P., et al. · 2020 · NeurIPS 2020

RecursiveCharacterTextSplitter

LangChain · 2023

Late Chunking: Contextual Chunk Embeddings Using Long-Context Embedding Models.

Günther, M., et al. · 2024 · arXiv preprint

Chunking Strategies

What a useful chunk must preserve

Fixed windows show the boundary failure

Prefer policy structure when you have it

Evaluate boundaries with labeled questions

Search small, answer with enough context

Escalate only when the baseline exposes a gap

Ship a chunking decision, not a guess

Mastery check

Key concepts

Evaluation rubric

Follow-up questions

Why is a chunk containing 48 hours with photos insufficient by itself?

When should overlap be set to zero?

Why retain the locator from the ingestion record in every chunk?

What problem does parent-child retrieval solve?

What evidence justifies switching from structure-aware splitting to late chunking?

Common pitfalls

Chunking Strategies

What a useful chunk must preserve

Fixed windows show the boundary failure

Prefer policy structure when you have it

Evaluate boundaries with labeled questions

Search small, answer with enough context

Escalate only when the baseline exposes a gap

Ship a chunking decision, not a guess

Mastery check

Key concepts

Evaluation rubric

Follow-up questions

Why is a chunk containing 48 hours with photos insufficient by itself?

When should overlap be set to zero?

Why retain the locator from the ingestion record in every chunk?

What problem does parent-child retrieval solve?

What evidence justifies switching from structure-aware splitting to late chunking?

Common pitfalls