GraphRAG & Knowledge Graphs

Advanced RAG made retrieval adaptive: rewrite the query, generate hypothetical evidence, critique retrieved context, and retry when evidence is weak. GraphRAG changes a different layer. It indexes relationship structure and summaries alongside text so a system can retrieve connected evidence or corpus-level themes, at additional extraction and query cost. This chapter explains when that tradeoff complements vector search and when it adds needless complexity.

Imagine running an online store with 50,000 customer support tickets. A shopper asks: "Why did my order take twelve days when the website promised five?" A simple retrieval system can find the ticket about that delay. But when an analyst asks: "What are the top three reasons our deliveries miss the promised window?" a small top-k result set may represent only a few incidents. It doesn't itself provide coverage over recurring themes across the corpus.

Microsoft's GraphRAG architecture addresses that query class by building a graph-based index of entities and relationships, pregenerating hierarchical community reports, and embedding text artifacts for retrieval ^[1][2][3]. In the original paper's global sensemaking evaluation, GraphRAG improved answer comprehensiveness and diversity over its vector RAG baseline ^[1]. Its index also supplies structured artifacts for entity-focused local context construction. A broader roadmap by Pan et al. places this in the wider effort to combine LLMs with structured knowledge graphs ^[4].

The mental model is simple: vector RAG is a filing cabinet; GraphRAG is a case board. Vector search finds chunks that look like the question. A graph index records connections and summary layers that a query method can use when evidence spans multiple documents. Standard GraphRAG local search builds ranked mixed context from graph and text artifacts; a product that needs explicit path traversal must implement and evaluate that behavior.

Why small top-k retrieval can be insufficient

The global query problem

Consider a corpus of 10,000 e-commerce support tickets. We'll use this as our running example throughout the article.

Local queries (vector search handles well)

For a specific question about a single issue, vector search excels:

"What's the return policy for opened electronics?" Retrieval path: top-5 similar tickets, then answer from local context. This works because the needed evidence is concentrated in a few chunks.

The question is self-contained. A few chunks about returns and electronics contain everything the system needs.

Global queries (small top-k retrieval can miss coverage)

For a question that requires synthesizing across the entire corpus, vector search struggles:

"What are the top 3 recurring reasons orders miss the delivery promise?" Failure mode: no single chunk contains a cross-corpus summary, and a top-5 result set is unlikely to represent themes across 10,000 tickets. The resulting answer can be incomplete or misleading.

Why similarity-only retrieval struggles

Vector search retrieves the most similar chunks, not necessarily the most representative ones. When answering a global query, no single chunk may contain the complete answer. One way to create broader retrieval units is to add:

1. Entity extraction across the entire corpus
2. Relationship modeling between entities
3. Community detection to find natural clusters of related information
4. Hierarchical summarization at different levels of abstraction

A concrete multi-hop example

Before we define terms, let's see the difference in action. Here's a tiny support corpus:

Ticket 1842: "Order #9021 shipped via FastCarrier. The package sat at the Memphis hub for five days because of a winter storm." Ticket 2031: "FastCarrier's Memphis hub uses outdated sort scanners. Replacement parts are back-ordered." Ticket 3155: "Order #9021 arrived twelve days late. Customer wants a refund on shipping fees."

Now ask:

"Why did Order #9021 arrive late, and is this likely to happen again?"

Illustrative vector-only result: Suppose retrieval finds Ticket 3155 (mentions "Order #9021" and "late") and Ticket 1842 (mentions "Order #9021" and "Memphis hub"), but not Ticket 2031 because it doesn't mention the order number. The answer can identify the weather delay yet miss the scanner problem.

Graph-enriched context: Suppose extraction produced these entities and relationships, with source-chunk provenance:

• (Order #9021) --[shipped_via]--> (FastCarrier)
• (Order #9021) --[delayed_at]--> (Memphis Hub)
• (Memphis Hub) --[operated_by]--> (FastCarrier)
• (Memphis Hub) --[affected_by]--> (Winter Storm)
• (Memphis Hub) --[has_issue]--> (Outdated Scanners)
• (Outdated Scanners) --[status]--> (Back-ordered Parts)

An implementation that supports evidence-backed traversal could start at Order #9021, follow delayed_at to Memphis Hub, and retrieve evidence about both Winter Storm and Outdated Scanners. It may then answer: "Order #9021 was delayed at the Memphis hub. The source material mentions a winter storm and also an unresolved scanner issue." It should not predict recurring delays unless source evidence supports that claim.

This is relationship-heavy retrieval: useful evidence is connected through entities that weren't all present in the query. Microsoft's standard local-search dataflow ranks graph and text artifacts into a context window; it isn't a promise that every answer executes a deterministic path. If explicit multi-hop paths matter, preserve edge provenance and test the traversal policy.

What is a knowledge graph?

A knowledge graph represents information as a network of entities (nodes) and relationships (edges). Unlike a flat vector database that stores text chunks as isolated embeddings, a knowledge graph preserves the structure of how facts connect.

The fundamental unit is the triple: (Subject) - [Predicate] -> (Object). The edge direction should reflect the meaning of the relation itself, not some generic left-to-right convention.

From our running example:

• (Order #9021) --[shipped_via]--> (FastCarrier)
• (Memphis Hub) --[has_issue]--> (Outdated Scanners)
• (Outdated Scanners) --[status]--> (Back-ordered Parts)

In production, you're usually working with a property graph rather than bare triples. That means nodes and edges carry metadata like type, description, source_chunk_ids, timestamps, or confidence scores. Those extra fields matter for filtering, provenance, and ranking during retrieval.

This graph structure mirrors how operations teams reason through associations. When a query mentions "Memphis hub," a retrieval system can use typed relationships and provenance to distinguish carrier delays, scanner outages, return-center capacity, and inventory transfers instead of relying on city-name similarity alone.

The GraphRAG pipeline

Phase 1: Graph construction (indexing)

The indexing phase transforms raw document chunks into a structured graph index. In Microsoft's GraphRAG docs, the standard pipeline extracts entities and relationships, optionally extracts claims, builds hierarchical communities, generates community reports, and embeds the resulting artifacts for downstream retrieval ^[2][3]:

Step 1: Entity & relationship extraction

The first step is to pass each document chunk to an extraction system that identifies key entities and the relationships between them. In standard GraphRAG, this is LLM-driven. In FastGraphRAG, some of this work is replaced with cheaper NLP-based heuristics to cut indexing cost ^[5]. A simple extraction prompt looks like this:

In production, don't rely on raw string prompting for JSON. Use structured output frameworks like Instructor (Python) or Zod (TypeScript) with function calling or tool-use APIs to enforce schema adherence. This prevents syntax errors when processing thousands of chunks.

The runnable version below uses a deterministic extractor so you can test the graph contract without calling an LLM. In production, replace extract_from_chunk() with a structured-output model call and keep the same Entity and Relationship objects.

Step 2: Build the knowledge graph

Once the entities and relationships are extracted and deduplicated, we can construct the graph data structure. Mapping isolated facts from individual chunks into an interconnected network gives later retrieval policies access to relationships that weren't explicit in any single chunk.

This function takes the lists of entities and relationships as input. Using a graph processing library like NetworkX, it adds each entity as a node and each typed relationship as a directed edge with provenance and weighting metadata:

Step 3: Community detection (Leiden algorithm)

After the graph is built, we need to find natural clusters of related entities. Think of it like grouping the support tickets into themes: shipping problems, refund problems, product defects. The difference is that the algorithm decides the groups based on how densely entities connect to each other, not on keyword matching.

The Leiden algorithm identifies these clusters. It's preferred over the classic Louvain method because Louvain can produce disconnected communities (nodes in the same group that aren't connected by edges). Traag et al. show that Leiden adds a refinement phase and guarantees connected communities ^[6]. That avoids a structural defect before report generation, although it can't correct bad entity extraction or unsupported edges. Microsoft's GraphRAG pipeline applies hierarchical Leiden recursively until communities hit a size threshold, which is how it gets both coarse and fine-grained views of the same corpus ^[3].

The algorithm optimizes modularity ($Q$), a measure of community structure quality in the extracted graph. In plain terms, modularity asks: "Are there more edges inside each group than we'd expect by chance?" A high $Q$ indicates strong clustering in that graph; it doesn't prove that extraction captured the real-world topic correctly.

For an unweighted, undirected graph, the formula is:

$Q = \frac{1}{2m} \sum_{i,j} \left[ A_{ij} - \frac{k_i k_j}{2m} \right] \delta(c_i, c_j)$

Where $A_{ij}$ is 1 if nodes $i$ and $j$ are directly connected and 0 otherwise; $k_i$ and $k_j$ are the number of edges each node touches; $m$ is the total number of edges in the graph; and $\delta(c_i, c_j)$ is 1 if the two nodes are in the same community, 0 otherwise. Higher modularity indicates better community structure.

The runnable example below computes one weighted Leiden partition with python-igraph. Standard GraphRAG recursively applies community detection to build a hierarchy; keeping this example at one level makes the clustering contract visible and keeps it fast enough to run locally.

The following diagram visualizes how entities are grouped into hierarchical communities, from specific concepts at the bottom to broad themes at the top:

Step 4: Community summarization

This function takes the entities, their descriptions, and their relationships within a community as input. It passes these to a summarizer, returning a community report that can later be used for global search ^[1][3]:

Phase 2: Query processing (runtime)

GraphRAG's query engine exposes Basic Search, Local Search, Global Search, and DRIFT Search. Local and global search are the core mental model here: local search builds context for entity-centric questions, global search synthesizes over community reports, Basic Search is a text-focused baseline, and DRIFT combines community-level entry points with local exploration ^[7][8][9].

Local search (specific questions)

Local search isn't just "do NER on the query and grab neighboring nodes." In Microsoft's GraphRAG docs, local search maps the query into semantically related entities, then prioritizes a mixed context from connected entities, relationships, community reports, linked text units, and optionally covariates if claim extraction is enabled ^[7].

The runnable example below demonstrates a bounded neighborhood expansion for intuition. It's a custom graph-walk sketch, not an exact reproduction of the GraphRAG context builder or its ranking policy:

The actual context-builder problem includes a token budget: relevant entities, relationships, text units, and reports compete for room in one generation request. A production route needs ranking and provenance, not unbounded expansion.

Global search (broad questions)

For queries that require cross-corpus understanding, GraphRAG uses a map-reduce pattern over community reports from a selected hierarchy level. The global-search docs batch community reports into chunks, produce rated intermediate points during the map step, then aggregate the highest-value points in the reduce step ^[1][8]:

Hybrid graph-vector architecture

A useful production design doesn't treat vector search and GraphRAG as mutually exclusive. The GraphRAG stack already mixes graph structure with embeddings during both indexing and query-time context building ^[2][7]. One hybrid architecture looks like this:

A router is a policy to evaluate, not a guarantee. Measure supported-answer accuracy, latency, and spend for each query class. In a workload dominated by direct evidence lookups, a text route may cover most requests; an analyst-heavy workload may justify more graph/report queries.

One subtle but important point: GraphRAG the technique doesn't require a dedicated graph database. Microsoft's reference implementation writes structured output tables to disk and builds query context from those artifacts directly. A graph database like Neo4j or Neptune becomes useful when you need custom traversals, shared KG infrastructure, or analyst-facing graph queries outside the stock pipeline ^[10].

Query expansion and context bridging

Beyond simple routing, the knowledge graph layer can actively enrich vector results.

Query expansion uses entities found in initial vector results to expand the search query, finding semantically related but textually distinct content.

Context bridging is a custom retrieval option when two retrieved chunks don't directly connect. If edges retain supporting chunks, a traversal can identify intermediate entities for retrieval and citation. For example, if one chunk mentions "Order #9021 shipped via FastCarrier" and another mentions "FastCarrier's Memphis hub has outdated scanners," a policy can inspect Order #9021 -> FastCarrier -> Memphis Hub -> Outdated Scanners, then fetch source text before making a claim about the order.

Result ranking can boost text results with short, supported paths to query entities. Treat this as a ranker feature to evaluate, not proof that a nearby node supports the answer.

Performance considerations

Indexing cost

The most significant barrier to adopting standard GraphRAG is the upfront indexing cost. Compared with vector-only RAG, its standard pipeline adds LLM-heavy graph extraction, summarization, community report generation, and multiple embedding passes ^[1][3]. Whether the resulting query quality justifies that work is an evaluation question.

Mitigation strategies

To make this viable at scale, teams need to optimize the ingestion pipeline. Microsoft explicitly recommends starting with fast, inexpensive models while you learn the system, and their docs estimate that graph extraction (entity and relationship extraction plus their summarization) is roughly 75% of standard indexing cost ^[11][5]. If your use case is mostly global summarization, FastGraphRAG can reduce cost further: it replaces LLM-based entity extraction with NLP noun-phrase extraction (using libraries like NLTK or spaCy) and defines relationships by entity co-occurrence within a text unit. The graph is noisier and less reusable outside GraphRAG, but indexing is much cheaper ^[5].

LazyGraphRAG in practice

Microsoft Research introduced LazyGraphRAG in November 2024 ^[12]. It targets a central cost concern with standard GraphRAG: paying for an LLM-driven index before knowing how often graph-assisted queries will run.

LazyGraphRAG defers LLM use to query time. Its index uses NLP noun-phrase extraction and graph statistics for community structure, without entity summaries or precomputed community reports. In Microsoft's reported experiment, its indexing cost matched the vector RAG setup and was approximately 0.1% of full GraphRAG indexing cost ^[12]. Treat that as a benchmark result, not a universal constant for every corpus and deployment. At query time it blends vector similarity with community structure and exposes a relevance test budget that trades query cost for quality. Microsoft reports that, at evaluated budget levels, LazyGraphRAG was competitive with or better than the tested alternatives on local and global query criteria ^[12].

The practical takeaway is that precomputed community reports aren't the only candidate for global sensemaking. If the corpus changes frequently or global queries are rare, benchmark a lazy approach against standard GraphRAG. If reusable community reports are a product output, the standard index provides artifacts the lazy path intentionally omits.

Use observed quality and workload volume to compare indexing strategies rather than choosing from architecture labels:

Query cost

While indexing represents the bulk of the computational expense, runtime query costs can also be higher than standard vector search. The cost depends heavily on the query strategy used:

• Local search: Usually moderate. You still have to build a mixed context from entity embeddings, graph neighborhoods, text units, and community reports, but the response path is much narrower than global search ^[7].
• Global search: Potentially expensive. Cost grows with the number of community-report batches you need to map over and the hierarchy level you choose ^[8].

Mitigation strategies

To manage these runtime costs, standard GraphRAG precomputes community reports at multiple hierarchical levels. At query time, evaluate whether a coarse level answers the question adequately before paying for more detailed reports. Lower levels tend to yield more thorough responses, but they can also increase report volume and LLM work ^[8].

Graph maintenance

Knowledge graphs aren't static; they must evolve as the underlying document corpus changes. Keeping the graph in sync with a live, mutating dataset introduces significant engineering complexity. Current GraphRAG releases now expose explicit update flows and standard-update / fast-update methods, and the output tables include fields used for incremental update merges ^[13][10].

• Additions: When new documents arrive, the system must extract entities and relationships, then merge them into the existing graph. Significant new connections may shift community structure, requiring reclustering and regeneration of affected reports.
• Deletions: Removing a document isn't as simple as deleting a row in a database. The system must trace and remove nodes or edges supported solely by that document. If removed evidence changes graph structure, communities and reports may need recomputation.
• Strategy: Even with update support, a deployment may prefer scheduled refreshes for structural artifacts. Entity merges, community boundaries, and report summaries can shift when new documents arrive, so online reclustering is harder to operate than plain vector re-indexing.

When to use GraphRAG

Because of the high indexing and maintenance costs, GraphRAG shouldn't be treated as a default replacement for all retrieval tasks; it's a specialized tool for complex analytical workloads. Evaluating the query profile of your application is essential before committing to a knowledge graph architecture. The following decision matrix highlights where each approach shines:

Use this framework to decide:

The pragmatic path: Start with vector search. It's cheaper, faster, and easier to maintain. Only add the graph layer when you hit specific limitations:

1. Users ask questions that require synthesizing information from many documents
2. You need to trace influence chains (who reports to whom, what depends on what)
3. Global sensemaking queries are a core use case, not an edge case
4. The corpus is large enough to justify the indexing investment

Mastery check

By the end of this lesson, you should be able to explain GraphRAG as an engineering tradeoff, not as a buzzword.

Key concepts

• GraphRAG adds entities, typed relationships, and community reports on top of chunk retrieval.
• Local search answers entity-centric questions with mixed graph and text context.
• Global search answers corpus-level questions with map-reduce over community reports.
• Hierarchical Leiden avoids disconnected communities before report generation; extraction quality still determines whether reports are useful.
• LazyGraphRAG shifts most LLM cost from indexing time to query time.

Evaluation rubric

• Foundational: Explain why small top-k similarity retrieval can underrepresent global sensemaking queries.
• Intermediate: Describe the GraphRAG indexing pipeline: entity extraction, relationship extraction, entity and relationship summarization, community detection, community reports, and retrieval-time embeddings.
• Advanced: Explain why hierarchical Leiden matters for community detection and report generation.
• Advanced: Compare Local Search, which builds mixed entity/text/report context, with Global Search, which map-reduces over community reports.
• Advanced: Analyze cost and latency tradeoffs: expensive indexing buys query capabilities that plain vector search doesn't have.
• Advanced: Design a hybrid graph-vector architecture where simple lookups stay cheap and structural questions get graph context.
• Advanced: Explain how to compare LazyGraphRAG with a precomputed-report pipeline using measured quality, query volume, and index/query cost.

Follow-up questions

Common pitfalls

When engineers first encounter knowledge graphs and GraphRAG, they often bring assumptions from traditional vector search or long-context LLMs. Here are the most common traps, what they look like in practice, and how to avoid them.

Mistake 1: "Use a bigger context window"

Symptom: You stuff 100,000 tokens into a long-context model and ask for a summary. The output misses key themes or contradicts itself.

Cause: A 1M token context window doesn't automatically solve global sensemaking. You still have to decide what to include, very large prompts are expensive to run, and models can still underuse information buried in the middle of long contexts ^[14].

Fix: Benchmark selective retrieval and structured indexing against long-context prompting. Standard GraphRAG is one candidate when repeated global questions justify precomputed reports; it isn't required for every long document.

Mistake 2: Treating GraphRAG as a replacement for vector search

Symptom: You replace your entire vector index with a knowledge graph. Simple lookups become slow and expensive.

Cause: GraphRAG is complementary, not substitutive. Local search uses entity-description embeddings and linked text units as part of context construction. Global search adds report synthesis for questions that require coverage across many documents.

Fix: Use a hybrid router. Route simple fact lookups to vector search, entity questions to local search, and global summaries to global search.

Mistake 3: Ignoring entity resolution

Symptom: Your graph has three separate nodes for "FastCarrier," "Fast Carrier," and "fastcarrier.com." Queries that should traverse through the carrier fail because the path is broken.

Cause: Entity extraction from unstructured text is inherently noisy. Skipping deduplication leaves near-duplicate nodes that fragment the graph.

Fix: Implement a three-stage resolution pipeline: (1) normalize names (lowercase, strip punctuation), (2) use embeddings or string distance to find candidate duplicates, (3) use a lightweight LLM verification step to confirm whether two names refer to the same real-world entity before merging.

Mistake 4: Extracting every noun as a node

Symptom: The graph balloons to millions of nodes, most of them useless. Common words like "customer," "order," and "email" become nodes with thousands of meaningless edges.

Cause: Without a schema, the extractor treats every noun as an entity.

Fix: Define an ontology (a typed schema) before extraction. Restrict nodes to domain-relevant types like Order, Carrier, Warehouse, ProductDefect, and RefundPolicy. Filter low-information entities in a post-processing step.

Mistake 5: Underestimating indexing cost

Symptom: The prototype works on 200 documents, then the first full-corpus run burns budget on extraction, summarization, and embedding passes.

Cause: Standard GraphRAG does much more than embed chunks. It asks models to extract entities, extract relationships, summarize repeated descriptions, generate community reports, and embed several downstream artifacts.

Fix: Measure indexing cost before committing to the architecture. Start with a small representative corpus, use inexpensive models while tuning prompts, consider FastGraphRAG for global-summary-heavy workloads, and set a refresh cadence instead of pretending every graph update is free.

Mistake 6: Ignoring graph maintenance

Symptom: Search results become stale or paths break after documents are added, deleted, or corrected.

Cause: A graph index has structure. New evidence can merge entities, change edge weights, move community boundaries, and invalidate old community reports.

Fix: Treat graph refresh as a product requirement. Track provenance with text_unit_ids, run incremental update flows where they are good enough, and schedule full rebuilds when entity resolution or community boundaries drift.

Try it yourself

Here's a short exercise to check your understanding. Work through it on paper or in a text editor before reading the solution sketch.

The corpus

Doc A: "Warehouse B handles returns for the West Coast. Warehouse B recently switched to AutoSort robots." Doc B: "AutoSort robots sometimes mislabel fragile items." Doc C: "Order #7712 (fragile glassware) was routed to Warehouse B, arrived broken, and received an approved refund."

The question: "Why did Order #7712 arrive broken, and what systemic issue might cause similar problems?"

Step 1: List the entities and relationships you'd extract from each document.

Step 2: Draw (or list) the triples that connect Order #7712 to the systemic issue.

Step 3: Explain why standard vector RAG might miss the systemic issue.

Solution sketch

Entities: Warehouse B, AutoSort robots, Order #7712, fragile glassware, refund.

Key triples

• (Order #7712) --[contains]--> (fragile glassware)
• (Order #7712) --[routed_to]--> (Warehouse B)
• (Warehouse B) --[uses]--> (AutoSort robots)
• (AutoSort robots) --[problem]--> (mislabels fragile items)
• (Order #7712) --[result]--> (arrived broken)

Multi-hop path: Order #7712 --[routed_to]--> Warehouse B --[uses]--> AutoSort robots --[problem]--> mislabels fragile items. The contains -> fragile glassware edge explains why that robot failure is relevant to this order.

Why similarity retrieval may miss it: No single document contains both "Order #7712" and "mislabels fragile items." A search for "Why did Order #7712 arrive broken?" may retrieve Doc C while missing Docs A and B because neither mentions the order. If extraction is accurate, a graph makes the path from order to warehouse to robot issue explicit and retrievable with provenance.

Key takeaways

1. GraphRAG addresses a global query problem that small top-k vector retrieval can handle poorly. Standard GraphRAG adds structure and precomputed summaries for corpus-level synthesis.

2. The pipeline is more than graph extraction. Standard GraphRAG extracts entities and relationships, can optionally extract claims, builds hierarchical Leiden communities, generates community reports, and embeds multiple artifacts for retrieval.

3. Local and global search are the core mental model. Local search (entity-centric + vector) handles specific questions like "What's the refund policy for opened electronics?" Global search (map-reduce over community summaries) handles broad questions like "What are the main themes?" DRIFT adds a hybrid middle ground, and Basic Search remains the plain vector fallback.

4. Indexing is expensive, and query cost depends on the search mode. Local search is usually manageable. Global search can get expensive if you have to map over many community-report batches.

5. Hybrid routing is an evaluation choice. Test simple questions on text retrieval and structural questions on graph/report context, then release routes that meet support, cost, and latency targets.

6. Start with vector search, add graphs when needed. GraphRAG shines for complex, interconnected datasets and global sensemaking. For simple fact retrieval, vector search remains faster and more cost-effective.

7. The expensive standard index isn't the only option. LazyGraphRAG (2024) moves LLM work to query time; Microsoft reported vector-RAG-like indexing cost in its comparison. Benchmark lazy and precomputed-report approaches on your query mix rather than assuming that ratio transfers unchanged.