GraphRAG & Knowledge Graphs

An AI platform has 10,000 incident reports, runbooks, and trace summaries. An on-call engineer asks: "Why did inference-api breach the latency service-level objective (SLO) during release 2026.06.14?" Ordinary retrieval can find the write-up about that service.

Then an analyst asks: "What are the top three recurring reasons model-serving incidents breach latency SLOs?" A small top-k result set can look convincing while covering only a few incidents. Similarity doesn't by itself give you theme coverage over the whole corpus.

The previous chapter made retrieval-augmented generation (RAG) adaptive: rewrite the query, generate hypothetical evidence, critique retrieved context, and retry when evidence is weak. GraphRAG changes the representation layer. It indexes relationship structure and summaries alongside text so a system can retrieve connected evidence or corpus-level themes, at extra extraction and query cost.

The engineering question is when that tradeoff complements vector search and when it adds work you don't need.

Microsoft's GraphRAG architecture builds a graph index of entities and relationships, pregenerates hierarchical community reports, and embeds those artifacts for retrieval ^{[1]Reference 1From Local to Global: A Graph RAG Approach to Query-Focused Summarization.https://arxiv.org/abs/2404.16130[2]Reference 2GraphRAG Indexing Overviewhttps://microsoft.github.io/graphrag/index/overview/[3]Reference 3GraphRAG Default Dataflowhttps://microsoft.github.io/graphrag/index/default_dataflow/}. The same index supplies structured artifacts for entity-focused local context.

In the original paper's global-sensemaking evaluation on podcast and news corpora in the roughly 1 million token range, GraphRAG beat a vector RAG baseline on LLM-judge comprehensiveness (about 72-83% win rate) and diversity (about 62-82% win rate) ^{[1]Reference 1From Local to Global: A Graph RAG Approach to Query-Focused Summarization.https://arxiv.org/abs/2404.16130}. That's a result for that query class and those datasets, not a promise that every product question wants a graph.

A broader roadmap by Pan et al. places this work in the wider effort to combine LLMs with structured knowledge graphs ^{[4]Reference 4Unifying Large Language Models and Knowledge Graphs: A Roadmap.https://arxiv.org/abs/2306.08302}.

Vector search ranks chunks that resemble the question. A graph index adds typed entity relationships and summary layers for cases where evidence spans multiple incidents. Standard GraphRAG local search builds ranked mixed context from graph and text artifacts. If your product needs an explicit path walk, you have to implement that policy and test it. The graph isn't a free multi-hop oracle.

Why small top-k retrieval can be insufficient

The global query problem

Keep those 10,000 incident reports, runbooks, traces, and postmortems as the running corpus. First ask what shape of evidence each question needs.

Local queries (vector search handles well)

For a specific question about one issue, vector search usually earns its keep:

"What is the timeout for the embeddings API?" Retrieval path: top-5 similar documents, then answer from local context. This works because the needed evidence is concentrated in a few chunks.

The question is self-contained, so a few chunks about the API contract and runbook contain everything the system needs.

Global queries (small top-k retrieval can miss coverage)

For a question that must synthesize across the entire corpus, small top-k vector retrieval has a different problem:

"What are the top 3 recurring reasons model-serving incidents breach the latency SLO?" Failure mode: no single chunk contains a cross-corpus summary, and a top-5 result set is unlikely to represent themes across 10,000 documents. The resulting answer can be incomplete or misleading.

Before raising k, predict what changes: coverage may grow, but top-k still has no mechanism to count or summarize every theme in 10,000 documents.

Why similarity-only retrieval struggles

Vector search retrieves the most similar chunks, not the most representative set. For "top three recurring SLO reasons," no single incident write-up is the answer. The index has to turn many local mentions into structures that can be grouped and summarized. GraphRAG builds those broader units in four offline steps:

1. Entity extraction across the entire corpus
2. Relationship modeling between entities
3. Community detection to find natural clusters of related information
4. Hierarchical summarization at different levels of abstraction

A concrete multi-hop example

Before naming graph pieces, predict what each route can see. Compare the two approaches on this tiny incident corpus:

Incident 1842: "inference-api release 2026.06.14 ran in us-east-1 and hit p95 latency after redis-cache began evicting hot keys." Incident 2031: "redis-cache in us-east-1 still uses the old maxmemory policy. The config migration is blocked by a compatibility test." Incident 3155: "inference-api breached the inference latency SLO twelve minutes after deploy. Rollback restored latency."

The decision question is:

"Why did inference-api breach the latency SLO, and is this likely to happen again?"

Illustrative vector-only result: Suppose retrieval finds Incident 3155 (mentions "inference-api" and "latency SLO") and Incident 1842 (mentions "inference-api" and "redis-cache"), but not Incident 2031 because it doesn't mention the service name.

That answer can identify the rollout and cache eviction while missing the unresolved maxmemory policy.

Now add graph structure. Suppose extraction produced these entities and relationships, with source-chunk provenance:

• (inference-api) --[ran_in]--> (us-east-1)
• (inference-api) --[depends_on]--> (redis-cache)
• (inference-api) --[triggered_by]--> (release 2026.06.14)
• (redis-cache) --[located_in]--> (us-east-1)
• (redis-cache) --[has_issue]--> (old maxmemory policy)
• (old maxmemory policy) --[blocked_by]--> (compatibility test)

An evidence-backed traversal can start at inference-api, follow depends_on to redis-cache, and retrieve evidence about both the eviction event and the unresolved maxmemory policy.

It may then answer: "inference-api breached the latency SLO after a rollout, and source material ties the incident to cache evictions plus an unresolved cache policy." It shouldn't predict another breach unless source evidence supports that claim.

This is relationship-heavy retrieval: useful evidence is connected through entities that weren't all present in the query.

Microsoft's standard local-search dataflow ranks graph and text artifacts into a context window; it isn't a promise that every answer executes a deterministic path. If explicit multi-hop paths matter, preserve edge provenance and test the traversal policy.

Entities, edges, and provenance

A knowledge graph represents information as a network of entities (nodes) and relationships (edges). A vector index stores chunks as isolated embeddings.

The graph's extra value is the connection between facts.

The basic unit is the triple: (Subject) - [Predicate] -> (Object). Edge direction should follow the meaning of the relation, not a generic left-to-right habit.

From our running example:

• (inference-api) --[depends_on]--> (redis-cache)
• (redis-cache) --[has_issue]--> (old maxmemory policy)
• (old maxmemory policy) --[blocked_by]--> (compatibility test)

A live index is usually a property graph rather than bare triples. Nodes and edges carry metadata such as type, description, source_chunk_ids, timestamps, or confidence.

Those fields are what you filter, cite, and rank at query time.

Give every entity a stable identity such as (canonical_title, canonical_type), not a title-only key. Keep the source and target type on each relationship, resolve both endpoints through that key, and merge repeated edges only after resolution. If a malformed edge omits a type, accept the name only when it maps to one candidate; reject ambiguous or unknown references so a convenient first match can't connect the wrong nodes.

This graph structure mirrors how an on-call engineer traces an incident. When a query mentions redis-cache, typed relationships and provenance can separate service dependencies, region-specific incidents, config drift, and release side effects.

Service-name similarity alone can't do that.

Graph traversal isn't learned message passing

Following inference-api -> redis-cache -> old maxmemory policy traverses existing relationships and returns their cited source chunks. That lookup doesn't train a model or update a node embedding.

A graph neural network (GNN) adds a separate learned model that combines node features, edges, and usually neighboring information to predict properties such as incident impact or service relevance.^{[5]Reference 5Semi-Supervised Classification with Graph Convolutional Networkshttps://arxiv.org/abs/1609.02907}

A message-passing neural network (MPNN) makes that distinction concrete: each layer computes messages from neighboring node features, aggregates them, and updates the destination node's learned representation.^{[6]Reference 6Neural Message Passing for Quantum Chemistryhttps://arxiv.org/abs/1704.01212}

Suppose redis-cache and another dependency contribute scalar messages 0.4 and 0.8; mean aggregation produces 0.6, which becomes an input to the next inference-api update. That learned update can help rank candidates, but it doesn't establish source provenance or prove a predicted relationship is true.

Standard GraphRAG can extract entities, follow stored edges, cluster communities, and prepare reports without training either a GNN or an MPNN.

What makes a knowledge graph useful beyond storing triples?

Answer

The graph connects entities through typed relationships and provenance. Retrieval can then follow meaningful paths, filter by metadata, and explain which source chunks support the path.

The GraphRAG pipeline

Phase 1: Graph construction (indexing)

The indexing phase turns raw document chunks into a structured graph index.

Microsoft's default dataflow extracts entities and relationships, optionally extracts claims (covariates), uses entity resolution to identify descriptions of the same object, builds hierarchical communities, generates community reports, and embeds the artifacts used at query time ^{[2]Reference 2GraphRAG Indexing Overviewhttps://microsoft.github.io/graphrag/index/overview/[3]Reference 3GraphRAG Default Dataflowhttps://microsoft.github.io/graphrag/index/default_dataflow/}:

Step 1: Entity & relationship extraction

Start with one text unit. An extractor names entities and the relationships between them. Standard GraphRAG does this with an LLM. FastGraphRAG replaces that step with cheaper NLP heuristics so indexing costs less ^{[7]Reference 7GraphRAG Indexing Methodshttps://microsoft.github.io/graphrag/index/methods/}. A simple extraction prompt looks like this:

That prompt isn't a parser. Use structured output (JSON schema or tool calling) so a missing brace doesn't kill a 10,000-chunk index job. Keep the contract the same either way: typed entities, typed edge endpoints, and source chunk IDs. An edge that says only source: "gateway" is incomplete when both a Service and a Datastore use that title.

The runnable version below uses a deterministic extractor so you can test that contract without calling a model. When you wire a real extractor, replace extract_from_chunk() with a structured-output call and keep the same Entity and Relationship objects.

After per-chunk extraction, GraphRAG merges entities that share a title and type, concatenates their descriptions, then asks a model to compress that list into one description per entity and relationship ^{[3]Reference 3GraphRAG Default Dataflowhttps://microsoft.github.io/graphrag/index/default_dataflow/}. The merge below does the bookkeeping; the LLM summary is the step you'd swap in later. It uses a typed canonical key, carries endpoint types on every edge, and fails closed when an untyped name matches more than one entity.

Why should extracted relationships store source chunk IDs and endpoint types?

Answer

Graph paths still need evidence and unambiguous endpoints. Source chunk IDs let the answer cite the text that produced each edge, while endpoint types let the resolver distinguish same-titled entities such as a Service and a Datastore. If an edge omits a type and the title has multiple candidates, reject the edge instead of guessing.

Step 2: Build the knowledge graph

Once entities and relationships are extracted and merged, store them as a directed property graph. Nodes hold type and description; edges hold relation type, weight, and source chunk IDs.

Later retrieval can follow relationships that never appeared together in one chunk.

The next function takes those lists and builds a small adjacency structure. It keeps typed canonical IDs internally and uses display names only when printing a result. Production GraphRAG writes the same facts to tables (entities, relationships, text units) rather than requiring a graph database ^{[8]Reference 8GraphRAG Outputshttps://microsoft.github.io/graphrag/index/outputs/}:

Step 3: Community detection (Leiden algorithm)

Retrieval walks the directed property graph. Community detection usually clusters an undirected (often weighted) view of the same nodes: ignore edge type for clustering, then keep it for later context packing.

In this corpus, dense clusters become themes such as cache pressure versus eval flakiness. The groups come from connection density, not keyword matching.

The Leiden algorithm finds those clusters. Traag et al. prefer it to classic Louvain because Louvain can emit disconnected communities (nodes in the same group with no path between them). Leiden adds a refinement phase and guarantees connected communities ^{[9]Reference 9From Louvain to Leiden: guaranteeing well-connected communities.https://arxiv.org/abs/1810.08473}. That removes a structural defect before report generation.

Leiden still can't fix bad extraction or unsupported edges. Microsoft's GraphRAG pipeline applies hierarchical Leiden recursively until communities hit a size threshold, which gives it coarse and fine-grained views of the same corpus ^{[3]Reference 3GraphRAG Default Dataflowhttps://microsoft.github.io/graphrag/index/default_dataflow/}.

One quality function Leiden can optimize is modularity ($Q$). In plain terms: are there more edges inside each group than chance would predict in this extracted graph? A high $Q$ means the clustering is strong on that graph. It doesn't prove the extractor captured a real topic.

For an unweighted, undirected graph:

$Q = \frac{1}{2m} \sum_{i,j} \left[ A_{ij} - \frac{k_i k_j}{2m} \right] \delta(c_i, c_j)$

$A_{ij}$ is 1 if nodes $i$ and $j$ share an edge and 0 otherwise. $k_i$ and $k_j$ are node degrees. $m$ is the number of undirected edges. $\delta(c_i, c_j)$ is 1 when both nodes sit in the same community. Put every node in one community and $Q$ is identically 0, which is a useful sanity check.

What does high modularity mean in a GraphRAG entity graph?

Answer

Entities inside the same community connect more densely than random chance would predict in the extracted graph. Validate the entity and edge evidence before treating that cluster as a real topic.

Score two candidate partitions of the same undirected graph. Production GraphRAG runs hierarchical Leiden; scoring $Q$ by hand shows what that algorithm is optimizing, using only the standard library.

The clusters roll up in layers. Global search can start at a high report, then drill into the cache branch without paying for every eval leaf.

Step 4: Community summarization

The final indexing step compresses one community's entities, descriptions, and relationships into a report. Global search can map over that report later ^{[1]Reference 1From Local to Global: A Graph RAG Approach to Query-Focused Summarization.https://arxiv.org/abs/2404.16130[3]Reference 3GraphRAG Default Dataflowhttps://microsoft.github.io/graphrag/index/default_dataflow/}:

Authorization continuity for graph artifacts

Community reports compress many text units into one artifact. If any unit in that community is confidential, a global-search answer that reads the report can leak facts the user isn't allowed to see, even when chunk-level vector search is correctly ACL-filtered.

Carry grants through the whole index, including derived artifacts beyond leaf chunks:

1. Text units: every chunk keeps tenant + principal ACL metadata (same contract as enterprise RAG).
2. Entities and edges: an edge is queryable only when the caller is allowed to see both endpoints' supporting evidence (or the edge fails closed).
3. Community reports: never materialize a mixed-privilege community report for global search. Either:
   • build reports only inside a single ACL equivalence class, or
   • recompute a per-principal report view from allowed text units at query time, or
   • drop the report and fall back to local search on authorized entities.
4. Revocation: when a document's ACL shrinks, reindex affected edges and community reports. Tombstoning the leaf chunk alone isn't enough if a summary still quotes it.

Worked fail-closed sketch. Three text units land in one Leiden community:

A global community report over all three would compress the restricted rotation policy into staff-visible theme text. Fail-closed choices for a non-security principal:

Don't map-reduce the mixed report and scrub afterward: the restricted fact is already in context.

The next lesson, RAG Security & Access Control, deepens prefilter vs post-filter vector search and operationalizes graph-artifact authorization. Graph modes need the same fail-closed habit: authz before map-reduce, not after the summary is already written into context.

Why can community reports leak data even when vector search is ACL-aware?

Answer

Reports summarize many chunks. If a community mixes public and restricted text units, global search can surface restricted facts through the summary without ever returning the restricted chunk as a retrieval hit.

Phase 2: Query processing (runtime)

At runtime, GraphRAG exposes Basic Search, Local Search, Global Search, and DRIFT Search. Local search builds mixed context for entity-centric questions, while global search synthesizes over community reports.

Basic Search is a text-unit vector baseline. DRIFT (Dynamic Reasoning and Inference with Flexible Traversal) starts from community reports, then uses local-search follow-ups ^{[10]Reference 10GraphRAG Local Searchhttps://microsoft.github.io/graphrag/query/local_search/[11]Reference 11GraphRAG Global Searchhttps://microsoft.github.io/graphrag/query/global_search/[12]Reference 12GraphRAG DRIFT Searchhttps://microsoft.github.io/graphrag/query/drift_search/}.

Local search (specific questions)

Local search isn't just named-entity matching plus one-hop neighbors. In Microsoft's docs, it maps the query into semantically related entities first.

It then ranks a mixed context from connected entities, relationships, community reports, linked text units, and optionally covariates if claim extraction is on ^{[10]Reference 10GraphRAG Local Searchhttps://microsoft.github.io/graphrag/query/local_search/}.

The runnable example below is a bounded neighborhood walk for intuition. For the incident question, predict that two hops from inference-api reach Redis Cache and Old Maxmemory Policy. The walk expands candidates; ranking and provenance still decide what enters context. It isn't a copy of GraphRAG's context builder or ranking policy:

The actual context-builder problem includes a token budget: relevant entities, relationships, text units, and reports compete for room in one generation request. A production route needs ranking and provenance, not unbounded expansion.

Global search (broad questions)

For questions that need cross-corpus coverage, GraphRAG uses a map-reduce over community reports at a chosen hierarchy level. The docs batch reports and produce rated intermediate points during map.

Reduce then keeps the highest-value points for the final answer ^{[1]Reference 1From Local to Global: A Graph RAG Approach to Query-Focused Summarization.https://arxiv.org/abs/2404.16130[11]Reference 11GraphRAG Global Searchhttps://microsoft.github.io/graphrag/query/global_search/}:

The sketch below does that map-reduce on four community reports from the incident corpus. Predict its shape first: map keeps latency-related points, then reduce returns the top three:

The level argument hides a cost problem. Static global search maps over every community report at the chosen level, so cost scales with that whole level even when most communities are irrelevant.

Dynamic community selection starts near the top of the hierarchy, rates each report against the query, prunes irrelevant branches, and expands only sub-communities under surviving reports.

On an AP News evaluation, Microsoft reported about 77% lower token cost at community level 1 with LLM-judge quality comparable to static search (win rates near 50%). Letting the search continue to level 3 improved comprehensiveness and empowerment (about 59% and 60% win rate vs static level 1) but raised cost about 34% because more reports entered map-reduce ^{[13]Reference 13GraphRAG: Improving global search via dynamic community selection.https://www.microsoft.com/en-us/research/blog/graphrag-improving-global-search-via-dynamic-community-selection/}.

Rating calls add work, so benchmark dynamic selection against static search on your workload.

DRIFT search

DRIFT sits between those modes. The primer compares the query with the top-k most similar community reports, drafts a broad answer, and emits follow-up questions.

Follow-up steps run local search to refine those questions, then the engine ranks the resulting question-answer tree ^{[12]Reference 12GraphRAG DRIFT Searchhttps://microsoft.github.io/graphrag/query/drift_search/}. Use it when a specific entity question still needs community-level breadth, and measure it against local search rather than assuming it's always better.

Hybrid graph-vector architecture

Don't treat vector search and GraphRAG as mutually exclusive. The GraphRAG stack already mixes graph structure with embeddings during both indexing and query-time context building ^{[2]Reference 2GraphRAG Indexing Overviewhttps://microsoft.github.io/graphrag/index/overview/[10]Reference 10GraphRAG Local Searchhttps://microsoft.github.io/graphrag/query/local_search/}. One hybrid architecture looks like this:

A router is a policy to evaluate, not a guarantee. Measure supported-answer accuracy, p95 latency, and spend for each query class. In a workload dominated by direct evidence lookups, a text route may cover most requests; an analyst-heavy workload may justify more graph/report queries.

One subtle detail: GraphRAG the technique doesn't require a dedicated graph database. Microsoft's reference implementation writes structured output tables to disk and builds query context from those artifacts directly. A graph database like Neo4j or Neptune becomes useful when you need custom traversals, shared KG infrastructure, or analyst-facing graph queries outside the stock pipeline ^{[8]Reference 8GraphRAG Outputshttps://microsoft.github.io/graphrag/index/outputs/}.

Query expansion and context bridging

Beyond simple routing, the knowledge graph layer can enrich vector results after initial retrieval.

Query expansion uses entities found in initial vector results to expand the search query, finding semantically related but textually distinct content.

Context bridging matters when two retrieved chunks don't directly connect. If edges retain supporting chunks, a traversal can identify intermediate entities for retrieval and citation.

For example, if one chunk mentions "inference-api depends on redis-cache" and another mentions "redis-cache still uses the old maxmemory policy," a policy can inspect inference-api -> redis-cache -> old maxmemory policy, then fetch source text before making a claim about the latency incident.

Result ranking can boost text results with short, supported paths to query entities. Treat this as a ranker feature to evaluate, not proof that a nearby node supports the answer.

What the extra index costs

Indexing cost

The biggest barrier to a standard GraphRAG index is the offline bill. Compared with vector-only RAG, its standard pipeline adds LLM-heavy graph extraction, summarization, community report generation, and multiple embedding passes ^{[1]Reference 1From Local to Global: A Graph RAG Approach to Query-Focused Summarization.https://arxiv.org/abs/2404.16130[3]Reference 3GraphRAG Default Dataflowhttps://microsoft.github.io/graphrag/index/default_dataflow/}.

Whether the resulting query quality justifies that work is an evaluation question.

Mitigation strategies

To keep indexing affordable, start cheap. Microsoft recommends fast, inexpensive models while you learn the system ^{[14]Reference 14GraphRAG Getting Startedhttps://microsoft.github.io/graphrag/get_started/}. Their indexing-methods docs estimate that graph extraction (entity and relationship extraction plus their summarization) is roughly 75% of standard indexing cost ^{[7]Reference 7GraphRAG Indexing Methodshttps://microsoft.github.io/graphrag/index/methods/}.

If your use case is mostly global summarization, FastGraphRAG can cut that further. It replaces LLM-based entity extraction with NLP noun-phrase extraction (NLTK or spaCy) and defines relationships by entity co-occurrence inside a text unit. The graph is noisier and less reusable outside GraphRAG, but indexing is much cheaper ^{[7]Reference 7GraphRAG Indexing Methodshttps://microsoft.github.io/graphrag/index/methods/}.

LazyGraphRAG in practice

Microsoft Research introduced LazyGraphRAG in November 2024 ^{[15]Reference 15LazyGraphRAG: Setting a New Standard for Quality and Costhttps://www.microsoft.com/en-us/research/blog/lazygraphrag-setting-a-new-standard-for-quality-and-cost/}. It targets a central cost concern with standard GraphRAG: paying for an LLM-driven index before knowing how often graph-assisted queries will run.

A June 2025 editor's note on that post says the method is available in Microsoft Discovery and as an Azure Local public preview. Check those product pages before assuming it's in the open-source GraphRAG CLI.

LazyGraphRAG defers LLM use to query time. Its index uses NLP noun-phrase extraction and graph statistics for community structure, without entity summaries or precomputed community reports.

In Microsoft's reported experiment, indexing cost matched the vector RAG setup and was about 0.1% of full GraphRAG indexing cost ^{[15]Reference 15LazyGraphRAG: Setting a New Standard for Quality and Costhttps://www.microsoft.com/en-us/research/blog/lazygraphrag-setting-a-new-standard-for-quality-and-cost/}. Treat that as a benchmark result, not a constant for every corpus.

At query time, LazyGraphRAG blends vector similarity with community structure and exposes a relevance test budget that trades query cost for quality.

In the same write-up, a low-budget setting was comparable to GraphRAG global search on global queries at more than 700 times lower query cost, and a higher budget (about 4% of global-search query cost at community level 2) beat the tested alternatives on both local and global criteria ^{[15]Reference 15LazyGraphRAG: Setting a New Standard for Quality and Costhttps://www.microsoft.com/en-us/research/blog/lazygraphrag-setting-a-new-standard-for-quality-and-cost/}. Those comparisons used LLM-as-judge win rates on 100 synthetic queries over 5,590 AP news articles.

Precomputed community reports aren't the only candidate for global sensemaking. If the corpus changes frequently or global queries are rare, benchmark a lazy approach against standard GraphRAG.

If reusable community reports are a product output, the standard index provides artifacts the lazy path intentionally omits.

Use observed quality and workload volume to compare indexing strategies rather than choosing from architecture labels:

What part of LazyGraphRAG makes its reported indexing cost close to the vector RAG comparison?

Answer

It skips LLM entity and relationship summarization and doesn't precompute community reports. Its index uses NLP noun-phrase extraction plus graph statistics, while LLM work happens at query time. The exact cost ratio still depends on the evaluated corpus and configuration.

Query cost

Indexing pays the bulk of the computational expense, but runtime queries can still cost more than standard vector search. The query strategy determines how much more:

• Local search: Usually moderate. You still have to build a mixed context from entity embeddings, graph neighborhoods, text units, and community reports, but the response path is much narrower than global search ^{[10]Reference 10GraphRAG Local Searchhttps://microsoft.github.io/graphrag/query/local_search/}.
• Global search: Potentially expensive. Cost grows with the number of community-report batches you need to map over and the hierarchy level you choose ^{[11]Reference 11GraphRAG Global Searchhttps://microsoft.github.io/graphrag/query/global_search/}.

Mitigation strategies

To manage these runtime costs, standard GraphRAG precomputes community reports at multiple hierarchical levels. At query time, evaluate whether a coarse level answers the question adequately before paying for more detailed reports. Lower levels tend to yield more thorough responses, but they can also increase report volume and LLM work ^{[11]Reference 11GraphRAG Global Searchhttps://microsoft.github.io/graphrag/query/global_search/}.

Graph maintenance

Knowledge graphs aren't static; they must evolve as the underlying document corpus changes. Keeping the graph in sync with a live, mutating dataset adds engineering complexity.

Current GraphRAG releases expose explicit update flows and standard-update / fast-update methods. The output tables also include fields used for incremental update merges ^{[16]Reference 16GraphRAG CLI Referencehttps://microsoft.github.io/graphrag/cli/[8]Reference 8GraphRAG Outputshttps://microsoft.github.io/graphrag/index/outputs/}.

• Additions: When new documents arrive, the system must extract entities and relationships, then merge them into the existing graph. New connections can shift community structure, requiring reclustering and regeneration of affected reports.
• Deletions: Removing a document isn't as simple as deleting a row in a database. The system must trace and remove nodes or edges supported solely by that document. If removed evidence changes graph structure, communities and reports may need recomputation.
• Strategy: Even with update support, a deployment may prefer scheduled refreshes for structural artifacts. Entity merges, community boundaries, and report summaries can shift when new documents arrive, so online reclustering is harder to operate than plain vector re-indexing.

When to use GraphRAG

GraphRAG isn't a default replacement for retrieval. It adds indexing and maintenance for analytical workloads. Label real queries before committing to the architecture:

The pragmatic path: start with vector search. Add a graph layer only when you can show a failure that reports or cited paths fix:

1. Users ask questions that need synthesis across many documents
2. You need to trace influence chains (what depends on what) with provenance
3. Global sensemaking is a core use case, not an edge case
4. The corpus is large and stable enough that indexing cost can amortize

Failures after the 200-document sandbox

A 200-document prototype hides different classes of failure. Once it leaves the sandbox, diagnose each trap as symptom, cause, and fix.

Mistake 1: "Use a bigger context window"

• Symptom: You stuff 100,000 tokens into a long-context model and ask for a summary. The output misses key themes or contradicts itself.

• Cause: A 1M token context window doesn't automatically solve global sensemaking. You still have to decide what to include, very large prompts are expensive to run, and models can still underuse information buried in the middle of long contexts ^{[17]Reference 17Lost in the Middle: How Language Models Use Long Contextshttps://aclanthology.org/2024.tacl-1.9/}.

• Fix: Benchmark selective retrieval and structured indexing against long-context prompting. Standard GraphRAG is one candidate when repeated global questions justify precomputed reports; it isn't required for every long document.

Mistake 2: Treating GraphRAG as a replacement for vector search

• Symptom: You replace your entire vector index with a knowledge graph. Simple lookups become slow and expensive.

• Cause: GraphRAG is complementary, not substitutive. Local search uses entity-description embeddings and linked text units as part of context construction. Global search adds report synthesis for questions that require coverage across many documents.

• Fix: Use a hybrid router. Route simple fact lookups to vector search, entity questions to local search, and global summaries to global search.

Mistake 3: Ignoring entity resolution

• Symptom: Your graph has three separate nodes for RedisCache, redis-cache, and redis.internal. Queries that should traverse through the cache dependency fail because the path is broken.

• Cause: Entity extraction from unstructured text is inherently noisy. Skipping deduplication leaves near-duplicate nodes that fragment the graph.

• Fix: Use (canonical title, canonical type) as the entity key, carry both endpoint types on every edge, and reject unknown or ambiguous references. Candidate aliasing can use embeddings or string distance within one tenant and environment, but stable identifiers, source provenance, and a human-reviewed policy must authorize each merge. A model suggestion alone can't authorize a cross-tenant or cross-environment merge.

Mistake 4: Extracting every noun as a node

• Symptom: The graph balloons to millions of nodes, most of them useless. Common words like "service," "alert," and "team" become nodes with thousands of meaningless edges.

• Cause: Without a schema, the extractor treats every noun as an entity.

• Fix: Define an ontology (a typed schema) before extraction. Restrict nodes to domain-relevant types like Service, Datastore, Region, Incident, ConfigIssue, and Runbook. Filter low-information entities in a post-processing step.

Mistake 5: Underestimating indexing cost

• Symptom: The prototype works on 200 documents, then the first full-corpus run burns budget on extraction, summarization, and embedding passes.

• Cause: Standard GraphRAG does much more than embed chunks. It asks models to extract entities, extract relationships, summarize repeated descriptions, generate community reports, and embed several downstream artifacts.

• Fix: Measure indexing cost before committing to the architecture. Start with a small representative corpus, use inexpensive models while tuning prompts, consider FastGraphRAG for global-summary-heavy workloads, and set a refresh cadence instead of pretending every graph update is free.

Mistake 6: Ignoring graph maintenance

• Symptom: Search results become stale or paths break after documents are added, deleted, or corrected.

• Cause: A graph index has structure. New evidence can merge entities, change edge weights, move community boundaries, and invalidate old community reports.

• Fix: Treat graph refresh as a product requirement. Track provenance with text_unit_ids, run incremental update flows where they are good enough, and schedule full rebuilds when entity resolution or community boundaries drift.

Try it yourself

Stay with the three incidents from the opening. Work this on paper before reading the sketch.

Queries

1. "What is the timeout for the embeddings API?"
2. "Why did inference-api breach the latency SLO, and is this likely to happen again?"
3. A staff user (not security) asks for "top recurring operational themes." One Leiden community mixed tu-public-slo, tu-security-key, and tu-public-runbook.

For each query, name the search mode you'd run first, which artifacts you'd admit into context, and what you must not claim.

Solution sketch

1. Basic / vector search. The timeout is a local fact. Don't pay for community map-reduce.

2. Local search with provenance. Map to inference-api, pack cited neighbors (depends_on redis-cache from 1842, has_issue maxmemory from 2031) under the token budget. Cite those two facts. Don't predict another SLO breach: the will_cause edge has no source chunks.

3. Fail closed before map-reduce. Drop the mixed community report. Use an all-staff equivalence-class report, recompute a per-principal view without tu-security-key, or fall back to authorized local search. Don't generate the mixed report and scrub the answer afterward.

When the graph earns its cost

Start by labeling 30 real queries as fact lookup, entity-specific investigation, or corpus-wide trend.

Use that table as a release gate: ship GraphRAG only for the bucket where vector retrieval fails and graph/report context improves cited-answer quality within your latency budget.