GraphRAG

GraphRAG, popularized by Microsoft Research, extracts entities and relationships from a corpus and builds a knowledge graph. Retrieval becomes graph traversal plus vector search over entity descriptions and community summaries. It's the best answer for corpus-wide queries that vector search handles poorly.

The problem it solves

Vanilla RAG works well for "find the chunk that answers this." It works poorly for:

• "What are the main themes in this corpus?"
• "How is X related to Y across all documents?"
• "Give me a summary of everything about Acme Corp."
• "What's the history of decisions about topic Z?"

These require aggregating information across many documents, not finding one chunk. Pure vector retrieval either misses most of the relevant material or returns too much noise.

The GraphRAG pipeline

Indexing phase

1. Chunk documents as usual
2. Extract entities from each chunk using an LLM (people, organizations, concepts, events)
3. Extract relationships between entities from each chunk
4. Build a graph: nodes are entities, edges are relationships
5. Community detection: cluster the graph into hierarchical communities (using Leiden or similar)
6. Generate community summaries: LLM-summarize each community into a description of its core content
7. Embed entities, relationships, community summaries for retrieval

Query phase

Two query modes in Microsoft's GraphRAG:

Local search

For specific entity-focused questions. Embed the query, find related entities in the graph, gather their connections and source chunks, synthesize. Best for "tell me about X" queries.

Global search

For corpus-wide questions. The LLM processes each community summary, generates partial answers, then aggregates them into a final response. Best for "themes," "patterns," "summarize across."

Strengths

• Handles corpus-wide synthesis better than any vector-only approach
• Explicit entity and relationship modeling enables traversal queries
• Community summaries provide abstract-level content retrieval
• Output tends to be more coherent on complex queries

Weaknesses

• Indexing is expensive: LLM calls for every chunk (entity + relationship extraction)
• Graph maintenance is complex when documents update
• Over-engineered for simple Q&A
• Community summaries can drift from actual document content
• Most teams don't need it, vanilla RAG is sufficient

Cost structure

GraphRAG indexing can cost 10-100x more than standard RAG indexing. For 1M chunks:

• Standard RAG: $20-100 in embedding costs
• GraphRAG: $2000-10000 in LLM extraction costs

At enterprise scale, this is significant. Use smaller/cheaper models for extraction if corpus is large.

Implementations

• Microsoft GraphRAG: the reference implementation. Open-source Python.
• LightRAG: lighter-weight alternative with similar ideas
• LlamaIndex PropertyGraphIndex: LlamaIndex's graph-based RAG
• Neo4j + custom pipeline: build your own on top of Neo4j or similar graph DB

Hybrid: GraphRAG + vector RAG

Serious systems combine both:

• Vector RAG for specific fact retrieval
• GraphRAG for thematic/relational queries
• Query classifier routes to the right approach

Or: use GraphRAG's community summaries as additional retrieval candidates alongside document chunks. At retrieval time, the system can return either granular chunks or high-level summaries depending on query type.

When GraphRAG is worth it

• Entity-heavy corpora (legal, research, historical records, news)
• Users ask cross-document thematic questions
• Relationships between entities are core to the value
• You have budget for significant indexing cost
• Corpus is relatively stable (frequent updates = painful graph maintenance)

When it isn't

• Simple Q&A corpora (product docs, FAQs)
• Constantly-changing corpora
• Low-volume applications where index cost dominates
• Teams without the ML engineering capacity to maintain it

GraphRAG is impressive and genuinely useful for the right use case. It's also often over-adopted by teams who would do better with well-tuned vanilla RAG.

Next: Adaptive RAG.