Lesson 4 retrieves facts about specific things. This lesson handles the question that breaks everything else: "What is this whole corpus about?"
Watch
Jonathan Larson of Microsoft on GraphRAG methods for building the context an LLM sees at retrieval time. A good companion to the community-summary approach in this lesson.
The idea from the Microsoft Research paper
Microsoft's GraphRAG treats corpus-wide questions as a query-focused summarization problem rather than a retrieval problem. The method, in the paper's own two stages:
- Index. An LLM extracts an entity knowledge graph from the source documents. The graph is clustered into communities of closely related entities, and a summary is pre-written for each community.
- Query. For a global question, each community summary produces a partial answer, and those partial answers are combined into the final response.
Because the expensive reading happens once at indexing time, a question like "what are the main themes?" no longer needs the model to see the entire corpus at query time.
Search modes you will see in the tool
The open-source implementation exposes several ways to query the same index. Check the docs for the exact list in your version; the main ideas are:
- Global search: answers corpus-wide questions from community summaries.
- Local search: answers questions about a specific entity by pulling that entity, its neighbors and linked text.
- DRIFT search: starts from community-level context, then follows up with local-style queries to add detail.
- Basic search: ordinary vector RAG, useful as a comparison baseline.
The cost reality
Indexing relies heavily on LLM calls across the whole corpus, so it can be expensive. Index a tiny sample first, read the token usage, and extrapolate before you point it at a large dataset. The payoff is cheap, broad answers afterwards.
Choose community summaries when users ask sensemaking questions over a large, fairly stable corpus. For fast-changing data or pinpoint lookups, the retrieval patterns from lesson 4 are usually the better fit.
Resources
Read the method and evaluation sections this time. Note how they judge answers (comprehensiveness and diversity).
The accessible companion to the paper.
Indexing and query guides, including the search modes.
Source code, install instructions and a getting-started path.
A reading list for going deeper after this lesson.
Check yourself
- Index a small dataset (a few dozen pages) with the Microsoft implementation. Record the token usage.
- Ask the same three corpus-wide questions with global search and with your lesson 4 basic retriever.
- Write down where the answers differ and which you would trust for a decision.