GraphRAG: From Better Retrieval to Better Knowledge
Over the past two years, many enterprises have adopted Retrieval-Augmented Generation (RAG) to help AI models interact with their internal documents. For simple, single-document searches, standard RAG works very well. However, as teams try to answer more complex questions that require connecting multiple pieces of information, standard RAG often reaches its limits. GraphRAG—which combines the structured data of knowledge graphs with Large Language Models (LLMs)—is emerging as the logical next step. It allows AI to map relationships between different facts before answering a question. However, many treat GraphRAG like a simple technology update, assuming a new database will automatically yield better answers. In reality, GraphRAG is a dedicated knowledge engineering project. Organizations that see a return on their investment succeed because of clear ownership, a well-defined schema, and ongoing maintenance. What's in this article: What’s driving the shift to GraphRAG What it takes to create successful GraphRAG projects A reference architecture for production GraphRAG Our preferred frameworks and implementation approach The Drivers Behind the Shift to GraphRAG GraphRAG is moving from an experimental concept to enterprise production because of three specific shifts in the technology landscape. 1. The Limits of Similarity Search Standard vector RAG is designed to retrieve text that is semantically similar to a user's query. This is excellent for finding a specific policy, procedure, or document. However, it struggles with two types of questions common in business: Multi-hop questions: Queries that require linking facts across several different documents. For example, "Which of our vendors' suppliers are exposed to the new EU regulation?" Aggregate questions: Queries where the answer requires summarizing trends across an entire dataset, such as "What are the most common themes in this quarter's IT support tickets?" 2. Transition from Pipelines to Agents The role of enterprise AI is also changing. Today's AI agents execute workflows that involve planning, retrieving information multiple times, calling external tools, and verifying intermediate results. This requires knowledge sources that expose explicit relationships and provenance rather than isolated text chunks. GraphRAG provides that structured foundation, allowing agents to navigate connected information. 3. Automated Construction of Knowledge Graphs Until recently, building a knowledge graph required a team of human data architects to manually enter relationships, which made it too expensive for many projects. Today, LLMs can scan unstructured text and extract entities (people, companies, products) and their relationships automatically. This automation has significantly lowered the cost of building a graph. What It Takes to Build GraphRAG The success of a GraphRAG project depends largely on how the data is prepared. Knowledge Extraction at Ingestion In standard RAG, the AI has to figure out relationships at the exact moment the user asks a question. In GraphRAG, you use computing power upfront—during data ingestion—to extract relationships and store them as definitive links. When an LLM extracts "Company A supplies Company B" and saves it as a connected line in the graph, a complex question becomes a simple, fast database lookup. Ontology Design An ontology is the set of rules that defines what categories and relationships are allowed in your graph. When extracting data, it is tempting to let the AI create categories automatically. However, an open schema usually creates chaos. The AI might create WORKS_AT, EMPLOYED_BY, and WORKS_FOR as three separate relationships. Although they mean the same thing, the graph treats them as different relationships. When a user queries the graph later, the system will miss data because the relationships are fragmented. The solution is to define a strict, small schema (for example, 10 to 15 entity types) designed specifically around the questions users will ask. Entity Resolution Entity resolution is the process of ensuring that different variations of a name point to the same record. For example, "IBM", "I.B.M.", and "International Business Machines" must be merged into one single node. If they are not merged, the graph breaks into disconnected pieces, and the AI agent will hit dead ends when trying to follow a relationship. In a recent engagement with a banking fintech client, we saw this failure mode play out. Their initial attempt at open extraction yielded a noisy graph that struggled to distinguish between overlapping product features and historical software versions. We paused the build to co-design a strict, closed schema alongside their internal domain experts. That single intervention turned the system around: entity detection accuracy stabilized, and the graph could finally trace how specific product features evolved across versions—a critical business insight that was previously lost in a sea of generic, misaligned nodes. Data Provenance Agents make knowledge graphs highly effective, but they also amplify data errors. Because an agent queries the database multiple times in a loop, a single piece of bad data can compound into a major error in the final answer. Therefore, data provenance is mandatory. Every fact extracted into the graph must contain a direct link back to the source text. If an auditor or a user cannot verify where a fact came from, the system cannot be trusted. Inside the Implementation: A Reference Architecture While architectures vary by use case, most production GraphRAG systems share the same core components: a graph structure that represents enterprise knowledge, an ingestion pipeline that builds and maintains it, and retrieval patterns that combine graph traversal with LLM reasoning. The Two-Layer Graph A production GraphRAG store is not one graph but two, linked together: Lexical layer: Mirrors the source documents as Document → Section → Chunk nodes, with vector embeddings associated with each chunk. This provides the semantic retrieval capabilities of a traditional vector store within the graph. Domain layer: Represents the entities (Company, Product, Regulation) and relationships extracted from the text, with every entity connected back to the chunks it appeared in via MENTIONED_IN edges. Those cross-layer edges provide the provenance described above. Any traversal through the domain layer can end by handing the LLM the actual source text, and any suspect fact can be audited against the document it came from. The Ingestion Pipeline 1. Parse and Chunk Documents Layout-aware parsing (using tools such as Docling or Unstructured) preserves document structure-headings, tables, sections rather than reducing documents to plain text. Chunks of roughly 300–800 tokens are written into the lexical layer with their hierarchy and reading order intact, and embedded for vector search. 2. Extract Against the Schema A fast, low-cost model processes each chunk, but its output is constrained through structured output so it is impossible for the extractor to emit an entity or relationship type outside the approved ontology. The difference between putting the schema in the prompt and enforcing it at the API level is the difference between a queryable graph and fragmentation like the WORKS_AT/EMPLOYED_BY problem described earlier. A typical extraction result looks like this: { "entities": [ {"type": "Company", "name": "Acme Corp"}, {"type": "Regulation", "name": "EU Regulation 2024/17"} ], "relations": [ {"source": "Acme Corp", "type": "SUBJECT_TO", "target": "EU Regulation 2024/17", "source_chunk": "doc12#c4"} ] }3. Resolve Entities before Writing Each extracted entity is checked against the existing graph: embedding similarity generates candidate matches, and a lightweight LLM adjudicates each candidate pair ("same real-world entity, yes or no?") using both entities' descriptions and neighboring relationships as context. Confirmed matches merge into the existing node, keeping all aliases and provenance links. Because this step runs during every ingestion, new documents are incorporated into the graph incrementally without requiring a full rebuild. 4. Update Graph Incrementally Once entity resolution is complete, the graph is updated using upsert (merge) semantics, so re-processing a document never duplicates nodes or edges. Every relationship carries its source_chunk reference. 5. Build Community Summaries For corpora where users ask trend and theme questions, we run community detection (the Leiden algorithm) over the domain graph and have an LLM write hierarchical summaries of each cluster. These summaries—not raw chunks—are what answer "what are the main themes?" questions. Query Time: How the Agent Actually Uses the Graph At query time, GraphRAG works best by combining vector search with graph traversal. User queries are expressed in natural language, while graphs excel at representing explicit relationships rather than fuzzy text matches. The system therefore begins with vector search over the lexical layer to identify relevant entry points, then traverses the graph to retrieve the connected information needed to answer the question. Rather than relying on a fixed retrieval pipeline, the agent is given a small set of graph operations: search_entities(text)→ candidate entity nodes (vector + name match) get_neighbors(node, rel_type?) → adjacent nodes and relationships get_source_chunks(node) → original text behind a fact (provenance) query_graph(question) → generated Cypher, read-only, as a fallbackMost questions can be answered using the first three operations. For more complex multi-hop and aggregation queries, the agent generates a Graph query directly. The graph schema is included in the model's context, the generated query is validated before execution, and any execution errors are fed back to the model for self-correction. In practice, two or three iterations resolve most failures. The vendor-exposure example introduced earlier compiles down to a single graph traversal: MATCH (v:Company {name: "Vendor X"})(p:Product)The multi-hop question that defeats similarity search becomes one declarative query because the reasoning was already done, once, at ingestion. We expose this toolbox to AI agents over the Model Context Protocol (MCP), which has become the standard integration layer: the graph database runs behind an MCP server, and any MCP-capable agent can inspect the schema and call the tools without custom glue code. This also keeps the security controls (read-only credentials, timeouts, result limits, query logging) in one enforceable place. Frameworks We Use LayerTypical ChoicesWhen and WhyGraph databaseNeo4j (with its native vector index); or FalkorDBNeo4j for enterprise deployments: mature Cypher tooling, vector search built in, official MCP server. FalkorDB for lighter-weight or embedded deployments.Graph constructionLlamaIndex PropertyGraphIndex; the neo4j-graphrag package; LangChain LLMGraphTransformerThese handle schema-constrained extraction, embedding, and upsert plumbing. Choice depends primarily on the client's orchestration stack.Indexing pipelinesMicrosoft GraphRAG; LightRAGMicrosoft GraphRAG for aggregate and thematic questions. LightRAG when frequent updates and incremental, lower-cost indexing are more important.Agent memoryGraphiti (Zep)Best for evolving data such as user preferences or account states. Its bi-temporal model (validity intervals on edges instead of overwrites) answers both "what is true now?" and "what was true then?"Agent integrationMCP servers; native function callingMCP for portability across agent frameworks; a single place to enforce read-only access and query limits.EvaluationRAGAS plus hand-built multi-hop test setsRAGAS measures faithfulness and relevance. Multi-hop graph scenarios require domain-specific hand-crafted test cases.One selection principle worth stating explicitly: the framework choice matters far less than the decisions covered in the previous section. A team with a clean closed schema and solid entity resolution will succeed with any of these stacks; a team without them might fail with all of them. Our Implementation Approach GraphRAG technology has matured significantly faster than the enterprise's understanding of what it requires. What remains scarce is the discipline to treat enterprise knowledge as a curated, continuously maintained asset rather than a pile of documents with an index on top. At QBurst, we put those principles into practice through a phased implementation approach. We begin by identifying the business questions the system must answer before designing the ontology and knowledge model. After validating the graph within a single business domain through User Acceptance Testing (UAT), we expand incrementally to additional domains. Our Institutional Knowledge Platform and deployment accelerators reduce implementation effort while preserving this staged approach, allowing clients to move from pilot to production more quickly.
