Ask a classic RAG this question: “if the Payment service goes down, which customers are impacted?”. It will look for passages that resemble the question… and bring you back the Payment service docs. The answer itself is written nowhere: it has to be deduced — Payment is called by Order, Order serves the shop, the shop has Premium customers. Three hops. An archivist doesn’t know how to do that.
You know the board from detective shows: photos, places, clues — and red strings between them. The detective doesn’t crack the case by owning the biggest box of archives, but by connecting the clues. Giving that board to an AI is what graph engineering is all about. Let’s build it together — you’ll see, it’s not rocket science.
The archivist’s blind spot
Vector RAG is a brilliant archivist: it instantly retrieves the passages semantically close to your question. As long as the answer is written somewhere in a document, it shines.
But similarity has a blind spot: relationships. Chopped into chunks and filed by resemblance, your documents become a bag of confetti — every piece of confetti is relevant, but the strings between them were cut during chunking. Hence two families of questions that stall the archivist:
- Multi-hop questions: “who approved the decision that introduced the dependency that’s causing trouble?” — the answer crosses three documents that look nothing alike.
- Global questions: “what are the main themes of this corpus?” — no chunk resembles the question, because the answer is everywhere.
Against those, fetching “the 10 most similar passages” no longer cuts it. You have to follow the strings.
The detective board, formally
A knowledge graph is three ingredients — not one more:
- the nodes: the entities (the Payment service, the customer ACME, ADR #12) — the photos on the board;
- the edges: the relationships, directed and named (calls, depends on, decided) — the red strings;
- the properties: the details carried by either (criticality, date, status).
The basic unit is written as a triple: subject → predicate → object: Order —calls→ Payment, ACME —subscribes to→ Premium. Nothing more. And yet this little format changes the nature of search: instead of asking “what resembles my question?”, you ask “what is connected to this node, and how?”. Resemblance becomes traversal.
Graph engineering: the craft of building the board
The board doesn’t build itself — and that’s where the discipline is born. You’ll recognize a cycle: it’s the same one as memory’s, applied to relationships.
- Model — choose the ontology: which entities and which relationships matter for your questions. It’s the most important and least technical move: a well-kept domain glossary is already its rough draft.
- Extract — fill the graph. Good news: reading text and producing triples is precisely what an LLM does well. You hand it the documents and the schema, it proposes
subject —relation→ object, the human decides on the doubtful ones. - Store and query — a graph database (Neo4j, Cosmos DB in Gremlin mode…) and its traversal language: “start from this node, follow depends on for three hops, bring back what you cross”.
- Maintain — the board ages: are “ACME”, “Acme Corp” and “acme-prod” the same node? Entity resolution and relationship expiry are the graph equivalent of tidying up the memory cards.
GraphRAG: the archivist AND the detective
The graph doesn’t replace vectors — the two snap together, and the assembly has a name: GraphRAG.
- Enter: the user’s question is fuzzy (“the payment thing that’s slow”) — the embedding finds the right entry node in the graph. The archivist opens the right door.
- Reason: from that node, you traverse — dependencies, decisions, owners — to assemble a context that similarity alone would never have gathered. The detective follows the strings.
- Summarize: for global questions, you pre-compute communities (the graph’s neighborhoods) and their summaries — answering “what is this corpus about?” becomes possible without re-reading the corpus.
The result then goes, as always, onto the model’s desk: the graph generates nothing, it selects better what the model gets to read. It’s also the graph-flavored version of Agent Framework’s context providers: connected memories, served at the right time.
And one secret to finish convincing you: your repository is already a graph. Calls between classes, references between projects, diagrams as code, commit history — when an agent chases “who calls this method?”, it’s doing graph traversal without saying so. Graph engineering is offering the same thing to your business knowledge.
A word of honesty
The detective board is impressive, but it has a price — three warnings before you pull out the red strings:
- Most needs don’t need it. If your questions are “where is it written?”, vector RAG is enough and costs ten times less to build. The graph earns its keep when your questions are multi-hop or global — not before. Start simple, measure with evals, add complexity only if the failures prove it.
- Automatic extraction gets proofread. An LLM extracting triples can hallucinate a relationship — and a false relationship in a graph is more toxic than a badly filed passage: it will be traversed with confidence on every query.
- The schema is a bet. Too fine an ontology costs a fortune to fill; too coarse, it answers nothing. Practical rule: model for the questions you actually ask, not for the beauty of the model.
In short
| Vector RAG | GraphRAG | |
|---|---|---|
| The metaphor | the archivist | the detective and the board |
| Retrieves | what resembles | what is connected |
| Shines on | “where is it written?” | multi-hop and global questions |
| Basic unit | the text chunk | the triple subject → relation → object |
| Build cost | low | high (schema, extraction, upkeep) |
| Typical risk | missing passage | false relationship traversed with confidence |
- A graph = nodes + edges + properties; the question shifts from “what resembles?” to “what is connected?”.
- Graph engineering = model, extract, query, maintain — memory’s cycle, applied to relationships.
- GraphRAG snaps the two together: vectors find the entry door, traversal does the reasoning.
- And the order of things: plain RAG first, evals — the detective board when multi-hop failures justify it.
The detective doesn’t throw away the archives: he connects them. Do the same for your AI — and the day someone asks “if Payment goes down, who do we call?”, it will follow the strings instead of handing you the docs. And that, honestly… is not rocket science.
