An LLM fresh out of the box is a brilliant colleague… amnesiac, cut off from the world, and frozen at its training date. It knows neither your documents, nor your database, nor this morning’s weather — and it does nothing: talking is all it has. All of AI application engineering consists of filling those gaps. The problem is the shelf: RAG, RIG, tool calling, MCP, fine-tuning, long context, memory… too often we pick the technology by buzz rather than by need — and end up with a vector database to query three SQL tables.
This guide puts each tool back in front of the gap it fills, with a decision table and — a first on this blog — an interactive box to walk you through the choice. Not rocket science.
The right question: which gap are you filling?
Everything gets simpler when you sort technologies by gap filled rather than popularity. An LLM has four gaps, and each family of tools targets one:
| The gap | The question to ask | The remedies |
|---|---|---|
| Knowing | “it doesn’t know my data” | direct context, RAG, RIG |
| Acting | “it can’t execute anything” | tool calling, MCP |
| Being | “it doesn’t talk the way it should” | prompt engineering, fine-tuning |
| Remembering | “it forgets everything between sessions” | conversational memory |
The classic — and costly — mistake is crossing the columns: fine-tuning to inject knowledge (stale at the first change), or vectorizing data that deserved a SQL query.
Filling the “knowing” gap: three tools, three situations
Direct context — the simplest thing that works. Your corpus is small and stable (a product doc, an FAQ, a policy)? Paste it wholesale into the system prompt. With today’s windows and prompt caching making repeated prefixes nearly free (some call it cache-augmented generation), this “dumb” approach beats a badly tuned RAG nine times out of ten. Its limits: corpus size and the model’s diluting attention.
RAG — the library with a librarian. Large document base, moving content, open questions: you chunk, embed, and retrieve the relevant passages at question time. It’s the standard for knowledge bases — we take it apart tomorrow in RAG and embeddings explained simply, then in code form on August 1.
RIG — the live fact-checker. Lesser known: Retrieval-Interleaved Generation. Where RAG retrieves once before generating, RIG lets the model query the source while it generates: it emits a mini-query at the moment of writing a number, and the real value replaces its estimate. It’s the approach popularized by DataGemma (Google) wired to Data Commons, and it targets one precise problem: the statistics and numeric values a classic RAG cites poorly. Promising, but young — little ready-made tooling.
Filling the “acting” gap: tools, and the socket that distributes them
Tool calling gives the model arms: you declare functions (SQL query, API call, computation, sending mail), the model picks which to call and with which arguments, your code executes. It’s also the remedy for exact questions: “customer X’s orders in March” is not a semantic question, it’s a WHERE — a structured question deserves a structured answer, not five approximate vector excerpts.
MCP is not one more capability — it’s a standard socket. The Model Context Protocol doesn’t make your model smarter: it packages your tools (and resources, and prompts) behind a protocol every client understands — IDEs, chatbots, agents. The decision rule is simple: a tool for one application → a native function is enough; the same tool for several clients or teams → an MCP server, written once, plugged in everywhere. We already built an MCP server in .NET, and its production version lands on July 30.
Filling the “being” and “remembering” gaps
Style, tone and format are not fixed with a vector database. The staircase: first a structured system prompt and a few well-chosen examples (few-shot); if — and only if — that fails to hold a format or a very specific domain jargon, fine-tuning enters the scene. Remember its golden rule: it teaches the model how to talk, not what to know — knowledge injected by fine-tuning goes stale and can’t be cited.
Memory, finally, fixes the amnesia between sessions: conversation summaries, user preferences, note files the agent re-reads. It’s a topic of its own — we devote the July 27 article to it.
The decision table
| Situation | The right reflex | Complexity |
|---|---|---|
| Corpus < ~100 pages, stable | everything in context + cache | ★ |
| Large doc base, open questions | RAG | ★★★ |
| Structured data (SQL, API) | tool calling | ★★ |
| Numbers/stats to cite precisely | RIG (watch this space) | ★★★ |
| The model must act | tool calling | ★★ |
| Tools shared across several apps | MCP | ★★ |
| Stubborn tone, format, jargon | prompt first, then fine-tuning | ★★★★ |
| Forgetting between sessions | memory | ★★ |
Your turn: the interactive guide
Answer two questions and the box suggests a starting point — with the link to dig deeper.
And in real life: you combine
Serious applications almost always stack several bricks: a RAG plus a SQL tool for exact questions, direct context plus memory, a light fine-tune plus a RAG for facts. The golden rule remains the staircase of simplicity: prompt → context → RAG or tools → fine-tuning, climbing one step only when the previous one has proven its limits — with numbers to show for it (evals arrive in the July 20 article). And when several bricks need orchestrating, that’s the job of agents — we laid the groundwork with the Microsoft Agent Framework.
A word of honesty
- RIG is the youngster of the list: solid concept, still-confidential tooling. I present it so the term doesn’t ambush you in a meeting — not as a default choice in 2026.
- The borders are porous: an “agentic RAG” is just a RAG driven by tool calling, web search is just one more tool. Don’t fight over labels, fight over the gap to fill.
- The interactive box gives a starting point, not an architect’s verdict: your volumes, your budget and your evals always have the last word.
In short
- Sort by gap filled: knowing (context, RAG, RIG), acting (tools, MCP), being (prompt, fine-tuning), remembering (memory).
- Small stable corpus → context + cache; large living base → RAG; structured data → SQL via tool calling; inline stats → RIG (watch this space).
- MCP doesn’t make the model more capable: it standardizes the socket — reserve it for shared tools.
- Fine-tuning teaches how to talk, never what to know.
- Climb the staircase of simplicity one step at a time, evals in hand — and combine without guilt.
Tomorrow morning, we dive into the most-used step of that staircase: RAG and its embeddings, explained simply. And that, honestly… is not rocket science.
