Build an AI Agent Knowledge Base: The Pattern That Cut Tokens 80%

TL;DR Instead of fine-tuning a model on organizational knowledge, split it into a wiki (dense, frequently-used facts and decision frameworks) and recipes (composable, step-by-step procedures that reference the wiki but hold no facts). Wrap both in a four-phase self-improvement flywheel — diagnose, compile, validate, land — with human checkpoints on genuine ambiguity. Meta reports this cut per-turn tokens 80%, turned days of expert assessment into minutes, and shipped zero regressions across its improvement cycles in a six-week rollout. None of it needs new model weights.

An AI agent knowledge base is a structured set of text files — not fine-tuned weights — that an agent reads at runtime to reason like a domain expert. The idea sounds almost too plain to be an architecture: put the facts in files, put the reasoning in separate procedure files, and let both be edited the way you’d edit code. But Meta’s engineering team reported building exactly this for tasks like regulatory compliance and financial risk review — domains where a wrong answer is expensive and “the model felt off” isn’t an acceptable bug report — and the numbers are concrete enough to be worth stealing the pattern rather than the headline.

If you’re already writing a CLAUDE.md for your coding agent or maintaining tool instructions it reads at runtime, you’re doing a smaller version of this. This post is what the same idea looks like once it has to scale past a few hundred lines and stay correct without you personally re-reading every file.

How to build an AI agent knowledge base without fine-tuning

The starting complaint is universal: specialist knowledge “lives in people’s heads and rarely gets captured anywhere durable,” so experts spend their time answering the same routine questions instead of doing the genuinely novel work only they can do. Fine-tuning looks like the obvious fix and is usually the wrong one — a fine-tune is opaque (you can’t diff why the model changed its mind), slow to update (a new fact means a new training run), and unauditable in a regulated domain where you need to show a reviewer exactly which rule fired.

The alternative keeps every fact as text a human can read and a linter can check, split across two kinds of files:

1. Knowledge files — positions, taxonomy/vocabulary definitions, routing indexes, and gateway files that point to the right position file for a given question. Each carries YAML frontmatter declaring its dependencies and consumers, which turns the whole set into a bidirectional dependency graph instead of a folder of loose Markdown.
2. Recipes — composable procedures that prescribe a multi-step analytical workflow. A recipe references knowledge files by name and contains no domain facts of its own. That separation is the whole trick: you can correct a fact without re-verifying the reasoning that uses it, and refine the reasoning without re-checking every fact it touches.

Meta’s implementation runs to 200+ files at this split. Yours doesn’t have to start anywhere near that size — the pattern holds at ten files as well as two hundred.

Why context stuffing breaks down at scale

The naive version of “give the agent knowledge” is to paste everything relevant into the system prompt and let a long context window sort it out. Anthropic’s own guidance on context engineering names the failure mode directly: model performance degrades as context grows — “context rot” — so the fix isn’t a bigger window, it’s curating a smaller, higher-signal one. Anthropic’s specific recommendation is just-in-time retrieval: keep lightweight identifiers in context and load the actual data through a tool call only when a step needs it, the same way a person keeps a folder structure in their head instead of memorizing every file’s contents.

The wiki-and-recipes split is that recommendation made structural instead of aspirational. A recipe’s steps use progressive disclosure — each step loads only the knowledge file it needs for that step, not the whole knowledge base up front. That’s the mechanical reason Meta’s restructuring cut tokens consumed per turn by 80%: the agent stopped re-reading facts it wasn’t using yet.

The four-layer architecture: knowledge, recipes, flywheel, oversight

Four pieces, each doing one job:

• Knowledge system — the facts, versioned and dependency-tracked.
• Reasoning layer (recipes) — the procedures, kept fact-free so they stay stable while the facts underneath them change.
• Self-improvement flywheel — the process that turns an expert’s correction into a shipped edit.
• Human oversight — checkpoints on intermediate review, with escalation triggered by genuine ambiguity rather than every low-confidence output.

The design principle underneath all four, stated directly in Meta’s writeup: “keep the complexity in text files that are readable by both humans and agents, rather than fine-tuned model weights.” That sentence is the whole architectural bet — complexity you can git diff is complexity you can fix in minutes; complexity baked into weights is complexity you retrain for.

How do you decide what goes in the wiki vs. RAG?

This is the question that trips up most first attempts, because “put everything in a vector store” is the default and it’s the wrong default for facts an agent needs on nearly every turn.

The split Meta uses is by access frequency and density, not by document type:

• Wiki (high-density, frequent use) — distilled positions and decision frameworks. Small enough to load in full for the recipe step that needs them. If an agent needs a fact on more than an occasional turn, it belongs here, written as a short, direct statement, not a paragraph of hedging.
• RAG (sparse, situational) — reference material and historical records an agent needs occasionally: past cases, one-off precedents, long documents nobody reads end-to-end. Retrieval is the right tool here precisely because the material isn’t dense enough to justify permanent residence in the wiki.

Get this backwards — RAG for the frequent facts, wiki-style full-load for the sparse archive — and you reproduce the exact problem the split was supposed to fix: either a bloated context on every turn, or a retrieval miss on the fact that mattered most.

The 5-step self-improvement flywheel

Static knowledge bases rot the moment an edge case appears that nobody wrote down. The flywheel is what keeps this one current without turning every correction into a full re-training cycle:

1. Collect an expert’s correction or dissatisfaction with an output.
2. Diagnose the feedback down to its root cause — which knowledge file or recipe step actually produced the wrong answer.
3. Compile the issues into a minimal, verified edit — not a rewrite, the smallest change that fixes the diagnosed cause.
4. Validate the edit with targeted replay against past cases plus regression testing against the existing suite.
5. Land the change with an audit trail: what changed, why, and which expert flagged it.

Meta reports zero regressions across improvement cycles over a six-week rollout, with domain experts rating outputs “useful almost all the time” and assessment time dropping from days to minutes. The zero-regression number is the one worth taking seriously — it’s the difference between a knowledge base you can safely edit weekly and one you’re afraid to touch.

What breaks if you skip the review step?

Everything downstream of it, eventually. The two checks doing the load-bearing work here are independent adversarial review (a second pass, deliberately trying to break the proposed edit) and deterministic structural linting — a program, not a model, checking for contradictions between files, dangling references to knowledge files that no longer exist, and dependency cycles in the frontmatter graph.

Skip adversarial review and you get edits that look correct to the person who wrote them but quietly contradict an existing position file — the exact “undocumented tribal knowledge” problem this architecture exists to fix, just relocated into Markdown instead of someone’s head. Skip structural linting and a renamed or deleted knowledge file leaves every recipe that referenced it silently broken, because nothing enforced the dependency graph declared in the frontmatter. Both checks are cheap relative to the failure they prevent, which is why they run on every edit rather than periodically.

Human oversight sits on top of both: checkpoints for intermediate review on high-stakes steps, and escalation triggered specifically by ambiguity the automated checks can’t resolve — not by every low-confidence output, which would just reintroduce the bottleneck the whole system exists to remove.

Wiki-and-recipes vs. fine-tuning vs. plain RAG

	Wiki + recipes	Fine-tuning	Plain RAG (no recipes)
Update latency	Minutes (edit + lint + review)	A full training run	Minutes (re-index)
Auditability	High — git diff on plain text	Low — opaque weight deltas	Medium — retrieved chunk is visible, reasoning isn’t
Works across model swaps	Yes — knowledge lives outside weights	No — retrain per model	Yes
Per-turn token cost	Low — progressive disclosure loads only what a step needs	Lowest — no extra context needed	Variable — depends on chunk size and top-k
Best for	Dense facts used on most turns, plus multi-step reasoning	Style/format changes, not fast-moving facts	Sparse, occasional reference material
Failure mode if under-maintained	Dependency-graph rot — caught by linting	Silent staleness — model doesn’t know what it doesn’t know	Retrieval miss — the right chunk exists but isn’t fetched

The honest takeaway from this table isn’t “wiki-and-recipes wins” — it’s that the three aren’t substitutes. Meta’s own split routes the dense, frequent facts to the wiki and the sparse, situational material to RAG, and neither one uses fine-tuning for facts that change. Reach for fine-tuning when you’re changing how the model writes, not what it knows.

Common mistakes when you build this yourself

• Letting recipes carry facts. The moment a procedure hardcodes a threshold instead of referencing the wiki, you’ve lost the property that made this maintainable — now a fact lives in two places and they will eventually disagree.
• Skipping the dependency graph. Frontmatter that isn’t actually machine-checked is documentation, not infrastructure. If nothing lints it, dangling references accumulate silently.
• Escalating everything instead of the genuinely ambiguous. Human-in-the-loop on every uncertain output just moves the bottleneck from “the expert answers routine questions” to “the expert reviews routine flags.” The oversight layer only pays off when escalation is selective.
• Treating this as a one-time migration. The value is in the flywheel, not the initial file set. A wiki that isn’t being edited weekly from real feedback is a snapshot, not a knowledge base.

FAQ

Do I need to fine-tune a model to give it organizational knowledge?

No. The pattern this post describes keeps every fact in plain text files the agent reads at runtime, not in model weights. That means updates ship in minutes through a normal edit-and-review cycle, and any reasoning-capable model can use the same knowledge base without retraining.

What is the difference between a wiki file and a recipe in this pattern?

A wiki file holds domain facts — positions, definitions, thresholds — and nothing about how to reason. A recipe is a composable procedure that references wiki files by name but contains no facts of its own. Splitting the two means you can fix a fact without touching the reasoning, and fix the reasoning without re-verifying every fact.

When should I use RAG instead of a wiki file?

Use the wiki for the small set of high-density facts an agent needs on nearly every turn — decision frameworks, thresholds, definitions. Use RAG for sparse, situational material: historical records, one-off precedents, reference documents an agent needs occasionally and can afford to fetch on demand. Loading everything into the wiki defeats the point of keeping it small.

How do you stop a knowledge base from drifting out of date?

With a flywheel, not a schedule: diagnose expert feedback to a root cause, compile the minimal edit that fixes it, validate the edit against replay and regression tests, then land it with an audit trail. Independent adversarial review plus structural linting catches contradictions and dangling references before they ship, which is what makes frequent small edits safe.

Doesn’t a 200-file knowledge base just become the thing it was supposed to replace — undocumented tribal knowledge, but in Markdown?

Only if nothing enforces structure. The pattern requires YAML frontmatter declaring each file’s dependencies and consumers, which turns the knowledge base into a bidirectional dependency graph a linter can check. Without that graph and without human checkpoints on ambiguous cases, yes, it degrades into the same mess it replaced — just harder to search.

If you’re deciding what to feed the agent versus what to let it fetch on demand, that’s the same trade-off covered in building a RAG pipeline from scratch and in cutting agent tool-call cost with a prompt rewrite — and if the agent’s context is getting summarized out from under it before the knowledge base can even be consulted, see what actually survives agent context compaction.

Sources

• An Organizational Second Brain: Building an AI That Learns From Experts — Meta Engineering
• Effective context engineering for AI agents — Anthropic Engineering
• Writing effective tools for agents — Anthropic Engineering

Going deeper on agentic coding? See AI Coding Agents — Agentic AI for Developers and LLM Engineering.

Explore more: AI Coding Agents · LLM Engineering · Claude Code