How we moved our company documents from Confluence to a self-maintaining LLM wiki
Company wikis often do not fail all at once, but they decay gradually.
A deployment procedure changes, but three related pages still describe the old process. A customer call contains an important product insight, but it remains buried in the meeting notes. A service is retired, yet it continues to appear in infrastructure and security documentation.
Nobody deliberately creates this mess. It is what you get when hundreds of small maintenance tasks compete with customer work, product work, and the occasional desire to go home.
We had accumulated 310 company pages and 400 attachments in Confluence. Moving those files into another documentation tool would have preserved the same maintenance problem. Instead, we moved them into a Git repository and gave an LLM agent the tedious job: filing new information, updating related pages, maintaining links, and pointing out where our documents disagree.
Company wikis decay because maintenance costs compound
Writing the first version of a document is usually straightforward. The costly part begins when something changes.
Suppose an engineer moves a service to a new host. The deployment runbook needs an update, but so do the infrastructure overview, disaster recovery notes, vendor inventory, customer environment page, and perhaps an accepted technical decision. Missing one of those edits leaves the wiki internally inconsistent.
The problem gets worse as the number of pages grows. Every new page adds useful knowledge, but it also adds relationships that somebody has to remember. Our fallback was to write fewer updates, create another summary page, or ask the person who "just knows how it works." None of those options aged well.
We also found that missing documentation was hard to see. An empty account inventory can look like a completed page unless it is explicitly marked as an open gap. An old runbook looks authoritative until somebody tries the command. Or a decision without its original context becomes hearsay.
Storage was adequate. The recurring cost came from maintaining relationships and current claims. That issue finally led us to the LLM Wiki pattern described by Andrej Karpathy.
An LLM wiki integrates source-backed knowledge across topics
The basic idea is simple. Keep the original sources, then maintain a separate set of pages that explains what the company currently knows.
When a new source arrives, the agent reads it and looks for every page it might affect. It can revise a topic page, update a runbook, add a source to a customer page, flag a contradiction, and link a decision to the experiment that produced it. One source can produce changes across ten files without asking a person to remember all ten.
A customer call is a good example. The transcript may contain a deployment
constraint, a request for a product feature, a complaint about onboarding, and
a signal about pricing. Filing the transcript under meetings/ preserves the
record, but it does not integrate any of those facts into the places where
people will look for them later.
Our agent treats the transcript as some sort of raw evidence. It proposes updates to the customer, product, runbook, and experiment pages that depend on it. A human reviews one large change set rather than opening four parts of the wiki and trying to remember what belongs where.
This is the main advantage of the LLM. It is willing to do the bookkeeping. Humans still choose the sources, correct the interpretation, and approve the result.
One additional note here: Using agentic search, the LLM can search through vast parts of the Wiki for potential relevant content. Of course the better the indexes are, the less the LLM needs to search, but LLMs are very good in finding content even in the largest of knowledge bases. They simply search, validate, repeat, and then propose a change set for human review.
Separate sources, current knowledge, and history
The repository has several layers because each one answers a different question.
raw/ holds original evidence: Confluence exports, meeting transcripts,
customer calls, vendor documents, contracts, screenshots, research, and files
from source repositories. These files are append-only. The agent can read and
cite them, but it cannot quietly improve an inconvenient source.
wiki/ holds the maintained pages that people and agents actually read. The
folders follow company subjects such as customers, engineering, products,
security, experiments, and decisions. Knowledge no longer has to live where a
Confluence space happened to put it.
archive/ holds retired procedures, old product work, and superseded pages. We
want that history, especially when it explains why something exists. We do not
want an agent presenting a deployment guide for a dead product as current
advice.
index.md is the front door. It lists the available pages, their status, and a
one-line description. log.md records ingests, maintenance operations, and
checks in chronological order.
Finally, AGENTS.md and a company-wiki skill tell the agent how to operate.
They define the directory structure, required metadata, source rules, and the
steps for ingesting or checking material. The storage layer does not depend on
one model, although changing models still requires us to test the ingest,
editing, and checking workflows. The sources, pages, history, and maintenance
contract stay in the repository. The skill is basically the operation instruction for the whole knowledge base.
This borrows heavily from the established Docs as Code approach. Plain text gives us Git history, pull request review, and automated checks without a custom documentation platform.
LLM Wiki Architecture
The Confluence export preserved evidence before we reorganized it
We treated the move from Confluence as a one-way cutover.
The exporter used the official Confluence Cloud content API and retained three forms of every page:
- Readable Markdown with provenance metadata
- Confluence's storage-format XHTML for the page body
- Every attachment stored alongside its page
The metadata included the original page ID and URL, page hierarchy, version, authors, and creation and update dates. We exported 310 pages and 400 attachments. Personal spaces were excluded because this repository was for company knowledge, not personal notes.
Preserving the raw export first gave us room to be opinionated about the active wiki. We could merge five related Confluence pages into one infrastructure overview without losing the originals. We could split a sprawling page into a runbook and a reference page. We could remove Atlassian boilerplate from the active layer while retaining it in the snapshot.
The first compilation produced roughly 160 active pages and 90 archived pages. The remaining source pages were merged into other topics or kept only as raw evidence. The migration reorganized the evidence by subject, merging and splitting source pages where necessary.
The agent also carried useful screenshots into the curated pages at the steps where they mattered. That detail sounds boring until somebody has to follow a finance runbook whose six screenshots were "successfully migrated" into an unlabelled asset folder.
Some questions could not be answered from the export. Was a service still running? Had a proposed browser extension ever been built? Which of two different hostnames was current? The agent collected those questions for human triage. We recorded the answers as new sources before updating the wiki.
The agent sends ambiguity to human triage because resolving it without new evidence can introduce errors.
Migration Architecture
Self-maintenance is a controlled Git workflow
In our current workflow, a human adds a source and invokes the coding agent. The agent does not autonomously monitor every company system. The source might be a meeting transcript, a customer call, a vendor document, or a change in an application repository.
The agent reads the complete source, identifies what changed, and searches the wiki for affected subjects. It checks whether the source contradicts any existing page. Then it proposes edits, new cross-references, archival changes, or a new page if the knowledge has no suitable home.
Every active page has frontmatter that records:
- A human owner
- The date a human last verified it
- Its sources
- Its status
- Open-ended tags
Status does not describe writing status, but is more of a 'trust layer'. current means verified
and in use. stale means the page was true once but has not been re-verified.
stub means the missing information is itself worth recording. superseded
points readers to the replacement.
Accepted decisions are immutable. If the company changes its mind, the agent creates a new decision and links the two records. That follows the same logic as Architectural Decision Records: preserve the context and consequences instead of rewriting history until the old choice looks irrational.
Before a change is ready for review, two scripts inspect it:
The first checks metadata, source paths, broken links, index coverage, orphaned pages, and path-specific rules. The second looks for factual literals such as dates, hostnames, commands, and identifiers that do not appear in the cited sources.
The evidence check reports suspects rather than pretending it can prove truth. A paraphrase may be legitimate. A command copied from an uncited document is not. A human makes that call in review.
After approval, the agent updates the relevant index and appends the operation to the log. That closes the loop. The next agent session can see both the current knowledge and how it got there.
Source Update Architecture
Hierarchical indexes give plain files room to grow
The suspiciously primitive part of this design is index.md. One Markdown
file cannot remain the complete catalog for a company wiki containing
thousands of pages, so its job has to change as the wiki grows.
At moderate size, the root index can list every page. As the wiki grows, it becomes a routing index that points to smaller domain indexes:
The root index answers which part of the company probably contains the answer. A domain index narrows that to a small set of pages. The page itself carries the details, sources, status, owner, tags, and links.
We do not keep stretching one index until it becomes useless. We repeat the same pattern at the level where another routing decision becomes helpful. The agent updates these indexes when pages move or change, and the linter catches pages that disappear from navigation.
Agents do not need a perfect first search
Conventional search interfaces put a lot of pressure on one query. A person enters a phrase and expects the correct page near the top of one ranked list.
Coding agents can search differently. They can run several searches in parallel, inspect the vocabulary in early results, reformulate weak queries, follow links, compare candidate pages, and check Git history. The first search only needs to provide a useful clue for the next one.
An agent looking for deployment instructions might search for deploy,
release, docker stack, the service name, its hostname, and a repository
identifier. Exact strings often beat clever abstractions here. You do not need
advanced mathematics to find a hostname in a Markdown file. grep is having a
perfectly good day.
Agent coding harnesses already provide fast file discovery, parallel reads,
frontmatter parsing, and content search with tools such as
ripgrep. The wiki gives those tools
clean text, predictable paths, and explicit links.
This does not make primitive search inherently more accurate. The reliability comes from the process around it. The agent can reject stale results, triangulate across several pages, inspect cited evidence, and continue when the first answer is incomplete.
Difficult searches become maintenance input
When an agent struggles to find a page, the failure tells us something useful. The page may need a better summary, an alias, another cross-reference, a more specific filename, or a different location.
After answering the question, the agent can propose that improvement. The next search starts from a better wiki.
We have not validated this design with thousands of active pages yet. Our expected path is to add generated catalogs, ownership views, full-text indexes, or specialized retrieval services when navigation and latency measurements justify them. Markdown remains the durable knowledge layer underneath. That lets us add infrastructure in response to observed limits rather than beginning with a search project before we have a wiki worth searching.
How simple search can scale with agentic search
Dedicated knowledge systems can put the same controls behind a better interface
Git works for us because we are a technical company. We already use branches, diffs, pull requests, and coding agents every day. Asking somebody to review a Markdown change feels normal to us. In many companies, it would feel like being asked to review the accounting policy in a terminal.
The architecture depends on the controls rather than the git command. A
dedicated knowledge management system can support the same model if it provides:
- Complete version history
- Reviewable changes rather than silent overwrites
- Rollback to an earlier accepted version
- An acceptance workflow with named reviewers
- Sources and provenance attached to the proposed change
- Ownership, trust status, and an audit log
- Permissions and an API that agents can use without bypassing human review
A purpose-built interface could make this workflow better than our current repository. It could show an agent's proposal beside the accepted page, explain which source supports each change, and let a reviewer accept or reject individual parts. Changes could be routed to the page owner automatically. Contradictions and stale claims could appear in a review queue instead of a terminal report.
The same applies to navigation. A dedicated system does not need a literal
index.md, but it does need simple front doors that agents can inspect. These
might be domain home pages, catalogs, filtered collections, or generated views
that list documents with short summaries, owners, trust status, and links to
related areas. The format matters less than giving the agent a cheap way to
narrow the search before opening full documents.
Search should also expose useful primitives rather than only one answer box. An agent needs to run several queries, inspect candidate results, try another term, follow relationships, apply metadata filters, and compare sources. A knowledge management system designed for this workflow would make those steps available through a stable API while showing the resulting search trail to the human reviewer.
This could be a better future for knowledge management: approachable navigation for people, simple machine-readable front doors for agents, and agentic search that improves through several fast attempts instead of betting everything on one search term.
That experience would be easier for finance, sales, operations, and other teams that should not have to learn Git vocabulary to maintain company knowledge. It could also provide more granular access control than a repository where clone access usually exposes every file.
Any replacement still has to keep the controls. A pleasant editor that hides provenance or lets an agent overwrite approved content would be a regression. The agent should submit a proposal, the responsible person should accept it, and the system should retain both the previous version and the reason for the change.
Our Git repository is therefore a useful implementation and a stepping stone. A dedicated knowledge system may be the better endpoint as the wiki spreads beyond the engineering team. Its job is to make review, navigation, and acceptance more approachable without weakening the audit trail underneath.
The practical gain is less documentation bookkeeping
The immediate benefit is mundane, which is usually a good sign for internal tools. People provide a source and review the proposed integration. They no longer have to remember every page affected by it.
Runbooks now state whether they are current, stale, incomplete, or superseded. An old procedure can remain available for historical context without appearing next to today's instructions as an equally valid option.
Knowledge gaps are visible too. Unknown infrastructure ownership, incomplete account inventories, and unverified procedures become named gaps. That makes them assignable work rather than surprises waiting for an audit or incident.
The cross-functional updates have been particularly useful. A vendor change can affect infrastructure documentation, security approval records, and a customer-specific deployment page. One reviewed ingest can update all three.
Experiments also get stricter treatment. A running experiment must declare its metric before it starts. A concluded experiment records results and an outcome, including failure. The failed work stays in the wiki so the next person does not rediscover it with great confidence six months later.
We have deliberately not attached a percentage to the productivity gain. We have not measured one. What we can observe is the mechanism: maintenance tasks that used to depend on memory are now proposed together, checked, and reviewed as a change set.
After several years, the wiki becomes company memory
Over time, we expect the repository to hold thousands of pages connected to tens of thousands of sources. It would record how the company operates, what it decided, what customers requested, which experiments failed, and how products and infrastructure changed.
A three-year-old customer call is not useful merely because the transcript still exists. It becomes useful when its relevant details have already reached the customer, product, and decision pages that people consult today.
This resembles Vannevar Bush's 1945 description of the memex in "As We May Think": a durable body of records connected by associative trails. Bush left the work of building and maintaining those trails to the owner. We use an LLM agent for that maintenance work.
The interface will evolve. We expect more domain indexes, ownership dashboards, review queues, and generated catalogs. The foundation can stay boring: sources below, maintained knowledge above, and Git history connecting the two.
Company Wiki vision
Human verification remains necessary
An agent cannot verify that an old procedure still works by reading the old procedure. It cannot detect a system change that nobody recorded. It cannot know which side of a genuine contradiction is correct, or whether the source itself contains a mistake.
Our migration made that painfully clear. At one point the compiled wiki inferred that retired infrastructure was still costing money because no source recorded its decommissioning. The inference sounded reasonable and was wrong. A human answered the question, and that answer became a new source.
This is why every page has a human owner and a verification date. The linter can find a broken link. The evidence checker can find a hostname absent from the cited documents. Neither can confirm that production still matches the runbook.
Access control is another limit. A Git repository does not provide the same page-level permissions as a hosted wiki. Anyone who can clone the repository can usually read its raw attachments, including historical files. Repository boundaries and access policies have to reflect that.
The model endpoint and agent harness are part of the security boundary too. Our sources can contain contracts, customer calls, and attachments. A production setup needs an approved data-retention policy, authorization appropriate to the source, imported text treated as untrusted data, restricted tool permissions, and human review before writes. A prompt injection hidden in a source document must not become an instruction to the maintenance agent.
The system assumes agents need constraints and review. Automating the tedious parts does not remove responsibility from the humans who understand the company.
Start with one source and one maintenance contract
You do not need to migrate 310 pages to test this pattern.
Create a repository with raw/, wiki/, index.md, log.md, and an
AGENTS.md file. Put one source in raw/, tell the agent that sources are
immutable, and ask it to create a grounded page with a visible source link.
Then add a second source that changes something in the first. This is the useful test. Watch whether the agent updates the existing page, records the conflict, and preserves both sources. A system that only produces another summary has missed the point.
Once the workflow feels right, add checks for source existence, broken links, index coverage, and required metadata. Let the structure grow in response to real documents. Our own schema became simpler after the migration because an early page-type taxonomy created work without improving retrieval.
Karpathy's original LLM Wiki idea file is intentionally abstract. Give it to your coding agent, add the rules your company needs, and learn from the first few ingests before designing the grand knowledge architecture.
FAQ about LLM-maintained company wikis
What is an LLM-maintained wiki?
It is a persistent collection of interlinked pages that an LLM agent updates as new sources arrive. The agent handles filing, synthesis, cross-references, and routine checks. Humans provide sources, resolve ambiguity, and review changes.
How do you stop the LLM from inventing company facts?
Every material claim must point to an original source. Pages carry owners, verification dates, and trust statuses. Automated checks find missing sources and suspicious literals, while Git review keeps a human in the approval path. These controls reduce unsupported claims; they cannot prove that every source or interpretation is correct.
Can a Markdown company wiki scale to thousands of pages?
We have tested this structure with roughly 160 active pages, not thousands. Our expected path is to route a small root index to domain and subdomain indexes, then add generated catalogs or full-text search when observed navigation and latency data justify them. The Markdown source of record does not have to change when those retrieval tools arrive, but the design still needs validation at larger scale.
Interested in building high-quality AI agent systems?
We prepared a comprehensive guide based on cutting-edge research for how to build robust, reliable AI agent systems that actually work in production. This guide covers:
- Understanding the 14 systematic failure modes in multi-agent systems
- Evidence-based best practices for agent design
- Structured communication protocols and verification mechanisms