Agent memory is moving to Markdown files — from Karpathy's LLM wiki to Google's OKF 0.2: what the spec covers and what it doesn't
Keeping an agent's long-term memory in a folder of Markdown files: from Karpathy's LLM wiki gist to Google's Open Knowledge Format (OKF) v0.2.
On this page
What does it take for an agent to remember things after a session ends? Through the first half of 2026, the usual answer was a vector database, a knowledge graph, and a separate service to manage them. Since April this year, a different answer has been taking hold: a single folder of Markdown files. It began with the "LLM wiki" idea file Andrej Karpathy posted on April 4. In June, Google Cloud turned this pattern into an open file convention, Open Knowledge Format (OKF) v0.1, and in July it announced v0.2, which adds fields for provenance, verification and expiry. In late August an essay titled "Agent memory as a file format" got 191 points on Hacker News, and in September a git-based memory tool implementing OKF gathered more than 700 stars. After reading the OKF spec itself, the two official announcement posts, Karpathy's gist, the essay and the tool repositories, we separate what the spec actually defines from what tools and the community claim. We did no measurements of our own.
1. Executive summary
type, and nothing elseSpec §4.1, §11 [1]
Spec §1, §5, §10 [1][3]
verifiedUnverified → machine-confirmed → human-reviewed, spec §5.3 [1]
type, and not to reject anything unknown. It deliberately does not define search, embeddings or serving. That is why it fits any agent framework, and also why there is no evidence yet that "using OKF improves memory." What v0.2 added is not performance but trust. By recording in the frontmatter who created a document, who checked it and until when it is valid, it lets you tell documents an agent wrote apart from documents a person approved. The practical takeaway for now is those trust fields; figures the tools advertise, such as "Slashes token bloat by 80%," should be read as developer claims.Some definitions first. Frontmatter is the YAML metadata block wrapped in --- at the top of a Markdown file. A bundle is a directory tree of OKF files, distributed as a git repository or an archive. Producer (the side that writes the files: people, agents, export pipelines) and consumer (the side that reads them: agents, UIs, search indexes) are the spec's terms. A concept is the unit one file describes; it can be a table, a metric, an API, a playbook, a runbook or anything else. The LLM wiki is a pattern Karpathy named, in which an agent reads source material and itself builds and revises a set of interlinked Markdown pages. Vector search turns text into numeric vectors to find items with similar meaning, and BM25 is a traditional full-text search score based on word matches.
Spec quotations in this report are from the original SPEC.md (v0.2) in the GoogleCloudPlatform/open-knowledge-format repository, and the tools' performance figures are claims from each repository's README. Star counts are GitHub API values from 2026-09-21.
2. Six months — from a gist to a spec to tools
| When | What happened | Source |
|---|---|---|
| 2026-04-04 | Karpathy publishes the gist llm-wiki.md, an idea file in which "the LLM incrementally builds and maintains a persistent wiki — a structured, interlinked collection of markdown files that sits between you and the raw sources." Not runnable code but a description to paste into an agent's prompt | Gist [4] |
| 2026-06-12 | Google Cloud's data analytics team announces OKF v0.1, "an open specification that formalizes the LLM-wiki pattern into a portable, interoperable format." The only required field is type. 87 points on Hacker News (HN) | Announcement [2], HN [9] |
| 2026-07-24 | OKF v0.2 announced. Adds five trust signals (provenance, trust, freshness, lifecycle, attestation) as frontmatter fields and a new concept type. v0.1 bundles are read as they are | Announcement [3] |
| 2026-08-11 | The spec moves to its own repository, GoogleCloudPlatform/open-knowledge-format (Apache-2.0), including a reference agent and four example bundles | Repository [1] |
| 2026-08-13 | MCP Memory released. OKF v0.2 Markdown + SQLite FTS5, giving Claude Desktop and Cursor six memory tools. 70 points on HN | Repository [6], HN [12] |
| 2026-08-31 | Cal Paterson's essay "Agent memory as a file format." The memoryfield proposal: distribute Markdown pages plus an optional SQLite vector index as a single zip. 191 points and 96 comments on HN | Essay [5], HN [7] |
| 2026-09-05 | OKF Agent Memory released. A single Go binary, OKF v0.2 bundles + in-memory BM25 search, an MCP server. 81 points on HN, 707 stars (09-21) | Repository [8], HN [10] |
The three threads point the same way. Karpathy proposed a way of working in which the agent writes the wiki itself, Google Cloud defined a convention for representing that wiki as files, and independent developers attached search and MCP servers to that convention. GBrain, which this blog covered in June, belongs to the third of these layers, the product layer. This report deals with the format layer beneath it.
3. What the OKF spec actually defines — one required field
The spec's opening paragraph sums up its character: "The format is intentionally minimal: a directory of markdown files with YAML frontmatter. There is no schema registry, no central authority, and no required tooling. If you can cat a file, you can read OKF; if you can git clone a repo, you can ship it" [1].
| Item | What the spec says | Notes |
|---|---|---|
| Bundle structure | A directory tree of Markdown files. Grouped into subdirectories, but the producer decides the structure, independent of the domain. Distributed as a git repository (recommended), a tar/zip archive, or a subdirectory of a larger repository | §3 |
| Reserved filenames | Only index.md (a directory listing, for progressive disclosure) and log.md (update history) are reserved. Every other .md is a concept document | §3.1, §8, §9 |
| Required field | Just type. The producer picks the value, with no central registration (for example Metric, Playbook, API Endpoint). Consumers must treat an unknown type as a generic concept document | §4.1 |
| Recommended fields | title, description (one sentence), resource (URI of the asset described), tags | §4.1 |
| Extensions | Producers may add any key. Consumers must preserve keys they don't know and must not reject them | §4.1 |
| Links | Ordinary Markdown links. The file path is the concept's identity, so the directory itself becomes the graph | §6 |
| Conformance | Conformant if every non-reserved .md has parseable frontmatter and a non-empty type. Consumers must not reject a bundle for missing optional fields, unknown types, unknown keys, broken links or a missing index.md | §11 |
| Version | The bundle-root index.md may declare okf_version: "0.2". Even an unknown version should be read on a best-effort basis | §12 |
| What it does not define | A taxonomy of types; storage, serving and query infrastructure; replacing domain schemas such as Avro, Protobuf or OpenAPI; packaging of executable code | §1 Non-goals |
The last two rows of the table are the point. The conformance rules are almost all of the form "do not reject," and what the spec leaves out includes search and serving. OKF is a minimal convention that writers and readers must follow, not a way to retrieve memories. So you can view a bundle in Obsidian, search it with ripgrep, or layer an embedding index on top of it without violating the spec. The announcement post's example is a data catalog: concept files sit in datasets/, tables/ and metrics/ directories under sales/, linked like [customers](/tables/customers.md) [2]. Using the same frame for a coding agent's project memory is the tools' choice, not the spec's.
4. v0.2's five trust signals and three trust tiers
The motivation for v0.2 is in §1 of the spec: "Increasingly, a knowledge corpus is not authored once and then read: it is continuously written and maintained by agents. When most concepts are machine-generated, a consumer needs answers that a plain markdown-plus-frontmatter convention does not make first-class: What was this created from, and how was it verified? How much should I trust it? Is it still true? Is it the current version? Was this number produced the way we said it must be?" [1]. The announcement post calls the mechanisms that answer these five questions the five trust signals [3]: Provenance, Trust & Verification, Freshness, Lifecycle, and Attested Computations. The first four are frontmatter fields; the last is a new concept type.
| Question | Field | What the spec says |
|---|---|---|
| Created from what? (provenance) | sources | The list of materials the concept derives from: external documents, paths inside the bundle, or scope descriptors such as "all queries in BigQuery project X." Each entry can carry objective credibility signals such as author, usage_count and last_modified. Replaces v0.1's # Citations list in the body |
| How far can it be trusted? (trust) | generated: { by, at }verified: [ { by, at } ] | generated records who produced the current content and when its last meaningful change happened (replacing v0.1's timestamp). verified is a list of events in which the content was confirmed against its sources or its asset. The two are deliberately kept apart: "who wrote a concept need not be who confirmed it." Actors are written producer/version for agents, human:<id> for people, and process:<name> for processes |
| Is it still true? (freshness) | stale_after | A single absolute instant. The concept is stale when now >= stale_after. An absolute instant was chosen over a relative TTL to keep the check "a plain comparison with no reference to when the concept was read" |
| Is it the current version? (lifecycle) | status | draft → stable → deprecated. stable if absent. Deprecated concepts are kept for links and history |
| Was it computed correctly? (attestation) | Attested Computation type, executor, attester | A new concept type that carries not just what a value means but "a sanctioned way to compute it" and "the means to check it," that is, to confirm that the sanctioned way was actually run. The runtime protocol and sandboxed execution are left to a future version |
Sources: [1] §5, §10, §13; [3]. Row names follow the five questions in §1 of the spec; the labels in parentheses follow the announcement post's categories.
Trust tiers are derived from verified. No key means unverified; confirmation only by actors other than human: means machine-confirmed; confirmation by a human:<id> means human-reviewed. The spec states plainly that "Trust tiers are advisory signals, not access control," but the announcement post describes their use this way: a consumer can express "only surface human-reviewed metrics in the executive dashboard" as a single frontmatter filter [3]. This is where it connects with agents overclaiming "review complete", which this blog covered two days earlier. When a person's confirmation is recorded in a separate field on documents an agent wrote, what the agent said it checked and what a person actually checked become distinguishable at the file level.
Compatibility is explicit: "A v0.1 bundle drops in unchanged." Only two things were renamed (timestamp → generated.at, # Citations → sources), and v0.2 consumers may fall back to reading the old forms [1][3].
5. How the three approaches differ — LLM wiki, memoryfield, OKF
All three proposals can be summed up as "remember with files," but they deal with different layers.
| Item | LLM wiki (Karpathy, April) | memoryfield (Paterson, August) | OKF v0.2 (Google Cloud, July) |
|---|---|---|---|
| What it is | An operating pattern. Three layers: raw sources (immutable) → wiki (written by the agent) → schema file (CLAUDE.md, AGENTS.md) | A proposed distribution format. Markdown pages + an optional SQLite vector index, packaged as *.memoryfield.zip | A file convention. Markdown + frontmatter, path as identity, trust fields |
| What it defines | Three operations: ingest, query and lint. People supply sources and questions; the agent summarizes, cross-references and organizes | A page size cap of about 8KB (2,000 tokens), YAML frontmatter (title, timestamps, UUID, summary), nomic-embed-text-v1.5 embeddings (270MB, runs without a GPU) | Required type, reserved filenames, link rules, trust and lifecycle fields, conformance |
| Search | Not defined. Obsidian sits alongside, used like an IDE | One semantic search + one parallel read: at most two tool calls. The point is to cut the repeated tool calls spent following related documents | Outside the spec. Any index can be layered on top |
| Trust and lifecycle | The lint operation checks for contradictions, stale claims and gaps | No explicit fields | sources, generated, verified, stale_after, status |
| Distribution | Local folder | zip, local, S3, GitHub, HTTP | git (recommended), tar/zip, subdirectory |
| Intended audience | Knowledge building by individuals and teams | Memory that can move between agents | Knowledge exchange between organizations, data catalogs |
Sources [4][5][1]. The memoryfield figures are as stated in the essay, and the essay does not mention OKF.
Paterson's essay criticizes existing memory systems on three points: the lock-in that comes with the move into a platform business; the complexity of needing pgvector, Neo4j and a dedicated model; and the slowness of calling tools many times while walking a knowledge graph. His answer is a single one: Markdown with simple embeddings [5]. The most common reaction on HN boiled that down to one line: "A whole lot of text to say 'Just use text'" (thm), and "It feels like 50% of AI 'progress' is just finding different ways to say 'tell the model things in English'" (andyfilms1). On the other side were comments that took issue with who writes the memory, such as "'harness managed' memory is utter garbage, I am convinced … auto managed memory was always the problem" (JohnMakin), and a quality concern that mistaken past explorations rank high in semantic search [7].
6. The tools and their claims — nothing verified yet
| Tool | What it does | README claims | Our assessment |
|---|---|---|---|
| OKF Agent Memory okf-memory, Go, MIT, 707 stars | Keeps an OKF v0.2 bundle in knowledge/ inside the project and searches it through an MCP server. A rule set of 100–150 tokens always sits in AGENTS.md (the push layer), while the bundle is a layer of knowledge files read on demand (the pull layer) | In-memory BM25 search under 300µs, "Slashes token bloat by 80%," memory under 15MB, no external database or dependencies | The README gives no denominator or measurement method for the 80%. A developer claim |
| MCP Memory fellowgeek, MIT, 217 stars | OKF v0.2 Markdown in memory/ and a SQLite FTS5 index in .mcp_memory/. Six tools: store, get, search, delete, read the last checkpoint, and update it | Key lookups under 20ms | No benchmark. A developer claim |
| Reference agent Google Cloud, Apache-2.0 | A proof of concept included in the spec repository. Generates OKF bundles automatically from BigQuery datasets. Four example bundles: GA4, Stack Overflow, Bitcoin, retail | "The format itself is the contribution; this agent and the visualizer exist to make the format tangible" | A catalog generator, not a memory tool |
Sources [8][6][1]. Star counts from the GitHub API on 2026-09-21. Both tools were created between 2026-08 and 2026-09, so each has a little over a month of operating history.
What was valuable in the HN discussion of OKF Agent Memory was not the tool's numbers but the debate over retrieval methods. One user (vshulcz) reported success rates of 61% for BM25 and 37% for semantic search on a personal corpus and wrote: "The real cliff is reranking … and staleness. Vector search has zero concept of 'superseded info'." Another user (langs) said: "What nobody has gotten close to solving is maintenance and provenance - what goes into memory, what qualifies as truth, how stale memory gets invalidated/superseded" [10]. Both are one person's experience, but the second point is exactly what v0.2 turned into fields. The format does not supply the answers, but there is now a place to write them down.
Adoption signals are still at the margins. GitBook says "A GitBook space already looks like the directory OKF describes" and presents itself as "an OKF-ready knowledge source," but it does not describe an export feature [11]. Beyond that, there is a PR adding v0.2 support to a wiki tool (nexwiki), a static publishing engine (Kiso), and Obsidian-style plugins, each drawing a few points on HN. At the time of writing, we could not find any major agent framework or coding agent that has announced OKF as its memory format.
7. Checklist to apply today
These are operating recommendations we drew from the spec and the three proposals. None of them has had its effect verified by a controlled measurement.
- Start memory as a folder. Put a directory such as
knowledge/in the project and write atypein each file's frontmatter. That alone makes it an OKF-conformant bundle, which any tool that reads OKF must not reject. Individual tools may add conventions of their own. - Write
generatedon files an agent wrote andverifiedon files a person checked. Keeping the two apart is the core of v0.2. If you tell the agent in its prompt, "When you create a new concept file, addgenerated: { by: <agent/version>, at: <ISO 8601> }and leaveverifiedempty," the file itself records whether a person has reviewed it. - For facts that will obviously go stale, write
stale_afteras an absolute time. Prices, model names and API limits are examples. On the consuming side, warn before putting a file withnow >= stale_afterinto the context. - Use
index.mdfor progressive disclosure. A list with a one-line description in each directory makes it easier to have prompts and tools open only the files they need instead of reading everything. Karpathy's schema file (CLAUDE.md, AGENTS.md) should say only how to use these lists. - Make lint a recurring job. Have the agent check for contradictions, stale claims and broken links, and record the results in
log.mdin date order. The format only gives stale memories a place to be marked; this job is what actually finds them, and whether to invalidate them has to be decided and applied through a separate policy. - Search is an index, not the source. Whether it is BM25 or embeddings, keep it as something you can rebuild from the files, and put the files in git. When you switch tools, the memory stays.
- Measure figures like "Slashes token bloat by 80%" on your own repository. Run the same task with no folder, with the folder only, and with the folder plus a search tool, and compare input tokens and results. None of the material we read contains that comparison.
- Build one path that uses only human-reviewed files. For work where errors are costly, such as deployment decisions or customer support, feed in only files whose
verifiedincludes ahuman:actor. It is the same structure as the dashboard example in the announcement post.
8. What this is not yet
- We did no measurements of our own. This report cross-checks the spec, the announcement posts, the gist, the essay, repository READMEs and HN discussions. None of the material we read contains a controlled comparison showing that file-based memory beats memory built on a vector database.
- OKF is at 0.2. The spec itself calls it "a starting point, not a finished standard," and leaves the runtime protocol, the attester ABI, sandboxing, caching and semantic-layer templates under "Considered and deferred." Fields may change further before 1.0.
- The starting point is the data catalog. The announcement posts' examples and the reference agent describe BigQuery tables and metrics, and Google's own adoption is in Knowledge Catalog. Using OKF for a coding agent's project memory is an extension by independent tools; the spec has not validated that use.
- Every tool figure is a developer claim. 300µs, 80% and 20ms are all sentences in READMEs, with no measurement conditions. Both tools are one to two months old.
- The HN retrieval comparison is one person's experience. BM25 at 61% versus semantic search at 37% comes from a personal corpus, and the method was not published. We cite only the direction.
- The date of Karpathy's gist is as the page displays it. The GitHub API returned no metadata, so we used 2026-04-04 as shown on the page. For stars, we saw only the page's "5,000+" display.
- Repository figures are a one-day snapshot. Star counts and recent commits are values from 2026-09-21. The announcement posts' dates follow the page metadata (datePublished); because of the time zone difference, the pages display them one day later (June 13, July 25).
- What this report does not cover. Feature comparisons with managed memory services such as mem0, Letta and Zep, and results on long-term memory benchmarks such as LoCoMo and LongMemEval, are out of scope. The debate over formats and the debate over benchmarks have not come together yet.
9. Sources
Checked on 2026-09-21. [1]–[3] are the spec and the vendor's announcement posts, and [4] and [5] are the proposers' originals. [6] and [8] are the tools' descriptions of themselves, [7], [9], [10] and [12] are community discussions, and [11] is a tool vendor's blog.
| Source | What we checked | Scope of the evidence |
|---|---|---|
| [1] GoogleCloudPlatform/open-knowledge-format · SPEC.md (OKF v0.2) · Repository | Motivation, goals and non-goals, bundle structure, reserved filenames, required and recommended fields, link rules, trust and lifecycle fields, trust tiers, attested computations, conformance, versioning, changes from v0.1. Repository created 2026-08-11, Apache-2.0, 525 stars | The spec itself |
| [2] Google Cloud · How the Open Knowledge Format can improve data sharing (2026-06-12) | The v0.1 definition ("formalizes the LLM-wiki pattern"), fields, the sales/ example structure, Knowledge Catalog integration, authors (McVeety, Hormati) | Vendor announcement post |
| [3] Google Cloud · OKF v0.2 adds trust signals (2026-07-24) | The five trust signals and their fields, how trust tiers are used (dashboard filter), v0.1 compatibility, a round-trip demo through Knowledge Catalog | Vendor announcement post |
| [4] Andrej Karpathy · llm-wiki.md (gist, 2026-04-04) | The three-layer structure (raw sources, wiki, schema), the ingest, query and lint operations, using Obsidian alongside, example uses | Proposer's original |
| [5] Cal Paterson · Agent memory as a file format (2026-08) | The memoryfield format (8KB pages, frontmatter, nomic-embed-text-v1.5, zip), criticism of existing systems, the two-tool-call argument | Proposer's original |
| [6] fellowgeek/mcp-memory | OKF v0.2 + SQLite FTS5, six tools, MIT, 217 stars, created 2026-08-13 | Tool README (self-description) |
| [7] Hacker News · Agent memory as a file format discussion (2026-08-31) | 191 points and 96 comments, the "Just use text" reaction, criticism of automatically managed memory, concerns about semantic search quality | Secondary source. Community discussion |
| [8] okf-memory/okf-agent-memory | The DMAA structure (AGENTS.md push layer + bundle pull layer), the BM25 300µs, 80% and 15MB claims, MCP server, MIT, 707 stars, created 2026-09-05 | Tool README (self-description) |
| [9] Hacker News · Google proposes Open Knowledge Format based on Markdown (2026-06-13) | 87 points, 15 comments | Secondary source |
| [10] Hacker News · OKF Agent Memory discussion (2026-09-05) | 81 points and 32 comments, an experience report on BM25 versus semantic search (vshulcz), the maintenance and provenance problem (langs), praise for the single binary | Secondary source. Community discussion |
| [11] GitBook · What is OKF? (2026-06-18) | The "OKF-ready" wording; no export feature specified | Tool vendor's blog. Has a stake |
| [12] Hacker News · Show HN: MCP Memory (2026-08-13) | 70 points, 36 comments | Secondary source |
Editorial standard: in each section we kept what the spec defines separate from what tools and the community claim, and we labeled every tool performance figure as a "claim." Where secondary blogs call OKF a "Google standard," we described it by the actual scope of the announcement posts instead.