An agent stored 10,134 memories in 32 days. Only 38 were clean enough to keep.

Every agent memory store has a number nobody checks: how much of what it holds is true, current, and worth retrieving. One mem0 user checked. They ran mem0 in production for 32 days, from February 23 to March 26, 2026, with one AI agent, one human, daily conversations, and a Qdrant backend, then audited the store and filed the results as a GitHub issue.

Read the provenance before the numbers. This is one user’s self-reported audit of their own store, not a benchmark, not a mem0 measurement, and not a controlled study. The counts below are the reporter’s own, from the issue. That framing matters, and the numbers are still worth your attention, because the failure modes are structural and they are not unique to one stack.

What the audit found

The store wrote 10,134 entries in 32 days. Automated triage cut that to 6,264. The reporter then read the rest entry by entry, in eight batches. 224 entries survived the full process. Of those, only 38 were clean enough to keep as-is; the other 186 had to be deleted and rewritten from scratch. By the reporter’s count, 97.8 percent of the store was junk.

The biggest single category was not hallucination. It was restatement: 52.7 percent of the junk was the store re-saving the agent’s own boot-file and system-prompt facts, session after session. The setup the agent runs on got extracted, stored, retrieved, and extracted again, until more than half the memory was the agent remembering its own configuration back to itself. Nobody wrote a bad entry. The pipeline just never asked whether an entry was already known, or worth knowing at all.

The tell: a preference nobody has

The clearest exhibit in the issue is a single sentence: “User prefers Vim.” The 2B extraction model invented it once. The store saved it. On later sessions, that entry came back in recall, the extractor treated it as conversation content, and saved it again. By the end of the audit it existed 808 times, and 191 of those were exact copies of the same sentence.

Nobody in the system uses Vim. The detail that should stay with you is the mechanism, not the joke. A memory store with a recall step and no provenance check is a feedback loop: whatever gets in, true or invented, gets cited by the system itself and re-enters as fresh evidence. The model wrote the first copy. The pipeline wrote the other 807.

A bigger model did not fix it

The obvious fix is a better extraction model, and the reporter tried it mid-run. Extraction ran on gemma2:2b for the first 20 days, then Claude Sonnet 4.6 for the last 12. The hallucinations stopped. The junk rate barely moved.

The issue’s explanation is the part worth quoting in your next design review: the stronger model followed the permissive extraction prompt more faithfully. Given instructions to extract freely and no way to decline, a better model extracts more, more accurately, including the boot files, the restatements, and the recalled entries sitting in context. The bottleneck the reporter names is the extraction prompt and the missing quality gate, not model size. There was no step in the pipeline whose job was to say no.

It is not one tool, and not one layer

The mem0 audit is about extraction quality. The same week of issue trackers shows the neighboring layers failing in their own way.

At the storage layer, an openclaw issue reports a memory index table, SQLite’s memory_index_chunks, growing unbounded on a production instance: 38,985 rows, 3.3 GB, flagged P1 for crash-looping the app. Partial fixes shipped September 30; automatic retention is still unresolved. The entries may or may not be junk, but nobody had put a ceiling on the table, so growth itself became the incident.

At the context layer, an openai/codex issue reports compaction retaining unbounded image payloads: 92 images kept, a 4.16 GB session file, and about 8,000 tokens of headroom left after compacting. The summary shrank. The payload did not. And Anthropic’s own tooling has an open issue for an auto-memory index outgrowing its 200-line read window, where the index grows past what the reader ever loads.

Three layers, three failure modes. Extraction writes junk. Storage grows unbounded. Compaction keeps the payload. Each layer needs its own audit, and each audit is a count before it is anything else.

Why this is a data engineering problem

Translate the story out of agent vocabulary and it is familiar. A memory store is a table. The extractor is an ingest job. Recall is a read path that feeds the write path. What was missing is the stack every production ingest pipeline is expected to have: a schema for what qualifies as a record, a reject path for what does not, deduplication against what is already stored, and a periodic quality audit on the output. Nobody would let an ETL job write unvalidated rows into a production table for 32 days and call the result a data asset. Agent memory is getting a pass because the rows are sentences.

Grading a sample of stored entries against the conversations that produced them is eval work, plain and simple, and it is the same discipline as grading any other model output before you trust it downstream. That is the job the AI Eval Harness was built to template, if you want the graded-review structure instead of a spreadsheet.

Our reading, not the reporter’s

The reporter’s conclusion is about one pipeline’s prompt and gate. Ours is broader: an agent’s memory is a production data store that happens to be written by a model, and it inherits every obligation that comes with that. Retention, deduplication, provenance, quality sampling. The tooling is new. None of the obligations are.

So the question from the carousel stands. What is living in your agent’s memory store right now: audited on a schedule, capped but never read, growing while nobody looks, or not built yet? Whatever the answer, start with the count. The reporter did, and 38 out of 10,134 is what the count had to say.

This post started as an Instagram post →