How to Build Agent Memory Without Letting It Drift or Corrupt Itself

Overview

Agent memory sounds simple: store what matters, retrieve it later, done. A single-turn prompt can only be wrong once, though. A memory store that's wrong stays wrong, and keeps getting handed back to the model until someone notices.

Take a scheduling agent that remembers a user's stated timezone. Say it reads "I'm usually free after 5, my time" as after 5pm Eastern, when the user is on the West Coast, and writes that into durable memory. Every meeting it books after that inherits the same three-hour error, and the agent has no reason to doubt itself: the memory it reads back agrees with what it just did.

That's memory drift. The cause is almost always architectural rather than a model problem: a store the agent can write to freely, with nothing between a candidate memory and the record except the model's confidence in itself. Anthropic's memory tool documentation is unusually direct about where that responsibility sits. The tool is client-side, and the docs say it plainly: "Your application executes every file operation Claude requests, so these safeguards are your responsibility." The model asks. Your code decides.

Context-window management is a different subject and we treat it as one. What follows is about what survives a run.

What Most Agents Need

Most of what gets written about agent memory assumes a scale of problem most teams don't have. Episodic versus semantic versus procedural stores, tiered retrieval, a vector database with its own uptime to manage, all of it is infrastructure for an agent juggling many users and sessions, with a store large enough that retrieval precision is a real concern.

It helps to know what "large" means in the papers. LoCoMo, the long-term conversational memory benchmark, builds dialogues averaging 300 turns and 9,000 tokens across up to 35 sessions, and reports that long-context models and RAG both improve on the task while still lagging "substantially" behind human performance. Mem0, published against that benchmark in 2025, reports a 26% relative improvement over OpenAI on an LLM-as-judge metric, 91% lower p95 latency and over 90% token savings against passing the full conversation. Real gains, measured on 35-session dialogues. That is not where most single-workflow agents live.

If your agent runs one workflow and its memory need amounts to "what did this user tell me a few minutes ago, plus a handful of durable preferences," we'd build two things: a transient scratchpad discarded every run, and a short, append-only list of validated facts. Build the gate that decides what earns a place on that list first.

The four-way taxonomy is still worth knowing, because it tells you what a given record is for. LangGraph's documentation splits long-term memory into semantic memory, "the retention of specific facts and concepts", episodic memory, "recalling past events or actions", and procedural memory, "remembering the rules used to perform tasks", with short-term memory scoped to a single thread. Mapped onto storage, that gives four homes and four different lifetimes.

Table 1 — Four memory layers, their real lifetimes, and the specific failure that follows from confusing them.

The signal to add more isn't the calendar. It's evidence: once the durable list is big enough that retrieval starts missing things, or you've caught two entries disagreeing, add embeddings and pruning. Not before. A dedupe system built against forty facts solves a problem you don't have, and it's one more thing that can go wrong.

Transient vs. Durable

Most agent failures start when everything, scratch reasoning, half-finished tool output, and verified user facts, lives in one blob called memory. The agent can't tell a guess from a fact when it reads back, and neither can your retrieval layer.

Split the store into two lifecycles instead. Transient memory is short-term, task-scoped, and discarded when the task ends; it's allowed to be messy, since nothing reads it after the run. Durable memory is long-term and only updated after validation; every entry is something you're willing to let the agent believe weeks from now.

class Memory:
    def __init__(self):
        self.transient = {}  # reset every run
        self.durable = []    # validated, persistent

def reset_transient(memory):
    memory.transient = {}

def add_durable_memory(memory, item):
    memory.durable.append(item)

There's no path for the agent loop to write straight into durable because the only way in runs through the validator built next. Keeping add_durable_memory out of the model's reach is the cheapest guardrail in this entire system.

Give every durable entry a shape before you write the first one. Retrofitting provenance onto ten thousand rows is a migration; specifying it now is a dataclass.

@dataclass
class MemoryItem:
    text: str
    source: str          # tool call id, document id, or "user_utterance"
    source_span: str     # the exact text the claim came from
    created_at: float
    confidence: float
    validator: str       # which gate version admitted it
    superseded_by: str | None = None   # append-only: never overwrite

source_span answers "where did the agent get this", and superseded_by turns a correction into a record instead of an overwrite.

Validating Writes

A candidate memory earns its place by passing three checks, cheapest first: what kind of statement is it, does it contradict what's stored, is the model confident enough to trust. We'd order it that way not because the order changes the outcome, but because it wastes the fewest tokens on a candidate that was always going to be rejected.

Type first. A lexical filter catches the obvious anthropomorphic case for nothing: block "I feel," "I want," "I am becoming," and similar before any model runs. Those are the seeds of an agent reading its own invented internal state back in.

BLOCKLIST = ["i feel", "i want", "i am becoming", "as an ai"]

def is_self_referential(text):
    return any(p in text.lower() for p in BLOCKLIST)

What survives goes to a cheap classifier. Reject speculation, opinion and chain-of-thought before spending anything on the expensive checks:

CLASSIFIER_PROMPT = """
Classify the memory candidate as one of:
A) factual  B) hypothetical/speculative  C) opinion/emotion  D) internal reasoning
Candidate: "{candidate}"
Respond with a single letter.
"""

def classify_memory(candidate, model_call):
    return model_call(CLASSIFIER_PROMPT.format(candidate=candidate)).strip().upper()

Contradiction second. Ask the model directly whether the candidate conflicts with what's already stored, and make it commit to a confidence:

VALIDATION_PROMPT = """
Does "{candidate}" contradict any of these known facts?
{facts}
Respond as JSON: {{"contradiction": true|false, "confidence": 0-1}}
"""

def validate_memory(candidate, durable_facts, model_call):
    try:
        return json.loads(model_call(VALIDATION_PROMPT.format(
            candidate=candidate, facts="\n".join(durable_facts))))
    except (json.JSONDecodeError, TypeError):
        return {"contradiction": True, "confidence": 0.0}  # fail closed

A malformed response fails closed, aact:failure-behavior/fail-closed in the taxonomy's terms. An unparseable verdict is a contradiction, never permission to write.

def safe_write(memory, candidate, model_call):
    verdict = validate_memory(candidate, [m.text for m in memory.durable], model_call)
    if verdict["contradiction"] or verdict["confidence"] < 0.7:
        return False
    add_durable_memory(memory, candidate)
    return True

The 0.7 threshold isn't a law: raise it where a wrong durable memory is expensive, lower it where a missing one hurts more. A stricter version also requires the candidate to cite the tool output or document it came from, and rejects anything that can't. Anthropic's memory documentation lands in the same place for a different reason, noting that while the model "usually refuses to write sensitive information to memory files", the stronger guarantee is validation that "strips sensitive data before your handler writes the file." A gate that rejects a hallucination can also redact a credential.

This catches the largest failure, a hallucination stored as fact, but it has a blind spot: a confident model can still contradict a memory that wasn't in the facts list it was shown. Retrieval decides what the validator sees, so a contradiction check over a truncated candidate set is one you can defeat by having too much memory.

Duplicates and Contradictions

Second tier: don't build it until the rejection log shows duplicates or lookalike contradictions slipping through.

An LLM validator is good at meaning but too slow to run on every pair of memories. Embeddings give a cheap first pass:

from sentence_transformers import SentenceTransformer, util
model = SentenceTransformer("all-MiniLM-L6-v2")

def is_duplicate(candidate, memory, threshold=0.85):
    c = model.encode(candidate, convert_to_tensor=True)
    return any(util.cos_sim(c, model.encode(m.text, convert_to_tensor=True)) > threshold
               for m in memory.durable)

Two properties of that model matter before you trust its output. It maps text to a 384-dimensional vector, and, in the model card's words, "input text longer than 256 word pieces is truncated." Truncation is the trap. Feed it a 400-word summarized memory and the tail has no influence on the vector, so two entries differing only in their endings look like duplicates. Keep durable entries to one claim each and this never comes up.

High similarity isn't agreement either. "Alice lives in NY" and "Alice lives in SF" score nearly identical to a similarity model and directly contradict each other. Catching that needs a natural-language-inference model, scoring a pair across contradiction, entailment, and neutral, run only on the handful of candidates the embedding pass flags as plausibly about the same thing:

from sentence_transformers import CrossEncoder
nli = CrossEncoder("cross-encoder/nli-deberta-v3-base")
LABELS = ["contradiction", "entailment", "neutral"]

def nli_contradicts(candidate, stored, threshold=0.5):
    scores = nli.predict([(stored, candidate)])[0]
    return LABELS[scores.argmax()] == "contradiction" and scores.max() > threshold

The label order in LABELS is not guessable: that model's card documents its outputs as "contradiction, entailment, neutral", and getting the order wrong gives you a gate that admits exactly the writes it should reject. Its reported accuracy is 92.38 on SNLI-test and 90.04 on the MNLI mismatched set. Roughly one pair in ten is misjudged on clean benchmark sentences, and memory candidates are messier than that. Use embeddings to narrow the field and NLI to adjudicate, then treat the verdict as a filter rather than an oracle.

Memory as Attack Surface

Everything above treats a bad memory as an accident. The research says to plan for it being deliberate.

AgentPoison, from 2024, attacks an agent by poisoning its memory or knowledge base rather than the model. It reports average attack success above 80% across three real agents, under 1% degradation on normal tasks, and fewer than 0.1% of instances poisoned, with "no additional model training or fine-tuning". It works on the retrieval layer you added to make memory better.

MINJA, from a 2025 paper revised in early 2026, tightens the threat model. The attacker never touches the store: they inject records "by only interacting with the agent via queries and output observations". Any user of a shared-memory agent can write to what the next user retrieves.

OWASP's 2025 list names both halves. LLM04:2025 Data and Model Poisoning covers manipulation of "pre-training, fine-tuning, or embedding data" to introduce "vulnerabilities, backdoors, or biases". LLM08:2025 Vector and Embedding Weaknesses describes the agent-memory case directly: in multi-tenant environments "there's a risk of context leakage between users or queries", and retrieval augmentation "can inadvertently alter the foundational model's behavior". Its first prevention item is not a model change. It is to "implement fine-grained access controls and permission-aware vector and embedding stores", followed by "maintain detailed immutable logs of retrieval activities".

Three defences follow from the papers rather than from taste. Partition durable memory by identity, so one user's write cannot be another's retrieval. Log retrievals, not just writes, because both attacks are visible in what came back. And keep the gate on writes that originate in tool output or fetched documents, which is where injected text arrives.

Figure 1 — Both documented attacks arrive through the same two doors, and partitioning is what stops the second one becoming a channel between users.

Decay, Pruning, Provenance

A store that only grows drowns its own signal: retrieval precision falls as size climbs, the agent conditions on weaker context, and drift accelerates. A positional effect sits underneath this too. "Lost in the Middle" found model performance "is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts." Stuffing thirty retrieved memories into a prompt does not give the model thirty memories. Three mechanisms keep the store bounded, and none is worth building until it is large enough to need it.

Decay removes stale entries on a schedule. Uniform expiry is simplest and rarely right, since a stated allergy should outlive a preferred meeting time. Anthropic's memory-tool guidance gives the operational form: "periodically delete memory files that haven't been accessed in a long time", alongside capping how large any one file may grow.

Pruning keeps only what's relevant to the current task once the store exceeds a budget, scored by similarity to the live query. It's aggressive: anything off-topic gets dropped, so reserve it for stores you can rebuild, or pair it with a "pinned" flag.

Summarization compresses what remains, with one instruction doing all the work: don't infer. Compression is the moment a model is tempted to invent connective tissue that was never in the source, so the prompt has to forbid it outright. It runs rarely, and it goes through the same gate as any other write. A summarizer is the one component whose entire job is generating new sentences about stored facts.

Whatever survives still needs to be auditable. Make durable memory append-only, log every accepted and rejected candidate with the verdict that produced it, and attach provenance. A corrupt memory then becomes one traceable event rather than a silent overwrite.

{"ts": "2026-05-07T09:14:22Z", "candidate": "User prefers 5pm ET meetings",
 "verdict": "rejected", "gate": "nli_contradiction", "against": "mem_8831",
 "nli_scores": {"contradiction": 0.94, "entailment": 0.02, "neutral": 0.04},
 "source": "tool:calendar.list#c91f", "actor": "user_204", "agent": "sched-v7"}

That record is what makes the gate auditable, and its shape is aact:audit-granularity/per-action-with-principal-chain applied to a write. Without one, "the gate is working" is a belief.

The Pipeline, Assembled

Order the gates by cost and bail early: lexical checks, then the type classifier, then embedding dedupe, then the LLM validator, which is the most expensive call in the chain.

Figure 2 — Every candidate memory passes through the same four gates, cheapest first, before it's allowed to become durably true.

def safe_memory_update(memory, candidate, model_call):
    if is_self_referential(candidate):
        return False, "self_referential"
    if classify_memory(candidate, model_call) != "A":
        return False, "non_factual"
    if is_duplicate(candidate, memory):
        return False, "duplicate"

    verdict = validate_memory(candidate, [m.text for m in memory.durable], model_call)
    if verdict["contradiction"] or verdict["confidence"] < 0.7:
        return False, "failed_validation"

    item = MemoryItem(candidate, source="agent", created_at=time.time())
    add_durable_memory(memory, item)
    return True, "accepted"

This is deliberately conservative: when in doubt, it rejects. A missed memory costs one re-derivation. A corrupt one costs every future decision that touches it.

Telling Drift Apart

Drift is a symptom, not a diagnosis, and the causes below have different fixes. This is the table we'd keep next to the rejection log.

Table 2 — Four drift causes with opposite fixes, and the cheapest check that separates each from its neighbours.

The instrumentation that makes the table usable is two counters and a histogram: accepted writes, rejected writes by reason, and the age distribution of what retrieval returns. No agent framework ships them by default.

Where Fabriq Fits

One class of content should never reach a memory store at all. Agentic Fabriq holds each user's credentials in a vault and attaches them server-side at the moment of the call, so the agent never handles the secret and has nothing to write down. Every call also carries two identities, the agent and the acting user, which is what makes per-identity partitioning enforceable rather than a naming convention.

Frequently Asked Questions

How do I stop my AI agent from remembering wrong information? Put a validator between the agent and the store, and give it the authority to say no. Classify the candidate, check it against what's stored, require a source span, reject anything below a confidence threshold. Every other mechanism here operates on records the gate already let through.

What's the difference between an agent's context window and its memory? The context window is what the model sees on this call, and it's gone afterwards. Memory is what you persist and retrieve later. An overfull context window degrades one answer; a corrupt memory degrades every future answer that retrieves it.

Do I need a vector database for agent memory? Not at the start. A short list of validated facts in the database you already run is enough until retrieval starts missing things. Add vector search when you can show a recall problem, and note that it is also the layer both documented poisoning attacks target.

Should I separate episodic and semantic memory? Eventually, for a concrete reason rather than tidiness. They carry different lifetimes and different authority, and confusing them lets "the agent did this once" become "this is true". Table 1 has the split.

Can agent memory be poisoned? Yes, and the published attacks are stronger than most teams assume. AgentPoison reports over 80% attack success at under 0.1% poisoning, and MINJA injects through ordinary queries with no access to the store. OWASP's LLM08 entry recommends permission-aware vector stores and immutable retrieval logs.

How long should a memory live? Long enough to be useful, short enough that a wrong one expires on its own. Vary lifetime by category rather than setting one global TTL, and record an access timestamp so the policy is enforceable.

How do I test a memory system before it ships? Write the adversarial cases as fixtures: a hallucinated fact, a direct contradiction of a stored entry, a near-duplicate that differs only after 256 word pieces, a malformed validator response. Assert a rejection for each. The malformed-response case is the one most implementations fail.

Conclusion

We think memory is the easiest way for an agent to become a system nobody trusts, not because remembering is hard, but because writing without a gate turns the agent into something that edits its own beliefs with no one watching.

Most of what's above is optional. The gate in front of durable writes is not: classify the candidate, check it against what's stored, require provenance, reject anything the model isn't confident about. Add embeddings, decay and pruning once the rejection log says the gate alone isn't enough. Partition by identity before two users share a store, because the poisoning literature says that channel gets used.

Sources

All URLs read 2026-09-29.

• Evaluating Very Long-Term Conversational Memory of LLM Agents (LoCoMo) — 300-turn, ~9,000-token, up-to-35-session dialogues, and the finding that long-context models and RAG still lag human performance.
• Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory — 26% relative improvement on the LOCOMO LLM-as-judge metric, 91% lower p95 latency, over 90% token savings.
• MemGPT: Towards LLMs as Operating Systems — the virtual-context-management framing that most tiered memory designs descend from.
• Generative Agents: Interactive Simulacra of Human Behavior — the memory stream and reflection pattern, with ablations showing observation, planning and reflection each matter.
• AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases — over 80% average attack success, under 1% benign impact, under 0.1% of instances poisoned, no model fine-tuning.
• Memory Injection Attacks on LLM Agents via Query-Only Interaction (MINJA) — injection into a memory bank "by only interacting with the agent via queries and output observations".
• Lost in the Middle: How Language Models Use Long Contexts — retrieval position effects, and why more retrieved memories is not more usable memory.
• OWASP LLM04:2025 Data and Model Poisoning — manipulation of embedding data to introduce backdoors and biases.
• Agent Access Control Taxonomy — the failure behavior and audit granularity dimensions; fail-closed for the unparseable-verdict case, and per-action-with-principal-chain for the shape of the gate's own record.
• OWASP LLM08:2025 Vector and Embedding Weaknesses — multi-tenant context leakage, retrieval altering model behaviour, permission-aware stores and immutable retrieval logs.
• Memory tool, Anthropic platform documentation — client-side execution, validation as the application's responsibility, sensitive-data stripping, size caps, expiry of unread files, path-traversal protection.
• Add and manage memory, LangChain / LangGraph documentation — the semantic, episodic and procedural split and the hot-path versus background write trade-off.
• sentence-transformers/all-MiniLM-L6-v2 model card — 384-dimensional embeddings and truncation of input longer than 256 word pieces.
• cross-encoder/nli-deberta-v3-base model card — label order "contradiction, entailment, neutral", 92.38 on SNLI-test and 90.04 on MNLI mismatched.