Agent Memory

Agent memory is everything that lets an agent use information beyond what fits in, or survives, a single model call: the working context of the current step, the state of the current task, and long-term memories of facts, past episodes and procedures. This chapter gives one taxonomy used throughout the course, then the mechanisms for writing, retrieving, updating and forgetting memories, and the risks that come with each.

One Taxonomy

The model's weights hold parametric memory - what it learned in training - which you change only by training. Everything else is external memory managed by the harness. We classify it on two axes: its scope (how long it lives) and, for long-term memory, its kind, following the CoALA framework (Sumers et al., 2024), itself borrowed from cognitive science.

mindmap
  root(("🧠 Agent memory"))
    🪟 Working memory
      The context window now
      Instructions and tools
      Recent turns and tool results
    📋 Task state
      Plan and todo list
      Variables and intermediate results
      Checkpoints for resume
    📚 Semantic
      Facts about the user and world
      Preferences and profile
    🎞️ Episodic
      What happened in past episodes
      Successful trajectories
    🛠️ Procedural
      Instructions and rules
      Skills and code

Memory	Scope	Holds	Lives in	Example
Working	One model call	Everything the model sees now	The context window	The last 8 turns plus 3 tool results
Task state	One task or thread	Plan, progress, intermediate results; resumable checkpoints	Harness state, checkpointer (database)	"Steps 1-3 done; step 4 waiting for approval"
Semantic (long-term)	Across sessions	Facts and preferences	Profile document, KV store, vector index, knowledge graph	"Prefers aisle seats; employer is Acme"
Episodic (long-term)	Across sessions	Records of past experiences	Log store, vector index of episode summaries	"Last month's refund dispute was resolved by escalating to billing"
Procedural (long-term)	Across sessions	How to do things	System prompt, skill files, code, tool definitions	"When booking travel for this team, always check the policy doc first"

The terms short-term memory (working memory plus task state, scoped to one conversation thread) and long-term memory (the last three rows) are also common - LangGraph, for example, implements short-term memory with a checkpointer and long-term memory with a store.

---

Working Memory: Managing the Context Window

Every token in the context is paid for on every call and competes for the model's attention; quality degrades as contexts grow long and noisy (Context Engineering). In a long episode, working memory must be managed actively:

Technique	How it works	Trade-off
Trim	Drop the oldest turns beyond a token budget, keeping the system prompt and the goal	Simple; loses information silently
Truncate tool outputs	Cap each tool result; return a summary plus a handle to fetch more	Needs tools designed for pagination
Context editing	Clear stale tool results (and old thinking) from the transcript while keeping the structure - Claude's API offers this as a server-side strategy	Keeps recent reasoning intact; cleared content is gone
Compaction	When near the limit, summarise the history into a condensed state and continue from it	Preserves key facts if the summary prompt is good; costs a model call
Offload to files or state	Write large intermediate results (a downloaded document, a table) to a file or state field and keep only a reference in context	The agent must know to read it back
Pinned task summary	Keep the goal, constraints and progress in a compact block that is never trimmed	Must be updated as the task progresses

A good compaction summary keeps: the goal and constraints, decisions made and why, facts established (with ids), what was tried and failed, and the next steps. It drops raw tool output that has already been used.

Task State: Surviving Steps and Failures

Task state is structured data your harness owns - not just messages. Keep the plan, the list of completed steps, and key intermediate results in explicit fields. Persist it after every step (a checkpoint) so that a crash, a timeout or a pause for human approval doesn't lose the work:

from langchain.agents import create_agent
from langgraph.checkpoint.memory import InMemorySaver   # use a Postgres/SQLite saver in production

agent = create_agent(model, tools=[...], checkpointer=InMemorySaver())
config = {"configurable": {"thread_id": "ticket-8812"}}

agent.invoke({"messages": [{"role": "user", "content": "Refund my broken poles"}]}, config)
# ...process restarts, or a human approves an action...
agent.invoke({"messages": [{"role": "user", "content": "Yes, go ahead"}]}, config)  # resumes the thread

Checkpoints make agents resumable and debuggable (you can replay from any step). For long-running agents in production, durable execution engines take this further - see Production Agents.

---

Long-Term Memory

The write path: when and how memories are created

Approach	How	Pros	Cons	Examples
Hot path (agent-managed)	The agent has memory tools and decides during the conversation what to save, update or delete	Immediate; the agent chooses what matters	Adds latency and tool calls; the agent may save too much or too little	MemGPT / Letta memory blocks; Claude's memory tool (a /memories file directory the agent reads and edits)
Background (extraction)	After or between conversations, a separate process extracts memories from the transcript	No latency on the conversation; consistent extraction prompts	Memories lag; extraction quality depends on the pipeline	Mem0's extraction pipeline; LangMem background memory manager

Mem0 (Chhikara et al., 2025) is a representative extraction design: an LLM extracts candidate facts from recent messages, retrieves similar existing memories, and chooses an operation for each - ADD, UPDATE, DELETE or NOOP - so the store stays consistent rather than accumulating contradictions. On the LOCOMO benchmark its authors report 26% relative improvement in LLM-judge score over OpenAI's memory feature, and over 90% token savings versus sending full conversation history.

The read path: bringing memories back

• Always loaded: a small profile or core memory block in the system prompt (Letta's "core memory"; user preferences). Keep it short.
• Retrieved on demand: search semantic and episodic stores by similarity to the current request, as in RAG. Generative Agents (Park et al., 2023) scored memories by relevance, recency and importance combined - a pattern many systems still use.
• Agent-initiated: the agent calls search_memory(query) when it decides it needs to.

Structure: flat, graph or files

Structure	Good at	Example systems
Flat facts in a vector/KV store	Preferences and simple facts	Mem0, LangGraph store
Temporal knowledge graph	Facts that change over time and relations between entities; each fact has a validity interval, so "worked at Acme until March" is kept, not overwritten	Zep / Graphiti (Rasmussen et al., 2025)
Self-organising notes	Linking related memories and evolving them as new ones arrive	A-MEM (Xu et al., 2025)
Files the agent edits	Transparent, inspectable, versionable memory	Claude's memory tool; project files such as AGENTS.md for coding agents

Procedural memory: learning how

Procedural memory changes how the agent behaves: updated instructions, new rules learned from feedback, reusable skills. LangMem implements it as prompt optimisation from trajectories and feedback; coding agents implement it as instruction files and skills - folders of instructions and scripts loaded when relevant (Agent Engineering). Procedural memory is powerful and risky: a bad update affects every future episode, so gate changes behind review or evaluation.

# LangGraph long-term store: namespaced per user; search by filter or semantic query
from langgraph.store.memory import InMemoryStore

store = InMemoryStore()   # production: a Postgres store with an embedding index
ns = ("users", "u42", "memories")
store.put(ns, "seat", {"text": "Prefers aisle seats", "source": "conversation 2026-09-12"})
hits = store.search(ns, limit=5)   # with an index configured: store.search(ns, query="flight booking")

---

Updates, Conflicts and Forgetting

Memory that only grows becomes wrong. Plan for:

• Updates and contradictions - "moved to Berlin" must supersede "lives in Paris". Use update operations (Mem0) or validity intervals (Zep) rather than appending.
• Provenance - store where and when each memory came from, so it can be audited and corrected.
• Expiry - time-to-live for volatile facts; decay for rarely used episodic memories.
• Deletion on request - privacy law (for example the GDPR's right to erasure) applies to memories about people. Deletion must reach every store and derived index, including summaries.
• Consolidation - periodically merge duplicates and summarise old episodes, much as some systems run "sleep-time" background agents to reorganise memory.

Risks

Risk	What happens	Defence
Memory poisoning	An attacker gets malicious instructions or false facts stored (via a document, a web page or crafted queries), which then influence future episodes. AgentPoison (Chen et al., 2024) poisoned memory and knowledge bases; MINJA (Dong et al., 2025) injected memories through ordinary queries alone	Treat memories as untrusted data, not instructions; store provenance; validate before writing; separate procedural memory writes behind review
Cross-user leakage	One user's memories retrieved for another	Namespace by user and tenant; enforce the filter in the store, not in the prompt
Stale or wrong memories	The agent confidently uses outdated facts	Update operations, validity intervals, expiry; let users view and correct memories
Over-personalisation	Irrelevant memories distract the model	Retrieve few, relevant memories; measure with and without memory

Evaluating memory

Test memory with multi-session benchmarks - LoCoMo (Maharana et al., 2024) and LongMemEval (Wu et al., 2024) test recall of facts, temporal reasoning, updates and knowing when information is absent - and with your own scenarios: seed facts in session 1, update one in session 2, ask in session 3.

---

Check Yourself

Exercises

Study Notes

• Parametric memory (weights) vs external memory (managed by the harness)
• Scope: working (this call) → task state (this thread, checkpointed) → long-term (across sessions)
• Long-term kinds (CoALA): semantic (facts), episodic (experiences), procedural (how-to: instructions, skills)
• Working memory tools: trim, truncate outputs, context editing, compaction, offloading, pinned task summary
• Write path: hot path (agent tools - Letta, Claude memory tool) vs background extraction (Mem0, LangMem)
• Mem0 operations: ADD / UPDATE / DELETE / NOOP; Zep: temporal knowledge graph with validity intervals
• Read path: always-loaded core, retrieved on demand (relevance, recency, importance), agent-initiated search
• Plan for updates, provenance, expiry, deletion; defend against poisoning and cross-user leakage
• Evaluate with LoCoMo, LongMemEval and multi-session scenarios

References

• Sumers et al., Cognitive Architectures for Language Agents (CoALA) (TMLR 2024)
• Packer et al., MemGPT: Towards LLMs as Operating Systems (2023); Letta documentation
• Chhikara et al., Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory (2025)
• Rasmussen et al., Zep: A Temporal Knowledge Graph Architecture for Agent Memory (2025)
• Xu et al., A-MEM: Agentic Memory for LLM Agents (2025)
• Park et al., Generative Agents: Interactive Simulacra of Human Behavior (UIST 2023)
• LangChain, LangMem SDK launch (2025) and Memory overview (docs)
• Anthropic, Memory tool (docs, 2026)
• Chen et al., AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases (NeurIPS 2024)
• Dong et al., Memory Injection Attacks on LLM Agents via Query-Only Interaction (2025)
• Maharana et al., Evaluating Very Long-Term Conversational Memory of LLM Agents (LoCoMo) (ACL 2024); Wu et al., LongMemEval (ICLR 2025)

Last reviewed: 2026-09