AI Agent Memory Explained

What does memory mean in an AI agent? AI agent memory is the application's controlled ability to preserve and retrieve useful state across steps, turns or sessions. It can include recent conversation, workflow checkpoints, approved user preferences and past decisions. The model itself is not privately recollecting events. Software decides what is stored, where it is stored, who may read it, when it is retrieved and when it must be corrected or deleted.

Memory is a system design, not a human trait

A raw model call receives an input and produces an output. Continuity appears when the surrounding application sends previous messages again, references a server-managed conversation, restores a checkpoint or retrieves selected records from a database. Calling all of this “memory” is convenient, but it hides important differences in reliability, permissions and lifecycle.

For a business, the useful question is not “Can the agent remember?” It is “Which fact or state should remain available for which job, under whose authority, for how long?” An agent that remembers everything creates noise and risk. An agent that remembers nothing forces customers and employees to repeat themselves and makes multi-step work fragile.

OpenAI's Agents SDK illustrates the application boundary clearly: session implementations store conversation history for a specific session and retrieve that history before later runs. The SDK also offers different storage patterns, compaction and explicit operations to inspect, remove or clear session items. Those are memory-management choices made by the application, not evidence that the model has human-like recall.

Five layers people often call agent memory

1. Working context: what the model can use now

Working context is the material assembled for the current model call: system instructions, the user's request, recent messages, selected tool results, relevant memories and retrieved documents. It is closer to a workbench than an archive. If a fact is stored elsewhere but not retrieved into this context, the model cannot use it for that decision.

More context is not always better. Long histories can repeat obsolete instructions, bury the latest customer state and increase latency and cost. Summarization or compaction helps, but compression can omit evidence and exceptions. Preserve authoritative identifiers and source links so the agent can re-read important facts rather than trusting a lossy summary.

2. Conversation state: continuity inside one case

Conversation state keeps the ordered history of a thread: messages, tool calls, approvals, interruptions and results. It lets a support agent understand that identity was already verified, a quotation was already drafted or an approval is still pending. This state should normally be isolated by thread, customer case or workflow instance.

Do not confuse continuity with truth. A prior message may be speculative, a tool result may be stale and a customer may correct an earlier statement. Before an agent changes an external record, it should retrieve fresh state from the system of record and treat conversation history as context, not final authority.

3. Durable memory: selected facts across sessions

Durable memory stores information intended to influence later sessions: a customer's preferred language, an account's approved delivery window, a recurring workflow decision or a lesson from a completed run. Microsoft's current agent-memory guidance similarly distinguishes recent thread context from longer-lived information such as preferences and historical summaries.

This layer needs a write policy. The safest design does not turn every sentence into memory. It extracts a candidate, identifies its subject and source, checks whether the user or business process permits retention, resolves conflicts with existing entries and records when it should expire. High-impact memories may require explicit confirmation before they can shape later actions.

4. Retrieved knowledge: evidence, not personal memory

A policy manual, product catalogue, CRM record or contract is usually knowledge managed by another system. Retrieval-augmented generation can find a relevant passage and place it into working context, but it should not copy that material into a free-floating personal memory store. Keep the source authoritative, enforce its permissions during retrieval and show citations when a user needs to verify the answer.

This distinction matters when a document changes. If the agent stored “the refund window is 30 days” as durable memory and the policy becomes 14 days, the old memory can override the current source. For policy and transactional facts, retrieve from the system of record at decision time.

5. Operational state: what the workflow has done

Pending approval, last successful step, invoice ID, retry count and idempotency key are operational state. They should live in a database, queue or business system with deterministic consistency—not only in a model transcript or vector store. This lets a workflow resume safely, prevents duplicate actions and supports audit and recovery.

When teams call operational state “agent memory,” they sometimes reach for semantic search where an indexed field or transaction is required. Use ordinary software controls for exact state. Use model-facing memory only to supply the context needed to interpret or communicate that state.

How retrieval turns stored information into behaviour

Durable storage alone changes nothing. At runtime, the application forms a query from the current task and identity, filters candidate memories by tenant, user, role, purpose and time, ranks relevant entries, applies safety checks and injects a small selection into working context. The agent then produces a response or plan influenced by those entries.

That retrieval path is part of the decision system. A false or malicious memory can be recalled later, after the original interaction has ended. Microsoft's memory-safety guidance treats persistent memory as a configuration-like layer because an entry can influence future tool choice, refusal behaviour and reasoning. It recommends provenance, gated writes, isolation, retrieval-time checks and lifecycle observability.

Evaluate memory at two levels. First, test retrieval: did the system find the right entry, exclude another user's data and ignore expired or superseded facts? Second, test outcome influence: did the memory improve task success without causing unsafe actions, over-personalization or misplaced confidence? “The agent mentioned my preference” is not sufficient evidence of a good memory system.

Privacy, permission and deletion are core architecture

Memory expands the period during which information can affect a system. That makes data minimization and separation practical engineering requirements. Scope stores and queries by tenant and subject. Give the agent's service identity only the read and write rights required for its job. Encrypt sensitive stores, protect credentials, log memory creation and retrieval, and avoid placing secrets in model-facing context.

Retention should follow purpose. A temporary troubleshooting detail might expire with the ticket; a preference may remain until changed; a regulated record may follow a defined business retention schedule. “Keep it because it might help” is not a retention rule.

Deletion must cover more than the primary row. Identify indexes, embeddings, caches, summaries, backups and derived memories that may contain or reproduce the fact. Define whether deletion is immediate, queued or subject to a legitimate retention exception. Give authorized users a way to view, correct and remove durable memories about them, and record completion without preserving the deleted content in the audit event.

A practical memory contract

Before adding a memory store, write one contract for each memory class. This nine-question version keeps product, operations, security and privacy decisions in the same review.

If a team cannot answer the writer, reader, retention and deletion questions, the memory is not ready for production. Start with thread-scoped state and a narrow set of confirmed durable facts. Expand only when evaluation shows a real improvement.

Three business examples

Customer support

Keep the current ticket's messages and tool results as conversation state. Retrieve the latest order and policy from their systems of record. Store a confirmed communication preference across tickets only when permitted. Do not retain an emotional complaint as a permanent personality label.

Sales follow-up

Store contact status, consent and the next agreed action in the CRM. The agent may use a thread summary to draft the next message, but it should re-read the current opportunity stage before sending. Expire temporary research and prevent one account's notes from appearing in another account.

Internal operations

Use checkpoints for a long-running workflow, durable records for approvals and transactions, and retrieved documents for policies. A cross-session memory can preserve a team's confirmed reporting preference; it should not replace the current finance record or access-control system.

Choose the smallest memory that completes the job

1. Start with authoritative state. Identify the CRM, ERP, document store or workflow database that already owns the fact.
2. Add thread continuity. Preserve only what one case needs to resume safely.
3. Retrieve knowledge on demand. Keep permissions and freshness attached to the original source.
4. Add durable memory selectively. Retain confirmed preferences, decisions or lessons only when they improve a measured outcome.
5. Design correction and deletion before launch. Include indexes, summaries and derived stores.
6. Test isolation and poisoning. Try cross-user queries, conflicting memories, malicious entries, stale facts and revoked access.
7. Measure accepted outcomes. Compare task success, corrections, latency, cost and privacy incidents with and without memory.

Use the RAG explainer to separate evidence retrieval from cross-session memory, the knowledge-base implementation guide for permissions and source ownership, and the agent security checklist to review access and incident controls.

Primary sources checked

Checked 12 September 2026. The five-layer map and memory contract are Shuhaib's practitioner synthesis; product features should be rechecked before implementation.

• OpenAI Agents SDK — sessions and memory operations
• OpenAI Agents SDK — state and conversation-management strategies
• OpenAI Agents SDK — encrypted sessions and expiration
• Microsoft Azure Cosmos DB — agent memory patterns
• Microsoft Security — managing memory safety in agentic systems