Let's talk

Memory in Agents

Agent Memory: Short Term vs Long Term

To build reliable AI agents, you must understand how they store and retrieve information through different memory layers.

By Divyaprakash JhaUpdated 4 min readTopic: Agentic AI

Photo for Agent Memory: Short Term vs Long Term

Working memory is the agent's immediate context window, containing the current goal and recent actions. It is fast and essential for the current task but has a limited capacity, requiring you to summarize old steps to prevent the agent from losing focus.

Working Memory: The Current Context

Think of working memory as the "mental scratchpad" an agent uses while performing a specific task. When you give an agent a goal—for example, "Reconcile last month's GST invoices against the bank statement and flag mismatches"—the agent needs to keep track of what it is doing right now.

This memory layer includes the original goal, the steps the agent has already taken, and the results it received from using tools. If the agent calls a tool to search_invoices(gstin, month), the data returned by that tool becomes part of its working memory so it can use that information in the next step.

However, there is a catch: working memory is fast but limited in capacity. Every model has a "context window," which is the maximum amount of information it can process at one time. If an agent performs dozens of steps, the conversation history can become too long, causing the agent to "forget" the original goal or become confused.

To manage these limits, you should not simply keep every single interaction in the history. Instead, you should implement a strategy where old steps are summarized. This allows the agent to retain the essential "gist" of what happened earlier without wasting precious space on every word of a long technical log.

Episodic Memory: The Record of Experience

While working memory is about the *current* task, episodic memory is about the *history* of past runs. It is the agent's ability to look back at previous attempts to see what worked and, more importantly, what failed.

Imagine a small business owner in Mumbai using an agent to manage customer refunds. On Monday, the agent tries to process a refund, but it fails because it doesn't have the correct authorization code. If the agent has episodic memory, it "remembers" this failure. When it attempts a similar task on Tuesday, it won't repeat the exact same mistake; it will check for the authorization code first or follow a different path.

Episodic memory tracks:

  • What the agent tried to do in previous sessions.
  • Which specific actions led to successful outcomes.
  • What a human had to step in and correct.

By recording these "episodes" of experience, the agent becomes more autonomous over time. It learns from human feedback, ensuring that if a manager corrects an agent's mistake once, the agent doesn't make that same error every morning when it wakes up on a "heartbeat" to check the inbox.

Long-Term Memory: Persistent Knowledge

Long-term memory is the agent's permanent library. It does not change based on the current task; instead, it provides the foundational facts and documents the agent needs to do its job accurately.

This is where you store your business's "source of truth." Examples include:

  • Product price lists and discount policies.
  • Company HR policies and compliance documents.
  • Detailed customer histories and past order data.
  • Standard Operating Procedures (SOPs) for specific business processes.

Unlike working memory, which is loaded automatically, long-term memory is typically accessed "on demand." This is often achieved through a technique called Retrieval-Augmented Generation (RAG), where the agent uses search or embeddings to find the specific piece of information it needs from a vast knowledge base.

It simply searches its long-term memory, finds the relevant paragraph, and brings that specific fact into its working memory to answer the question.

Summary of Memory Types

To build a robust agentic AI system, you must understand how these three layers interact. You can think of it as a human employee: working memory is their focus on the task at hand, episodic memory is their experience from previous jobs, and long-term memory is the reference manual on their desk.

Memory Type Primary Purpose Duration Analogy
Working Memory Immediate task execution and context. Short-term (Current session) A mental scratchpad or sticky note.
Episodic Memory Learning from past experiences and errors. Medium-term (Past runs) A diary of past workdays.
Long-Term Memory Accessing permanent facts and documents. Permanent (Knowledge base) A library or reference manual.

Why Layered Memory Matters

Building an agent is not just about picking a powerful model; it is about designing the agentic architecture that surrounds it. A common mistake for builders is trying to cram everything into the model's context window. This leads to "loop problems," where the agent gets stuck in a repetitive cycle or loses track of the original goal because it is overwhelmed by irrelevant data.

Layered memory is the solution to this problem. By separating concerns, you create a system that is both focused and knowledgeable. This architecture allows you to build agents that are:

  • Reliable: They stay on track because their working memory is clean and summarized.
  • Efficient: They don't waste expensive tokens by reading entire manuals when they only need one sentence.
  • Adaptive: They improve over time by learning from human corrections via episodic memory.

When you design with layered memory, you move away from simple chatbots and toward true agentic AI. You transition from a system that just "answers questions" to a system that "completes goals" by intelligently navigating its own history and knowledge base.

If you are moving from simple workflows toward more complex, autonomous agents, understanding this hierarchy is the first step in moving from a prototype to a production-grade business asset.

Yellow robotic arm operating inside a glass enclosure with a monitor above.
Photo: Robot arm works on a small component in a lab setting by Shixart1985, CC BY 2.0

Questions people ask

What is the difference between working and long-term memory in AI agents?

Working memory is the immediate context window used for the current task, while long-term memory is a persistent store of facts and documents retrieved via search or RAG.

How does episodic memory help an AI agent?

Episodic memory records past attempts and human corrections, allowing the agent to avoid repeating previous mistakes in new runs.

What should I do if the context window is getting too full?

Instead of keeping every single step in the context, you should summarize older steps to save space while retaining essential information.

Can an agent use long-term memory to look up company policies?

Yes, long-term memory is designed to hold documents like policies and price lists, which the agent retrieves on demand.

Is memory part of the model or the code?

In agentic architecture, the model is inside a loop, but the code around it manages the memory, deciding what gets saved and when.