Let's talk

Reliable Agentic AI

Durable Execution for AI Agents

Most AI agent demos are single loops in a notebook. Production-grade agents require durable execution to survive restarts and wait for human input.

By Divyaprakash JhaUpdated 7 min readTopic: Agentic AI

Photo for Durable Execution for AI Agents

What is Durable Execution for AI Agents?

Durable execution is a system design that ensures an AI agent's progress is saved at every step. This allows an agent to survive server restarts, handle long-running tasks that take days, and resume exactly where it left off without losing its place.

In a simple AI demo, the code runs in a single loop. If your internet drops or the server reboots, the agent "dies," and you have to start the entire process from scratch. For a business, this is a disaster if the agent was halfway through a complex task.

Durable execution changes this by using checkpoints. After every single action the agent takes, the system saves its current state. If the system fails, the agent doesn't start over; it simply picks up from the last saved checkpoint.

  • Persistence across time: Agents can manage tasks that take days, such as waiting for a manager to approve a refund or a customer to confirm a booking.
  • Resilience to failure: If a server restarts or a connection drops, the agent resumes its task exactly where it left off.
  • Time travel: Because every step is saved, you can "fork" a run. This means you can go back to an earlier step, try a different path, and see if the agent performs better.

Why Agents Need More Than a Simple Loop

Most AI agent demos are impressive in a notebook, but they are not production-ready. A production-grade agent is an "always-on" employee, not a script that runs once and disappears. For businesses, reliability is the difference between a toy and a tool.

Consider a real-world task like reconciling GST invoices. The agent must read files, query a ledger, compare numbers, and flag mismatches. If the process fails halfway through due to a timeout, you cannot afford to have the agent duplicate the work or lose the context of what it already checked. You need a system that maintains a continuous, reliable record of progress.

Durable systems also solve the problem of "idle time." Many agentic tasks require human intervention. An agent might perform a task, then need to wait two days for a manager to approve a high-value transaction. A simple loop would require keeping a server active and consuming resources for two days. A durable system allows the process to pause and wait without consuming active resources.

Furthermore, reliability requires accountability. When an agent acts on your behalf, you need to know exactly why it made a specific decision. This requires a robust architecture that tracks every step of the process from start to finish.

Core Components of a Durable Agentic Architecture

To build a reliable agent, you must wrap the Large Language Model (LLM) in a layer of "ordinary" software. The model makes the decisions, but the software handles the heavy lifting of reliability and safety. According to agentic architecture principles, a robust system requires several key components:

1. Checkpoints

As mentioned, these are the "save points." The system saves the state after every step, ensuring that the agent's memory and current task progress are never lost.

2. Human-in-the-loop

This is a vital integration point where the agent pauses. For risky actions—like sending a large payment or deleting a database entry—the system stops and waits for a human to approve or deny the action. This ensures that the agent's autonomy is bounded by human oversight.

3. Layered Memory

Agents need different types of memory to function effectively:

  • Working Memory: This is the current context window, containing the immediate goal, recent steps, and tool results.
  • Episodic Memory: This is the record of past runs. It allows the agent to learn from what it tried previously, what worked, and what a human had to correct.
  • Long-term Memory: This holds permanent facts, documents, or customer history, often retrieved on demand.

4. Guardrails

Guardrails are rules enforced outside of the model's control. You should never rely on the model to "promise" to stay within a budget or avoid a certain website. Instead, you implement hard limits in your code, such as spending caps or blocked domains, to ensure safety regardless of what the model suggests.

Workflows vs. Agents: The Hybrid Approach

A common mistake for builders is trying to make everything an "agent." To build a stable system, you must understand the difference between a workflow and an agent. The most effective production systems use a hybrid approach.

Feature Workflow Agent
Decision Maker The developer (in code) The model (at runtime)
Predictability High (fixed paths) Lower (variable paths)
Step Count Fixed and known Variable and often unknown
Use Case Standardized processes Complex, open-ended tasks

A Workflow follows predefined, predictable code paths. It is highly reliable and perfect for tasks like "send a WhatsApp template when a new lead arrives."

An Agent directs its own process. It is better for tasks where the steps aren't known in advance, such as "research this company and find three potential reasons they might be a good client."

The Hybrid Approach is the gold standard. You use a workflow to create a reliable "outer shell" that handles the predictable parts of a business process, and you place flexible agentic steps inside that shell to handle the unpredictable parts. This gives you the reliability of code with the intelligence of an LLM.

Governance, Safety, and Auditability

As agents move from experimental scripts to autonomous workers, governance becomes non-negotiable. You cannot simply give an agent a "brain" and hope for the best; you must give it boundaries.

One effective way to manage this is by assigning a risk class to every tool the agent can use. For example:

  • Read/Fetch: Low risk (e.g., reading an invoice).
  • Write: Medium risk (e.g., updating a CRM).
  • External/Critical: High risk (e.g., making a payment or sending an email to a client).

You can then set autonomy levels for the agent. An agent with a "low" autonomy level might only be allowed to "Observe" and suggest actions, while a "high" autonomy agent might be allowed to "Act" on low-risk tools automatically. For critical tools, the system should always require human approval.

Additionally, businesses must consider data residency. If you are operating in India, you may need to ensure that sensitive personal data is handled by models or services that comply with local data residency laws, keeping that data within specific geographic boundaries.

Finally, every autonomous system needs a kill switch. If an agent enters an infinite loop or begins behaving unexpectedly, a human must be able to stop the process immediately.

Patterns for Reliable Agentic Systems

When moving beyond a single loop, you should use proven patterns to manage complexity. These patterns help increase reliability and make the agent's behavior more predictable.

Plan-and-Execute

In this pattern, the model first writes out a full, detailed plan for the entire task. This plan can be reviewed and approved by a human before any actions are taken. Once approved, an "executor" works through the plan, only re-planning if a specific step fails. This is ideal for tasks like "migrate these 300 product listings," where you want to verify the approach before the agent starts working.

Router

A router uses a small, fast model to act as a traffic controller. It looks at the incoming request and directs it to the correct specialized path. For example, a customer query about "billing" goes to a billing-specific workflow, while a "technical support" query goes to a different specialized agent. This increases reliability because each path uses a highly focused prompt and a specific set of tools.

Orchestrator-Workers

This pattern is used for complex, multi-part tasks. An "orchestrator" tracks the overall goal, the facts discovered so far, and the current progress. It then breaks the task into subtasks and delegates them to "worker" agents. The orchestrator then combines the results from all workers to complete the final goal. This is particularly useful when you cannot know the exact subtasks in advance, such as when an agent is helping to edit several different files in a codebase.

To learn more about how to implement these in a larger system, see our guide on building multi-agent systems.

Frequently Asked Questions

Why can't I just use a standard LLM loop for my agent?
Standard loops lose all progress if the server restarts or the connection breaks. Durable execution uses checkpoints to ensure the agent can resume its task from the exact point of failure.
How do agents handle human intervention?
Durable execution allows for "pause points" where an agent waits for human approval. The agent's state is saved, allowing it to remain idle for days without losing its place in the workflow.
What is the benefit of an audit log in agentic AI?
A secure audit log provides a record of every action an agent takes, which is vital for governance and debugging complex multi-step tasks.
Can I resume an agentic workflow if it fails?
Yes. With durable execution, you can use "time travel" to fork a run from any earlier step to try a different path or correct a mistake.
How do you prevent an agent from spending too much money?
You implement guardrails and budgets outside of the model's control, such as token budgets and step limits, to ensure the agent stays within its constraints.
Yellow robotic arm operating inside a glass enclosure with a monitor above.
Photo: Robot arm works on a small component in a lab setting by Shixart1985, CC BY 2.0

Questions people ask

Why can't I just use a standard LLM loop for my agent?

Standard loops lose all progress if the server restarts or the connection breaks. Durable execution uses checkpoints to ensure the agent can resume its task from the exact point of failure.

How do agents handle human intervention?

Durable execution allows for 'pause points' where an agent waits for human approval. The agent's state is saved, allowing it to remain idle for days without losing its place in the workflow.

What is the benefit of an audit log in agentic AI?

A hash-chained audit log provides a secure, uneditable record of every action an agent takes, which is vital for governance and debugging complex multi-step tasks.

Can I resume an agentic workflow if it fails?

Yes. With durable execution, you can use 'time travel' to fork a run from any earlier step to try a different path or correct a mistake.

How do you prevent an agent from spending too much money?

You implement guardrails and budgets outside of the model's control, such as token budgets and step limits, to ensure the agent stays within its constraints.