Let's talk

Design for Autonomy

Agentic Architecture

Agentic architecture is the set of design decisions that make AI agents reliable, safe, and affordable by wrapping models in controlled loops.

By Divyaprakash JhaUpdated 6 min readTopic: Agentic AI

Photo for Agentic Architecture

What is Agentic Architecture?

Agentic architecture is the set of design decisions that make AI agents reliable, safe, and affordable by wrapping models in controlled loops. It is how you design software where an LLM works toward a goal by itself through planning, tool use, and verification.

Unlike a standard chatbot that simply replies to messages, an agentic system takes a complex goal and executes the steps required to reach it. For example, instead of just answering a question, an agent might be tasked to "reconcile last month's GST invoices against the bank statement and flag mismatches." To do this, the system must read files, query a ledger, compare numbers, and perhaps even ask an accountant about an odd entry.

The architecture wraps the model in a loop consisting of clear tools, layered memory, a verifier, and durable checkpoints. This ensures the model isn't just "chatting," but is actually performing a job.

Feature Chatbot Workflow AI Agent
Who decides the next step? The user, one message at a time The developer, in code The model, at run time
Use of tools Rarely Yes, at fixed points Yes, chosen per step
Steps per task One reply Fixed and known Variable, often unknown in advance
Predictability High High Lower; needs guardrails

The Core Agent Loop

Every agent, no matter how advanced, operates within a fundamental loop. While the model acts as the decision-maker inside this loop, the loop itself is composed of ordinary, testable software code. This code manages how many steps are allowed, which tools are available, and when to involve a human.

  1. Goal: The system receives a task with a clear definition of done, specific constraints, and a set budget.
  2. Plan: The model decides the next immediate step or drafts a full plan for the entire task.
  3. Act: The system calls a tool, such as a database query, an API call, or a file write.
  4. Observe and Verify: The system reads the tool result and checks it. It asks: Did the invoice total match the sum? Did the API return an error?
  5. Remember: The system records what happened so that later steps—or even later runs—can use that information.
  6. Repeat or Stop: The loop continues, re-plans, asks a human for help, or finishes and reports the result.

Key Components of the Architecture

To move from a simple demo to a production-grade system, your architecture needs several distinct components working in harmony.

The Model

The language model is the brain. It reads the goal, the history, and the tool results to choose the next action. When choosing a model, prioritize tool-calling reliability and instruction following over raw benchmark scores. Many systems use a powerful, expensive model for complex planning and a smaller, cheaper model for routine tasks like message classification.

Tools

Tools are functions the model can call, such as search_invoices(gstin, month) or send_whatsapp(template_id, phone). For a tool to work well, it needs a narrow scope, clear names, typed parameters, and helpful error messages. Avoid tools that return massive amounts of data, like 40,000 rows, as this will break the model's context window.

Memory

Effective agents use different types of memory to stay on track:

  • Working Memory: The current context window containing the goal and recent steps. Since this is limited, old steps should be summarized rather than kept in full.
  • Episodic Memory: A record of past runs—what the agent tried, what worked, and what a human had to correct. This prevents the agent from repeating the same mistakes.
  • Long-term Memory: Holds permanent facts, policies, or customer history, usually retrieved via search or embeddings.

Planner and Verifier

The Planner turns a high-level goal into actionable steps, either implicitly or through an explicit, reviewable plan. The Verifier is the quality control. It uses code, a second model, or a test suite to check work before it is finalized. Whenever possible, use cheap, deterministic code (like math checks) rather than asking a model to "verify" its own logic.

Safety, Guardrails, and Human Approval

When agents have the power to act, you must implement strict safety measures. These measures must exist outside the model's control; you cannot rely on the model simply "choosing" to follow a rule.

Guardrails

Guardrails are hard rules enforced by your software. These include spending limits (to prevent runaway costs), allowed tool lists, and blocked domains. For example, you might allow an agent to "read" a customer's history but never "write" or "delete" it without specific permission.

Human Approval

Human approval acts as a critical pause point. For risky actions—like issuing a refund or sending a mass WhatsApp message—the system should pause and wait for a person to say "yes" or "no." In production-grade systems, these approvals can stay pending for days without losing the agent's progress.

Single-Agent Design Patterns

Before building complex multi-agent teams, master these single-agent patterns. They are the building blocks of most successful agentic workflows.

  • ReAct: The model interleaves reasoning steps with tool calls and observations. This is the standard for research or troubleshooting tasks.
  • Plan-and-Execute: The model writes a full plan first, then an executor works through it. This is ideal for tasks like "migrate 300 product listings," where a human wants to review the plan before execution begins.
  • Router: A small, fast model classifies an input and sends it to a specialized path (e.g., sending billing questions to a billing flow and complaints to a human).
  • Orchestrator-Workers: An orchestrator breaks a task into subtasks and hands them to various workers, then combines the results.
  • Evaluator-Optimizer: One step produces a draft, and a second step critiques it against specific criteria, forcing the first step to revise the work.

Multi-Agent Topologies

When a single agent becomes too complex to manage, you can move toward multi-agent systems. This involves splitting work across several agents, each with its own specific role and tools.

Common Team Structures

  • Assembly-line: Roles hand a specific, typed artifact to the next agent in a sequence, much like a factory line.
  • Group Chat: A "speaker selector" picks which agent should speak next in a conversation.
  • Debate: Two agents argue different sides of an issue while a third "judge" agent decides the outcome.
  • Manager and Crew: A manager agent plans the high-level strategy and assigns specific tasks to specialized worker agents.
  • Handoff Network: Agents pass control to the most appropriate specialist as the task evolves.

Building for Production

Moving from a notebook demo to a reliable business tool requires a shift in focus. A production-grade AI agent must be more than just "smart"; it must be robust.

Reliability and Durability

Reliability in production often comes from making the system "durable." This means the system must survive a server restart and be able to resume a task exactly where it left off. If an agent is waiting two days for a manager to approve a refund, the system must "sleep" and remember its state without consuming active memory.

Cost and Control

Uncontrolled loops can lead to massive API bills. You must implement token budgets and step limits on every run to prevent an agent from getting stuck in an infinite, expensive loop.

The Hybrid Approach

The most effective architecture is often a hybrid. Use a structured workflow as the outer shell—providing a predictable path for the most important business logic—and place agentic steps inside that shell where flexibility is actually needed. This provides the "best of both worlds": the reliability of code and the intelligence of AI.

Frequently Asked Questions

What is the difference between a chatbot and an agent?
A chatbot typically provides one reply per user message, whereas an agent takes a goal and decides its own sequence of steps and tool calls to reach it.
Should I use a multi-agent system or a single agent?
Start with one agent and simple patterns. Add more agents only when roles truly differ and a single agent becomes too complex to manage reliably.
How do you prevent an AI agent from spending too much money?
Implement guardrails like token budgets, step limits, and spending limits that are enforced by code outside the model's control.
What is the role of a verifier in agentic architecture?
A verifier checks the agent's work—using code, a second model, or a test suite—to ensure the output meets specific criteria before it is accepted.
Can agents work together in a team?
Yes, through multi-agent topologies like 'Manager and Crew' or 'Handoff Networks' where agents delegate tasks or pass control to specialists.
Yellow robotic arm operating inside a glass enclosure with a monitor above.
Photo: Robot arm works on a small component in a lab setting by Shixart1985, CC BY 2.0

Questions people ask

What is the difference between a chatbot and an agent?

Should I use a multi-agent system or a single agent?

How do you prevent an AI agent from spending too much money?

What is the role of a verifier in agentic architecture?

Can agents work together in a team?