Let's talk

Master AI Design

Agentic Architecture Course

Learn how to design software where LLMs plan, use tools, and self-correct to achieve complex goals autonomously.

By Divyaprakash JhaUpdated 6 min readTopic: Agentic AI

Photo for Agentic Architecture Course

What is Agentic Architecture?

Agentic architecture is the design of software where a large language model acts as a decision-maker to achieve a specific goal. Instead of just answering questions, these systems use planning, tool use, and self-verification to complete complex, multi-step tasks autonomously.

Most people are used to simple chatbots that provide a single reply to a single question. Agentic architecture moves beyond this. It is about building systems that can actually do work, such as reconciling GST invoices against a bank statement or managing a clinic's appointment schedule.

The fundamental difference is the starting point. A standard chatbot receives a question; an AI agent receives a goal. A goal might be: "Reconcile last month's invoices, flag any mismatches, and send a summary to the accountant via WhatsApp." This requires the system to read files, query a ledger, compare numbers, and execute actions—not just generate text.

To understand how this differs from traditional AI, it is helpful to look at the distinction between workflows and agents. You can read more about the difference between AI agents and agentic AI to understand how these concepts overlap.

Feature Chatbot Workflow AI Agent
Who decides the next step? The user, one message at a time The developer, hardcoded in logic The model, at run time
Tool Use Rarely Yes, at fixed points Yes, chosen per step
Predictability High High Lower; requires guardrails

The Fundamental Agent Loop

An agent is not just a model; it is a model wrapped in a loop of ordinary, testable software code. While the model makes the decisions, the code manages the execution. Every reliable agent follows a cycle that looks like this:

  1. Goal: The system receives a task with a clear definition of "done," specific constraints, and a set budget.
  2. Plan: The model decides what the next step should be or drafts a full strategy for the entire task.
  3. Act: The system calls a tool. This might be an API call, a database query, or writing a file to a disk.
  4. Observe and Verify: The system reads the result of the tool. It checks: "Did the invoice total match?" or "Did the API return an error?"
  5. Remember: The system records what happened. This history is vital so the agent doesn't repeat the same mistake twice.
  6. Repeat or Stop: The loop continues until the goal is met, the budget is exhausted, or a human must intervene.

Core Components of an AI Agent

To build a production-ready agent, you need more than just an API key. You need a structured architecture consisting of several specialized components.

The Model (The Decision-Maker)

The model reads the goal, the history, and the tool results to choose the next action. When choosing a model, prioritize tool-calling reliability and instruction following over raw benchmark scores. Many advanced systems use a "strong" model (like a large reasoning model) for complex planning and a "small, cheap" model for routine tasks like classifying a message.

Tools (The Hands)

Tools are functions the model can trigger, such as search_invoices(gstin, month) or send_whatsapp(template_id, phone). For a tool to work well, it must have a narrow scope, a clear name, typed parameters, and helpful error messages. Note: A tool that returns 40,000 rows of data is considered a bug, as it will overwhelm the model's context window.

Memory (The Context)

Agents use different types of memory to stay efficient:

  • Working Memory: This is the current context window (the immediate conversation and recent tool results). Because this is limited, you should summarize old steps rather than keeping everything.
  • Episodic Memory: A record of past runs. This allows the agent to remember that "last week, the accountant corrected this specific entry," preventing it from making the same error again.
  • Long-term Memory: This holds massive amounts of data like price lists or company policies, usually retrieved on demand using search or embeddings.

Verifier and Guardrails (The Safety Net)

A Verifier checks the work before it is finalized. This can be a simple piece of code (e.g., "Does the sum of these items equal the total?") or a second model acting as a reviewer. Guardrails are rules enforced outside the model, such as spending limits or blocked domains. You should never rely on the model to "promise" to stay within budget; the code must enforce it.

Proven Agentic Design Patterns

Before building a complex multi-agent system, master these single-agent patterns. Most production value comes from these reliable structures.

  • ReAct (Reason + Act): The model interleaves a short reasoning step with a tool call. It reasons, acts, observes, and then reasons again. This is the standard for research or troubleshooting.
  • Plan-and-Execute: The model writes a full plan first. An executor then works through the steps. If a step fails, the system re-plans. This is excellent for tasks like "migrate these 300 product listings" because a human can review the plan before it runs.
  • Router: A small, cheap model acts as a traffic controller. It looks at an input and sends it to the right path—for example, sending billing questions to a billing flow and complaints to a human.
  • Orchestrator-Workers: An orchestrator breaks a large task into subtasks and hands them to different "workers" (which might be the same model with different prompts). This is useful for tasks where you don't know the subtasks in advance, like editing multiple files in a codebase.
  • Evaluator-Optimizer: One step creates a draft, and a second step critiques it against concrete criteria. The first step then revises the draft based on that critique.

For more on how these patterns apply to real-world scenarios, see our guide on agentic AI examples in real life.

Multi-Agent System Topologies

When a single agent becomes too complex, you split the work across multiple agents, each with its own specific role and prompt. Here are the topologies that actually work in production:

  • Assembly-line SOP: Each agent performs a specific role and hands a "typed artefact" (a structured result) to the next agent in line.
  • Group Chat: A "speaker selector" decides which agent should speak next in a conversation, similar to a moderated meeting.
  • Debate: Multiple agents argue different sides of an issue. A "judge" agent listens to the debate and makes the final decision.
  • Manager and Crew: A high-level manager agent plans the work and assigns specific tasks to a crew of specialized agents.
  • Handoff Networks: Agents pass control to the next specialist only when they reach the limit of their expertise, similar to a customer service agent transferring you to a manager.

Building for Production-Grade Reliability

Moving from a demo in a notebook to a business-critical tool requires "production-grade" engineering. If you are building for a company, you must address these four pillars:

1. Durability

Agents should be able to survive a server restart. This requires checkpoints—saving the state of the agent after every single step so it can resume exactly where it left off without starting over and wasting tokens.

2. Governance and Human-in-the-Loop

Agents that "act" in the real world (like sending money or deleting data) need limits. Every tool should have a risk class (e.g., Read, Fetch, Write, External, or Critical). Critical actions should require a Human-in-the-loop approval, where a person reviews the proposed action and clicks "Approve" before it executes.

3. Auditability

You must maintain a record of everything the agent did. A professional implementation uses a hash-chained audit log, meaning the history of actions is cryptographically secured so it cannot be quietly edited or deleted after the fact.

4. Cost Control

To prevent "infinite loops" where an agent spends hundreds of dollars trying to solve a problem it can't fix, you must implement token budgets and bounded cycles. This ensures the agent stops after a predefined number of steps or a specific dollar amount is reached.

Frequently Asked Questions

What is the difference between a workflow and an agent?
Workflows follow predefined code paths where the developer decides the steps. Agents direct their own process at run time, deciding the next step themselves based on the goal.
When should I use a multi-agent system?
Use multi-agent systems when roles truly differ and tasks can be split into specialized sub-tasks that require different prompts, tools, or levels of autonomy.
How do I prevent an agent from spending too much money?
Implement guardrails outside the model, such as step budgets, token limits, and mandatory human approval for high-cost or high-risk actions.
Can an agent work without a human?
Agents can operate autonomously, but for production, it is best to use human-in-the-loop checkpoints for risky actions or when the agent reaches a specific decision threshold.
What are the most important parts of an agentic loop?
The most reliable systems focus on the loop itself—ensuring the code correctly handles tool results, manages memory, and implements robust error recovery.
Yellow robotic arm operating inside a glass enclosure with a monitor above.
Photo: Robot arm works on a small component in a lab setting by Shixart1985, CC BY 2.0

Questions people ask

What is the difference between a workflow and an agent?

Workflows follow predefined code paths where the developer decides the steps. Agents direct their own process at run time, deciding the next step themselves.

When should I use a multi-agent system?

Use multi-agent systems when roles truly differ and tasks can be split into specialized sub-tasks that require different prompts or tools.

How do I prevent an agent from spending too much money?

Implement guardrails outside the model, such as step budgets, token limits, and mandatory human approval for high-cost or high-risk actions.

Can an agent work without a human?

Agents can operate autonomously, but for production, it is best to use human-in-the-loop checkpoints for risky actions or when the agent reaches a decision threshold.

What are the most important parts of an agentic loop?

The most reliable systems focus on the loop itself—ensuring the code correctly handles tool results, memory, and error recovery.