Let's talk

Agentic AI Architecture

Coding Agents

Coding agents are programs that take a goal—not just a question—and use an iterative loop to plan, act, and verify code changes until the job is done.

By Divyaprakash JhaUpdated 6 min readTopic: Agentic AI

Photo for Coding Agents

What are Coding Agents?

Coding agents are programs that take a goal—not just a question—and use an iterative loop to plan, act, and verify code changes until the job is done.

Unlike standard chatbots that provide a single reply to a prompt, coding agents are designed to achieve a specific, complex goal. For example, instead of just answering a question about a bug, an agent can be tasked with "Fix the login error in the production environment" or "Migrate all product listings from the old CSV to the new database."

They operate within a continuous loop. The process typically follows these steps:

  • Receive a task: The agent is given a clear definition of "done," including constraints and a budget.
  • Plan: The agent decides the next step or drafts a full plan for the task.
  • Act: The agent calls a tool, such as a database query, an API call, or a file write.
  • Observe and verify: The agent reads the tool result to see if it worked. It checks: Did the invoice total match? Did the API return an error?
  • Repeat or stop: The agent continues, re-plans, asks a human for help, or finishes and reports the result.

A successful agent requires more than just a smart model. It requires a loop wrapped in ordinary, reliable code to manage execution budgets, checkpoints, and human approvals.

Agents vs. Workflows vs. Chatbots

Understanding the difference between these three types of AI software is essential for builders to avoid over-engineering. While they are often confused, they serve very different business needs.

Feature Chatbot Workflow Agent
Who decides the next step? The user, one message at a time The developer, in code The model, at runtime
Use of Tools Rarely Yes, at fixed points Yes, chosen per step
Steps per task One reply Fixed and known Variable, often unknown in advance
Predictability High High Lower; needs guardrails

In practice, the most robust systems are hybrids. The outer shell is often a workflow that you can reason about and control, while agentic steps sit inside it to provide flexibility where steps are not known in advance.

The Anatomy of a Coding Agent

To move beyond a simple demo in a notebook, a production-grade agent needs several core components working in harmony.

The Model

The language model is the decision-maker. It reads the goal, the history, and the tool results to choose the next action. When building, it is often better to pick a model based on its reliability in tool-calling and instruction following rather than just raw benchmark scores.

Tools

Tools are specific functions the agent can call, such as search_invoices(gstin, month) or write_file(path, content). For an agent to work effectively, tools must have narrow scopes, clear names, typed parameters, and helpful error messages. A tool that returns too much data—like 40,000 rows—can break the agent's ability to process the information.

Memory

Agents require different layers of memory to function over long tasks:

  • Working Memory: This is the current context window. It includes the goal, recent steps, and tool results. Because this is limited, it is best to summarize old steps rather than keeping everything.
  • Episodic Memory: A record of past runs. This allows the agent to remember what it tried, what worked, and what a human corrected, preventing the same mistake twice.
  • Long-term Memory: This holds facts and documents, such as company price lists or policies, usually retrieved on demand.

Verifier

A verifier checks the agent's work against concrete criteria before the task is considered complete. This can be a piece of deterministic code (e.g., checking if a sum of line items equals an invoice total), a test suite, or a second model acting as a reviewer.

Proven Agentic Patterns for Coding

Building an agent from scratch can be difficult. Most successful systems use established patterns to manage how the model thinks and acts.

  • Plan-and-Execute: The model drafts a full plan first. An executor then works through the plan step-by-step, re-planning only if a specific step fails. This is highly useful for tasks like "migrate these 300 product listings" because the plan is visible and reviewable by a human before any changes are made.
  • Orchestrator-Workers: An orchestrator breaks a large, complex task into smaller subtasks and hands them to specialized workers. This is ideal when the subtasks cannot be known in advance, such as when an agent is editing multiple files in a large codebase.
  • Evaluator-Optimizer: This pattern uses a loop where one step produces a draft, and another step critiques that draft against clear criteria, forcing the first step to revise its work.
  • ReAct (Reason + Act): This pattern interleaves reasoning steps with tool calls. The model reasons about the current state, performs an action, observes the result, and reasons again. This is the standard approach for troubleshooting and research.

Ensuring Reliability and Safety

When an agent has the power to write code or move money, safety cannot be left to the model's "good behavior." You must implement guardrails that exist outside the model's control.

Guardrails: These are rules enforced by your software to set spending limits, block unauthorized tools, or filter specific types of output. For example, you might allow an agent to "read" a database but never "delete" from it without special permission.

Human-in-the-loop: This creates necessary pause points. For high-risk actions—like issuing a refund or deploying code to a live server—the system should pause and require a human to review and approve the action.

Durability: Production agents must be able to survive a server restart. This requires using checkpoints and state management so an agent can resume its work exactly where it left off, even if a task takes several days to complete.

Moving from Notebooks to Production

There is a massive gap between an agent running in a Jupyter Notebook and an agent running as a reliable part of a business. To bridge this gap, builders must focus on the "operating system" of the agent.

Reliability often depends more on the "loop"—the ordinary code surrounding the model—than the model itself. You need a system that handles retries, timeouts, and step budgets to ensure an agent doesn't get stuck in an infinite loop and drain your API budget.

For enterprise-grade operations, consider these requirements:

  • Audit Trails: Every action taken by the agent must be recorded in a hash-chained log. This ensures the history of what the agent did is unchangeable and can be audited later.
  • Task Management: Use a system that can handle long-running tasks and provide a "live operations floor" to monitor what is happening in real-time.
  • Governance: Implement "autonomy levels." For example, an agent might have "Observe" autonomy for reading data, but require "Human Approval" for any "Write" or "External" tool.

If you are building complex multi-agent systems, you may need to look into agent orchestration platforms or advanced agent meshes to manage how different specialized agents communicate and hand off tasks to one another.

Frequently Asked Questions

How is a coding agent different from a standard AI chatbot?
A chatbot provides a single response to a prompt, whereas a coding agent is given a goal and autonomously decides which steps, tools, and loops are needed to reach it.
Can coding agents be trusted to write code autonomously?
Reliability is achieved by wrapping the model in a loop with verifiers, guardrails, and human approval points for risky or critical actions.
What are the most common failure points in AI agents?
Most reliability problems stem from the "loop"—the logic that manages how many steps are allowed, how tools are called, and how errors are handled.
What is the benefit of a 'Plan-and-Execute' pattern?
It makes the agent's approach visible and reviewable by humans before any code is actually changed or executed.
Do agents need to be told every single step?
No, in an agentic architecture, the model decides the next step at runtime based on the goal, the tools available, and the results of previous actions.
Yellow robotic arm operating inside a glass enclosure with a monitor above.
Photo: Robot arm works on a small component in a lab setting by Shixart1985, CC BY 2.0

Questions people ask

How is a coding agent different from a standard AI chatbot?

A chatbot provides a single response to a prompt, whereas a coding agent is given a goal and autonomously decides which steps, tools, and loops are needed to reach it.

Can coding agents be trusted to write code autonomously?

Reliability is achieved by wrapping the model in a loop with verifiers, guardrails, and human approval points for risky or critical actions.

What are the most common failure points in AI agents?

Most reliability problems stem from the 'loop'—the logic that manages how many steps are allowed, how tools are called, and how errors are handled.

What is the benefit of a 'Plan-and-Execute' pattern?

It makes the agent's approach visible and reviewable by humans before any code is actually changed or executed.

Do agents need to be told every single step?

No, in an agentic architecture, the model decides the next step at runtime based on the goal, the tools available, and the results of previous actions.