Let's talk

Building with LLMs

AI Founder & Agentic Architect

Divyaprakash Jha is an AI founder and architect specializing in agentic systems, transforming large language models into autonomous, tool-using agents.

By Divyaprakash JhaUpdated 5 min readTopic: Large language models

Photo for AI Founder & Agentic Architect

An AI founder is a builder who creates products leveraging Large Language Models (LLMs) to solve complex business problems. Modern AI founders focus on agentic architecture, moving beyond simple chat to build systems that use tools to complete multi-step, autonomous jobs.

What is an AI Founder?

An AI founder is not just someone who uses a chatbot; they are architects of intelligent systems. They look at a manual business process—like checking inventory or following up on unpaid invoices—and design a way for a model to handle it.

Building with modern AI involves several layers of complexity:

  • Problem Solving: Using LLMs to automate reasoning tasks rather than just generating text.
  • Agentic Workflows: Moving from a "one-shot" prompt to a system that can plan, execute, and correct itself.
  • Real-world Integration: Connecting models to actual business tools like payment gateways or CRM systems.

For example, instead of just drafting a polite reminder, an AI founder builds a system that identifies which customers have overdue payments and sends a customized message via WhatsApp.

From LLMs to AI Agents

To understand an AI founder's work, you must understand the transition from a standard Large Language Model (LLM) to an AI agent. An LLM on its own is a sophisticated text predictor. It produces the most likely next token based on the patterns it learned during training. It does not "know" facts; it generates plausible text.

The breakthrough for builders is tool use, also known as function calling. By describing specific functions to the model—such as check_stock(sku_id) or create_invoice(customer_id, amount)—a founder can give the model a way to interact with the real world.

This creates a loop that defines an agentic system:

  1. Planning: The model looks at the user's request and decides which steps are needed.
  2. Action: The model outputs a structured request to call a specific tool.
  3. Observation: Your code runs the tool and feeds the result (like a stock count or a database record) back to the model.
  4. Decision: The model reads the result and decides if the job is done or if another step is required.

This loop allows for complex workflows, such as a clinic assistant that can check a doctor's schedule, verify a patient's appointment, and then book a new slot—all while keeping a human in the loop for final approvals on sensitive tasks.

How to Build with LLMs

Founders must choose the right strategy depending on whether they need the model to know new facts, follow a specific style, or simply follow instructions. There are three primary methods for making a general model work for a specific business use case.

Method What it changes Best for... Effort Level
Prompting The instructions and examples sent in the request. Controlling format, tone, and simple tasks. Low
RAG (Retrieval-Augmented Generation) The documents added to the context at question time. Answering questions from your own, changing business data. Medium
Fine-tuning The actual weights of the model. Consistent style or teaching narrow, specialized skills. High

For most businesses, RAG is the preferred starting point. If you want a model to answer questions based on your current price list or a specific set of medical records, RAG allows you to provide that data as context without the high cost of retraining a model.

If you need a model to always output data in a very specific JSON format for your software to read, or to speak in a very specific brand voice, fine-tuning becomes the more effective tool.

The Core of Modern AI

At the heart of these systems is the Transformer architecture. This architecture uses a mechanism called attention, which allows the model to weigh the importance of different words in a sentence. For instance, in the sentence "The clinic called the patient because she was late," attention allows the model to understand that "she" refers to the patient.

When building, founders must navigate several technical trade-offs:

  • Closed vs. Open-weight Models: Closed models are accessed via an API. They are easy to start with but your data is sent to a provider. Open-weight models allow you to host the model on your own servers, which is vital for data residency and privacy.
  • Reasoning Models: Some newer models use "test-time compute." This means they spend more time "thinking" through a problem—running an internal chain of reasoning—before they give you an answer. This is great for hard math or coding problems, but it is slower and more expensive than a standard model.
  • Context Windows: This is the amount of text a model can "keep in mind" at once. While windows are getting larger, it is important to remember that models still perform best when the most relevant information is placed clearly near the question.

Optimization for Real-World Use

Running large models is expensive and memory-intensive. To make AI viable for small businesses or mobile devices, founders use several optimization techniques.

Quantization

Every parameter in a model is a number. By using quantization, we can reduce the precision of these numbers (for example, from 16-bit to 4-bit). This significantly reduces the memory required. A 4-bit model might take up only a quarter of the memory, allowing it to run on cheaper hardware or even offline on a smartphone.

Mixture-of-Experts (MoE)

Instead of activating every single part of a model for every word, MoE models use a "router" to send the task to only a small group of "experts" within the model. This allows the model to have a huge amount of total knowledge while only using a fraction of the computing power for any single task, making it much faster.

The KV Cache

During the generation of text, the model has to re-calculate the "attention" for every previous word over and over again. The KV cache stores these values so the model doesn't have to recompute them, making the text generation feel much faster and more fluid.

By combining these techniques—RAG for knowledge, tool use for action, and quantization for speed—AI founders are building the next generation of autonomous business tools.

Frequently Asked Questions

What is the difference between an LLM and an AI agent?
An LLM is a model that predicts the next token in a sequence. An AI agent is a system that uses an LLM to call tools and execute multi-step workflows.
How do I stop an AI from hallucinating?
Use Retrieval-Augmented Generation (RAG) to provide the model with specific source material and instruct it to only answer from those facts. Always keep a human in the loop for legal, medical, or financial decisions.
Should I use RAG or fine-tuning for my business?
Use RAG to keep facts current and provide your own data. Use fine-tuning to make a model behave consistently or to teach it a specific style or narrow skill.
What are open-weight models?
Open-weight models allow you to run the trained weights on your own servers, which is ideal for data residency, offline use, and high-volume work.
Why are some languages more expensive to process in LLMs?
Some tokenizers require more tokens to represent the same sentence in certain languages, making them slower and costlier to process.
Close-up of a computer screen displaying lines of CSS code with color swatches.
Photo: Free computer code screen image by unknown, CC0

Questions people ask

What is the difference between an LLM and an AI agent?

An LLM is a model that predicts the next token in a sequence. An AI agent is a system that uses an LLM to call tools and execute multi-step workflows.

How do I stop an AI from hallucinating?

Use Retrieval-Augmented Generation (RAG) to provide the model with specific source material and instruct it to only answer from those facts.

Should I use RAG or fine-tuning for my business?

Use RAG to keep facts current and provide your own data. Use fine-tuning to make a model behave consistently or to teach it a specific style.

What are open-weight models?

Open-weight models allow you to run the trained weights on your own servers, which is ideal for data residency and offline use.

Why are some languages more expensive to process in LLMs?

Some tokenizers require more tokens to represent the same sentence in certain languages, making them slower and costlier to process.