Let's talk

The Engine of Intelligence

AI: Understanding Large Language Models

At its core, modern AI is powered by Large Language Models (LLMs)—neural networks trained to predict the next piece of text in a sequence.

By Divyaprakash JhaUpdated 7 min readTopic: Large language models

Photo for AI: Understanding Large Language Models

What is an LLM?

A Large Language Model (LLM) is a neural network trained on massive amounts of text to predict the next piece of text in a sequence. By scaling this prediction skill and tuning it to follow instructions, these models can answer questions, write, translate, code, and reason.

At its simplest, an LLM is a "language model," meaning it is designed to predict what text is likely to come next. When you ask a model to draft a GST reminder for a customer, it does not retrieve a stored letter from a database. Instead, it produces the reply one token at a time, choosing the most likely next piece of text based on everything that came before it.

The "Large" in LLM refers to the number of parameters—the adjustable numbers the model learns during training. These can range from a few hundred million in small models to hundreds of billions or more in the largest ones. While these models are incredibly capable, it is important to remember that they do not "look up" facts; they generate likely text. This is why they can sometimes hallucinate, producing a fluent but entirely false answer, such as an invented citation or a wrong tax rate.

To understand how these models function in a business environment, it is helpful to look at the different ways they are built and used. For a deeper dive into the broader field, see our guide on artificial intelligence.

How Models Read: Tokens and Embeddings

Models do not read letters or whole words the way humans do. Instead, they use a tokenizer to split text into smaller units called tokens. A common word might become a single token, while a rare or complex word might be split into several pieces.

This has significant practical implications for business owners, especially regarding cost and speed. Most AI providers measure pricing, speed, and context limits in tokens rather than words. Furthermore, languages that were under-represented in a tokenizer's training data—including many Indian languages written in their own scripts—often require more tokens to express the same sentence. This can make them slower and more expensive to process on certain models.

Once a word is broken into tokens, each token is mapped to an embedding. An embedding is a list of numbers, also known as a vector, with hundreds or thousands of dimensions. In this mathematical space, tokens with similar meanings sit close together. This concept is what powers "semantic search," allowing a system to find documents based on their meaning rather than just matching exact keywords.

The Transformer and Attention

Almost every modern LLM is built on the transformer architecture. The breakthrough of the transformer is a mechanism called attention. When a model processes a word, attention allows it to look at every other word in the sentence to decide which ones are most relevant to the current word being processed.

For example, in the sentence, "The clinic called the patient because she was late," the attention mechanism allows the model to work out that "she" most likely refers to the patient. Without attention, the model would struggle to maintain context in long, complex sentences.

Technically, for every token, the model computes three vectors: a query (what am I looking for?), a key (what do I offer?), and a value (what information do I carry?). By comparing the query of one token with the keys of all earlier tokens, the model calculates weights that determine how much information to take from each value. This process happens across dozens of layers, building up a sophisticated understanding of grammar, facts, and style.

From Pretraining to Instruction Tuning

Building a useful AI involves several distinct stages of training. You can learn more about the underlying mechanics in our guide to machine learning.

1. Pretraining

In the pretraining stage, a model is shown trillions of tokens of text and code. The goal is simple: predict the next token. Each time the model makes a wrong guess, it nudges its billions of parameters slightly to improve. The result is a base model. A base model is incredibly knowledgeable but is not very good at following directions; if you ask it a question, it might just try to continue the text rather than answering you.

2. Instruction Tuning

To make a model useful for humans, it undergoes instruction tuning. This is a supervised fine-tuning process where the model is trained on examples of instructions and the correct way to respond. This teaches the model the "format" of being an assistant—knowing when to answer a question, how to follow a specific format, and when to stop talking.

3. Preference Tuning

Finally, models undergo preference tuning to shape their helpfulness, tone, and safety. One common method is RLHF (Reinforcement Learning from Human Feedback), where humans rank different possible answers. A "reward model" learns these human preferences, and the LLM is optimized to match them. Other methods, like RLAIF, use AI feedback to guide this process.

Reasoning and Test-Time Compute

A new family of models, emerging in late 2024, is changing how we think about AI intelligence. These are often called reasoning models. Unlike standard LLMs that predict the next word almost instantly, reasoning models are trained to "think" before they answer. This is achieved through reinforcement learning on problems with checkable answers, such as math or code.

When faced with a hard problem, these models generate an internal "chain of reasoning." They try different approaches, catch their own mistakes, and refine their logic before they ever show you a final answer. This process uses more test-time compute—meaning the model uses more processing power at the moment you ask the question—to solve complex, multi-step problems that would trip up a standard model.

For a business, this creates a trade-off: reasoning models are better for hard logic and math, but they are often slower and more expensive. For simple tasks like "rewrite this email to be more polite," a standard model is usually more efficient.

Making AI Useful for Business

Simply having access to an LLM isn't enough; you need to make it work for your specific business needs. There are three primary ways to do this, and they should generally be attempted in this order:

Method What it changes Best for... Effort Level
Prompting The instructions and examples you send in the chat. Format, tone, and simple tasks. Low
RAG (Retrieval-Augmented Generation) The documents you add to the context at the moment of the question. Answering from your own, changing data (e.g., current price lists). Medium
Fine-tuning The model's actual internal weights. Consistent style or narrow, specialized skills. Higher

For most businesses, RAG is the most effective way to make an AI "know" your data. It allows the model to look at your specific documents (like a current fee list or a product catalog) and answer questions based only on those facts, which significantly reduces the chance of hallucinations.

From Text to Agents

On its own, an LLM is just a text generator. It can talk about the weather, but it cannot check the weather. However, through tool use (also known as function calling), an LLM can become much more powerful. You can provide the model with descriptions of functions—such as check_stock(sku) or create_invoice(customer, items)—and the model can respond with a structured request to run those functions.

When you put this in a loop, you create an AI agent. An agent can plan a multi-step job: it can check a stock level, realize an item is out of order, decide to draft a follow-up email to the customer, and then wait for a human to approve the action. This allows AI to move from being a "chatbot" to a "worker" capable of handling workflows like reconciling payments or following up on unpaid invoices via WhatsApp.

As you explore these capabilities, you may want to look into specific ai tools designed to facilitate these workflows.

Frequently Asked Questions

Why do AI models hallucinate?
Hallucinations happen because models are trained to produce plausible-sounding text rather than to "know" facts. They are essentially making the most likely guess for the next token, which can result in a fluent but false answer.
What is the difference between an LLM and a base model?
A base model is the result of pretraining on massive data to predict text; an instruction-tuned model has undergone further training to act as a helpful assistant that follows specific directions.
What is a context window?
The context window is the maximum number of tokens a model can consider at one time. This includes your instructions, the documents you've uploaded, the conversation history, and the model's own previous replies.
Should I use RAG or fine-tuning for my business data?
Use RAG to keep facts current and provide specific documents for the model to read. Use fine-tuning if you want to teach the model a specific behavior, a very particular style, or a highly specialized format.
Can AI run offline on a phone?
Yes. Small models with a few billion parameters, which have been "quantized" (compressed) to 4-bit precision, can run fully offline on recent mobile devices. This is useful for privacy or areas with patchy connectivity.
Close-up of a computer screen displaying lines of CSS code with color swatches.
Photo: Free computer code screen image by unknown, CC0

Questions people ask

Why do AI models hallucinate?

Hallucinations happen because models are trained to produce plausible-sounding text rather than to 'know' facts. They are essentially making the most likely guess for the next token.

What is the difference between an LLM and a base model?

A base model is the result of pretraining on massive data to predict text; an instruction-tuned model has undergone further training to act as a helpful assistant.

What is a context window?

The context window is the maximum number of tokens a model can consider at one time, including your instructions, documents, and conversation history.

Should I use RAG or fine-tuning for my business data?

Use RAG to keep facts current and provide specific documents; use fine-tuning to teach the model a specific behavior, style, or format.

Can AI run offline on a phone?

Yes. Small models with a few billion parameters, quantized to 4-bit precision, can run fully offline on recent mobile devices.