A Large Language Model (LLM) is a neural network trained on massive amounts of text to predict the next piece of text in a sequence. By scaling this single skill, these models can perform complex tasks like reasoning, coding, and following instructions.
What is an LLM?
At its core, an LLM is a neural network, typically using a transformer architecture. It is trained on enormous amounts of text and code to learn the patterns of human language.
It is important to understand that an LLM does not "look up" facts in a database like a traditional search engine. Instead, it works by predicting the next "token" in a sequence. For example, if you ask a model to draft a GST reminder for a customer, it does not retrieve a stored letter from a folder; it produces the reply one piece at a time, choosing the most likely next piece based on everything that came before it.
The term "large" refers to the number of parameters—the adjustable numbers the model learns during its training. These can range from a few hundred million in small models to hundreds of billions or more in the largest, most capable models.
How Models Read: Tokens and Embeddings
When you type a prompt into an AI, the model does not see words or letters the way humans do. It uses a tokenizer to break your text down into smaller units called tokens. A token might be a common word, a part of a rare word, or even a piece of punctuation.
Once the text is tokenized, each token is mapped to an embedding. An embedding is a list of numbers (a vector) that represents the meaning of that token. In this mathematical space, tokens with similar meanings end up with similar vectors.
This concept is the foundation of semantic search. By turning documents and questions into these mathematical vectors, a system can find the most relevant information by identifying which vectors are mathematically closest to each other.
| Concept | What it is | Why it matters for business |
|---|---|---|
| Token | A chunk of text (word or part of a word). | Pricing, speed, and context limits are all measured in tokens. |
| Embedding | A list of numbers representing meaning. | Allows the AI to understand "intent" rather than just matching keywords. |
Note for builders: Languages that are under-represented in a tokenizer's training data—such as many Indian languages written in their own scripts—often require more tokens to represent the same sentence. This can make these languages slower and more expensive to process on some models.
The Engine: Transformers and Attention
Modern LLMs are powered by the transformer architecture. The "secret sauce" of the transformer is a mechanism called attention.
Attention allows a model to look at every other word in a sentence to decide which ones are most relevant to the current word it is processing. It helps the model understand context and relationships. For example, in the sentence, "The clinic called the patient because she was late," attention is what allows the model to work out that "she" most likely refers to the patient.
Technically, for every token, the model computes three vectors: a query (what am I looking for?), a key (what do I offer?), and a value (what information do I carry?). By comparing the query of one token against the keys of all earlier tokens, the model calculates how much "attention" to pay to each piece of information.
How LLMs are Trained
Building a model happens in several distinct stages, moving from general knowledge to specific utility.
- Pretraining: The model is shown trillions of tokens of text and code. It plays a massive game of "guess the next word." The result is a base model. This model is incredibly knowledgeable but is designed to continue text patterns rather than follow specific instructions.
- Instruction Tuning: To make the model useful as an assistant, it undergoes supervised fine-tuning. It is trained on examples of instructions and high-quality responses, teaching it to follow commands and stop when a task is done.
- Preference Tuning (RLHF/RLAIF): This stage shapes the model's tone, helpfulness, and safety. In Reinforcement Learning from Human Feedback (RLHF), humans rank different answers, and the model is optimized to favor the ones humans prefer. RLAIF uses AI feedback to achieve similar results.
For a deeper dive into the mechanics of these models, see our guide on AI models.
Reasoning and Test-Time Compute
Since late 2024, a new family of reasoning models has emerged. Unlike standard models that respond almost instantly, these models are trained to "think" before they answer using reinforcement learning.
They produce an internal chain of reasoning, allowing them to try different approaches, catch their own mistakes, and only then provide a final reply. This process uses test-time compute, meaning the model spends more processing power during the actual answering phase to solve complex, multi-step problems.
This is a significant trade-off: reasoning models are ideal for hard tasks like math or complex coding, but they are slower and more expensive. For simple tasks like "rewrite this email to be more polite," a standard model is usually more efficient.
Understanding Hallucinations
A hallucination occurs when an AI produces a fluent, confident answer that is factually false. This might look like an invented legal citation, a wrong tax rate, or a policy your business never actually implemented.
Hallucinations happen because LLMs are trained to produce plausible-sounding text, not necessarily truthful text. If a model hasn't seen enough data on a specific topic, it will still attempt to generate a "best-sounding guess."
Making LLMs Useful for Business
To move from a general chatbot to a specialized business tool, you generally use three methods. You should typically try them in this order:
| Method | What it changes | Best Use Case |
|---|---|---|
| Prompting | The instructions and examples you send in the chat. | Changing the tone or format (e.g., "Reply in polite Hindi"). |
| RAG (Retrieval-Augmented Generation) | The documents you add to the context at the moment you ask a question. | Answering questions based on your current price lists or schedules. |
| Fine-tuning | The actual internal weights (the "brain") of the model. | Achieving a very specific, consistent style or narrow technical skill. |
For most businesses, RAG is the most effective way to make a model "know" your specific data without the high cost of retraining a model.
If you are interested in how these models can perform actions like checking stock or creating invoices, read our guide on AI agents.
Frequently Asked Questions
- What is the difference between a base model and an assistant?
- A base model is trained to continue text patterns (it just predicts what comes next), while an assistant is fine-tuned to follow instructions and answer questions.
- Why do LLMs sometimes make mistakes or "hallucinate"?
- LLMs generate the most likely next token based on patterns; they do not "look up" facts, so they may produce plausible-sounding but incorrect information.
- What is a token in AI?
- A token is the basic unit of text a model processes. It can be a whole word, a part of a word, or even punctuation.
- What is RAG?
- Retrieval-Augmented Generation (RAG) is a method where you provide specific documents to the model's context, allowing it to answer questions based on your actual data.
- What is the purpose of 'attention' in a transformer?
- Attention allows the model to weigh the importance of different words in a sentence, helping it understand context and relationships between ideas.

Questions people ask
What is the difference between a base model and an assistant?
A base model is trained to continue text patterns, while an assistant is fine-tuned through instruction tuning to follow commands and answer questions.
Why do LLMs sometimes make mistakes or 'hallucinate'?
LLMs generate the most likely next token based on patterns; they do not 'look up' facts, so they may produce plausible-sounding but incorrect information.
What is a token in AI?
A token is the basic unit of text a model processes. It can be a whole word, a part of a word, or even punctuation.
What is RAG?
Retrieval-Augmented Generation (RAG) is a method where you provide specific documents to the model's context, allowing it to answer questions based on your actual data.
What is the purpose of 'attention' in a transformer?
Attention allows the model to weigh the importance of different words in a sentence, helping it understand context and relationships between ideas.
