What are LLM Models?
A Large Language Model (LLM) is a neural network trained on massive amounts of text to predict the next token in a sequence. By scaling this prediction skill and tuning it to follow instructions, these models can reason, code, and hold conversations.
- An LLM is a neural network, usually based on the transformer architecture, trained on massive amounts of text.
- They function by predicting the most likely next piece of text, known as a token. For example, if you ask a model to draft a GST reminder for a customer, it does not retrieve a stored letter; it produces the reply one token at a time.
- The term "Large" refers to the number of parameters—the adjustable numbers the model learns during training—which can range from a few hundred million in small models to hundreds of billions or more in the largest ones.
How LLM Models Work
Most modern models use the transformer architecture and the "attention" mechanism. This mechanism allows the model to look at every other word in a sentence and decide how much each one matters to the current word being processed. For example, in the sentence "The clinic called the patient because she was late," attention helps the model work out that "she" most likely refers to the patient.
Models do not read whole words or letters. Instead, they process text in tokens. A common word might become one single token, while a rare word might be split into several pieces. This has practical implications for business: because pricing, speed, and context windows are measured in tokens, using languages that require more tokens per sentence—such as certain Indian languages—can make models slower and more expensive to use.
During the generation process, the model produces text one token at a time based on probability, choosing the most likely next piece given everything that came before it.
The Training Process
Building a model involves several distinct stages to move from raw text to a useful assistant:
- Pretraining: The model is shown trillions of tokens of text and code. It learns by constantly trying to predict what comes next, and every wrong guess nudges its billions of parameters slightly. The result is a "base model" that is knowledgeable but simply continues text rather than following instructions.
- Instruction Tuning: The model is further trained on examples of instructions and good responses. This supervised fine-tuning teaches it the format of being an assistant—learning to answer questions, follow a specific format, and stop when the task is done.
- Preference Tuning: This stage shapes the model's tone, helpfulness, and safety. In techniques like RLHF (Reinforcement Learning from Human Feedback), humans rank different answers, and the model is optimized to favor the higher-ranked responses.
Reasoning and Test-Time Compute
Newer reasoning models represent a shift in how AI solves problems. Since late 2024, a family of models has been trained—largely using reinforcement learning on problems with checkable answers like math or code—to "think" before they speak.
These models produce a long internal chain of reasoning, where they can try different approaches and catch their own mistakes before they give you a final answer. This process uses more "test-time compute"—meaning more processing power is spent at the moment the answer is being generated—to solve complex, multi-step problems. While this improves accuracy for hard tasks, it comes with a trade-off in cost and latency.
Open-Weight vs. Closed Models
When choosing a model for your business, you must decide how you want to access the technology. This choice impacts your data privacy and your infrastructure costs.
| Model Type | Access Method | Pros | Cons |
|---|---|---|---|
| Closed Models | Via API (e.g., OpenAI, Anthropic) | Easiest to start; highest capability. | Data goes to a provider; dependent on their pricing. |
| Open-Weight Models | Hosted on your own servers | Better data residency; works offline; full control. | Requires managing your own hardware/cloud. |
You may also encounter Mixture-of-Experts (MoE) models. Instead of using all parameters for every word, an MoE model uses a "router" to send tokens to only a few "experts" within the model. This means an MoE model can have a massive total parameter count (which requires more memory) but runs at the speed of a much smaller model because it uses less active compute for each token.
Making Models Useful for Business
A general model is rarely ready for a specific business task out of the box. You can make it useful through three primary methods, which should generally be attempted in this order:
- Prompting: This is the simplest method. You change the instructions or the examples you send in the prompt to guide the model's behavior. It is best for changing the format or tone of a response.
- RAG (Retrieval-Augmented Generation): This involves adding your own documents to the context at the moment a question is asked. The model "retrieves" the relevant facts from your data and uses them to answer. This is the best way to give a model knowledge of your specific, changing business data (like a current price list).
- Fine-tuning: This involves actually updating the weights of the model. This is a higher-effort task used to teach a model a very specific, narrow skill or a highly consistent style.
How to Choose an LLM Model
Choosing a model is not just about looking at public leaderboards. For a business owner or builder, the following steps are more effective:
- Define the task and create a test set: Collect 30 to 50 real-world examples of inputs and the "perfect" answers you expect. Test your models against this set.
- Consider language requirements: If your customers write in Hindi, Marathi, or Hinglish, test specifically for those. Model quality varies significantly depending on how efficiently the tokenizer handles those scripts.
- Start large, then shrink: It is often best to get your logic working with a large, highly capable model first. Once the workflow is proven, try to move to a smaller, cheaper, or quantized model to save costs.
- Evaluate by task, not by token: A reasoning model that gets the answer right on the first try might actually be cheaper than a fast, cheap model that requires three attempts to get the logic correct.
Frequently Asked Questions
- What is the difference between a base model and an instruction-tuned model?
- A base model is trained on next-token prediction and simply continues text. An instruction-tuned model is further trained to follow directions and act as an assistant.
- Why do LLMs hallucinate?
- Hallucinations happen because models are trained to produce plausible-sounding text rather than to strictly verify facts. They generate the most likely next token, which may be factually incorrect.
- What are tokens in an LLM?
- Tokens are the basic units of text a model reads and writes. They can be whole words or pieces of words, and they are used to measure pricing, speed, and context limits.
- What is the benefit of quantization?
- Quantization reduces the number of bits used to store model weights (e.g., from 16-bit to 4-bit). This allows large models to fit on smaller, cheaper hardware with modest quality loss.
- Can LLMs work offline?
- Yes, small models with a few billion parameters, when quantized, can run fully offline on recent mobile phones.

Questions people ask
What is the difference between a base model and an instruction-tuned model?
A base model is trained on next-token prediction and simply continues text. An instruction-tuned model is further trained to follow directions and act as an assistant.
Why do LLMs hallucinate?
Hallucinations happen because models are trained to produce plausible-sounding text rather than to strictly verify facts. They generate the most likely next token, which may be factually incorrect.
What are tokens in an LLM?
Tokens are the basic units of text a model reads and writes. They can be whole words or pieces of words, and they are used to measure pricing, speed, and context limits.
What is the benefit of quantization?
Quantization reduces the number of bits used to store model weights (e.g., from 16-bit to 4-bit). This allows large models to fit on smaller, cheaper hardware with modest quality loss.
Can LLMs work offline?
Yes, small models with a few billion parameters, when quantized, can run fully offline on recent mobile phones.
