Let's talk

AI Architecture Explained

Large Language Models

A large language model (LLM) is a neural network trained on massive amounts of text to predict the next token in a sequence.

By Divyaprakash JhaUpdated 5 min readTopic: Large language models

Photo for Large Language Models

What is a Large Language Model?

A large language model (LLM) is a neural network, usually a transformer, trained on enormous amounts of text to predict the next token. That one skill, scaled up and then tuned to follow instructions, lets it answer questions, write, translate, code and reason.

  • An LLM is a neural network, usually based on the transformer architecture, trained to predict the next token in a sequence.
  • The term "large" refers to the number of parameters—the adjustable numbers the model learns during training—which can range from a few hundred million in small models to hundreds of billions or more in the largest ones.
  • By scaling this prediction skill and tuning it to follow instructions, models can answer questions, write, translate, code, and reason.

How LLMs Process Information

Models do not read whole words; they use a tokenizer to split text into tokens. Common words become one token, while rare words are split into several pieces. This has practical implications for cost and speed; for example, languages that are under-represented in a tokenizer's training data, such as many Indian languages written in their own scripts, often require more tokens for the same sentence. This makes them slower and more expensive to process on certain models.

The transformer architecture uses "attention" to weigh the importance of every other word in a context when processing a specific word. In a sentence like "The clinic called the patient because she was late," attention is what allows the model to work out that "she" most likely refers to the patient.

Each token is mapped to an embedding, which is a list of numbers (a vector) with hundreds or thousands of dimensions. In this mathematical space, tokens with similar meanings end up with similar vectors. This concept is what powers semantic search: by embedding your documents and a question, you can find the documents whose vectors are closest to the question.

From Pretraining to Instruction Tuning

The journey of a model begins with pretraining. During this stage, the model is shown trillions of tokens of text and code. The goal is simple: predict what comes next. Every time the model makes a wrong guess, it nudges its billions of parameters slightly to improve. This results in a "base model" that is incredibly knowledgeable but tends to simply continue text rather than following instructions or answering questions.

To make a model useful for humans, developers use instruction tuning. This is a supervised fine-tuning process where the model is trained on examples of specific instructions and good responses. This teaches the model the format of being an assistant—knowing when to answer a question, how to follow a requested format, and when to stop talking.

Finally, models undergo preference tuning. In a process called Reinforcement Learning from Human Feedback (RLHF), people rank different alternative answers provided by the model. A reward model learns these human preferences, and the LLM is optimized toward them. This stage is critical for shaping the model's helpfulness, tone, and safety.

The Rise of Reasoning Models

A new family of reasoning models has emerged, trained largely with reinforcement learning on problems with checkable answers, such as mathematics and code. These models are designed to "think" before they answer.

Instead of immediately generating a response, they produce a long internal chain of reasoning. This allows them to try different approaches, catch their own mistakes, and only then provide a final reply. This approach uses more "test-time compute"—spending more computational power at the moment the answer is generated—to solve complex, multi-step problems. While this leads to better results on hard tasks, it comes with a trade-off of higher cost and increased latency compared to standard models.

Making LLMs Useful for Business

For business owners and builders, there are three primary ways to adapt a general model to specific business needs. The best approach depends on whether you want to change the model's behavior, its knowledge, or its format.

Method What it changes Best for... Effort Level
Prompting The instructions and examples you send Format, tone, and simple tasks Low
RAG The documents you add to the context at question time Answering from your own, changing data Medium
Fine-tuning The model's actual weights Consistent style or narrow, specialized skills Higher

Retrieval-Augmented Generation (RAG)

RAG is often the best solution for the question, "How do I make it know our data?" Instead of retraining the model, you provide the relevant information in the context window at the moment the question is asked. This allows the model to answer questions using your specific, constantly changing data, such as a clinic's current fee list or a doctor's schedule.

Fine-tuning

Fine-tuning is best when you need a model to behave in a very specific, consistent way—such as always extracting invoice fields into a specific JSON format—or when you want to teach a smaller, cheaper model a narrow skill. While powerful, it requires more effort in data preparation and evaluation.

Understanding Hallucinations

A hallucination occurs when a model generates a fluent, confident answer that is factually false. It might invent a citation, provide a wrong GST rate, or claim a policy exists that your company never actually had. This happens because the model is designed to produce plausible-sounding text, not necessarily to "know" facts.

To reduce these errors in a business environment, consider these strategies:

  • Use RAG: Provide the model with the specific source material and instruct it to answer only from that text.
  • Encourage Honesty: Explicitly tell the model in the prompt that it is allowed to say "I don't know" if the answer isn't in the provided context.
  • Human in the Loop: Always have a human verify outputs for anything involving legal, medical, or financial decisions.

How to Choose the Right Model

Choosing a model is not just about looking at public leaderboards. Builders should follow a structured approach to ensure the model fits their specific business constraints.

  1. Start Large, Then Shrink: Get your task working perfectly with a large, highly capable model first. Once the logic is proven, test if a smaller or quantized model can maintain that performance at a lower cost.
  2. Consider Data Residency: If your data is sensitive and cannot leave your servers or your country, look for open-weight models that you can host on your own infrastructure.
  3. Check Language Support: If your customers interact in Hindi, Marathi, or Hinglish, test the models specifically in those languages. Quality and token efficiency can vary significantly between different models.
  4. Evaluate the Task: For simple tasks like "rewrite this message politely," a standard model is fine. For hard, multi-step logic, a reasoning model might be more cost-effective because it gets the answer right on the first try.

Finally, consider how you will use the model. If you plan to build "agents" that can call functions—like check_stock(sku)—ensure the model has strong capabilities for tool use and structured output.

Close-up of a computer screen displaying lines of CSS code with color swatches.
Photo: Free computer code screen image by unknown, CC0

Questions people ask

Do LLMs look up facts in a database?

No, they generate likely text one token at a time. They do not retrieve stored letters or facts unless paired with a system like RAG.

What is a token in an LLM?

A token is the unit of text a model reads and writes. It can be a common word or a piece of a rare word.

What is the difference between open-weight and closed models?

Closed models are accessed via API and your data goes to the provider. Open-weight models allow you to run the model on your own servers for better data residency.

What are hallucinations?

A hallucination is a fluent, confident answer that is factually false, such as an invented citation or an incorrect tax rate.

Can LLMs run on a phone?

Yes, small models with a few billion parameters, quantized to around 4 bits, can run fully offline on recent mobile devices.