Let's talk

AI Fundamentals

What is LLM in AI?

A Large Language Model (LLM) is a neural network trained on enormous amounts of text to predict the next token in a sequence. By scaling this single skill, models can reason, write, and code.

By Divyaprakash JhaUpdated 5 min readTopic: Large language models

Photo for What is LLM in AI?

A Large Language Model (LLM) is a neural network trained on massive amounts of text to predict the next token in a sequence. By scaling this single skill, these models can perform complex tasks like reasoning, writing, and coding.

Defining the Large Language Model

At its core, an LLM is a type of neural network, most commonly built using the transformer architecture. This architecture allows the model to process language by looking at the relationships between all words in a sentence simultaneously.

  • The "Large" in LLM: This refers to the number of parameters—the internal, adjustable numbers the model learns during its training process. While small models might have a few hundred million parameters, the most powerful models can have hundreds of billions or more.
  • The "Language Model" aspect: A language model is a system designed to predict which text is likely to come next based on the preceding context. It is essentially a highly advanced autocomplete engine.

How LLMs Generate Text

It is a common misconception that LLMs are search engines or databases. They do not "look up" facts; they generate text through probabilistic prediction. To understand how they create coherent sentences, we need to look at two key concepts: tokens and attention.

The Power of Attention

Modern LLMs use a mechanism called "attention" to understand context. When the model processes a word, it looks at every other word in the sentence to decide which ones are most relevant. For example, in the sentence, "The clinic called the patient because she was late," the attention mechanism allows the model to understand that "she" refers to the patient, not the clinic.

Each layer of the model refines this understanding, stacking dozens of layers to build up a grasp of grammar, style, and even reasoning patterns. You can learn more about the underlying mechanics in our guide on how LLMs work.

From Pretraining to Instruction Tuning

Building a useful LLM happens in stages. A model does not start out as a helpful assistant; it starts as a raw predictor of text.

  1. Pretraining: The model reads trillions of tokens of text and code. Its only goal is to predict the next token. Through this, it learns the patterns of human language. The result is a "base model"—it is incredibly knowledgeable but often fails to follow instructions, often just continuing a prompt rather than answering it.
  2. Instruction Tuning: To make the model useful for humans, it undergoes supervised fine-tuning. It is trained on examples of specific instructions and the ideal responses to those instructions. This teaches the model how to act as an assistant.
  3. Preference Tuning: To refine the model's tone and safety, developers use methods like RLHF (Reinforcement Learning from Human Feedback). Humans rank different model answers, and the model is optimized to favor the types of responses humans prefer.

The Building Blocks: Tokens and Embeddings

To a computer, words are just strings of characters. To process them, LLMs use a two-step translation process involving tokens and embeddings.

Tokens: The Units of Text

Models do not read whole words. Instead, a tokenizer splits text into "tokens." A common word might be a single token, while a rare or complex word might be split into several pieces. This is why pricing and speed in AI are often measured in tokens rather than words.

For business owners in India, it is important to note that languages like Hindi or Marathi may require more tokens for the same sentence compared to English. This can make these languages slightly slower or more expensive to process on some models.

Embeddings: The Meaning in Numbers

Once a word is turned into a token, it is mapped to an embedding. An embedding is a list of numbers (a vector) that represents the meaning of that token. In this mathematical space, tokens with similar meanings—like "doctor" and "physician"—end up with similar vectors. This is the foundation of semantic search.

To understand the technical relationship between these elements, see what a token is in an LLM.

Understanding Hallucinations

One of the biggest risks for businesses using LLMs is the "hallucination." A hallucination occurs when a model produces an answer that is fluent and confident but factually incorrect—such as inventing a fake GST rate or a non-existent company policy.

This happens because the model is designed to produce plausible text, not necessarily true text. It is predicting what sounds right, not what is factually verified.

Problem Why it happens How to fix it
Hallucination Model predicts plausible but false text. Use RAG to provide specific source material.
Lack of Context The model "forgets" earlier parts of a long conversation. Manage the context window effectively.

To reduce these errors, businesses should use Retrieval-Augmented Generation (RAG). This allows you to give the model specific documents to reference, forcing it to answer based on your actual data rather than its own "memory."

Making LLMs Useful for Business

For a business owner, an LLM is a raw engine. To make it a useful tool, you must decide how to apply it. There are three primary methods, and you should generally try them in this order:

  1. Prompting: This involves changing the instructions or examples you send to the model. It is low effort and great for changing the tone or format of a response.
  2. RAG (Retrieval-Augmented Generation): This is the best way to make a model "know" your business. You connect the model to your own documents (like a fee list or a product catalog). The model looks up the relevant info and then answers the user.
  3. Fine-tuning: This is the most advanced method. It involves updating the actual weights of the model to achieve a very specific, consistent style or a narrow, specialized skill.

From Text to AI Agents

An LLM on its own is a "brain in a box"—it can think and talk, but it cannot act. To move from a chatbot to an AI Agent, you must give the model "tools."

Through a process called tool use (or function calling), a model can output a structured request to perform an action. For example, instead of just telling you that a customer has an unpaid invoice, the model can request to run a function that checks your accounting database.

When a model can plan a series of steps, call tools, read the results, and react to those results in a loop, it has become an agent. This allows for complex, multi-step workflows, such as reconciling payments or following up on unpaid invoices via WhatsApp.

For a deeper dive into these capabilities, read our LLM guide.

Close-up of a computer screen displaying lines of CSS code with color swatches.
Photo: Free computer code screen image by unknown, CC0

Questions people ask

Do LLMs actually 'know' facts?

No, they generate the most likely next token based on patterns. They can hallucinate incorrect information because they are predicting plausible text rather than retrieving stored facts.

What is the difference between a parameter and a token?

Parameters are the internal adjustable numbers the model learns during training that determine its capacity. Tokens are the units of text (words or parts of words) that the model reads and writes.

What is a context window?

The context window is the maximum number of tokens a model can consider at one time, including your instructions, conversation history, and the model's own previous replies.

How can I stop an LLM from making mistakes?

You can use Retrieval-Augmented Generation (RAG) to give the model specific documents to reference, or use tools to verify calculations and data via code.

What is the difference between an LLM and an AI agent?

An LLM is a model that generates text; an AI agent is a system where the LLM uses tools to perform multi-step tasks, like reconciling payments or checking stock.