Let's talk

The Core Architecture

Neural Networks

At the heart of every Large Language Model is a neural network. These mathematical structures allow machines to learn patterns from massive amounts of data.

By Divyaprakash JhaUpdated 5 min readTopic: Large language models

Photo for Neural Networks

A neural network is a computational architecture used to process data and recognize patterns. In the context of Large Language Models (LLMs), these networks are typically built using a "transformer" architecture and are trained to predict the next token in a sequence.

What is a Neural Network?

At its simplest, a neural network is a mathematical structure designed to mimic how information is processed. It is not a database of facts; rather, it is a system that learns to recognize patterns through massive amounts of data.

For a business owner, think of it as an incredibly fast pattern-recognition engine. If you show a system millions of examples of invoices, it eventually learns the "pattern" of what an invoice looks like, even if the layout changes.

  • A neural network is a computational architecture used to process data and recognize patterns.
  • In the context of Large Language Models (LLMs), the network is typically a "transformer" architecture.
  • It is trained on enormous amounts of text to perform a single core skill: predicting the next token in a sequence.

To understand how these networks relate to the broader field, you can explore our guides on artificial intelligence and AI.

How Neural Networks Learn

Learning in a neural network is a process of trial and error on a massive scale. The "knowledge" of the model is stored in parameters—these are adjustable numbers that the model fine-tunes during its training phase.

Imagine a small business owner trying to predict daily sales. Every time they make a wrong prediction, they adjust their mental model of the market. A neural network does this mathematically.

  • Learning happens through parameters—adjustable numbers the model learns during training.
  • During pretraining, the model is shown trillions of tokens; every time it makes a wrong prediction, the parameters are nudged slightly to improve accuracy.
  • Large models can have hundreds of billions of parameters, while smaller models may have a few hundred million.
Model Scale Parameter Count (Approx.) Typical Use Case
Small Models A few hundred million Running offline on mobile phones or simple tasks.
Large Models Hundreds of billions or more Complex reasoning, coding, and deep conversation.

The Transformer and Attention

Modern LLMs use a specific type of neural network called the transformer architecture. The breakthrough that made current AI possible is a mechanism called attention.

Without attention, a model might struggle to understand the relationship between words that are far apart in a sentence. Attention allows the model to look at every word in a context and decide which ones are most relevant to the word currently being processed.

For example, in the sentence, "The clinic called the patient because she was late," attention is what allows the model to work out that "she" refers to the patient, not the clinic.

  • Modern LLMs use the transformer architecture, which relies on a mechanism called "attention".
  • Attention allows a model to look at every word in a sentence to decide which ones are most relevant to the current word being processed.
  • This allows the network to understand context, such as determining what a pronoun like "she" refers to in a complex sentence.

Layers and Data Representation

To make sense of text, a neural network doesn't see letters or words the way humans do. Instead, it converts every "token" (a unit of text like a word or part of a word) into an embedding.

An embedding is a list of numbers, known as a vector, that represents the meaning of that token. In this mathematical space, tokens with similar meanings—like "doctor" and "physician"—end up with similar vectors, placing them close to each other.

These embeddings pass through dozens of layers. Each layer refines the representation, building up complex patterns of grammar, facts, style, and even reasoning.

  • Each token is mapped to an "embedding"—a list of numbers (a vector) that represents its meaning.
  • Tokens with similar meanings end up with similar vectors in this multi-dimensional space.
  • Dozens of layers are stacked together to build up complex patterns of grammar, facts, style, and reasoning.

From Base Models to Assistants

Creating a useful AI assistant is a multi-stage process. You cannot simply feed a model the internet and expect it to be a helpful employee; it requires specific tuning to follow instructions.

The process generally follows these three steps:

  1. Pretraining: This creates a "base model." It is incredibly knowledgeable because it has seen trillions of words, but it is designed only to predict the next word. If you ask a base model a question, it might just respond with more questions because it is simply "continuing" the text.
  2. Instruction Tuning: This is supervised fine-tuning. We show the model examples of instructions and the correct way to respond, teaching it to act like an assistant that follows directions.
  3. RLHF (Reinforcement Learning from Human Feedback): Humans rank different answers from the model. This shapes the model's tone, helpfulness, and safety, ensuring it behaves in a way that humans find useful and polite.

For more on how these models are used in real-world applications, see our guide on AI models.

New Frontiers: Reasoning Models

A new family of models has emerged that behaves differently than standard chat models. While standard models predict the next word almost instantly, reasoning models are designed to "think" before they speak.

These models use reinforcement learning on problems with checkable answers, such as math or code. When you ask them a difficult question, they generate a long internal chain of reasoning. They try different approaches, catch their own mistakes, and only provide a final answer once they have worked through the steps.

  • A new family of models uses reinforcement learning to "think" before they speak.
  • These models generate a long internal chain of reasoning to solve hard, multi-step problems like math or code.
  • This approach uses more "test-time compute" (processing power at the moment of answering) to achieve better results on hard problems.

Frequently Asked Questions

What is the difference between a parameter and a token?
A parameter is an adjustable number inside the model learned during training, while a token is a unit of text (like a word or part of a word) that the model reads and writes.
Why do LLMs sometimes hallucinate?
Hallucinations happen because the neural network is trained to produce plausible-sounding text rather than to verify facts; it is essentially making its best-sounding guess.
What is an embedding in a neural network?
An embedding is a vector of numbers assigned to a token. It maps the meaning of text into a mathematical space where related concepts are positioned close to each other.
What does "attention" actually do?
Attention is a mechanism that allows a model to weigh the importance of different words in a context, helping it understand relationships between distant words in a sentence.
What is the role of the context window?
The context window is the maximum number of tokens a model can consider at one time, including your instructions, the conversation history, and the model's own previous replies.
Close-up of a computer screen displaying lines of CSS code with color swatches.
Photo: Free computer code screen image by unknown, CC0

Questions people ask

What is the difference between a parameter and a token?

A parameter is an adjustable number inside the model learned during training, while a token is a unit of text (like a word or part of a word) that the model reads and writes.

Why do LLMs sometimes hallucinate?

Hallucinations happen because the neural network is trained to produce plausible-sounding text rather than to verify facts; it is essentially making its best-sounding guess.

What is an embedding in a neural network?

An embedding is a vector of numbers assigned to a token. It maps the meaning of text into a mathematical space where related concepts are positioned close to each other.

What does 'attention' actually do?

Attention is a mechanism that allows a model to weigh the importance of different words in a context, helping it understand relationships between distant words in a sentence.

What is the role of the context window?

The context window is the maximum number of tokens a model can consider at one time, including your instructions, the conversation history, and the model's own previous replies.