Let's talk

AI Architecture Explained

What is a Transformer in AI?

The transformer is the core architecture that powers almost all modern Large Language Models (LLMs). It allows AI to understand context by weighing the importance of different words in a sentence.

By Divyaprakash JhaUpdated 4 min readTopic: Large language models

Photo for What is a Transformer in AI?

The Core Idea: Attention

A transformer is a specific type of neural network architecture that serves as the engine for modern AI. It uses a mechanism called "attention" to understand how words in a sentence relate to one another, regardless of how far apart they are.

Before transformers, AI models often read text like a conveyor belt—one word at a time, from left to right. If a sentence was too long, the model would "forget" the beginning by the time it reached the end. This made understanding complex context very difficult.

The transformer architecture, introduced in the 2017 paper "Attention Is All You Need", changed everything. Instead of reading linearly, the model looks at every word in a sentence simultaneously. It uses the "attention" mechanism to decide which other words are most relevant to the one it is currently processing.

Consider this example: "The clinic called the patient because she was late."

When the model processes the word "she," the attention mechanism allows it to look back at all previous words. It calculates that "she" has a very strong relationship with "patient" and a weak relationship with "clinic." This allows the model to understand that the patient is the one who is late, providing the context necessary for a coherent conversation.

How Transformers Process Information

To make "attention" work mathematically, the transformer doesn't just look at words; it converts them into complex mathematical representations. For every single token (a piece of a word), the model calculates three distinct vectors:

  • Query: What is this token looking for?
  • Key: What information does this token offer to others?
  • Value: What is the actual information this token carries?

The model compares the Query of one token against the Keys of all the other tokens in the sequence. This comparison produces a score, or a "weight." These weights determine how much of each token's Value is passed forward to the next step.

Think of it like a business networking event. Each person has a "query" (what kind of partner are they looking for?) and a "key" (what skills do they have?). When a person's query matches another person's key, they pay "attention" to that person, and information is exchanged via their "value" (their actual expertise).

To handle complex language, transformers use multiple "attention heads" running in parallel. One head might focus on grammar, another on the relationship between people, and another on the timing of events. This allows the model to track many different types of relationships at the exact same time.

Layers, Knowledge, and Output

A transformer is not just one single calculation; it is a stack of many layers. As data passes through these dozens of layers, the model builds a deeper understanding of the text. The first layers might recognize basic grammar, while deeper layers grasp complex reasoning, style, and factual patterns.

After the attention mechanism has finished weighing the relationships between words, the tokens pass through a feed-forward network. This component acts like a massive storage block that holds much of the model's learned knowledge. While attention handles the *context* of the current sentence, the feed-forward network helps provide the *facts* needed to complete the thought.

In the specific type of transformer used by most modern chat tools—known as a decoder-only model—there is an additional step called "masking." A mask ensures that when the model is learning to predict the next word, it can only look at the words that came before it. It cannot "cheat" by looking at the answer it is supposed to be predicting.

To understand the relationship between these components and the larger models, see our guide on how large language models work.

Transformer vs. LLM: What's the difference?

It is common to use these terms interchangeably, but they refer to different things. In simple terms, one is the engine, and the other is the vehicle.

Feature Transformer Large Language Model (LLM)
Definition The underlying mathematical architecture (the engine). A massive neural network trained on huge datasets (the vehicle).
Primary Role To process sequences and apply "attention." To predict the next token and follow instructions.
Relationship The blueprint used to build the LLM. A specific, massive application of the transformer.

While almost all modern LLMs are built using the transformer architecture, not all transformers are LLMs. You can use a transformer for tasks that have nothing to do with language, such as analyzing patterns in stock market data or identifying objects in images.

How Transformers Learn

Building a transformer requires a massive amount of data and a multi-stage training process. This process turns a raw mathematical structure into a useful assistant.

1. Pretraining: The Foundation

In the first stage, the model is shown trillions of tokens of text and code. It is given one simple task: predict the next token. By trying to guess the next word over and over again, the model gradually learns grammar, facts, reasoning, and even coding logic. The result is a "base model"—a highly knowledgeable system that is great at continuing text but not very good at following specific commands.

2. Instruction Tuning: Becoming an Assistant

To make the model useful for humans, it undergoes instruction tuning. During this stage, the model is trained on specific examples of instructions and high-quality responses. This teaches the model the "format" of being an assistant—understanding that when a user asks a question, they want an answer, not just a continuation of the prompt.

3. Preference Tuning: Refining Behavior

The final stage involves shaping the model's tone, safety, and helpfulness. One common method is Reinforcement Learning from Human Feedback (RLHF). In this process, humans rank different potential answers from the model. A separate "reward model" learns these human preferences, and the LLM is then optimized to produce answers that humans are likely to rate highly.

This multi-step journey is what transforms a raw mathematical architecture into the conversational AI we interact with today. To learn more about the individual units of text used in this process, check out our explanation of what is a token in LLM.

Close-up of a computer screen displaying lines of CSS code with color swatches.
Photo: Free computer code screen image by unknown, CC0

Questions people ask

Is a transformer the same as an LLM?

No. A transformer is the specific neural network architecture, while an LLM is a large-scale model built using that architecture.

What is the 'attention' mechanism in a transformer?

Attention is a mechanism that allows the model to weigh the importance of different words in a sequence, helping it understand context and relationships.

Why are transformers better than older AI models?

Transformers can process entire sequences of text at once and use attention to understand complex relationships, making them much more efficient at handling context.

What are the three vectors used in attention?

For every token, the model computes a query (what it's looking for), a key (what it offers), and a value (the information it carries).

What is the role of the feed-forward network in a transformer?

After the attention mechanism processes the relationships between tokens, the feed-forward network helps hold much of the model's stored knowledge.