Let's talk

Optimizing AI behavior

Fine-tuning Large Language Models

Fine-tuning is the process of updating a model's weights to teach it specific styles, formats, or narrow skills. It is best used when you need consistent behavior rather than just new facts.

By Divyaprakash JhaUpdated 4 min readTopic: Large language models

Photo for Fine-tuning Large Language Models

Fine-tuning is the process of updating a model's weights to teach it specific styles, formats, or narrow skills. It is best used when you need consistent behavior rather than just providing the model with new facts.

What is Fine-tuning?

To understand fine-tuning, you first need to understand what a LLM is. A large language model is a neural network, usually a transformer, trained on enormous amounts of text to predict the next token. This training creates a "base model" that is knowledgeable but often just continues text rather than following specific instructions.

Fine-tuning is how we take that base model and specialize it. Instead of just predicting the next word in a random sentence, we train the model to act in a specific way.

  • Weight Updates: Fine-tuning updates the actual weights (parameters) of a model. These are the adjustable numbers the model learns during training.
  • Specialized Skills: It is used to teach a model a specific format, tone, or narrow skill. For example, if you want a model to always respond to customer queries in a very specific brand voice, fine-tuning is the tool for the job.
  • Behavior vs. Knowledge: Unlike RAG, which provides new data to the model, fine-tuning changes how the model itself behaves.
  • Efficiency: It is the right choice when you want a model to act consistently and cheaply for a specific task.

Fine-tuning vs. RAG vs. Prompting

When building an AI application, you have three main ways to make a general model work for your business. You should generally try them in this order: Prompting, then RAG, then Fine-tuning.

Method What it changes Best for... Effort
Prompting The instructions and examples you send Format, tone, or simple tasks via text instructions. Low
RAG (Retrieval-Augmented Generation) The documents you add to the context at question time Answering questions from your own, changing data, and keeping facts current. Medium
Fine-tuning The model's weights Consistent style, narrow skills, and making smaller models perform better on specific tasks. Higher

A simple way to remember the difference: Retrieval-augmented generation is usually the right answer to "how do I make it know our data." Fine-tuning is the right answer to "how do I make it behave consistently and cheaply."

Efficient Fine-tuning: LoRA and QLoRA

In the past, fine-tuning required massive amounts of computing power. If you wanted to update a model, you often had to perform "full fine-tuning," which updates every single parameter in the model. This requires massive GPU memory and is very expensive.

Modern developers use more efficient methods to make this process accessible to smaller businesses and individual builders:

  • LoRA (Low-Rank Adaptation): Instead of updating everything, LoRA freezes the original weights of the model and trains tiny "adapter" matrices instead. This makes the resulting file much smaller and the training much faster.
  • QLoRA: This is an even more efficient version. It keeps the base model in 4-bit precision (a way of compressing the model's numbers) while training the adapters. This allows you to fine-tune fairly large LLM models on a single GPU.

The Stages of Model Training

Models go through different stages of learning before they are ready for your specific business use case. Understanding these helps you know where your custom training fits in.

1. Pretraining

The model learns to predict the next token across trillions of words and code. This creates a "base model" that has a vast amount of general knowledge but doesn't necessarily know how to be a helpful assistant.

2. Instruction Tuning

This is a form of supervised fine-tuning. The model is trained on examples of instructions and good responses. This is what teaches the model the format of being an assistant—learning to answer questions, follow a requested format, and stop when the task is done.

3. Preference Tuning

In this final stage, the model learns from human or AI preferences. Using methods like RLHF (Reinforcement Learning from Human Feedback) or RLAIF (AI Feedback), the model is shaped to be more helpful, maintain a certain tone, and stay safe.

When to Choose Fine-tuning

Fine-tuning is a powerful tool, but it is not always necessary. For most businesses, starting with a large, capable model and using good prompts is the best first step. However, you should move to fine-tuning in the following scenarios:

  • Structured Data Extraction: Use it when you need a small model to extract data into a specific JSON format every single time without fail.
  • Brand Voice: Use it to achieve a very specific brand voice or a unique communication style that is hard to describe in a prompt.
  • Cost and Latency Optimization: This is a major business advantage. You can take a very large, expensive model and use it to generate perfect data. Then, you can use that data to fine-tune a much smaller, cheaper model to perform that same narrow task. This makes your AI application faster and significantly cheaper to run.

How to Start

If you have decided that fine-tuning is the right path, follow these practical steps to ensure your project succeeds:

  1. Define your task and create a test set: You cannot improve what you cannot measure. Create a list of 30 to 50 real-world examples that include the input and the "perfect" correct answer. This is your gold standard for testing if your fine-tuning actually worked.
  2. Decide on data constraints: If your data is highly sensitive and must stay on-site or within your own servers, you should look into open-weight models that you can host yourself.
  3. Start large, then shrink: A common successful strategy is to use a large, highly capable model to get the logic and the "perfect" answers right first. Once you have a high-quality dataset, try to shrink that capability down into a smaller, fine-tuned model to save on costs.
Close-up of a computer screen displaying lines of CSS code with color swatches.
Photo: Free computer code screen image by unknown, CC0

Questions people ask

Is fine-tuning better than RAG?

Not necessarily. RAG is better for giving a model new, changing facts. Fine-tuning is better for teaching a model a specific behavior or format.

Do I need a lot of GPUs for fine-tuning?

Not with modern methods like LoRA or QLoRA, which allow you to fine-tune large models on a single GPU by only training a small fraction of the parameters.

Can fine-tuning prevent hallucinations?

It helps with consistency, but it is not a complete fix. Providing source material via RAG is often more effective for preventing factual errors.

What is the difference between full fine-tuning and LoRA?

Full fine-tuning updates all the model's parameters, while LoRA only updates a small set of 'adapter' weights, making it much faster and cheaper.

When should I use a reasoning model instead of fine-tuning?

Use reasoning models for hard, multi-step problems that require thinking. Use fine-tuning for tasks like consistent data extraction or specific tone.