Let's talk

Hardware Guide

Best Laptop for Running LLMs

Choosing a laptop for local LLMs requires prioritizing memory capacity and bandwidth over raw compute. This guide helps you match hardware to your model needs.

By Divyaprakash JhaUpdated 5 min readTopic: AI hardware

Photo for Best Laptop for Running LLMs

The Golden Rule: Memory First, Bandwidth Second

To run an LLM, you need enough fast memory to hold the model weights plus the KV cache for your context length. Prioritize memory capacity to ensure a model fits, then look at bandwidth to ensure it runs fast.

When building local AI workflows, business owners often make the mistake of looking at raw compute power or "teraflops" first. For LLMs, the math is different. You are essentially playing a game of moving massive amounts of data from memory to the processor as quickly as possible.

To run a large language model (LLM), you need enough fast memory to hold the model weights plus the KV cache for your context length. Memory capacity determines if a model will fit; memory bandwidth determines how many tokens per second you get. If you choose a laptop that "just fits" the model, you will find that the performance drops significantly once you start a long conversation. This is because the KV cache—the memory used to remember the conversation history—grows as you talk. Always leave headroom for this overhead.

How Much VRAM Do You Need?

Calculating your hardware requirements is a matter of simple arithmetic. The rule of thumb is: parameters × bytes per parameter. However, you cannot simply use the raw file size; you must account for quantization and runtime overhead.

Quantization is the process of storing model weights at lower precision (like 4-bit or 8-bit) to reduce the memory required to run the model. This is why you will often see people running "4-bit" versions of models—it allows a massive model to fit on much smaller, cheaper hardware.

Memory Estimation Guide

When planning your purchase, remember to budget roughly 4.5 to 5 bits per weight to account for real-world file sizes. Additionally, always add 1 to 2 GB of runtime overhead for the framework and activations to ensure stability.

Model Size 4-bit Weights (approx.) 8-bit Weights (approx.) Typical Hardware (at 4-bit)
3B Model 1.5–2 GB 3 GB Any 6–8 GB card/laptop
8B Model 4–5 GB 8 GB 8–12 GB card/laptop
14B Model 7–9 GB 14 GB 12–16 GB card/laptop
32B Model 16–20 GB 32 GB 24–32 GB VRAM
70B Model 35–42 GB 70 GB 48 GB+ VRAM or multiple GPUs

For a business owner in India looking to set up a small office workstation, understanding VRAM is critical. If you want to run a 70B parameter model locally, a standard consumer laptop with 8GB or 16GB of RAM simply will not be able to load the model weights, let alone handle a long conversation.

The Apple Silicon Advantage

Apple's M-series chips offer a unique architecture that is highly beneficial for LLM builders. Unlike traditional PCs where the CPU and GPU have separate pools of memory, Apple uses unified memory. This allows the GPU to access a single large pool of system RAM.

This architecture has a massive advantage for large models. A Mac with a high memory configuration can load models that would require several expensive consumer GPUs in a traditional PC setup. For example, high-end M-series chips can offer significantly more capacity than most laptops.

  • High Capacity: Models like the M4 Max can offer up to 128 GB of unified memory, allowing you to run very large models.
  • Efficiency: Because the memory is integrated, the machine stays quiet and uses less power, making it ideal for developers working in a studio environment.
  • Software Support: Tools such as llama.cpp, Ollama, and Apple's MLX framework work exceptionally well on Apple Silicon.

While the memory bandwidth on a Mac is often lower than a top-tier NVIDIA desktop card, the sheer capacity makes it an excellent choice for running large-scale models that simply won't fit anywhere else in a laptop form factor.

NVIDIA Laptops and VRAM Limits

If you prefer the Windows or Linux ecosystem, NVIDIA is the industry standard. Consumer NVIDIA GPUs in laptops provide excellent performance and the best bandwidth per rupee, but they come with a significant limitation: VRAM capacity.

In a laptop, the GPU memory is soldered and limited. While a desktop can house multiple high-end cards, a laptop is usually restricted to a single GPU with a fixed amount of VRAM. For a single user, VRAM capacity is usually the first limit you hit. If you try to run a model that is larger than your available VRAM, you can use tools like llama.cpp to split layers between the GPU and the CPU. However, the CPU half will set the pace, and your generation speed will drop significantly.

To understand the deeper technical differences between these hardware tiers, you can read our detailed breakdown of GPU for AI.

NPUs vs. GPUs for AI

You may see "AI PCs" marketed with a dedicated NPU (Neural Processing Unit). It is important for business owners to understand that an NPU is not a replacement for a GPU when it comes to heavy LLM work.

NPUs in laptops are small, highly efficient accelerators designed for low-power, compact AI tasks. They are perfect for "on-device" features such as:

  • Real-time transcription
  • Smart replies in messaging apps
  • Summarizing small notifications
  • Background blur in video calls

However, NPUs are generally designed for models with roughly 1B to 4B parameters. They are not built for the heavy lifting required to run large-scale LLMs like Llama 3 or Mistral. For serious development or running large models, you still require a dedicated GPU or high-capacity unified memory.

How Bandwidth Affects Token Speed

When you are chatting with an AI, you notice two distinct phases: the "prefill" (the pause while it reads your prompt) and the "decoding" (the steady stream of words appearing). For most users, the decoding speed is the most important metric.

Generating text is often limited by memory bandwidth rather than raw compute power. Because the chip has to read every single weight of the model from memory to generate just one token, the speed at which those weights can travel is the bottleneck. A useful ceiling to estimate your speed is: max tokens per second ≈ memory bandwidth ÷ bytes read per token.

This is why a high-end card with massive bandwidth will feel much "snappier" than a lower-end card, even if they both have enough VRAM to hold the model. If you are building an application that requires real-time interaction, bandwidth is your most important metric for user experience.

Quick Hardware Reference

Use this table to quickly match your target model to the approximate hardware requirements. Note that these are estimates for 4-bit quantized models.

Model Size Approx. 4-bit Requirement Recommended VRAM/RAM
3B Model 1.5–2 GB 6–8 GB
8B Model 4–5 GB 8–12 GB
14B Model 7–9 GB 12–16 GB
32B Model 16–20 GB 24–32 GB

If you are unsure which specific components to look for in a workstation, check our guide on AI Hardware for more depth.

Close-up of a computer processor chip mounted in a socket.
Photo: File:AMD Fiji GPU package with GPU, HBM memory and interpose by C. Spille/pcgameshardware.de, CC BY-SA 4.0

Questions people ask

Will any laptop run an LLM?

Any laptop can run small models using the CPU and system RAM via llama.cpp, but performance will be significantly slower than using a GPU.

What is the most important spec for LLMs?

Memory capacity is the most important spec because it determines if the model can load; memory bandwidth is second as it determines generation speed.

What is quantization?

Quantization is storing model weights at lower precision (like 4-bit or 8-bit) to reduce the memory required to run the model.

Can I use my laptop's RAM for AI?

Yes, through tools like llama.cpp, you can run models on your system RAM, but it is much slower than using dedicated VRAM.

Do I need a dedicated GPU for AI?

A dedicated GPU is not strictly required for small models, but it is highly recommended for speed and for running larger models that exceed system RAM.