The Golden Rule: Memory First, Bandwidth Second
To run an LLM, you need enough fast memory to hold the model weights plus the KV cache for your context length. Prioritize memory capacity to ensure a model fits, then look at bandwidth to ensure it runs fast.
When building local AI workflows, business owners often make the mistake of looking at raw compute power or "teraflops" first. For LLMs, the math is different. You are essentially playing a game of moving massive amounts of data from memory to the processor as quickly as possible.
To run a large language model (LLM), you need enough fast memory to hold the model weights plus the KV cache for your context length. Memory capacity determines if a model will fit; memory bandwidth determines how many tokens per second you get. If you choose a laptop that "just fits" the model, you will find that the performance drops significantly once you start a long conversation. This is because the KV cache—the memory used to remember the conversation history—grows as you talk. Always leave headroom for this overhead.
How Much VRAM Do You Need?
Calculating your hardware requirements is a matter of simple arithmetic. The rule of thumb is: parameters × bytes per parameter. However, you cannot simply use the raw file size; you must account for quantization and runtime overhead.
Quantization is the process of storing model weights at lower precision (like 4-bit or 8-bit) to reduce the memory required to run the model. This is why you will often see people running "4-bit" versions of models—it allows a massive model to fit on much smaller, cheaper hardware.
Memory Estimation Guide
When planning your purchase, remember to budget roughly 4.5 to 5 bits per weight to account for real-world file sizes. Additionally, always add 1 to 2 GB of runtime overhead for the framework and activations to ensure stability.
| Model Size | 4-bit Weights (approx.) | 8-bit Weights (approx.) | Typical Hardware (at 4-bit) |
|---|---|---|---|
| 3B Model | 1.5–2 GB | 3 GB | Any 6–8 GB card/laptop |
| 8B Model | 4–5 GB | 8 GB | 8–12 GB card/laptop |
| 14B Model | 7–9 GB | 14 GB | 12–16 GB card/laptop |
| 32B Model | 16–20 GB | 32 GB | 24–32 GB VRAM |
| 70B Model | 35–42 GB | 70 GB | 48 GB+ VRAM or multiple GPUs |
For a business owner in India looking to set up a small office workstation, understanding VRAM is critical. If you want to run a 70B parameter model locally, a standard consumer laptop with 8GB or 16GB of RAM simply will not be able to load the model weights, let alone handle a long conversation.
The Apple Silicon Advantage
Apple's M-series chips offer a unique architecture that is highly beneficial for LLM builders. Unlike traditional PCs where the CPU and GPU have separate pools of memory, Apple uses unified memory. This allows the GPU to access a single large pool of system RAM.
This architecture has a massive advantage for large models. A Mac with a high memory configuration can load models that would require several expensive consumer GPUs in a traditional PC setup. For example, high-end M-series chips can offer significantly more capacity than most laptops.
- High Capacity: Models like the M4 Max can offer up to 128 GB of unified memory, allowing you to run very large models.
- Efficiency: Because the memory is integrated, the machine stays quiet and uses less power, making it ideal for developers working in a studio environment.
- Software Support: Tools such as
llama.cpp,Ollama, and Apple'sMLXframework work exceptionally well on Apple Silicon.
While the memory bandwidth on a Mac is often lower than a top-tier NVIDIA desktop card, the sheer capacity makes it an excellent choice for running large-scale models that simply won't fit anywhere else in a laptop form factor.
NVIDIA Laptops and VRAM Limits
If you prefer the Windows or Linux ecosystem, NVIDIA is the industry standard. Consumer NVIDIA GPUs in laptops provide excellent performance and the best bandwidth per rupee, but they come with a significant limitation: VRAM capacity.
In a laptop, the GPU memory is soldered and limited. While a desktop can house multiple high-end cards, a laptop is usually restricted to a single GPU with a fixed amount of VRAM. For a single user, VRAM capacity is usually the first limit you hit. If you try to run a model that is larger than your available VRAM, you can use tools like llama.cpp to split layers between the GPU and the CPU. However, the CPU half will set the pace, and your generation speed will drop significantly.
To understand the deeper technical differences between these hardware tiers, you can read our detailed breakdown of GPU for AI.
NPUs vs. GPUs for AI
You may see "AI PCs" marketed with a dedicated NPU (Neural Processing Unit). It is important for business owners to understand that an NPU is not a replacement for a GPU when it comes to heavy LLM work.
NPUs in laptops are small, highly efficient accelerators designed for low-power, compact AI tasks. They are perfect for "on-device" features such as:
- Real-time transcription
- Smart replies in messaging apps
- Summarizing small notifications
- Background blur in video calls
However, NPUs are generally designed for models with roughly 1B to 4B parameters. They are not built for the heavy lifting required to run large-scale LLMs like Llama 3 or Mistral. For serious development or running large models, you still require a dedicated GPU or high-capacity unified memory.
How Bandwidth Affects Token Speed
When you are chatting with an AI, you notice two distinct phases: the "prefill" (the pause while it reads your prompt) and the "decoding" (the steady stream of words appearing). For most users, the decoding speed is the most important metric.
Generating text is often limited by memory bandwidth rather than raw compute power. Because the chip has to read every single weight of the model from memory to generate just one token, the speed at which those weights can travel is the bottleneck. A useful ceiling to estimate your speed is: max tokens per second ≈ memory bandwidth ÷ bytes read per token.
This is why a high-end card with massive bandwidth will feel much "snappier" than a lower-end card, even if they both have enough VRAM to hold the model. If you are building an application that requires real-time interaction, bandwidth is your most important metric for user experience.
Quick Hardware Reference
Use this table to quickly match your target model to the approximate hardware requirements. Note that these are estimates for 4-bit quantized models.
| Model Size | Approx. 4-bit Requirement | Recommended VRAM/RAM |
|---|---|---|
| 3B Model | 1.5–2 GB | 6–8 GB |
| 8B Model | 4–5 GB | 8–12 GB |
| 14B Model | 7–9 GB | 12–16 GB |
| 32B Model | 16–20 GB | 24–32 GB |
If you are unsure which specific components to look for in a workstation, check our guide on AI Hardware for more depth.

Questions people ask
Will any laptop run an LLM?
Any laptop can run small models using the CPU and system RAM via llama.cpp, but performance will be significantly slower than using a GPU.
What is the most important spec for LLMs?
Memory capacity is the most important spec because it determines if the model can load; memory bandwidth is second as it determines generation speed.
What is quantization?
Quantization is storing model weights at lower precision (like 4-bit or 8-bit) to reduce the memory required to run the model.
Can I use my laptop's RAM for AI?
Yes, through tools like llama.cpp, you can run models on your system RAM, but it is much slower than using dedicated VRAM.
Do I need a dedicated GPU for AI?
A dedicated GPU is not strictly required for small models, but it is highly recommended for speed and for running larger models that exceed system RAM.
