Which LLM Can Your Laptop Run? A Practical Guide to RAM, VRAM, Parameters, Quantization, and GGUF

LM Studio and Ollama model lists are full of labels such as 7B, Q4_K_M, GGUF, and 128K. Larger numbers may look automatically better, but larger models also demand more memory, take longer to respond, and produce more heat.

Five factors determine whether a model is a good fit:

PC memory configuration × parameter count × quantization × context length × workload

The practical rule is simple: do not choose the largest local LLM your laptop can load. Choose one that preserves usable speed, sufficient context, and acceptable answer quality.

1. Start with RAM and VRAM

What the CPU and GPU do

The CPU handles general-purpose work, including the operating system and applications. A GPU is designed to run many similar operations in parallel, which makes it well suited to the large matrix operations used during LLM inference.

Inference repeatedly reads model weights from memory while performing calculations. Performance therefore depends not only on compute capability but also on memory bandwidth—how quickly data can reach the processor. Two laptops with the same memory capacity can have very different generation speeds because of their GPU, chip generation, power limits, and inference runtime.

RAM versus VRAM

RAM is the main memory used by the OS, browser, IDE, and other applications. On a conventional PC with a discrete GPU, the GPU's VRAM is physically separate from system RAM. The NVIDIA CUDA Programming Guide likewise distinguishes host memory attached to the CPU from device memory attached to the GPU.

Inference is usually faster when the model fits in VRAM. If it does not, some runtimes can offload layers to RAM and the CPU. The resulting slowdown depends on the runtime and the bandwidth of the connection between processors. Successfully loading a model does not prove that it is running comfortably on the GPU.

Unified memory on Apple silicon

On Apple silicon, the CPU and GPU share a unified memory pool. Apple's Metal documentation defines a unified-memory GPU as one that shares all its memory with the CPU.

That does not mean an LLM can use every byte of a MacBook's advertised 24GB. macOS, the browser, the IDE, and the inference engine all consume the same pool.

flowchart LR
    accTitle: Mac and Windows memory layouts
    accDescr: On Apple silicon, the CPU, GPU, operating system, and applications share unified memory. On a PC with a discrete NVIDIA GPU, the CPU and applications use system RAM while the GPU uses a separate VRAM pool.
    subgraph Mac[Apple silicon]
        MC[CPU] --> UM[Unified memory]
        MG[GPU] --> UM
        MO[OS and applications] --> UM
    end
    subgraph Win[PC with NVIDIA GPU]
        WC[CPU] --> RAM[System RAM]
        WO[OS and applications] --> RAM
        WG[GPU] --> VRAM[VRAM]
        RAM <--> WG
    end

Mac and Windows memory layouts

Consequently, a Mac with 16GB of unified memory is not equivalent to a Windows laptop with 16GB of RAM plus 8GB of VRAM.

2. What “3B” and “7B” mean

B stands for billion. Parameters are the weights adjusted while a model is trained.

• 3B: roughly 3 billion parameters
• 7B: roughly 7 billion parameters
• 14B: roughly 14 billion parameters
• 30B: roughly 30 billion parameters

All else being equal, a model with more parameters may be able to handle more complex instructions, knowledge, and reasoning. The cost is a larger file, higher memory use, longer loading time, more computation per generated token, greater heat, and higher power consumption.

Parameter count is not a quality ranking, however. A newer 7B model can outperform an older 14B model on a particular workload. Model generation, training data, instruction tuning, language coverage, and context design all matter.

Mixture-of-Experts models introduce another distinction: total parameters versus the parameters activated for each token. Inactive weights normally still need to be stored and loaded, so estimating memory from the active count alone is unsafe.

3. Relating model size to memory

A useful first approximation for the weights alone is:

weight size ≈ parameter count × bits per parameter ÷ 8

A simple 4-bit representation of a 7B model gives 7 billion × 4 ÷ 8 = 3.5 billion bytes, or about 3.5GB. Real quantization formats add per-block scales and other data, so a GGUF file may be larger than this calculation. For example, llama.cpp documents one Q4_K_M configuration at about 4.89 bits per weight rather than exactly four.

This table simply multiplies the parameter count by 4, 5, 8, or 16 bits. It is a theoretical weight-only estimate that ignores the difference between GB and GiB and the overhead of a particular quantization scheme.

Running the model requires additional memory for:

• the KV cache
• inference-engine workspaces
• model metadata and the tokenizer
• buffers used for GPU placement
• the operating system and other applications

An 8GB model file is therefore not guaranteed to run reliably with exactly 8GB of free memory. Leave headroom before downloading a model or buying hardware.

4. What quantization does

Quantization represents weights at lower precision to reduce file size and memory use. The llama.cpp quantization tool can convert high-precision GGUF models such as F32 or BF16 into lower-bit formats. This may reduce size and improve inference speed, but rounding away information can also reduce quality.

Q4_0, Q4_K_S, and Q4_K_M are not interchangeable. Their names describe more than a nominal bit count: they can use different block schemes and different precision allocations across tensors. Mixed schemes such as K_M do not necessarily store every weight at a uniform four bits.

For a first test, choose a Q4 variant. Compare Q5 if memory remains comfortable, and test Q8 when a possible quality gain justifies its size and speed costs. There is no universal winner between a heavily quantized large model and a smaller model at higher precision. Measure both with the same workload.

5. What GGUF is—and is not

Keep these three concepts separate:

• Model: the trained LLM itself
• Quantization: the process of reducing weight precision
• GGUF: a file format that stores weights, tensor information, metadata, and related data

According to the Hugging Face GGUF documentation, GGUF is a binary format designed for efficient model loading and saving, and it stores standardized metadata in addition to tensors. GGUF is therefore neither a model name nor a quantization method. The same model may be distributed as separate Q4, Q5, and Q8 GGUF files.

Choose an execution path that matches your hardware and workflow:

• GGUF across a broad range of CPUs and GPUs: llama.cpp-based runtimes
• Apple-silicon-optimized model distributions: MLX-LM
• GUI-based discovery and execution: tools such as LM Studio
• CLI and local API workflows: tools such as Ollama

MLX-LM is a Python package for generation, quantization, and fine-tuning on Apple silicon. It represents a different distribution and execution path from GGUF. Before downloading anything, verify that your runtime supports both the model architecture and its file format.

6. Context length and the KV cache

Context length is the number of tokens the model can reference in one request. The system prompt, conversation history, pasted documents or source code, and the generated response all consume the same token budget.

An autoregressive LLM generates one token at a time. The KV cache stores the Key and Value data from earlier attention calculations so the model does not have to recompute all of them at every step. The Hugging Face Transformers cache guide describes this reuse as an important generation optimization.

Model weights remain mostly fixed after loading, but a typical dynamic KV cache grows as the token sequence grows. Its size depends on the number of layers, KV heads, data type, batch size, runtime, and architectural features such as sliding-window attention. Parameter count alone is not enough to predict it.

A model card that advertises 128K context does not mean you should allocate 128K on a laptop. Start around 4K–8K for ordinary chat and increase to 16K or more only when long code or documents require it. When memory is tight, reduce the context allocation as well as considering a smaller model.

7. MacBook and Windows laptops require different strategies

MacBooks with Apple silicon

Because the CPU and GPU share unified memory, models are not confined to a separate, smaller VRAM pool. Metal-enabled llama.cpp runtimes and MLX-LM are common options. The tradeoff is that memory cannot be upgraded after purchase, and macOS and development tools consume the same capacity. Chip generation and memory bandwidth affect speed as well as capacity.

Windows laptops with NVIDIA GPUs

These systems have broad access to CUDA runtimes and are often fast when the model and KV cache fit in VRAM. Check RAM and VRAM independently. A runtime may offload overflow to system RAM, but CPU processing and transfers over PCIe can sharply reduce generation speed.

Two laptops with the same GPU product name may also have different power limits, cooling, and memory configurations. The name alone does not guarantee equal performance.

Windows laptops without an NVIDIA GPU

Use CPU inference, a supported integrated GPU, or another backend such as Vulkan. A small GGUF model is the safest starting point. A large model may load into system RAM yet still respond too slowly for interactive use.

The result depends on the combination of memory capacity + memory bandwidth + GPU type + runtime + quantization + context length.

8. Starting points for 16GB, 24GB, and 32GB systems

The following ranges are not guarantees. OS usage, model architecture, quantization, context length, runtime, and chip performance can all change the result. Treat them as candidates to test first.

For a discrete NVIDIA GPU, inspect VRAM first.

flowchart TD
    accTitle: Local LLM selection flow for a laptop
    accDescr: On Apple silicon, choose an initial candidate from unified memory capacity. On other systems, check for an NVIDIA GPU and use its VRAM when present; otherwise begin with a small GGUF model on the CPU.
    A[Inspect the PC] --> B{Apple silicon}
    B -->|Yes| C{Unified memory}
    C -->|16GB| D[Start with 3B–8B Q4]
    C -->|24GB| E[Start with 7B–14B Q4]
    C -->|32GB or more| F[Benchmark 14B and larger]
    B -->|No| G{NVIDIA GPU present}
    G -->|Yes| H[Select candidates by VRAM]
    G -->|No| I[Try a small GGUF model on CPU]
    D --> J[Measure speed, memory, and quality]
    E --> J
    F --> J
    H --> J
    I --> J

Laptop model-selection flow

Do not judge success by the largest parameter count that launches. Ask instead:

• Is time to first token acceptable?
• Is generation fast enough for chat or coding?
• Can the browser and IDE stay open?
• Does the system remain stable with long inputs and repeated use?
• Are heat, fan noise, and battery consumption acceptable?

9. Selecting for coding, Japanese, and summarization

Coding

Check whether the model received code-oriented training or fine-tuning, supports your languages, and can process realistically sized source files. Agent workflows may also require tool calling, structured output, patch generation, fill-in-the-middle completion, and a local API compatible with the client you use.

Test fixed tasks such as “explain this bug and return a minimal patch” and “produce output that strictly follows this JSON Schema.” Connecting a local endpoint to a client such as Claude Code or Codex does not guarantee that the underlying model can generate reliable tool calls.

Japanese

Do not stop at a language tag. Test Japanese instruction following, honorifics, proper nouns, and natural prose. Some tokenizers divide Japanese text into more tokens than comparable English text, consuming more context and prompt-processing time for the same number of characters.

Summarization

A useful summarizer must fit the document, preserve names and numbers, avoid adding unsupported claims, and follow length and formatting constraints. Test documents similar in length and structure to your real workload.

Model rankings age quickly. A fixed workload-specific prompt suite remains useful across model generations.

10. “Downloaded” is not the same as “usable”

Read the model card before choosing a download. The Hugging Face Model Cards guide describes model cards as documentation for intended use, limitations, training details, datasets, and evaluation results.

At minimum, check:

• parameter count and whether the model is dense or MoE
• whether it is a Base, Instruct, Code, or other variant
• supported languages and intended use
• context length and recommended chat template
• license and commercial-use conditions
• known limitations and evaluation results
• required inference libraries and versions
• GGUF, MLX, or other available formats
• who produced a quantized file and which source revision it used

There are three distinct milestones:

1. The file can be downloaded.
2. The model can be loaded into memory.
3. The model can be used repeatedly at the required speed and quality.

Only the third milestone represents a practical tool. After launch, test real code, documents, and conversation history for several minutes instead of judging the model from a short greeting.

11. What to record when benchmarking models

Keep the prompt, context allocation, maximum generated tokens, temperature, and runtime settings constant. llama.cpp also provides llama-bench for measuring inference performance.

Prompt tokens per second measures input processing; generation tokens per second measures answer production. Record them separately. CPU and GPU utilization, heat, fan noise, and time to first token provide additional clues about the bottleneck.

Evaluate quality with a fixed set of coding, Japanese, and summarization prompts. A fast model that ignores instructions—and a strong model that pauses too long after every sentence—may both be poor daily tools.

Conclusion: start with a smaller Q4 model

Use this sequence:

1. Check RAM, VRAM, or unified memory.
2. Reserve headroom for the OS, browser, and IDE.
3. Start with a 3B–8B Q4 model.
4. Measure speed, memory, and quality on the real workload.
5. Move to 14B or larger, or to Q5, when headroom remains.
6. Reduce model size or context length when memory is tight.

The best local LLM is not the largest model your laptop can technically run. It is the model with the speed and quality you can use every day.