LLM Memory (VRAM) Estimator

LLM Memory (VRAM) Estimator is a free, single-page calculator that answers the first question anyone asks before running an open-weight language model locally: will this model fit in my memory, and how fast could it possibly generate text? Pick a model, a quantization level and a context length, and the page shows how much GPU or unified memory to plan for, how that total breaks down, and whether it fits on seven common devices. It was built by Yunus Emre Vurgun and is published as a static, MIT-licensed Hugging Face Space.

The tool does not run a model or call any AI service. It is a planning calculator for people who work with LLMs, with real architecture numbers for popular open models filled in.

Who it is for

It is aimed at developers and hobbyists who run models with llama.cpp, Ollama, vLLM or MLX, and at anyone choosing hardware for that work. Typical questions: is a 24 GB card such as an RTX 3090 or 4090 enough for a 32B model at 4-bit, or does it take a 32 GB RTX 5090? Will a 70B model fit on a 64 GB Mac, or does it need the 128 GB configuration? Is a 128 GB unified-memory box in the NVIDIA DGX Spark class a sensible home for gpt-oss-120b?

Inputs

The left-hand panel has six main controls:

- Model preset: Llama 3.1 8B, Llama 3.3 70B, Qwen2.5 7B, 14B, 32B and 72B, Qwen3 8B, 14B and 32B, Mistral 7B v0.3, Mixtral 8x7B, the DeepSeek-R1 distills (Qwen 14B, Qwen 32B, Llama 70B), gpt-oss-20b and gpt-oss-120b, or "Custom".

- Weight precision: FP32 (4 bytes per parameter), FP16/BF16 (2), INT8/Q8_0 (about 1.06), Q6_K (about 0.82), Q4_K_M (about 0.60), MXFP4 (about 0.53, the native format of gpt-oss) and raw INT4 with no scale overhead (0.5). Quantized values include per-block scale overhead, which is why Q4_K_M is not simply half a byte.

- KV cache precision: FP16/BF16 (2 bytes), FP8/Q8 (1 byte) or Q4 (about 0.56 bytes).

- Context length: 512 to 131,072 tokens.

- Batch or parallel sequences: 1 to 32.

- Overhead: 0 to 30 percent, 15 percent by default, to cover activations, CUDA or Metal buffers and fragmentation.

An "Architecture" section holds total parameters, active parameters per token (for mixture-of-experts models), layer count, KV heads and head dimension. Presets fill these from each model's config.json on Hugging Face; you can overwrite them to estimate any other model.

How the numbers are calculated

The formulas are printed on the page and in the README, and the browser code is a one-to-one port of a short reference file, python/estimator.py:

- Weights = parameters × bytes per parameter.

- KV cache = 2 × layers × KV heads × head dimension × context × batch × KV bytes. The factor of 2 covers the separate key and value tensors.

- Total = (weights + KV cache) × (1 + overhead).

- Speed ceiling = memory bandwidth ÷ (active weights + KV cache) per generated token.

Two model details are handled explicitly. In gpt-oss models, half the layers use a 128-token sliding window, so their KV cache stays small even at long contexts. For mixture-of-experts models such as Mixtral 8x7B and gpt-oss, all parameters must sit in memory, but only the active parameters are read per token, so the speed ceiling uses the active count.

What you get back

The results panel shows a headline figure ("Plan for about X GB") and breaks it into weights, KV cache and overhead. It warns when your context exceeds the preset model's supported maximum. Below that, a table lists seven hardware targets: RTX 3090 (24 GB), RTX 4090 (24 GB), RTX 5090 (32 GB), a 64 GB and a 128 GB M4 Max Mac, NVIDIA DGX Spark (128 GB unified) and NVIDIA H100 80 GB SXM. For each one it shows the memory it assumes is usable, a verdict (Fits when the estimate is under 90 percent of usable memory, Tight between 90 and 100 percent, Doesn't fit above that), and a single-stream tokens-per-second ceiling where the model fits.

Worked examples

Computed with the tool's formulas at the default 15 percent overhead and FP16 KV cache (GB = 1024³ bytes, as on the page).

- Llama 3.1 8B at Q4_K_M with an 8,192-token context: about 4.5 GB of weights plus 1.0 GB of KV cache, so plan for about 6.3 GB. It fits on every listed device. At FP16 weights the same model needs about 18.4 GB, which still fits on a 24 GB card.

- Qwen3 32B at Q4_K_M with 16,384 tokens: about 18.3 GB of weights plus 4.0 GB of KV cache, about 25.7 GB in total. That does not fit on a 24 GB RTX 3090 or 4090, but it fits on the 32 GB RTX 5090.

- Llama 3.3 70B at Q4_K_M with 8,192 tokens: about 39.5 GB of weights plus 2.5 GB of KV cache, about 48.2 GB in total. That is just over the roughly 48 GB the tool treats as usable on a 64 GB Mac, so it shows "Doesn't fit" there and "Fits" on the 128 GB Mac, DGX Spark and H100. At 32,768 tokens the KV cache grows to 10 GB (total about 56.9 GB); an FP8 KV cache halves that to 5 GB (total about 51.1 GB).

- gpt-oss-120b at MXFP4 with 32,768 tokens: about 57.8 GB of weights, but only about 1.1 GB of KV cache because of its sliding-window layers, for about 67.7 GB in total. It fits on the 128 GB Mac, the DGX Spark and the H100. Because only about 5.1B parameters are active per token, its bandwidth ceiling on DGX Spark (273 GB/s) works out to roughly 70 tokens per second, compared with roughly 5 tokens per second for the dense 70B example above at the same context.

Limits and caveats

The page states that these are estimates for comparison, not guarantees, and that real usage depends on the runtime, its buffers and your operating system. Specifically:

- Usable memory per device is an assumption. Discrete GPUs are assumed to keep about 1 GB for the driver and display. Macs are assumed to let the GPU use about 75 percent of unified memory by default, which can be raised. DGX Spark is assumed to share its 128 GB with the operating system.

- The speed figure is a memory-bandwidth ceiling for single-stream decoding. Real tokens per second will be lower; compute limits, prompt processing time and runtime efficiency are not modeled.

- Overhead is a single percentage; runtimes vary.

- The hardware list is fixed at seven devices, and multi-GPU splitting, CPU offload and partial layer offload are not modeled.

Privacy and price

The Space is one static index.html with vanilla JavaScript and no dependencies. All calculations happen in your browser: no account, no sign-up, and nothing you enter is sent to a server. It is free to use.

The same calculator runs at https://sdfsdr2rd3wf-llm-memory-estimator.static.hf.space and on the Hugging Face Space page at https://huggingface.co/spaces/sdfsdr2rd3wf/llm-memory-estimator. A simpler single-page version is also hosted at https://yunus-free-tools.neocities.org/llm-memory-estimator.html. It takes a parameter count, precision, context, a KV-cache-per-1k-tokens figure, overhead and your own memory bandwidth, so it works for any device.

For readers who want the reasoning behind a hardware purchase rather than just the memory math, the author also wrote HARDWARE THEORY, a $4.99 ebook available at https://theknowledgeproject.gumroad.com/l/cjyyly.