Run a local LLM at home
From a $800 Mac mini to a 512GB Mac Studio: the hardware that actually fits the model you want.
Running a model locally means no per-token bills, nothing leaving your house, and the freedom to run fine-tuned or uncensored models. The whole game is memory: the model has to fit in VRAM (or Apple unified memory) at a quantization that keeps it useful. Pick your tier by the largest model you actually want to run, not by raw GPU speed.
Rule of thumb at Q4: budget 0.55–0.6 GB per billion parameters, plus 1–4 GB for context. So 8B ≈ 5 GB, 14B ≈ 9 GB, 32B ≈ 20 GB, 70B ≈ 40–42 GB, and DeepSeek V3/R1 at 671B ≈ 400 GB. Q8 doubles those, FP16 quadruples. Generation speed scales with memory bandwidth, not core count; 10 tokens/s is the readable floor and 20+ feels snappy. Not sure where your model lands? Use the VRAM calculator.
Software is free at every tier: Ollama for a one-command server with an OpenAI-compatible API, LM Studio if you want a GUI, llama.cpp underneath both when you need --tensor-split across two cards, and Open WebUI for a ChatGPT-style front end you can reach from your phone over Tailscale.
The kit
A dedicated PC with 48GB of VRAM from two used 3090s. Runs 70B at Q4 fully on GPU. You assemble it.
Budget is our editorial estimate (USD, Oct 2026) from the sum of items you keep. It is not an Amazon price; check live prices on Amazon.
- 01 · GPUs×2$$$$
NVIDIA GeForce RTX 3090 24GB (used)
Still the value king of local inference: 24GB and 936 GB/s for the price of a mid-range card.
Two of these give 48GB of VRAM, which runs Llama 3.3 70B at Q4 entirely on GPU at 15–18 tokens/s using llama.cpp's tensor split. Nothing newer offers 24GB at this price, and the 3090's bandwidth is within 10% of a 4090. Buy from a seller with returns, ask whether it was mined on, and budget 350 W each.
Two cards = 48GB. A single used 4090 is the quiet alternative for 32B-class models at 35+ tok/s. Do not pay 2026's $4,000+ street price for a 5090; its 32GB is a nicer 24GB, not a new tier.
- vram
- 24GB GDDR6X
- bandwidth
- 936 GB/s
- power
- 350 W
- 02 · CPU$$$
AMD Ryzen 7 7700
An efficient 8-core that stays out of the way.
The CPU barely matters for GPU inference; it shuffles tokens and runs your Docker containers. The 7700 is 65 W, comes with a cooler, and leaves money for the parts that do matter. If you plan to offload big MoE models to system RAM, step up to a 9950X for its extra memory bandwidth and cores.
- cores
- 8 / 16 threads
- tdp
- 65 W
- socket
- AM5
Find on Amazonpaid link - 03 · Motherboard$$$
ASUS ProArt B650-Creator
AM5 board with two x16-size slots that bifurcate to x8/x8 for dual GPUs.
Most consumer boards run a second GPU at x4 through the chipset. The ProArt B650-Creator splits the CPU lanes x8/x8, has 2.5GbE plus 10GbE, and leaves enough slot spacing for two three-slot cards. That combination at under $300 is why it shows up in so many r/LocalLLaMA dual-3090 builds.
- slots
- 2× PCIe 4.0 x16 (x8/x8)
- lan
- 10GbE + 2.5GbE
- 04 · RAM$$$
64GB DDR5-6000 kit (2×32GB)
Headroom to offload layers of models that do not fit in VRAM.
With 48GB of VRAM and 64GB of RAM you can run Qwen3-235B MoE at Q3 with CPU offload, slowly but at all. 6000 MT/s is the AM5 sweet spot. 2026 DRAM pricing is painful, so buy a matched kit once rather than adding sticks later.
- capacity
- 64GB
- speed
- DDR5-6000 CL30
- 05 · Power supply$$$
Corsair HX1500i (1500 W, ATX 3.1)
Dual 3090s spike well past 700 W; this gives margin and native 12V-2x6 cables.
Transient spikes on two 3090s can briefly pull 1,000 W from the wall. A 1,200–1,500 W ATX 3.1 unit with native 12V-2x6 or 8-pin PCIe cables avoids the adapter chains that caused the melted-connector saga. For a single-card build the 1000 W RM1000x is plenty.
- output
- 1500 W
- efficiency
- 80+ Platinum
- standard
- ATX 3.1
- 06 · Case$$$
Fractal Design Torrent (E-ATX airflow case)
Two 180 mm front fans aimed straight at 700 W of GPUs.
Dual-GPU rigs die from heat soak, not from any single component. The Torrent's open front and huge fans move more air at low RPM than any mesh mid-tower, and it has room for two three-slot cards plus a 1500 W supply. The Meshify 2 or Lian Li Lancool 216 work if you prefer a smaller footprint.
- fans
- 2× 180 mm + 3× 140 mm
- gpuLength
- up to 461 mm
Find on Amazonpaid link - 07 · Storage$$
WD Black SN850X 2TB NVMe
Models are 5–45GB each. 1TB fills in a month.
Loading a 40GB model from a fast NVMe takes seconds instead of minutes, and the SN850X is a proven TLC drive without the QLC endurance trap. The Samsung 990 Pro is the equivalent alternative; buy whichever is cheaper that week.
- capacity
- 2TB
- type
- PCIe 4.0 TLC
Find on Amazonpaid link
Skip this / common mistakes
- !
Don't buy a 16GB gaming card (4060 Ti, 5070) for LLMs. 24GB is the real floor for 32B models.
- !
Don't buy a base 16GB Mac mini expecting 14B models. 8B is its ceiling once you add context.
- !
Don't pay $4,000+ for an RTX 5090 in the 2026 shortage. Two 3090s or a Strix Halo box are better dollars per gigabyte.
- !
Don't skip the power-supply step. 8-pin to 12VHPWR adapter chains are the number-one cause of melted connectors.
- !
Don't expect CPU-only inference on 128GB of DDR5 to be usable: 1–3 tokens/s on 70B. RAM is for offload, not the main engine.
Questions people ask
How much VRAM do I need for a 70B model?
About 40–42 GB at Q4 (two 24GB cards or a 64GB+ Mac) and about 70 GB at Q8. A single 24GB or 32GB card cannot hold it without offloading to system RAM, which drops speed to a few tokens per second.
Mac or NVIDIA for a local LLM?
Mac for big models quietly and cheaply per gigabyte; NVIDIA for speed, fine-tuning, image and video generation, and the broadest software support. Many people end up with a Mac mini for daily use and a GPU box for experiments.
Is a used RTX 3090 still worth it in 2026?
Yes. It has the same 24GB and 936 GB/s as far newer cards at a fraction of the price. Check for mining wear (fan noise, thermal pads) and buy from a seller with returns.
What is the best model to run locally right now?
For code: Qwen3 32B or Qwen2.5-Coder 32B. For general chat: Gemma 3 27B and Llama 3.3 70B. For reasoning: DeepSeek R1 distills, or gpt-oss-120b if you have 64GB+. On 24GB Macs, Qwen3 14B is the sweet spot.
Sources
Heads up: links to Amazon are affiliate links. As an Amazon Associate I earn from qualifying purchases. It costs you nothing and does not change which products get picked. How this works
Pairs well with
Homelab starter kit
A sub-20 W mini PC running Proxmox, a NAS for the data, and a UPS so a power blip never eats your drives.
Ergonomic developer desk
Monitor at eye height, a chair that fits, a desk that lets you stand. Buy those first; cheap out on the rest until it bugs you.
Privacy & self-hosting hardware
Travel router, two security keys, an encrypted stick, a Pi-hole. Each one quietly closes a door.
Paul Graham
Y Combinator co-founder, essayist
Books Paul Graham has recommended in his essays and on X, each linked to the essay or post where he said it.
Patrick Collison
Stripe co-founder and CEO
Books Patrick Collison flags as particularly great on his personal bookshelf page, several of which Stripe Press later republished.
Readers of this kit also open
A sub-20 W mini PC running Proxmox, a NAS for the data, and a UPS so a power blip never eats your drives.
No soldering. A hot-swap kit, switches and keycaps you chose, and the three tools worth owning.
Monitor at eye height, a chair that fits, a desk that lets you stand. Buy those first; cheap out on the rest until it bugs you.
Travel router, two security keys, an encrypted stick, a Pi-hole. Each one quietly closes a door.
Spotted a better part, or a price that moved? Tell us. Every kit is re-checked when the hardware changes. All kits.