Choosing a graphics card for local models: VRAM beats everything

The gaming instinct — buy the newest generation — is the wrong instinct here. For running models, a graphics card is a box of memory with a processor attached, and the memory is what you are buying: how much decides what runs at all, and how fast that memory moves decides the speed. A three-year-old card with more VRAM beats this year's card with less, for this job, almost always.

Think in VRAM classes, not model numbers. Fit is binary, so the market divides into rungs: cards around 8 GB run the small models; 12–16 GB opens the popular mid-sized range that most people actually settle on; 24 GB is where the seriously capable quantized models start fitting whole. Which model lands on which rung changes as models improve — that is what the fit checker and each card's page in the hardware index are for — but the rungs themselves are stable.

The used market is where the value is. Because VRAM outranks generation, last-generation cards with big memory hold up remarkably well — the used 24 GB RTX 3090 has been the community's value pick for years for exactly this reason. Check the card you're eyeing in the hardware index: the fit verdicts don't care what year it shipped.

Bandwidth is the tiebreaker. Between two cards that both fit your model, the one with higher memory bandwidth generates faster — it is the physics of every token reading the whole model. The spec sheets bury it; our card pages lead with it.

The honest asterisk on brands. The local-model toolchain grew up on NVIDIA, and the smoothest path still runs through it. AMD and Intel cards increasingly work — the mainstream tools support them — but "increasingly work" means you will occasionally be the person filing the bug report. Factor your own patience in as a spec.

And compare against not buying a card. A Mac's unified memory is the same purchase in another shape, and for genuinely large models, renting or a hosted endpoint may beat owning. The right question is the one the catalogue answers: which models do you actually want to run?

Where next: Check a card against the catalogue · The hardware index · VRAM and what fits