When to consider running models locally

Running a model on your own machine sounds like the enthusiast's option. Sometimes it is. But there are four situations where it is simply the right answer, and one honest reason it might not be.

Privacy you can verify. A downloaded model runs with the network cable out, figuratively or literally. Prompts that never leave your machine cannot be logged, trained on, or subpoenaed — not as a policy, as a physical fact. If your work involves client data, patient notes or unreleased code, this is the strongest position available, and no provider's retention policy matches it.

Cost with a ceiling. Rented inference is a meter that never stops. Local inference costs what the hardware cost, once. Heavy daily use of a mid-sized model can repay a graphics card in months — and the hardware you already own may be enough: our fit verdicts cover machines from data-centre cards down to the phone in your pocket, and a surprising number of models run comfortably on an ordinary laptop.

Working offline. On a plane, on a site with no signal, or in a country where a provider geo-blocks: a local model does not notice.

Tinkering rights. Fine-tuning on your own data, wiring a model into a home automation, running a transcriber over years of voice notes — things that are awkward or expensive through an API become weekend projects locally.

The honest counter-argument: capability. The strongest models are not downloadable, and a laptop-sized model is not one of the frontier ones. If your work needs the best answer available, rented frontier models win. Many people land on both — a local model for the private and the routine, a rented one for the hard problems.

Where next: Check your own machine · What is quantization