Hardware / NVIDIA /GeForce RTX 4090

NVIDIA · GPU

GeForce RTX 4090

Runs 125 models fully in memory, 9 more with CPU offload. 9 of those sit close enough to 24 GB that our size estimate is what decides them.

Memory

24 GB

Bandwidth

1008 GB/s

Runs

125

01

Specifications

Memory

24 GB

Decides which models fit at all.

Memory bandwidth

1008 GB/s

How fast it can read the model. Sets the speed of the reply.

Compute

82.6 TFLOPS

Trillions of half-precision sums a second. Sets how fast it reads a long prompt, not how fast it replies.

Power draw

450 W

At full load, for sizing a power supply.

MSRP

$1,599

What the maker asked for it new. We hold no used or street prices, so this is not what one costs today.

Released

Oct 12, 2022

Architecture

Ada Lovelace

02

What it runs

Best quant per model. Text models first, then speech, then what does not fit.

What is quantisation? →

ModelParamsContextSpeedVerdict

Granite 4.1 8B8.8B66K129 tok/sFits in memory Qwen3.5-9B9.7B66K137 tok/sFits in memory Ministral 3 8B 25128.9B66K127 tok/sFits in memory Ministral 3 3B 25123.8B131K200 tok/sFits in memory gpt-oss-safeguard-20b21.5B131K314 tok/sFits in memory Granite 4.0 Micro3.2B66K416 tok/sFits in memory Qwen3 VL 8B Thinking8.8B66K129 tok/sFits in memory UI-TARS 7B8.3B66K160 tok/sFits in memory Llama 3.2 3B Instruct3.2B131K416 tok/sFits in memory Qwen3 VL 8B Instruct8.8B66K129 tok/sFits in memory Qwen3 8B8.2B66K162 tok/sFits in memory Gemma 3 4B4.3B131K264 tok/sFits in memory Llama 3.1 8B Instruct8B131K142 tok/sFits in memory Granite 4.2 8B8.8B66K129 tok/sFits in memory Nemotron 3.5 Content Safety4.3B131K264 tok/sFits in memory Llama Guard 4 12B12B131K111 tok/sFits in memory Gemma 3n 4B7.8B33K171 tok/sFits in memory Gemma 3 12B12.2B131K109 tok/sFits in memory Rocinante 12B12.2B66K109 tok/sFits in memory Llama 3.2 1B Instruct1.2B33K1109 tok/sFits in memory Mistral Nemo12.2B66K109 tok/sFits in memory Qwen2.5 7B Instruct7.6B33K175 tok/sFits in memory Gemma 4 26B A4B25.8B16K283 tok/sFits in memory Reka Edge7.1B16K160 tok/sFits in memory Nemotron 3 Nano 30B A3B31.6B16K377 tok/sFits in memoryest Ministral 3 14B 251213.9B66K96 tok/sFits in memory UnslopNemo 12B12.2B33K109 tok/sFits in memory Qwen3 30B A3B Thinking 250730.5B16K377 tok/sFits in memoryest Qwen3 Coder 30B A3B Instruct30.5B16K377 tok/sFits in memoryest Qwen3 30B A3B Instruct 250730.5B16K377 tok/sFits in memoryest Qwen3 14B14.8B66K90 tok/sFits in memory Qwen3 VL 30B A3B Thinking31.1B8K377 tok/sFits in memoryest Llama 3 8B Lunaris8B8K142 tok/sFits in memory Qwen3 VL 30B A3B Instruct31.1B8K377 tok/sFits in memoryest Hy-MT2-1.8B2B8K565 tok/sFits in memory Hy-MT2-30B-A3B30.1B8K377 tok/sFits in memoryest Hy-MT2-7B8B8K121 tok/sFits in memory Phi 414.7B16K91 tok/sFits in memory MythoMax 13B13B8K102 tok/sFits in memory ReMM SLERP 13B13B4K102 tok/sFits in memory Reka Flash 320.9B33K64 tok/sFits in memory Cydonia 24B V4.123.6B33K56 tok/sFits in memory Uncensored24B33K55 tok/sFits in memory Mistral Small 3.2 24B24B33K55 tok/sFits in memory Mistral Small 3.1 24B24B33K55 tok/sFits in memory Mistral Small 323.6B33K56 tok/sFits in memory Gemma 3 27B27.4B33K49 tok/sFits in memory Ternary Bonsai 2 27B27B33K49 tok/sFits in memory Voxtral Small 24B 250724.3B16K55 tok/sFits in memory Muse Glimmer 30B29.8B33K45 tok/sFits in memory Qwen3.5-27B27.8B8K48 tok/sFits in memory Qwen3.6 27B27.8B8K48 tok/sFits in memory Gemma 2 27B27.2B8K49 tok/sFits in memory Nemotron 3.5 Lightning31.6B16K36 tok/sFits in memoryest Qwen3.8 27B27.8B8K48 tok/sFits in memory GLM 4.7 Flash31.2B4K36 tok/sFits in memory Laguna XS 2.133.4B4K34 tok/sFits in memory Olmo 3 32B Think32.2B2K41 tok/sFits in memoryest

Speech models it runs67 modelsTranscription and speech models that fit in memory. Throughput figures for a transcription model come from its text decoder, so treat this as a fit answer rather than a speed one.

Niagara 19m Batch.en20M37948 tok/sFits in memory Niagara 38m Batch.en38M19973 tok/sFits in memory Higgs Audio v3 8b STT v28.9B150 tok/sFits in memory Higgs Audio v3 STT2.7B281 tok/sFits in memory Cohere Transcribe 03 20262.1B361 tok/sFits in memory Distil Large v3.50.8B949 tok/sFits in memory Lite Whisper Large v3 Acc1.4B810 tok/sFits in memory Owsm CTC v3.1 1B1.1B1209 tok/sFits in memory Owsm CTC v3.2 ft 1B1B759 tok/sFits in memory Owsm CTC v4 1B1B1330 tok/sFits in memory Hubert Large Ls960 ft0.3B2372 tok/sFits in memory MMS 1b All1B1134 tok/sFits in memory Wav2vec2 Large 960h Lv60 Self0.3B2372 tok/sFits in memory Hojo ASR V15.2B219 tok/sFits in memory Granite 4.0 1b Speech2.3B330 tok/sFits in memory Granite Speech 3.3 2b3B443 tok/sFits in memory Granite Speech 3.3 8b8.6B155 tok/sFits in memory Granite Speech 4.1 2b2.3B578 tok/sFits in memory Granite Speech 4.1 2b NAR2.3B578 tok/sFits in memory STT 2.6b en2.6B512 tok/sFits in memory Phi 4 Multimodal Instruct5.6B66K136 tok/sFits in memory VibeVoice ASR HF8.3B137 tok/sFits in memory Voxtral Mini 3B 25074.7B162 tok/sFits in memory Voxtral Mini 4B Realtime 26024.4B173 tok/sFits in memory Canary 180m Flash0.2B6299 tok/sFits in memory Canary 1b1B759 tok/sFits in memory Canary 1b Flash0.8B949 tok/sFits in memory Canary 1b v21B759 tok/sFits in memory Canary Qwen 2.5b2.6B292 tok/sFits in memory Nemotron 3.5 ASR Streaming 0.6b0.6B1265 tok/sFits in memory Nemotron Speech Streaming en 0.6b0.6B1265 tok/sFits in memory Parakeet CTC 0.6b0.6B2217 tok/sFits in memory Parakeet CTC 1.1b1.1B690 tok/sFits in memory Parakeet RNNT 0.6b0.6B1265 tok/sFits in memory Parakeet RNNT 1.1b1.1B1209 tok/sFits in memory Parakeet TDT CTC 110m0.1B6900 tok/sFits in memory Parakeet TDT 0.6b v20.6B1890 tok/sFits in memory Parakeet TDT 0.6b v30.6B1265 tok/sFits in memory Parakeet TDT 1.1b1.1B690 tok/sFits in memory Nyra Health CrisperWhisper1.6B474 tok/sFits in memory Whisper Large v31.5B756 tok/sFits in memory Whisper Large v3 Turbo0.8B1663 tok/sFits in memory MOSS Transcribe Diarize0.9B843 tok/sFits in memory Qwen3 ASR 0.6B0.9B1478 tok/sFits in memory Qwen3 ASR 0.6B HF0.8B1663 tok/sFits in memory Qwen3 ASR 1.7B2.3B493 tok/sFits in memory Qwen3 ASR 1.7B HF2B380 tok/sFits in memory Zipformer cr CTC Transducer XL 290M0.3B3910 tok/sFits in memory Zipformer Transducer XL 290M0.3B2617 tok/sFits in memory ASR Conformer Largescaleasr0.5B2771 tok/sFits in memory GLM ASR Nano 25122.3B493 tok/sFits in memory Granite Speech 5.0 470m Turboctc0.5B2268 tok/sFits in memory Granite Speech 5.0 470m Turboctc nc0.5B2660 tok/sFits in memory Fun ASR Nano 2512 HF0.8B1663 tok/sFits in memory Gemma 4 E2B it5.1B149 tok/sFits in memory Gemma 4 E4B it8B131K166 tok/sFits in memory Niagara 9m Batch.en9M84329 tok/sFits in memory Niagara 84m Batch.en84M9035 tok/sFits in memory ARK ASR 0.6B1.3B33K1023 tok/sFits in memory ARK ASR 3B4.1B33K324 tok/sFits in memory Audio8 ASR 0.1B0.3B33K2530 tok/sFits in memory MOSS Transcribe Preview 2B2.4B33K554 tok/sFits in memory Gemma 4 12B it12B111 tok/sFits in memory Moonshine Streaming Medium0.3B4K4434 tok/sFits in memory Moonshine Streaming Small0.1B4K7590 tok/sFits in memory Moonshine Streaming Tiny0B4Kno speed estimateFits in memory Moonshine Tiny0B194no speed estimateFits in memory

Runs only by spilling into system memory9 modelsThese load, but part of the weights sits in ordinary system memory, which is far slower than the chip.

Nex-N2-Mini35.1B—no speed estimateSpills to system RAMest Qwen3.6 35B A3B36B—no speed estimateSpills to system RAM Gemma 4 31B31.3B—no speed estimateSpills to system RAM Qwen3 VL 32B Instruct33.4B—no speed estimateSpills to system RAM Skyfall 36B V236.9B—no speed estimateSpills to system RAM Qwen2.5 Coder 32B Instruct32.8B—no speed estimateSpills to system RAMest Qwen3.5-35B-A3B36B—no speed estimateSpills to system RAMest Qwen3 32B32.8B—no speed estimateSpills to system RAM Nex-N2.5-Mini35.1B—no speed estimateSpills to system RAM

Too large for this device90 modelsThe weights do not fit even with part of them offloaded to system memory. Closest calls first.

Llama 3.1 70B Instruct70.6B—no speed estimateToo large Llama 3.3 Euryale 70B70.6B—no speed estimateToo large Llama 3.3 70B Instruct70.6B—no speed estimateToo large Qwen2.5 72B Instruct72.7B—no speed estimateToo large Qwen3 Coder Next79.7B—no speed estimateToo large Qwen3 Next 80B A3B Instruct81.3B—no speed estimateToo large GLM 4.6V108B—no speed estimateToo large Ling-2.6-flash107B—no speed estimateToo large Command A111B—no speed estimateToo large gpt-oss-120b120B—no speed estimateToo large Nemotron 3 Super124B—no speed estimateToo large Hermes 4 70B70.6B—no speed estimateToo large Llama 3.1 Euryale 70B v2.270.6B—no speed estimateToo large R1 Distill Llama 70B70.6B—no speed estimateToo large Hermes 3 70B Instruct70.6B—no speed estimateToo large Devstral 2 2512125B—no speed estimateToo large GLM 4.5V108B—no speed estimateToo large Llama 4 Scout109B—no speed estimateToo large Magnum v4 72B72.7B—no speed estimateToo large Qwen2.5 VL 72B Instruct73.4B—no speed estimateToo large Laguna S 2.1118B—no speed estimateToo large Hunyuan A13B Instruct80.4B—no speed estimateToo large Qwen3 Next 80B A3B Thinking81.3B—no speed estimateToo large Mixtral 8x22B Instruct141B—no speed estimateToo large GLM 4.5 Air111B—no speed estimateToo large Step 3.7 Flash201B—no speed estimateToo large Mistral Small 4119B—no speed estimateToo large Qwen3.5-122B-A10B125B—no speed estimateToo large Mistral Medium 3.5128B—no speed estimateToo large Ling-3.0-flash128B—no speed estimateToo large MiniMax M2229B—no speed estimateToo large Step 3.5 Flash199B—no speed estimateToo large Qwen3 VL 235B A22B Thinking236B—no speed estimateToo large MiniMax M2.5229B—no speed estimateToo large Qwen3 235B A22B Thinking 2507235B—no speed estimateToo large Qwen3 VL 235B A22B Instruct236B—no speed estimateToo large DeepSeek V4 Flash291B—no speed estimateToo large GLM 5.3 Flash321B—no speed estimateToo large Hy3299B—no speed estimateToo large GLM 4.5358B—no speed estimateToo large Laguna M.1226B—no speed estimateToo large Nex-N2-Pro397B—no speed estimateToo large MiniMax M2.7229B—no speed estimateToo large MiniMax M2.1229B—no speed estimateToo large Qwen3 235B A22B Instruct 2507235B—no speed estimateToo large GLM 4.6357B—no speed estimateToo large GLM 4.7358B—no speed estimateToo large ERNIE 4.5 VL 424B A47B424B—no speed estimateToo large MiniMax M3427B—no speed estimateToo large Nex-N2.5-Pro397B—no speed estimateToo large Inkling Small266B—no speed estimateToo large Trinity Large Thinking399B—no speed estimateToo large Jamba Large 1.7399B—no speed estimateToo large Hy3 preview299B—no speed estimateToo large DeepSeek V4 Flash Vision Exp305B—no speed estimateToo large MiniMax M1456B—no speed estimateToo large MiniMax-01456B—no speed estimateToo large MiMo-V2.5311B—no speed estimateToo large Qwen3 Coder 480B A35B480B—no speed estimateToo large DeepSeek V3.1 Terminus685B—no speed estimateToo large DeepSeek V3.2685B—no speed estimateToo large DeepSeek V3.2 Exp685B—no speed estimateToo large Llama 4 Maverick402B—no speed estimateToo large Qwen3.5 397B A17B403B—no speed estimateToo large Hermes 4 405B406B—no speed estimateToo large Hermes 3 405B Instruct406B—no speed estimateToo large GLM 5.1754B—no speed estimateToo large DeepSeek V3685B—no speed estimateToo large GLM 5754B—no speed estimateToo large DeepSeek V4.1 Flash763B—no speed estimateToo large GLM 5.2753B—no speed estimateToo large Nemotron 3 Ultra561B—no speed estimateToo large Kimi K2 07111T—no speed estimateToo large Ling-2.6-1T1T—no speed estimateToo large MiMo-V2.5-Pro1T—no speed estimateToo large R1 0528685B—no speed estimateToo large DeepSeek V3 0324685B—no speed estimateToo large DeepSeek V3.1685B—no speed estimateToo large Ring-2.6-1T1T—no speed estimateToo large Kimi K2 Thinking1.1T—no speed estimateToo large Kimi K2.7 Code1.1T—no speed estimateToo large Kimi K2.61.1T—no speed estimateToo large Kimi K2.51.1T—no speed estimateToo large Hy4 preview780B—no speed estimateToo large Inkling952B—no speed estimateToo large Kimi K2 09051T—no speed estimateToo large LongCat 2.01.8T—no speed estimateToo large Qwen3.8 2.4T A95B2.4T—no speed estimateToo large DeepSeek V4 Pro1.6T—no speed estimateToo large Kimi K32.8T—no speed estimateToo large

224 models have a verdict on this device: 125 fit in memory, 9 spill into system memory and 90 are too large. Every one of them is on this page.

03

Other Ada Lovelace cards

The same generation in other memory sizes. Memory is what decides which models fit.

GeForce RTX 4060 8GB8 GB · less memory GeForce RTX 4070 SUPER12 GB · less memory GeForce RTX 4060 Ti 16GB16 GB · less memory GeForce RTX 4070 Ti SUPER16 GB · less memory GeForce RTX 4080 SUPER16 GB · less memory RTX 6000 Ada48 GB · more memory L40S48 GB · more memory

Something wrong on this page? Tell us