Hardware / NVIDIA /GeForce RTX 4090

NVIDIA · GPU

GeForce RTX 4090

Runs 108 models fully in memory, 8 more with CPU offload. 7 of those sit close enough to 24 GB that our size estimate is what decides them.

Memory

24 GB

Bandwidth

1008 GB/s

Runs

108

01

Specifications

Memory

24 GB

Bandwidth

1008 GB/s

FP16 compute

82.6 TFLOPS

TDP

450 W

MSRP

$1,599

Street price

—

Released

Oct 12, 2022

Architecture

Ada Lovelace

02

What it runs

Best quant per model — sorted by fit score.

ModelParamsBest quantContextSpeedVerdict

Granite 4.1 8B8.8BQ4_K_M66K151 tok/sestFits in memory Qwen3.5-9B9.7BQ4_K_M66K137 tok/sestFits in memory Ministral 3 8B 25128.9BQ4_K_M66K150 tok/sestFits in memory Ministral 3 3B 25123.8BQ4_K_M131K350 tok/sestFits in memory Granite 4.0 Micro3.2BQ4_K_M66K353 tok/sestFits in memory Qwen3 VL 8B Thinking8.8BQ4_K_M66K151 tok/sestFits in memory UI-TARS 7B8.3BQ4_K_M66K160 tok/sestFits in memory Llama 3.2 3B Instruct3.2BQ4_K_M131K416 tok/sestFits in memory Qwen3 VL 8B Instruct8.8BQ4_K_M66K151 tok/sestFits in memory Qwen3 8B8.2BQ4_K_M66K162 tok/sestFits in memory Gemma 3 4B4.3BQ5_K_M131K264 tok/sestFits in memory Llama 3.1 8B Instruct8BQ4_K_M131K166 tok/sestFits in memory Niagara 19m Batch.en20MQ8_0~262K37948 tok/sestFits in memory Niagara 38m Batch.en38MQ5_K_M~262K29836 tok/sestFits in memory ARK ASR 0.6B1.3BQ5_K_M~262K872 tok/sestFits in memory ARK ASR 3B4.1BQ5_K_M~262K277 tok/sestFits in memory Audio8 ASR 0.1B0.3BQ5_K_M~262K3779 tok/sestFits in memory Higgs Audio v3 8b STT v28.9BQ4_K_M~66K150 tok/sestFits in memory Higgs Audio v3 STT2.7BQ4_K_M~131K493 tok/sestFits in memory Cohere Transcribe 03 20262.1BQ8_0~262K361 tok/sestFits in memory Distil Large v3.50.8BQ8_0~262K949 tok/sestFits in memory Lite Whisper Large v3 Acc1.4BQ8_0~262K542 tok/sestFits in memory Owsm CTC v3.1 1B1.1BQ5_K_M~262K1031 tok/sestFits in memory Owsm CTC v3.2 ft 1B1BQ8_0~262K759 tok/sestFits in memory Owsm CTC v4 1B1BQ8_0~262K759 tok/sestFits in memory Hubert Large Ls960 ft0.3BQ4_K_M~262K4157 tok/sestFits in memory MMS 1b All1BQ8_0~262K759 tok/sestFits in memory Wav2vec2 Large 960h Lv60 Self0.3BQ5_K_M~262K3543 tok/sestFits in memory Hojo ASR V15.2BQ5_K_M~262K219 tok/sestFits in memory Granite 4.0 1b Speech2.3BQ8_0~131K330 tok/sestFits in memory Granite Speech 3.3 2b3BQ8_0~131K253 tok/sestFits in memory Granite Speech 3.3 8b8.6BQ4_K_M~66K155 tok/sestFits in memory Granite Speech 4.1 2b2.3BQ8_0~131K330 tok/sestFits in memory Granite Speech 4.1 2b NAR2.3BQ8_0~131K330 tok/sestFits in memory STT 2.6b en2.6BQ4_K_M~262K512 tok/sestFits in memory Phi 4 Multimodal Instruct5.6BQ8_0~66K136 tok/sestFits in memory VibeVoice ASR HF8.3BQ4_K_M~262K160 tok/sestFits in memory Voxtral Mini 3B 25075BQ5_K_M~262K227 tok/sestFits in memory Voxtral Mini 4B Realtime 26024.4BQ4_K_M~131K302 tok/sestFits in memory Canary 180m Flash0.2BQ4_K_M~262K7390 tok/sestFits in memory Canary 1b1BQ5_K_M~262K1134 tok/sestFits in memory Canary 1b Flash1BQ4_K_M~262K1330 tok/sestFits in memory Canary 1b v21BQ4_K_M~262K1330 tok/sestFits in memory Canary Qwen 2.5b2.6BQ4_K_M~262K512 tok/sestFits in memory Nemotron 3.5 ASR Streaming 0.6b0.6BQ8_0~262K1186 tok/sestFits in memory Nemotron Speech Streaming en 0.6b0.6BQ8_0~262K1265 tok/sestFits in memory Parakeet CTC 0.6b0.6BQ5_K_M~262K1890 tok/sestFits in memory Parakeet CTC 1.1b1.1BQ5_K_M~262K1031 tok/sestFits in memory Parakeet RNNT 0.6b0.6BQ4_K_M~262K2217 tok/sestFits in memory Parakeet RNNT 1.1b1.1BQ5_K_M~262K1031 tok/sestFits in memory Parakeet TDT CTC 110m0.1BQ8_0~262K6900 tok/sestFits in memory Parakeet TDT 0.6b v20.6BQ5_K_M~262K1890 tok/sestFits in memory Parakeet TDT 0.6b v30.6BQ4_K_M~262K2217 tok/sestFits in memory Parakeet TDT 1.1b1.1BQ5_K_M~262K1031 tok/sestFits in memory Nyra Health CrisperWhisper1.6BQ5_K_M~262K709 tok/sestFits in memory Whisper Large v31.5BQ5_K_M~262K756 tok/sestFits in memory Whisper Large v3 Turbo0.8BQ8_0~262K949 tok/sestFits in memory MOSS Transcribe Diarize0.9BQ8_0~131K843 tok/sestFits in memory Qwen3 ASR 0.6B0.8BQ5_K_M~262K1454 tok/sestFits in memory Qwen3 ASR 0.6B HF0.8BQ4_K_M~131K1663 tok/sestFits in memory

A speed marked est is calculated from this card's memory bandwidth and the size of the weights it has to read per word. It is not a measurement.

Something wrong on this page? Tell us