Compatibility check
Can I run Llama Guard 4 12B on GeForce RTX 3060 8GB?
Partially — with CPU offload.best quant:
7.6 GB of weights, plus 1.6 GB for the software that runs it and the smallest conversation it can hold, comes to 9.2 GB against the 7.4 GB this 8 GB device leaves free.
Decode speed
—
Usable context
—
Memory
8 GB
Spills to system RAM
Spills to system RAMest
Too large
Plan B — rent it
Llama Guard 4 12B is hosted by 2 providers from $0.18 per 1M output tokens (OpenRouter).Compare providers →
