Compatibility check
Can I run Rocinante 12B on GeForce RTX 3060 8GB?
Partially — with CPU offload.best quant:
7.7 GB of weights, plus 1.8 GB for the software that runs it and the smallest conversation it can hold, comes to 9.5 GB against the 7.4 GB this 8 GB device leaves free.
Decode speed
—
Usable context
—
Memory
8 GB
Spills to system RAM
Spills to system RAMest
Too large
Plan B — rent it
Rocinante 12B is hosted by 2 providers from $0.50 per 1M output tokens (OpenRouter).Compare providers →
