Compatibility check
Can I run Hermes 3 70B Instruct on RTX 6000 Ada?
Probably partially, with CPU offload.best quant:
This one is close. The answer rests on a file size we calculated from the parameter count rather than measured. A 10% error in that size changes the verdict, so treat it as a maybe and test before you buy.
44.5 GB of weights, plus 3.2 GB for the software that runs it and the smallest conversation it can hold, comes to 47.7 GB against the 46.8 GB this 48 GB device leaves free.
Decode speed
—
Usable context
—
Memory
48 GB
Spills to system RAMest
Spills to system RAM
Too large
Plan B — rent it
Hermes 3 70B Instruct is hosted by 2 providers from $0.70 per 1M output tokens (OpenRouter).Compare providers →
