Can I run Llama 3.1 Euryale 70B v2.2 on L40S?

Q: Can I run Llama 3.1 Euryale 70B v2.2 on L40S?

Partially. Llama 3.1 Euryale 70B v2.2 needs CPU offload on L40S (48 GB) — expect reduced speed. It is a close call, computed from an estimated file size, so a 10% error either way would change the answer.

Probably partially, with CPU offload.best quant:

This one is close. The answer rests on a file size we calculated from the parameter count rather than measured. A 10% error in that size changes the verdict, so treat it as a maybe and test before you buy.

44.5 GB of weights, plus 3.2 GB for the software that runs it and the smallest conversation it can hold, comes to 47.7 GB against the 46.8 GB this 48 GB device leaves free.

Decode speed

—

Usable context

—

Memory

48 GB

Spills to system RAMest

Spills to system RAM

Too large

What is quantisation? →

Plan B — rent it

Llama 3.1 Euryale 70B v2.2 is hosted by 3 providers from $0.85 per 1M output tokens (OpenRouter).Compare providers →

Llama 3.1 Euryale 70B v2.2

70.6B · full specs & benchmarks →

L40S

48 GB · everything it runs →