Leaderboards
Benchmarks
Every score carries its origin.
Arena Hard Prompts168GPQA Diamond16LiveBench58LiveBench Data Analysis58LiveBench Reasoning58MMLU-Pro16
MMLU-Pro
% correct, 12k harder multi-task questionsProvenance16 modelsscores Jun 12, 2024 – Jan 22, 2025
1Qwen2.5 72B InstructQwen72.7BScore56.3Score / $141Score / GB VRAM~1.2ScoredOct 16, 20242Mistral LargeMistral AIScore55.6Score / $9ScoredNov 19, 20243Llama 3.3 70B InstructMeta70.6BScore53.3Score / $167Score / GB VRAM~1.2ScoredDec 3, 20244Llama 3.1 70B InstructMeta70.6BScore53.1Score / $133Score / GB VRAM~1.2ScoredAug 15, 20245Phi 4Microsoft14.7BScore52.9Score / $378Score / GB VRAM6ScoredJan 8, 20256R1 Distill Llama 70BDeepSeek70.6BScore47.5Score / $59Score / GB VRAM~1.1ScoredJan 22, 20257Hermes 3 70B InstructNous Research70.6BScore47.3Score / $68Score / GB VRAM~1.1ScoredAug 28, 20248Mixtral 8x22B InstructMistral AI141BScore44.8Score / $7Score / GB VRAM~0.51ScoredJun 12, 20249Gemma 2 27BGoogle27.2BScore44.5Score / $68Score / GB VRAM~2.6ScoredAug 7, 202410Qwen2.5 Coder 32B InstructQwen32.8BScore44.1Score / $44Score / GB VRAM~2.1ScoredDec 10, 202411Qwen2.5 7B InstructQwen7.6BScore42.9Score / $613Score / GB VRAM~9ScoredSep 18, 202412Command R7B (12-2024)CohereScore35.7Score / $238ScoredDec 20, 202413Mistral NemoMistral AI12.2BScore35.2Score / $1173Score / GB VRAM~4.6ScoredAug 29, 202414Ministral 8BMistral AIScore32.9Score / $299ScoredDec 1, 202415Llama 3.2 3B InstructMeta3.2BScore31.9Score / $638Score / GB VRAM~16ScoredSep 27, 202416Llama 3.2 1B InstructMeta1.2BScore16.8Score / $840Score / GB VRAM~22ScoredSep 23, 2024
