FuLLM benchmark of benchmarks

One ranking across689 benchmarks.

38,805 reported scores for 1,451 models, combined into one Fu Score.

Open the Fu Ranking Browse benchmarks 1328
Fu Score by rankupdated 2026-10-23
scores
38,805
benchmarks
689
models
1,451
#1 by Fu
GPT-6 Astra 90.7
Fu Ranking → Charts only → Benchmarks ↓

Benchmark leads Benchmarks won per group. Click a row to filter.

Meta Score Best Fu Score of any model in the group.

# Name Topic Description Relevance GitHub ★ Leader Top score
1 3DSRBench O Spatial reasoning 3D spatial reasoning benchmark for large multimodal models (height, location, orientation, multi-object relations). ★★★★★ 🇺🇸 Gemini 3 Pro 68.9%
2 AA-Briefcase O Knowledge work Artificial Analysis benchmark of long-horizon multi-week knowledge-work projects; Elo from rubric and pairwise judging. ★★★★★ 🇺🇸 Claude Sonnet 5.5 1824
3 AA-Index O Multi-domain QA Comprehensive QA index across diverse domains. ★★★★★ 🇺🇸 Grok 4 73.2%
4 AA-LCR O Long-context reasoning A challenging benchmark measuring language models' ability to extract, reason about, and synthesize information from long-form documents ranging from 10k to 100k tokens. ★★★★★ 🇨🇳 Kimi-K3 88.7%
5 AA-Omniscience O Knowledge and hallucination Benchmark measuring factual recall and hallucination across economically relevant domains. ★★★★★ 🇺🇸 Claude Fable 5 40.0
6 AA-Omniscience Accuracy O Knowledge and hallucination Share of AA-Omniscience questions answered correctly (accuracy component of the AA-Omniscience Index). ★★★★★ 🇺🇸 Gemini 3.5 Flash 51.4%
7 AA-Omniscience-Decision Accuracy O Knowledge and hallucination Accuracy on 500 private AA-Omniscience questions posed as five options (four answers plus "I don't know") for decision models. ★★★★★ 🇺🇸 Claude Opus 5.5 (max) 93.4%
8 AA-Omniscience-Decision Non-Hallucination Rate O Knowledge and hallucination Share of AA-Omniscience-Decision questions not answered correctly where the model chose "I don't know". ★★★★★ G d1-ultra 84.5%
9 AA-Omniscience Non-Hallucination Rate O Knowledge and hallucination Share of AA-Omniscience questions not answered incorrectly (abstains instead of hallucinating) among non-correct answers. ★★★★★ 🇺🇸 Nemotron 3 Ultra 78.7%
10 AA-Omniscience-Public O Knowledge and hallucination AA-Omniscience Index on the publicly released question subset (index from -100 to 100). ★★★★★ 🇨🇳 GLM-5.3 22.6
11 AA-Omniscience-Public Accuracy o Knowledge and hallucination Share of questions answered correctly on the publicly released AA-Omniscience subset. ★★★★★ 🇺🇸 GPT-6 Luna (medium) 46.9%
12 AceBench O Industry QA Industry-focused benchmark assessing domain QA and reasoning. ★★★★★ 🇨🇳 ERNIE 5.0 87.7%
13 ACEBench (ZH) o Tool use (Chinese) Chinese split of ACEBench tool-use evaluation. ★★★★★ 🇨🇳 ERNIE 5.0 89.6%
14 ACOS (decision) O Aspect sentiment Aspect-category-opinion-sentiment decisions; per-review F1. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. ★★★★★ G pplx-decider-v1.2 40.9
15 ACP-Bench Bool O Safety evaluation (boolean) Safety and behavior evaluation with yes/no questions. ★★★★★ 🇨🇳 Qwen3-32B 85.1%
16 ACP-Bench MCQ O Safety evaluation (MCQ) Safety and behavior evaluation with multiple-choice questions. ★★★★★ 🇺🇸 Llama 3.3 70B 82.1%
17 ActivityNet-QA O Video QA (open-ended) Open-ended QA on ActivityNet videos; GPT-judged accuracy. ★★★★★ 🇺🇸 GPT-4o 61.9%
18 ACVA o Arabic culture Arabic Cultural Value Alignment benchmark (true/false statements about Arab culture). ★★★★★ G Fanar-2-27B-Instruct 82.7%
19 AdvancedIF O Instruction following Meta's AdvancedIF rubric-based benchmark for complex, multi-turn and system-prompt instruction following. ★★★★★ 🇺🇸 Gemini 3.1 Pro 83.9%
20 AetherCode O Code generation Code generation benchmark for diverse coding tasks. ★★★★★ 🇺🇸 GPT-5.5 81.6%
21 AfriMed-QA o Medical QA Pan-African medical question answering benchmark. ★★★★★ 🇺🇸 MedGemma 27B 72.0%
22 AG News o Topic classification AG News four-class news topic classification. ★★★★★ G Laya 92.8%
23 AgentBench o LLM agents Multi-environment agent benchmark (OS, DB, KG, puzzles, household, web shopping, web browsing); overall weighted score. ★★★★★ 🇨🇳 GLM 4 (05-20) 3.79
24 AgentCompany o Agent reasoning Company-level agent reasoning and decision-making benchmark. ★★★★★ 🇺🇸 Claude Sonnet 4.5 41.0%
25 AgentDojo O Agent evaluation Interactive evaluation suite for autonomous agents across tools and tasks. ★★★★★ 🇺🇸 Claude 3.7 Sonnet 88.7%
26 Agentic Coding O Agentic coding Agentic coding benchmark for autonomous software tasks. ★★★★★ 🇺🇸 Gemini 3 Flash Preview 53.8%
27 AgentIF o Instruction following Instruction-following benchmark built from real agentic scenarios with long, complex system prompts and constraints; score is (CSR+ISR)/2. ★★★★★ G openPangu-2.0-Flash Thinking 44.9%
28 AgentIF-OneDay o Agentic instruction following Agentic instruction-following benchmark of full-day, multi-step everyday and office tasks for general agents. ★★★★★ G Nanbeige4.2-3B 67.5%
29 Agents' Last Exam O Agentic professional work Long-horizon agentic workflows across 55 professional sub-industries performed on a computer. ★★★★★ 🇺🇸 GPT-6 Astra 59.3%
30 Agents' Last Exam (average score) O Agentic professional work Agents' Last Exam (ALE-CLI, 105-task Linux split) average rubric score (as opposed to pass rate). ★★★★★ 🇺🇸 Claude Opus 5.5 (max) 63.7%
31 AgentWorldBench (Android) O Language world models AgentWorldBench Android domain: open-ended evaluation of language world models predicting next environment state in Android agent interactions; 5-dimension rubric mean normalized 0-100. ★★★★★ 🇺🇸 Claude Opus 4.6 61.7%
32 AgentWorldBench (MCP) O Language world models AgentWorldBench MCP domain: open-ended evaluation of language world models predicting next environment state in MCP agent interactions; 5-dimension rubric mean normalized 0-100. ★★★★★ 🇺🇸 GPT-5.4 70.1%
33 AgentWorldBench (OS) O Language world models AgentWorldBench OS domain: open-ended evaluation of language world models predicting next environment state in OS agent interactions; 5-dimension rubric mean normalized 0-100. ★★★★★ 🇺🇸 Claude Opus 4.6 70.2%
34 AgentWorldBench (Search) O Language world models AgentWorldBench Search domain: open-ended evaluation of language world models predicting next environment state in Search agent interactions; 5-dimension rubric mean normalized 0-100. ★★★★★ 🇨🇳 Qwen-AgentWorld-397B-A17B 37.8%
35 AgentWorldBench (SWE) O Language world models AgentWorldBench SWE domain: open-ended evaluation of language world models predicting next environment state in SWE agent interactions; 5-dimension rubric mean normalized 0-100. ★★★★★ 🇨🇳 Qwen-AgentWorld-397B-A17B 68.5%
36 AgentWorldBench (Terminal) O Language world models AgentWorldBench Terminal domain: open-ended evaluation of language world models predicting next environment state in Terminal agent interactions; 5-dimension rubric mean normalized 0-100. ★★★★★ 🇺🇸 Claude Opus 4.8 59.2%
37 AgentWorldBench (Web) O Language world models AgentWorldBench Web domain: open-ended evaluation of language world models predicting next environment state in Web agent interactions; 5-dimension rubric mean normalized 0-100. ★★★★★ 🇺🇸 Claude Opus 4.8 54.7%
38 AGIEval (English) O Exams English subset of AGIEval; academic and professional exam questions. ★★★★★ 🇨🇳 Qwen3-VL 32B Thinking 92.2%
39 AGIEval LSAT-AR o Law exam reasoning LSAT Analytical Reasoning subset from AGIEval benchmark. ★★★★★ 🇨🇳 Qwen2.5 32B Base 30.4%
40 AI2D O Diagram understanding (VQA) Visual question answering over science and diagram images. ★★★★★ 🇺🇸 Gemini 3 Pro 98.7%
41 AICodeKing Non-Agentic O Code generation (non-agentic) Non-agentic code generation benchmark from AICodeKing. ★★★★★ 🇺🇸 Claude Opus 4.6 100.0%
42 Aider Code Editing O Code editing Measures interactive code editing quality within the Aider assistant workflow. ★★★★★ 🇺🇸 Gemini 2.5 Pro 89.8%
43 Aider-Polyglot O Code assistant eval Aider polyglot coding leaderboard. ★★★★★ 🇺🇸 Gemini 3 Pro Preview 92.9%
44 Aider-Polyglot (Diff) O Code assistant eval Aider polyglot leaderboard using diff mode (pass@2). ★★★★★ 🇺🇸 Gemini 3 Pro Preview 91.9%
45 AIME 2024 O Math (competition) American Invitational Mathematics Examination 2024 problems. ★★★★★ 🇺🇸 GPT-OSS 120B 96.6%
46 AIME 2024-Ko o Math (competition, Korean) Korean translation of AIME 2024 problems. ★★★★★ 🇨🇳 Qwen3-30B-A3B-Thinking-2507 80.3%
47 AIME 2025 O Math (competition) American Invitational Mathematics Examination 2025 problems. ★★★★★ 🇺🇸 Claude Sonnet 4.5 100.0%
48 AIME 2026 I O Math (competition) American Invitational Mathematics Examination 2026 I problems. ★★★★★ 🇺🇸 GPT-5.5 100.0%
49 AIME 2026 I (with tools) o Math competition (with tools) AIME 2026 I problems solved with tool (code execution) access. ★★★★★ 🇺🇸 Gemini 3.1 Pro 99.9%
50 AInstein-SWE-Bench o Agentic coding AInstein agent coding benchmark. ★★★★★ 🇺🇸 Gemini 3 Pro 42.8%
51 AlignBench O Alignment and instruction following Benchmark for instruction-following quality and alignment behavior. ★★★★★ 🇨🇳 Hunyuan-Large-Instruct 8.30
52 All-Angles Bench O Spatial perception All-Angles benchmark for spatial recognition and 3D perception. ★★★★★ 🇨🇳 Qwen3.6-35B-A3B 64.0%
53 Almieyar o Arabic linguistics Arabic linguistic competence benchmark across phonology, morphology, syntax, semantics and pragmatics. ★★★★★ G Fanar-2-27B-Instruct 79.5%
54 AlpacaEval O Instruction following Automatic eval using GPT-4 as a judge. ★★★★★ 1849 🇺🇸 Llama-3_1-70B-TFree-HAT-SFT 99.4%
55 AlpacaEval 1.0 O Instruction following Original AlpacaEval: win rate against text-davinci-003 judged by GPT-4. ★★★★★ 🇺🇸 GPT-4 Turbo (11-06) 97.7%
56 AlpacaEval 2.0 O Instruction following Updated AlpacaEval with improved prompts and judging. ★★★★★ 🇨🇳 DeepSeek R1 87.6%
57 Amazon ESCI O Product search relevance Shopping query/product relevance (Exact, Substitute, Complement, Irrelevant); macro-F1 (Jev Decision Index suite, posed as typed decisions). ★★★★★ G pplx-decider-v1.2 62.4
58 AMC-23 O Math (competition) American Mathematics Competition 2023 evaluation. ★★★★★ G QwQ-32B 98.5%
59 AMO-Bench O Math (competition) Advanced math olympiad-style benchmark. ★★★★★ 🇺🇸 Gemini 3 Pro 72.5%
60 AMO-Bench CH O Math (competition) Chinese subset of AMO-Bench. ★★★★★ 🇺🇸 Gemini 3 Pro 74.9%
61 Android Daily o GUI agents Benchmark of long-horizon daily phone-use tasks across multiple Android apps. ★★★★★ 🇺🇸 Gemini 3 Flash 63.2%
62 AndroidBench o Software engineering (Android) Agentic Android app development benchmark; Qwen reports avg@3 on the 95-task public subset. ★★★★★ 🇺🇸 Claude Fable 5 84.5%
63 AndroidWorld O Mobile agents Benchmark for agents operating Android apps via UI automation. ★★★★★ 🇺🇸 Claude Fable 5 88.8%
64 ANLI O Natural language inference Adversarial NLI (entailment / neutral / contradiction); macro-F1 (Jev Decision Index suite, posed as typed decisions). ★★★★★ G AutoTrust JEV-27B 77.2
65 APEX-Agents O Long horizon professional tasks APEX benchmark evaluating agents on long-horizon professional tasks. ★★★★★ 🇺🇸 Claude Fable 5 59.2%
66 APEX-Agents 1.1 O Long horizon professional tasks Mercor APEX-Agents 1.1 revision: 240 long-horizon cross-application professional-services tasks (investment banking, consulting, corporate law) with updated task specs and judge; Pass@1. Not comparable to APEX-Agents v1.0. ★★★★★ 🇺🇸 Gemini 4 Argon 82.4%
67 APEX-SWE o Software engineering Mercor/Cognition benchmark of integration (shipping end-to-end systems on cloud/business/infra services) and observability (debugging production-style failures from telemetry) tasks; Pass@1. ★★★★★ 🇺🇸 Claude Fable 5 58.8%
68 API-Bank o Tool use API-Bank tool-use benchmark. ★★★★★ 🇺🇸 Llama 3.1 405B 92.0%
69 API-Bank (decision) O Tool use API-Bank cases answered as option choices; accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. ★★★★★ G Clef-Flash 93.1%
70 ArabicMMLU O Arabic knowledge Native Arabic multi-task knowledge benchmark (school exams across MENA countries), 40 tasks. ★★★★★ G Fanar-2-27B-Instruct 74.7%
71 ARC-AGI-1 O General reasoning ARC-AGI Phase 1 aggregate accuracy. ★★★★★ 🇺🇸 GPT-6 Astra 98.5%
72 ARC-AGI-2 O General reasoning ARC-AGI Phase 2 aggregate accuracy. ★★★★★ 🇺🇸 GPT-6 Astra 95.0%
73 ARC-AGI-3 O General reasoning ARC-AGI-3 interactive reasoning benchmark; Relative Human Action Efficiency (RHAE) score. ★★★★★ 🇺🇸 GPT-6 Astra 99.9%
74 ARC Average O Science QA (average) Average accuracy across ARC-Easy and ARC-Challenge. ★★★★★ 🇺🇸 SmolLM2 1.7B Pretrained 60.5%
75 ARC-Challenge O Science QA Hard subset of AI2 Reasoning Challenge; grade-school science. ★★★★★ G Clef-Flash 98.3%
76 ARC-Challenge (DE) o Science QA (German) German translation of the ARC Challenge benchmark. ★★★★★ 🇺🇸 Llama-3_1-70B-TFree-HAT-SFT 0.669
77 ARC-Easy O Science QA Easier subset of AI2 Reasoning Challenge. ★★★★★ G Clef-Omni 99.8%
78 ARC-Easy (DE) o Science QA (German) German translation of the ARC Easy science QA benchmark. ★★★★★ 🇺🇸 Llama-3_1-70B-TFree-HAT-SFT 0.848
79 Arena-Hard O Chat ability Hard prompts on Chatbot Arena. ★★★★★ 920 🇫🇷 Mistral Medium 3 97.1%
80 Arena-Hard V2 O Chat ability Updated Arena-Hard v2 prompts on Chatbot Arena. ★★★★★ 920 🇺🇸 GPT-OSS 120B 90.3%
81 Arena-Hard V2 Creative Writing O Creative writing Chatbot Arena Hard V2 creative writing win-rate subset. ★★★★★ 🇺🇸 Gemini 3 Pro 93.6%
82 Arena-Hard V2 Hard Prompt O Chat ability Chatbot Arena Hard V2 benchmark using the hard prompt win-rate subset. ★★★★★ 🇺🇸 Nemotron-Cascade-2-30B-A3B 88.2%
83 ARKitScenes O 3D scene understanding ARKitScenes benchmark for assessing 3D scene reconstruction and understanding from mixed reality captures. ★★★★★ 🇨🇳 Qwen2.5-VL 72B Instruct 61.5%
84 ART Agent Red Teaming O Agent robustness Evaluation suite for adversarial red-teaming of autonomous AI agents. ★★★★★ 🇺🇸 Claude Opus 4.5 ↓ 33.6%
85 ArtifactsBench O Agentic coding Artifacts-focused coding and tool-use benchmark evaluating generated code artifacts. ★★★★★ G TeleChat3-Coder-36B-Thinking 82.8%
86 ArxivMath O Research math Math problems derived from recent arXiv papers (reported in Anthropic Fable 5/Mythos 5 system card). ★★★★★ 🇺🇸 GPT-5.6 Sol 79.5%
87 ASR AMI o ASR Automatic speech recognition benchmark on AMI meeting speech. ★★★★★ 🇨🇳 Qwen2.5-Omni-3B ↓ 15.1%
88 ASR Common Voice 8 (Japanese) O ASR Japanese automatic speech recognition on the Common Voice 8.0 Japanese test set (character error rate). ★★★★★ 🇺🇸 LFM2.5-Audio-1.5B-JP ↓ 4.4%
89 ASR Earnings22 o ASR Automatic speech recognition benchmark on Earnings22 financial calls. ★★★★★ G Whisper-large-V3 ↓ 11.3%
90 ASR GigaSpeech o ASR Automatic speech recognition benchmark on GigaSpeech. ★★★★★ G Whisper-large-V3 ↓ 10.0%
91 ASR JSUT Basic5000 O ASR Japanese automatic speech recognition on JSUT basic5000 (character error rate). ★★★★★ G Whisper-large-V3 ↓ 7.1%
92 ASR LibriSpeech Clean O ASR Automatic speech recognition benchmark on LibriSpeech clean split. ★★★★★ 🇨🇳 Qwen3-Omni-30B-A3B-Instruct ↓ 1.2%
93 ASR LibriSpeech Other O ASR Automatic speech recognition benchmark on LibriSpeech other split. ★★★★★ G Ming-flash-omni 2.0 ↓ 2.2%
94 ASR ReazonSpeech O ASR Japanese automatic speech recognition on the ReazonSpeech held-out test set (character error rate). ★★★★★ G ReazonSpeech-NeMo-v2 ↓ 11.2%
95 ASR SPGISpeech o ASR Automatic speech recognition benchmark on SPGISpeech. ★★★★★ 🇺🇸 LFM2.5-Audio-1.5B ↓ 2.8%
96 ASR TED-LIUM o ASR Automatic speech recognition benchmark on TED-LIUM. ★★★★★ 🇺🇸 LFM2.5-Audio-1.5B ↓ 3.5%
97 ASR VoxPopuli o ASR Automatic speech recognition benchmark on VoxPopuli. ★★★★★ 🇨🇳 Qwen2.5-Omni-3B ↓ 5.6%
98 AstaBench O Agent evaluation Evaluates science agents across literature understanding, data analysis, planning, tool use, coding, and search. ★★★★★ 🇺🇸 Claude Opus 4.7 58.0%
99 AttaQ O Safety / jailbreak Adversarial jailbreak suite measuring refusal robustness against targeted attack prompts. ★★★★★ G Granite 3.3 8B Instruct 88.5%
100 Audio MultiChallenge o Spoken multi-turn conversation Scale AI Audio MultiChallenge: multi-turn spoken conversation benchmark; average pass rate (APR). ★★★★★ 🇺🇸 GPT-realtime-2.0 (xhigh) 48.5%
101 AutoCodeBench O Autonomous coding End-to-end autonomous coding benchmark with unit-test based execution across diverse repositories and tasks. ★★★★★ 🇺🇸 Claude Opus 4 (Thinking) 52.4%
102 AutoCodeBench-Lite O Autonomous coding Lite version of AutoCodeBench focusing on smaller tasks with the same end-to-end, unit-test-based evaluation. ★★★★★ 🇺🇸 Claude Opus 4 64.5%
103 AutoLogi o Logical reasoning AutoLogi benchmark evaluating automated logical reasoning accuracy. ★★★★★ 🇺🇸 Claude Sonnet 4 89.8%
104 AutomationBench O Agentic business workflows Zapier benchmark of end-to-end business workflows via REST APIs across 47 simulated apps; pass/fail on deterministic state assertions. ★★★★★ G MiniMax M3.1 Flash (max) 62.8%
105 AutomationBench-AA O Business workflows Artificial Analysis run of AutomationBench end-to-end business workflows. ★★★★★ 🇨🇳 Qwen3.8-Flash-Next 72.1%
106 AutomationBench (partial credit) O Business workflows AutomationBench end-to-end business workflows scored with partial credit per task. ★★★★★ 🇺🇸 GPT-5.6 Sol 81.2%
107 BABE O STEM reasoning STEM reasoning benchmark evaluating broad applied and basic engineering knowledge. ★★★★★ 🇺🇸 GPT-5.2 (high) 58.1%
108 BabyVision O Visual reasoning Visual reasoning benchmark testing basic visual perception and understanding. ★★★★★ 🇨🇳 Qwen3.8-Max 91.3%
109 BabyVision (With Tools) O Visual reasoning BabyVision visual perception benchmark evaluated with tool access in an agent harness. ★★★★★ 🇺🇸 GPT-6 Astra 97.7%
110 BALROG O Agent robustness Benchmark for assessing LLM agents under adversarial and out-of-distribution tool-use scenarios. ★★★★★ 🇺🇸 Grok 4 43.6%
111 BankerToolBench O Investment banking agent tasks BankerToolBench: tool-using agent tasks modeled on investment-banking workflows, scored with dataset rubrics. ★★★★★ 🇺🇸 Claude Opus 5 81.9%
112 BANKING77 O Intent classification Bank customer intent among 77 classes; macro-F1 (Jev Decision Index suite, posed as typed decisions). ★★★★★ G Clef-Omni 94.8
113 BBH O Multi-task reasoning Hard subset of BIG-bench with diverse reasoning tasks. ★★★★★ 510 🇨🇳 ERNIE 4.5 424B A47B 94.3%
114 BBH (decision) O Multi-task reasoning BIG-Bench Hard fixed-option tasks answered directly with no working; accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. ★★★★★ G Jev 1.14 93.2%
115 BBH-ZH O Multi-task reasoning (Chinese) Chinese translation of BIG-Bench Hard reasoning tasks. ★★★★★ G LLaDA2.0 Flash 87.5%
116 BBQ O Bias evaluation Bias Benchmark for Question Answering evaluating social biases across contexts. ★★★★★ 🇫🇷 Mixtral 8x 7B 56.0%
117 BEAM 128K o Long-context memory BEAM long-context memory benchmark at 128K tokens. ★★★★★ G Muse Glimmer 30B 65.1%
118 BeaverTails o Safety / harmfulness Safety benchmark evaluating harmfulness in model responses. ★★★★★ G IQuest-Coder-V1-40B-Thinking 76.7%
119 Belebele O Multilingual reading comprehension Parallel multiple-choice reading-comprehension benchmark over FLORES passages in 122 language variants. ★★★★★ 🇺🇸 Gemma 4 31B (IT) 95.9%
120 BenchCAD O CAD generation CAD modelling benchmark for agents producing 3D CAD artifacts. ★★★★★ 🇺🇸 GPT-6 Astra 95.9%
121 BenchCAD (python tool) o CAD BenchCAD with a Python tool available. ★★★★★ 🇺🇸 GPT-5.6 Sol 83.4%
122 BeyondAIME O Math (beyond AIME) Advanced math problems exceeding AIME difficulty. ★★★★★ 🇺🇸 GPT-5.5 91.0%
123 BFCL O Code reasoning Benchmark for functional code correctness and logic. ★★★★★ 🇨🇳 Qwen3-4B 95.0%
124 BFCL (decision) O Function calling Yes/no per supplied function on BFCL cases; case exact accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. ★★★★★ G Clef-Flash 98.8%
125 BFCL Live v2 O Finance QA Financial compliance and literacy questions from the BFCL Live v2 benchmark. ★★★★★ 🇺🇸 o1 Mini 81.0%
126 BFCL v2 o Code reasoning Second release of the BFCL benchmark focusing on functional code correctness and logic. ★★★★★ 🇨🇳 MiniCPM3-4B 76.0%
127 BFCL v3 O Code reasoning Benchmark for functional code correctness and logic (v3). ★★★★★ 🇨🇳 GLM 4.5 77.8%
128 BFCL v3 (Japanese) O Function calling (Japanese) Japanese translation of BFCL v3 function-calling benchmark. ★★★★★ 🇺🇸 NVIDIA-Nemotron-Nano-9B-v2-Japanese 64.9%
129 BFCL v3 (Live) O Tool calling BFCL v3 Live subset for real-time tool calling evaluation. ★★★★★ 🇺🇸 Gemini 3.1 Pro 84.9%
130 BFCL v3 (Multi-Turn) O Tool calling BFCL v3 Multi-Turn subset for multi-turn tool calling evaluation. ★★★★★ G MiniMax M2.5 76.8%
131 BFCL v4 O Code reasoning BFCL v4 benchmark for functional code correctness and logic. ★★★★★ 🇨🇳 Qwen3.6-27B 80.2%
132 BFCL v4 Multi-Turn o Function calling Multi-turn category of the Berkeley Function Calling Leaderboard v4. ★★★★★ G Mercury Voice 60.5%
133 BharatMath o Indic languages Math word problems in Indian languages. ★★★★★ 🇺🇸 Gemma 4 31B (IT) 78.6%
134 BIG-Bench o Multi-task reasoning BIG-bench overall performance (original). ★★★★★ 3110 🇺🇸 Gemma 2 7B 55.1%
135 Big Bench Audio O Speech reasoning Artificial Analysis Big Bench Audio: spoken-question reasoning benchmark adapted from BIG-Bench Hard. ★★★★★ 🇺🇸 Gemini 3 Flash 99.2%
136 BIG-Bench Extra Hard O Multi-task reasoning Extra hard subset of BIG-bench tasks. ★★★★★ 🇺🇸 Gemini 3 Pro 78.8%
137 Big Finance Bench o Finance Finance benchmark reported in the GPT-5.6 announcement. ★★★★★ 🇺🇸 GPT-5.6 Sol 53.0%
138 BigCodeBench O Code Generation BigCodeBench evaluates large language models on practical code generation tasks with unit-test verification. ★★★★★ G MiMo V2 Flash Base 70.1%
139 BigCodeBench Hard O Code generation (hard) Harder variant of BigCodeBench testing complex programming and library tasks with function-level code generation. ★★★★★ 🇺🇸 Claude 3.7 Sonnet (2025-02-19) 35.8%
140 BIOBench o Biology reasoning Biology knowledge and reasoning benchmark. ★★★★★ 🇺🇸 Gemini 3 Pro 51.9%
141 Biology-Instruction O Biology multi-omics Multi-omics sequence reasoning benchmark for biological data understanding. ★★★★★ G Intern-S2-Preview-397B 56.9%
142 BioLP-Bench o Biomedical NLP Comprehensive biomedical language processing benchmark evaluating LLMs across tasks like NER, relation extraction, and QA. ★★★★★ 🇺🇸 Grok 4 47.0%
143 BioMysteryBench O Biology research Anthropic biology research benchmark of open-ended data-analysis 'mysteries'; human-solved subset. ★★★★★ 🇺🇸 Claude Opus 5 90.1%
144 BioMysteryBench (Hard) O Biology research BioMysteryBench hard subset (problems not solved by human experts). ★★★★★ 🇺🇸 Gemini 3.8 Flash 56.5%
145 Bird-SQL O Text-to-SQL Natural language to SQL generation benchmark. ★★★★★ 🇺🇸 Gemini 2.0 Pro 59.3%
146 BixBench o Bioinformatics Benchmark of real-world bioinformatics and data-analysis tasks. ★★★★★ 🇺🇸 GPT-5.5 80.5%
147 BLEnD O Cultural understanding Benchmark assessing cultural knowledge, sensitivity, and language use across contexts. ★★★★★ 🇨🇳 Qwen2.5-72B-Instruct 75.4%
148 BLINK O Multimodal grounding Evaluates visual-language grounding and reference resolution to reduce hallucinations. ★★★★★ 🇺🇸 Gemini 3.6 Flash 89.2%
149 Blueprint-Bench 2 o Spatial reasoning Andon Labs agentic spatial-reasoning benchmark: reconstruct 2D floor plans from ~20 interior photos per apartment (50 apartments). ★★★★★ 🇺🇸 Claude Fable 5 38.6%
150 BoB-HVR O Composite capability index Hard, Versatile, and Relevant composite score across eight capability buckets. ★★★★★ 🇺🇸 Gemini 3 Pro 18.3
151 BOLD o Bias evaluation Bias in Open-ended Language Dataset probing demographic biases in text generation. ★★★★★ 🇫🇷 Mixtral 8x 7B ↓ 0.084
152 BoolQ O Reading comprehension Yes/no QA from naturally occurring questions. ★★★★★ 171 G ST-MoE-32B 92.4%
153 Borda Count (Multilingual) o Aggregate ranking Borda count aggregate ranking across multilingual benchmarks; lower is better. ★★★★★ 🇨🇳 Qwen3-32B ↓ 2.90
154 BPoMP (decision) O Preference prediction BPoMP preference decisions; accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. ★★★★★ G Clef 96.9%
155 BridgeBench o Reasoning BridgeBench evaluation benchmark. ★★★★★ 🇺🇸 Claude Opus 4.6 60.1%
156 BRIGHT (decision) O Reasoning-intensive retrieval Rate candidate documents for BRIGHT queries; nDCG@10. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. ★★★★★ G GLiDE (no thinking, 28B) 50.1
157 BrokenArXiv o Research math Benchmark of subtly broken/false statements derived from recent arXiv math papers; models must detect and prove or refute them. ★★★★★ 🇺🇸 Claude Opus 5 77.7%
158 BrowseComp O Web browsing Web browsing comprehension and competence benchmark. ★★★★★ G Nex-N2.5-Max 92.6%
159 BrowseComp (Agent Swarm) o Web browsing agents BrowseComp evaluated with Moonshot's Agent Swarm multi-agent orchestration (parallel sub-agents) instead of a single agent. ★★★★★ 🇨🇳 Kimi-K2.6 Thinking 86.3%
160 BrowseComp-Plus O Deep research BrowseComp-Plus: BrowseComp questions over a fixed, curated document corpus for reproducible deep-research agent evaluation. ★★★★★ G Jais-3-32B 71.9%
161 BrowseComp-VL O Multimodal browsing Vision-language variant of BrowseComp requiring multimodal web browsing. ★★★★★ 🇺🇸 GPT-5.5 65.7%
162 BrowseComp (With Content Manager) O Web browsing BrowseComp benchmark evaluated with content manager assistance. ★★★★★ 🇨🇳 Kimi-K3 91.2%
163 BrowseComp_zh O Web browsing (Chinese) Chinese variant of the BrowseComp web browsing benchmark. ★★★★★ G XYZ-Aquila-pro 85.1%
164 BRuMo25 O Math competition BruMo 2025 olympiad-style mathematics benchmark. ★★★★★ 🇨🇳 Kimi-K2.5 Thinking 98.3%
165 Business Arena O Business simulation Simulated business-operation arena; final profit/value in USD achieved by the agent. ★★★★★ 🇺🇸 GPT-5.6 Sol $168,867
166 BuzzBench O Humor analysis A humour analysis benchmark. ★★★★★ 🇺🇸 Gemini 2.5 Pro 71.1%
167 C-Eval O Chinese exams Chinese college-level exam benchmark. ★★★★★ 1768 🇨🇳 Kimi-K2.5 94.0%
168 C3 O Chinese reading comprehension Multiple-choice Chinese machine reading comprehension (CLUE C3). ★★★★★ 🇨🇳 InternVL2.5-26B 95.8%
169 C3-Bench o Reasoning (Chinese) Comprehensive Chinese reasoning capability benchmark. ★★★★★ 35 🇨🇳 GLM-4.5 Base 83.1%
170 CaseLaw v2 O Legal reasoning U.S. case law benchmark evaluating legal reasoning and judgment over court opinions. ★★★★★ 🇺🇸 Grok 4.3 79.3%
171 CaseReportBench o Medical information extraction Clinical information extraction from medical case reports into structured fields. ★★★★★ 🇺🇸 LFM2.5 350M 32.5%
172 CC-Bench-V2 Backend o Agentic coding Z.ai CC-Bench-V2 backend development tasks in a Claude Code-style agent harness. ★★★★★ 🇺🇸 Claude Opus 4.6 26.9%
173 CC-Bench-V2 Frontend o Agentic coding Z.ai CC-Bench-V2 frontend development tasks in a Claude Code-style agent harness. ★★★★★ 🇺🇸 Claude Opus 4.6 75.9%
174 CC-Bench-V2 Repo Exploration o Agentic coding Z.ai CC-Bench-V2 repository exploration tasks in a Claude Code-style agent harness. ★★★★★ 🇺🇸 Claude Opus 4.6 74.4%
175 CC-OCR O OCR (cross-lingual) Cross-lingual OCR benchmark evaluating character recognition across mixed-language documents. ★★★★★ 🇨🇳 Qwen3.6-Plus 83.4%
176 CC-OCR (KIE) o Key information extraction Key information extraction track of the CC-OCR benchmark. ★★★★★ 🇨🇳 Qwen3-VL-235B-A22B 95.1%
177 CC-OCR (Multilingual) o OCR (multilingual) Multilingual text recognition track of the CC-OCR benchmark. ★★★★★ G Qianfan-OCR 76.7%
178 CCBench O Chinese culture VQA MMBench-family benchmark on Chinese culture (painting, calligraphy, cuisine, history...). ★★★★★ G Step-1V 32K 71.2%
179 CCPM O Chinese classical poetry Chinese Classical Poetry Matching: match modern Chinese to classical poetry lines. ★★★★★ 🇨🇳 DeepSeek-V2 93.1%
180 cfcolor (decision) O Color perception Color-name / color judgement decisions; accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. ★★★★★ G pplx-decider-v1.2 68.2%
181 CFEval o Coding ELO / contest eval Contest-style coding evaluation with ELO-like scoring. ★★★★★ 🇨🇳 Qwen3-235B-A22B-Thinking-2507 2134
182 CGBench O Long video QA Cartoon/CG long video question answering benchmark. ★★★★★ G Seed2.0 Lite (0428) 65.5%
183 Charades-STA O Video grounding Charades-STA temporal grounding (mIoU). ★★★★★ 🇨🇳 Seed1.5-VL-Thinking 64.0%
184 ChartBench o Chart understanding Chart comprehension benchmark covering annotated and unannotated chart types. ★★★★★ G Qianfan-OCR 85.9%
185 ChartMuseum O Chart understanding Large-scale curated collection of charts for evaluating parsing, grounding, and reasoning. ★★★★★ 🇺🇸 Claude Mythos 5 85.9%
186 Chartography O Chart understanding Surge AI benchmark for professional chart understanding (no tools). ★★★★★ 🇺🇸 Gemini 4 Argon 71.6%
187 Chartography (With Tools) O Chart understanding Surge AI Chartography evaluated with tool (code execution) access. ★★★★★ 🇺🇸 Claude Sonnet 5.5 90.2%
188 ChartQA O Chart understanding (VQA) Visual question answering over charts and plots. ★★★★★ 🇨🇳 Keye-VL-1.5-8B 94.1%
189 ChartQA-Pro O Chart understanding (VQA) Professional-grade chart question answering with diverse chart types and complex reasoning. ★★★★★ 🇺🇸 Claude Mythos 5 71.6%
190 ChartQA-Pro (with tools) o Chart understanding (with tools) ChartQA-Pro evaluated with tool access. ★★★★★ 🇺🇸 Claude Opus 4.8 72.3%
191 CharXiv (DQ) O Chart description (PDF) Scientific chart/table descriptive questions from arXiv PDFs. ★★★★★ 🇺🇸 Gemini 3 Pro 96.0%
192 CharXiv (RQ) O Chart reasoning (PDF) Scientific chart/table reasoning questions from arXiv PDFs. ★★★★★ 🇺🇸 Claude Fable 5 93.5%
193 CharXiv (RQ, With Tools) O Chart understanding CharXiv reasoning questions evaluated with tool access. ★★★★★ 🇺🇸 Claude Mythos 5 93.5%
194 ChessBench (decision) O Chess Choose the best move among candidates (ChessBench); accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. ★★★★★ G Clef 24.7%
195 CheXpert o Medical imaging Chest X-ray condition classification, macro F1 over top 5 conditions. ★★★★★ 🇺🇸 MedGemma 27B 49.9%
196 CHID O Chinese idiom cloze Chinese IDiom cloze-test dataset (CLUE). ★★★★★ 🇨🇳 DeepSeek-V2 92.7%
197 Chinese SimpleQA O QA (Chinese) Chinese variant of the SimpleQA benchmark. ★★★★★ 🇨🇳 ERNIE 5.0 86.0%
198 CIBench O Code interpreter Evaluation of LLMs using a code interpreter for data-science tasks; EN/ZH average. ★★★★★ G InternLM2-Chat-20B 55.4%
199 Circa o Indirect answers Interpreting indirect yes/no answers in dialogue. ★★★★★ G pplx-decider-v1-27b 89.2%
200 Civil Comments o Toxicity classification Toxicity classification of online comments (Jigsaw Civil Comments). ★★★★★ G d1-omni-600M 95.8%
201 CL-Bench O Long-context reasoning Comprehensive long-context benchmark evaluating reasoning over extended contexts. ★★★★★ 🇺🇸 GPT-5.5 27.8%
202 CL-bench Life O Context learning Tencent Hy context-learning benchmark built from real-life scenarios (companion to CL-bench); models must learn and apply rules given in long, messy contexts. ★★★★★ 🇺🇸 GPT-5.5 21.1%
203 CLadder O Causal reasoning Causal reasoning questions across the ladder of causation; accuracy (Jev Decision Index suite, posed as typed decisions). ★★★★★ G Clef-Flash 97.7%
204 Claw-Eval O Agentic assistant tasks Benchmark of end-to-end OpenClaw-style personal-agent tasks; overall (EN) score. ★★★★★ 🇨🇳 GLM-5.1 84.4%
205 Claw-Eval Multimodal o Multimodal agent tasks Multimodal subset of Claw-Eval OpenClaw-style agent benchmark. ★★★★★ 🇺🇸 GPT-5.4 25.7%
206 Claw-Gym o Agentic assistant tasks OpenClaw-style personal-assistant agent environment suite measuring task success on daily tasks. ★★★★★ G Nanbeige4.2-3B 65.0%
207 ClawEval (Pass@3) o Agent tasks (OpenClaw) ClawEval OpenClaw agent benchmark, pass@3 (any of 3 trials succeeds). ★★★★★ 🇺🇸 Gemini 3.1 Pro 82.9%
208 ClawEval (Pass^3) O Agent tasks (OpenClaw) ClawEval OpenClaw agent benchmark, pass^3 (all 3 trials succeed). ★★★★★ 🇨🇳 DeepSeek V4-Flash-0731 78.9%
209 CLIcK O Korean instruction following Korean long-form instruction-following benchmark. ★★★★★ 🇨🇳 DeepSeek V4-Pro-Thinking (max) 91.6%
210 CLINC150+OOS O Intent classification CLINC150 intent detection with out-of-scope class; macro-F1 (Jev Decision Index suite, posed as typed decisions). ★★★★★ G Clef-Omni 97.7
211 CloningScenarios o Biosecurity refusal Safety benchmark that red-teams models with cloning-related misuse scenarios to measure compliance and refusal rates. ★★★★★ 🇺🇸 Grok 4 ↓ 45.0%
212 CLUEWSC O Coreference reasoning (Chinese) Chinese Winograd Schema-style coreference benchmark from CLUE. ★★★★★ 🇨🇳 DeepSeek R1 92.8%
213 CMath O Math (Chinese) Chinese mathematics benchmark. ★★★★★ G LLaDA2.0 Flash 96.9%
214 CMMLU O Chinese multi-domain Chinese counterpart to MMLU. ★★★★★ 781 🇨🇳 Qwen2.5 Max 91.9%
215 CMMMU o Chinese multimodal understanding Chinese Massive Multi-discipline Multimodal Understanding benchmark. ★★★★★ 🇺🇸 GPT-4V(ision) 42.5%
216 CMRC 2018 O Chinese reading comprehension Span-extraction Chinese machine reading comprehension (CMRC 2018). ★★★★★ G TigerBot-70B-Base 85.9%
217 CMT-Benchmark O Physics reasoning (condensed matter) Condensed Matter Theory benchmark of expert-written research-level theoretical physics problems. ★★★★★ G Sage-A1.5 80.0%
218 CNMO 2024 O Math (competition) China National Mathematical Olympiad 2024 evaluation set. ★★★★★ G openPangu-R-72B-2512 Slow Thinking 82.8%
219 COCO Captions o Image captioning COCO image captioning benchmark (Karpathy test split), CIDEr score. ★★★★★ 🇺🇸 T5Gemma 2 4B-4B 105.4
220 Codeforces O Competitive programming Competitive programming performance on Codeforces problems (ELO). ★★★★★ 🇨🇳 DeepSeek V4.1-Flash 3471
221 Codeforces (Percentile) O Competitive programming Competitive programming performance on Codeforces problems, reported as the percentile of human competitors outperformed. ★★★★★ G Ring-1T-preview 94.7%
222 CodeIF-Bench o Code instruction following Code-focused instruction following benchmark. ★★★★★ 🇨🇳 Qwen3-8B Non-Thinking 50.0%
223 CollegeMath O College-level math College-level mathematics problems (MathScale) covering algebra, calculus, linear algebra, probability, etc. ★★★★★ 🇨🇳 Qwen2.5-Math-72B-Instruct 49.5%
224 COLLIE O Instruction following Comprehensive instruction-following evaluation suite. ★★★★★ 55 🇺🇸 GPT-5 99.0%
225 Collie-Hard o Instruction following Hard subset of Collie instruction-following tasks. ★★★★★ 🇺🇸 GPT-5 (high) 99.0%
226 CommerceAgentBench O E-commerce agents Agentic benchmark of B2B/e-commerce sourcing and commerce workflows (Accio-Lab). ★★★★★ 🇺🇸 GPT-5.6 Sol 49.5%
227 CommonsenseQA O Commonsense QA Multiple-choice QA requiring commonsense knowledge. ★★★★★ 🇨🇳 Hunyuan-Large 92.9%
228 Complex Workflow o Complex workflows Complex workflow benchmark for economically valuable tasks. ★★★★★ 🇺🇸 Gemini 3 Pro 58.2%
229 ComplexFuncBench o Code reasoning Complex function reasoning and synthesis benchmark. ★★★★★ 🇺🇸 GPT-4o 66.5%
230 ContPhy O Video physical reasoning Continuum Physical Concept Learning and Reasoning from videos. ★★★★★ G Seed2.0 Pro 67.4%
231 Contract Scrub o Contract review Contract-review task: identify and redact/flag clauses in legal contracts (Thomson Reuters eval). ★★★★★ 🇺🇸 Gemma 4 31B (IT) 54.8%
232 ContractNLI O Legal NLI Document-level natural language inference over contracts; macro-F1 (Jev Decision Index suite, posed as typed decisions). ★★★★★ G Clef-Flash 84.3
233 ConvFinQA O Conversational QA Conversational numerical reasoning over financial reports (ChatRAG-Bench subset). ★★★★★ 🇺🇸 Claude Opus 4.6 86.5%
234 COPA O Causal reasoning Choice of Plausible Alternatives. ★★★★★ G Vega v2 99.4%
235 CoQA O Conversational QA Conversational question answering (ChatRAG-Bench subset). ★★★★★ G Yuan3.0 Ultra 94.6%
236 CORE o Ontological reasoning Comprehensive Ontological Relation Evaluation for Large Language Models. ★★★★★ G Nanbeige4.1-3B 53.5%
237 CorpFin o Corporate finance Corporate finance reasoning and numerical literacy benchmark. ★★★★★ 🇺🇸 Grok 4 Fast 73.3%
238 CorpusQA O Long-context QA Question answering over large text corpora. ★★★★★ 🇺🇸 GPT-5 81.6%
239 COSMOS (Cosmos-Reason benchmark) O Embodied reasoning NVIDIA Cosmos-Reason embodied physical-reasoning benchmark. ★★★★★ 🇺🇸 GPT-6 Astra 75.7%
240 CountBench O Visual counting Object counting and numeracy benchmark for visual-language models across varied scenes. ★★★★★ 🇨🇳 Qwen3.5-27B 97.8%
241 CountBenchQA O Visual counting QA Visual question answering benchmark focused on counting objects across varied scenes. ★★★★★ G dots.mocr 94.5%
242 Countdown o Planning and reasoning Countdown-style reasoning and planning benchmark. ★★★★★ G K2-V2 75.6%
243 Countix o Video counting Video-based counting benchmark for multiple objects. ★★★★★ G Seed1.8 31.0%
244 CountQA O Visual counting Visual question answering benchmark on object counting in real-world images. ★★★★★ 🇺🇸 GPT-6 Astra 86.4%
245 CoVoST o Speech translation CoVoST speech-to-text translation benchmark (BLEU). ★★★★★ 🇺🇸 Gemma 4 E4B IT 35.5
246 CRAG o Retrieval QA Complex Retrieval-Augmented Generation benchmark for grounded question answering. ★★★★★ G Jamba Mini 1.6 76.2%
247 Creative Story‑Writing Benchmark V3 O Creative writing Story writing benchmark evaluating creativity, coherence, and style (v3). ★★★★★ 291 🇨🇳 Kimi-K2-Instruct-0905 8.75
248 Longform Creative Writing O Creative writing Longform creative writing evaluation (EQ-Bench). ★★★★★ 20 🇺🇸 Claude Opus 5 86.3%
249 Creative Writing v3 O Creative writing A LLM-judged creative writing benchmark. ★★★★★ 54 🇺🇸 GPT-6 Astra 2173
250 Complex Research using Integrated Thinking – Physics Test O Reasoning CritPt (Complex Research using Integrated Thinking – Physics Test) benchmark. ★★★★★ 🇺🇸 GPT-5.6 Sol 32.3%
251 CrossTabBench Relation F1 o Cross-page tables CrossTabBench: F1 for detecting cross-page table continuation relations. ★★★★★ G OvisOCR3 97.1%
252 CrossTabBench Reconstruction TEDS o Cross-page tables CrossTabBench: TEDS of reconstructed cross-page tables. ★★★★★ 🇨🇳 Qwen3.8-Max 91.7%
253 CrossVid O Multi-video reasoning Benchmark for cross-video (multi-video) understanding and reasoning. ★★★★★ G Seed2.1 Pro 65.0%
254 CRPE O Relation hallucination Circular-based Relation Probing Evaluation (relation split) for object-relation hallucination. ★★★★★ 🇨🇳 InternVL2.5-26B 79.1%
255 CRUX-I O Code reasoning Code Reasoning and Understanding eXam – Interactive. ★★★★★ 🇺🇸 Gemini 3 Pro Preview 98.8%
256 CRUX-O O Code reasoning Code Reasoning and Understanding eXam – Offline. ★★★★★ G IQuest-Coder-V1-40B-Loop-Thinking 99.4%
257 CruxEval O Code reasoning Mathematical coding challenge set from the CruxEval benchmark. ★★★★★ 🇨🇳 Qwen3-32B Thinking 95.0%
258 CRUXEval (decision) O Code reasoning CRUXEval output prediction posed as choices among candidate outputs; accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. ★★★★★ G Clef-Omni 88.8%
259 CSimpleQA O QA Chinese SimpleQA benchmark variant (short factual questions). ★★★★★ G Ling 2.5 1T 79.0%
260 CultCat o Cultural knowledge Yandex 4-shot benchmark of (Russian) cultural facts. ★★★★★ G AliceAI-Foundation-80B-A3B-Base 86.5%
261 CursorBench 3.0 O Software engineering Cursor's coding-agent benchmark on ambiguous multi-file tasks from real Cursor sessions (v3.0, used in the Composer 2 technical report as CursorBench-3). ★★★★★ 🇺🇸 GPT-5.4 63.9%
262 CursorBench 3.2 o Software engineering Cursor's coding-agent benchmark on ambiguous multi-file tasks from real Cursor sessions (v3.2). ★★★★★ 🇺🇸 Claude Fable 5.1 73.4%
263 CursorBench 4.0 o Software engineering Cursor's coding-agent benchmark on ambiguous multi-file tasks from real Cursor sessions (v4.0). ★★★★★ 🇺🇸 Claude Opus 5.5 57.8%
264 Customer Support Q&A o Customer support QA Customer support question answering benchmark. ★★★★★ G Seed1.8 69.0%
265 CUTE o English characters CUTE aggregate capability score. ★★★★★ 🇺🇸 Bolmo 7B 78.6%
266 CV-Bench O Computer vision QA Diverse CV tasks for VLMs. ★★★★★ 🇺🇸 Gemini 3 Pro 92.0%
267 CVTG-2K CLIPScore o Text rendering CVTG-2K CLIPScore for text rendering in image generation. ★★★★★ G Seedream 4.5 0.807
268 CVTG-2K NED o Text rendering CVTG-2K normalized edit distance (NED) for text rendering. ★★★★★ 🇨🇳 GLM-Image 0.956
269 CVTG-2K Word Accuracy o Text rendering CVTG-2K word accuracy for text rendering in images. ★★★★★ 🇨🇳 GLM-Image 0.912
270 CWE-Bench O Cybersecurity CWE-Bench v1 vulnerability detection benchmark (cwe-bench.com, Collinear AI); pass@1. ★★★★★ 🇺🇸 Gemini 4 Argon 68.0%
271 ChestX-ray14 o Medical imaging NIH ChestX-ray14 classification, macro F1. ★★★★★ 🇺🇸 MedGemma 4B 50.1%
272 CyBench O Cybersecurity CTF Framework with 40 professional-level CTF tasks evaluating LLMs' practical cybersecurity capabilities. ★★★★★ 🇺🇸 Llama 3 70B ↓ 5.0%
273 CyberGym O Cybersecurity tasks Benchmark for cybersecurity-related coding and reasoning tasks. ★★★★★ G MiMo V2.6 Flash RL 95.1%
274 Cybersecurity Capture The Flag Challenges o Cybersecurity CTF Capture-the-flag challenge benchmark evaluating cybersecurity problem-solving skills. ★★★★★ 🇺🇸 GPT-5.6 Sol 96.7%
275 Cybersecurity CTF o Cybersecurity CTF Cybersecurity Capture The Flag challenges benchmark. ★★★★★ 🇺🇸 GPT-5.3 Codex 77.6%
276 D4LA o Layout analysis Diverse and detailed dataset for document layout analysis (12 document types, 27 categories). ★★★★★ G Infinity-Parser2-Pro 52.4%
277 DA-2K O Spatial reasoning 2D/3D spatial reasoning benchmark. ★★★★★ G HY-Embodied-0.5 MoT-2B 92.3%
278 Daily-Omni o Audio-visual QA Daily-Omni: audio-visual question answering on everyday videos with temporal alignment. ★★★★★ 🇺🇸 Gemini 3 Pro 84.2%
279 Deep Planning O Planning and reasoning Benchmark evaluating deep planning and multi-step reasoning capabilities. ★★★★★ 🇨🇳 Qwen3.7-Plus 62.3%
280 DeepCodeBench o Code QA Qodo's question-answering benchmark over real-world code repositories, requiring retrieval and reasoning across large codebases. ★★★★★ G openPangu-2.0-Flash Thinking 76.5%
281 DeepConsult o Agentic writing Agentic consulting and writing benchmark. ★★★★★ 🇺🇸 GPT-5 (high) 57.2%
282 DeepMind Mathematics O Math reasoning Synthetic math problem sets from DeepMind covering arithmetic, algebra, calculus, and more. ★★★★★ G Granite-4.1-30B 81.9%
283 DeepResearch Bench II o Deep research reports Second-generation DeepResearch Bench evaluating long-form deep-research reports with fine-grained rubrics. ★★★★★ G Xing4.0-29B-A4B 60.8%
284 DeepResearchBench O Agentic research writing Research-oriented agentic writing and planning benchmark. ★★★★★ G AgentCPM-Report 50.1%
285 DeepSearchQA O Deep web search QA Multi-step web search and question answering benchmark. ★★★★★ G Atria-Dawn-Preview 96.0%
286 DeepSearchQA (F1) O Deep web search QA DeepSearchQA multi-step web search benchmark, F1-score metric (the existing deepsearchqa entry is used for the accuracy metric). ★★★★★ 🇨🇳 Kimi-K3 95.0%
287 DeepShop o Web agents Online shopping web-agent benchmark with complex, deep product-search queries. ★★★★★ 🇺🇸 MolmoWeb-8B 42.3%
288 DeepSWE O Software engineering 113 long-horizon software engineering tasks written from scratch to avoid contamination (v1.1). ★★★★★ 🇺🇸 Claude Opus 5.5 74.2%
289 DeepSWE 1.0 O Software engineering Datacurve DeepSWE v1.0 long-horizon software-engineering tasks, run by Artificial Analysis with each provider's own harness (pass@1). ★★★★★ 🇺🇸 GPT-5.5 70.0%
290 DeepSWE v1.1 O Software engineering Original long-horizon software-engineering tasks in real codebases (Datacurve). ★★★★★ G Muse Wonder Full 79.0%
291 Defects4J O Program repair Real Java bugs from Defects4J used for automated program repair. ★★★★★ 🇺🇸 GPT-4o-2024-05-13 26.1%
292 DeR2 Bench o Long-context reasoning Dense retrieval and reasoning benchmark for long-context evaluation. ★★★★★ 🇺🇸 GPT-5.2 (high) 69.0%
293 Design2Code O Coding (UI) Translating UI designs into code. ★★★★★ 🇨🇳 GLM-5V-Turbo 94.8%
294 DesignArena O Generative design Leaderboard tracking generative design systems across layout, branding, and marketing tasks. ★★★★★ 🇺🇸 Claude Sonnet 4.5 (Thinking) 1410
295 DetailBench o Spot small mistakes Evaluates whether LLMs can notice subtle errors and minor inconsistencies in text. ★★★★★ 🇺🇸 Llama 4 Maverick 8.7%
296 DiscoX O Agentic writing DiscoX benchmark for agentic writing and reasoning. ★★★★★ G Seed2.0 Pro 82.0%
297 Do-Anything-Now o Safety / jailbreak Resistance to Do Anything Now (DAN) style jailbreak prompts. ★★★★★ G IQuest-Coder-V1-40B-Thinking 97.7%
298 Do-Not-Answer O Safety / refusal Evaluates a model's ability to refuse unsafe or disallowed requests. ★★★★★ 🇺🇸 Llama 2 7B Chat 99.4%
299 Doc2Dial O Conversational QA Document-grounded dialogue QA (ChatRAG-Bench subset). ★★★★★ G Yuan3.0 Ultra 55.8%
300 DocFigCapBench Caption Score o Document figure captioning DocFigCapBench: caption quality score (0-10 scale). ★★★★★ 🇨🇳 Qwen3.8-Max 7.30
301 DocFigCapBench Localization F1 o Document figure captioning DocFigCapBench: localization F1 of document figures. ★★★★★ G OvisOCR3 91.0%
302 DocLayNet o Layout analysis Human-annotated document layout segmentation dataset; layout analysis mIoU/mAP. ★★★★★ G PaddleOCR-VL-1.5 71.0%
303 DocMath O Document math Math reasoning on document-based problems. ★★★★★ 🇨🇳 Qwen3.5-35B-A3B 68.9%
304 Docmatix (multimodal RAG) o Multimodal RAG Docmatix document-VQA based multimodal retrieval-augmented QA evaluation. ★★★★★ G Yuan3.0 Ultra 67.4%
305 DocVQA O Document understanding (VQA) Visual question answering over scanned documents. ★★★★★ 🇨🇳 Seed1.5-VL-Thinking 96.9%
306 Dolphin-Page o Document OCR Dolphin Page benchmark measuring OCR fidelity and structured extraction on multi-layout documents. ★★★★★ 🇺🇸 Dolphin 1.5 ↓ 7.4%
307 DoQA O Conversational QA Domain-specific FAQ conversational QA (ChatRAG-Bench subset). ★★★★★ G Yuan3.0 Ultra 63.4%
308 DPG-Bench O Text rendering DPG-Bench score for text rendering in image generation. ★★★★★ G Seedream 4.5 88.6
309 DRACO O Deep research DRACO deep-research benchmark scored with per-question rubrics. ★★★★★ 🇺🇸 Claude Opus 5 88.6%
310 DREAM-1K o Video captioning DREAM-1K dense/detailed video description benchmark of 1,000 clips. ★★★★★ 🇺🇸 GPT-6 Astra 79.0%
311 DROP O Reading + reasoning Discrete reasoning over paragraphs (addition, counting, comparisons). ★★★★★ 🇨🇳 Kimi K2 Instruct 93.5%
312 DS-1000 o Data-science code generation DS-1000: 1000 data-science coding problems across 7 Python libraries. ★★★★★ 🇨🇳 DeepSeek-Coder-33B-base 40.2%
313 DSBench-FullStack (internal) o Coding DeepSeek internal full-stack development test set for coding agents. ★★★★★ 🇺🇸 Claude Fable 5 77.2%
314 DSBench-Hard (internal) o Coding DeepSeek internal test set of difficult coding-agent problems. ★★★★★ 🇺🇸 Claude Opus 4.8 71.7%
315 DUDE O Multimodal long-context Long-context multimodal understanding benchmark. ★★★★★ 🇺🇸 Claude Opus 4.7 84.5%
316 DynaMath O Math reasoning (video) Dynamic/video-based mathematical reasoning evaluating temporal and visual understanding. ★★★★★ 🇨🇳 Qwen3.6-Plus 88.0%
317 Economically important tasks o Industry QA (cross-domain) Evaluation suite of real-world, economically impactful tasks across key industries and workflows. ★★★★★ 🇺🇸 GPT-5 47.1%
318 EduBench English o Education (English) Yandex education benchmark built from Alice user queries: English language subset. ★★★★★ 🇨🇳 DeepSeek V4-Flash Base 82.9%
319 EduBench History o Education (history) Yandex education benchmark built from Alice user queries: history subset. ★★★★★ G AliceAI-Foundation-80B-A3B-Base 82.0%
320 EduBench Literature o Education (literature) Yandex education benchmark built from Alice user queries: literature subset. ★★★★★ G AliceAI-Foundation-80B-A3B-Base 73.8%
321 EduBench Math o Education (math) Yandex education benchmark built from Alice user queries: school math subset. ★★★★★ 🇨🇳 Qwen3.5-35B-A3B-Base 80.0%
322 EduBench Math University o Education (university math) Yandex education benchmark built from Alice user queries: university math subset. ★★★★★ G AliceAI-Foundation-80B-A3B-Base 70.1%
323 EduBench Russian o Education (Russian language) Yandex education benchmark built from Alice user queries: Russian language subset. ★★★★★ G AliceAI-Foundation-80B-A3B-Base 74.2%
324 Education o Economics/education Education field evaluation (economically valuable tasks). ★★★★★ G Seed1.8 60.8%
325 EEBench O Electrical engineering atopile's electrical-engineering agent benchmark: circuit/PCB design tasks expressed as declarative atopile code and verified automatically. ★★★★★ 🇺🇸 Claude Opus 5.5 75.0%
326 EGE o Russian state exam Part-A questions of the Russian Unified State Exam (EGE) across subjects. ★★★★★ G AliceAI-Foundation-80B-A3B-Base 90.5%
327 Ego3D-Bench (RMSE) o Driving spatial reasoning Ego-centric multi-view 3D spatial reasoning benchmark on outdoor driving scenes; metric = RMSE of numeric answers (lower is better). ★★★★★ 🇨🇳 Qwen-Drive-1.0-4B ↓ 7.78
328 EgoPlan-Bench2 O Embodied reasoning Egocentric embodied task-planning benchmark (v2). ★★★★★ 🇺🇸 GPT-6 Astra 69.3%
329 EgoSchema O Egocentric video QA EgoSchema validation accuracy. ★★★★★ 🇺🇸 GPT-5.2 81.2%
330 EgoTempo o Egocentric temporal reasoning Egocentric video temporal reasoning benchmark. ★★★★★ G Seed2.0 Pro 71.8%
331 EHRNoteQA o Medical records Clinical-note question answering over MIMIC-IV discharge summaries. ★★★★★ 🇺🇸 MedGemma 27B 90.7%
332 EIFBench o Instruction following Complex instruction-following benchmark. ★★★★★ 🇺🇸 GPT-5 (high) 66.7%
333 EmbSpatialBench O Spatial understanding Embodied spatial understanding benchmark evaluating navigation and localization. ★★★★★ 🇨🇳 Qwen3.5-Omni-Plus 85.4%
334 EMMA O Multimodal reasoning EMMA benchmark for multimodal reasoning. ★★★★★ G Seed2.1 Pro 79.3%
335 Enamel O Composite capability Composite capability benchmark capturing broad model performance (Enamel score). ★★★★★ G Rnj-1.5 50.7%
336 EnConda-Bench o Code editing English code editing benchmark for applying conditional modifications. ★★★★★ G Youtu-LLM-2B 21.5%
337 Encyclo-K o Encyclopedic knowledge Encyclopedic knowledge evaluation benchmark. ★★★★★ G Seed2.0 Pro 65.7%
338 EnigmaEval O Challenging puzzles Challenging puzzle benchmark. ★★★★★ 🇺🇸 Gemini 3 Pro 17.8%
339 Enterprise RAG o Retrieval-augmented generation Enterprise retrieval-augmented generation evaluation covering internal knowledge bases. ★★★★★ 🇺🇸 Apriel Nemotron 15B Thinker 69.2%
340 EnterpriseOps-Gym o Enterprise agent tasks Agentic benchmark of enterprise operations workflows with tool use. ★★★★★ G Spark-X2.6-4B 33.6%
341 EQ-Bench O Reasoning General reasoning benchmark assessing equation/logic capabilities. ★★★★★ 352 🇺🇸 Claude 3.5 Sonnet 20240620 86.4%
342 EQ-Bench 3 O Emotional intelligence (roleplay) A benchmark measuring emotional intelligence in challenging roleplays, judged by Sonnet 3.7. ★★★★★ 21 🇺🇸 Claude Fable 5 2050
343 ERQA O Spatial reasoning Spatial recognition and reasoning QA benchmark (ERQA). ★★★★★ G Glimpse 83.8%
344 ERQA-PLUS O Embodied reasoning Extended Embodied Reasoning QA benchmark. ★★★★★ 🇺🇸 Gemini 3.6 Flash 92.2%
345 E.T. Bench o Video event understanding E.T. Bench: event-level and time-sensitive video understanding benchmark. ★★★★★ G Glimpse 63.5%
346 EvalPerf O Code evaluation performance Measures performance of LLM code evaluation, including runtime, memory, and efficiency metrics. ★★★★★ 🇺🇸 GPT-4o (2024-08-06) 100.0%
347 EvalPlus O Code generation Aggregated code evaluation suite from EvalPlus. ★★★★★ 1577 🇺🇸 o1 Mini 89.0%
348 EVG o Document OCR EVG document OCR benchmark evaluating recognition accuracy and layout extraction. ★★★★★ 🇺🇸 Dolphin 1.5 ↓ 3.0%
349 EXECUTE o Multilingual character tasks Multilingual character-level evaluation benchmark. ★★★★★ 🇺🇸 Bolmo 7B 71.6%
350 ExpertFactsQA Law o Expert knowledge (law) Yandex expert-written factual QA benchmark, law (hard) subset. ★★★★★ G AliceAI-Foundation-80B-A3B-Base 49.6%
351 ExpertFactsQA Medicine o Expert knowledge (medicine) Yandex expert-written factual QA benchmark, medicine subset. ★★★★★ G AliceAI-Foundation-80B-A3B-Base 63.6%
352 ExploitBench O Cybersecurity Developing V8 vulnerabilities into increasingly capable exploits (capability coverage). ★★★★★ 🇺🇸 GPT-6 Astra 100.0%
353 ExploitBench (Jun-Aug 2026) o Cybersecurity ExploitBench restricted to vulnerabilities disclosed June-August 2026, after model training cutoffs. ★★★★★ 🇺🇸 GPT-6 Astra 39.0%
354 ExploitGym O Cybersecurity UC Berkeley-led benchmark: agents turn real known vulnerabilities into working exploits achieving arbitrary code execution. ★★★★★ 🇺🇸 GPT-6 Astra 42.4%
355 ExploitGym-Honeypot o Safety ExploitGym honeypot variant measuring how often agents take bait out of scope (lower is better). ★★★★★ 🇺🇸 GPT-6 Astra
356 FACTS Benchmark Suite O Held out internal grounding, parametric, MM, and search retrieval benchmarks Comprehensive factuality benchmark suite covering held-out internal grounding, parametric knowledge, multimodal understanding, and search retrieval benchmarks. ★★★★★ 🇺🇸 Gemini 3 Pro 70.5%
357 FACTS Grounding O Grounding / factuality Grounded factuality benchmark evaluating model alignment with source facts. ★★★★★ 🇨🇳 Kimi K2 Instruct 88.5%
358 FACTS Parametric o Factuality FACTS suite parametric-knowledge split: closed-book factoid questions answered without retrieval. ★★★★★ 🇨🇳 DeepSeek V4-Pro Base 62.6%
359 FActScore O Factuality on open-source prompts Measures factuality on an open-source prompt suite; scored as 100 - hallucination rate, higher is better. ★★★★★ 🇺🇸 GPT-5 99.0%
360 FairPoliticsQA o Political fairness (Japanese) Sakana AI benchmark of balanced, fair answers to Japanese political questions. ★★★★★ G Sakana Namazu 56.3%
361 FaithEval (Inconsistent) o Context faithfulness FaithEval inconsistent-context subset: detecting contradictory evidence in retrieved context. ★★★★★ G Thomson-1.0-Small 97.7%
362 FaithJudge (1-Hallu.) o Hallucination detection FaithJudge hallucination rate with 1-hallucination metric (lower is better). ★★★★★ G Moonlight-Instruct ↓ 56.0%
363 Meta Score Agent O Composite capability index ★★★★★ 🇺🇸 Claude Opus 4.5 100.0%
364 Meta Score Code O Composite capability index ★★★★★ 🇺🇸 Claude Opus 4.5 100.0%
365 Meta Score Math O Composite capability index ★★★★★ 🇨🇳 Qwen3-VL-235B-A22B Instruct 100.0%
366 Meta Score OCR O Composite capability index ★★★★★ 🇺🇸 o3 (low) 80.0%
367 Meta Score Overall O Composite capability index Overall Meta Score across the benchmarks of all Meta Score categories. ★★★★★ 🇨🇳 Qwen3.8-Max 51.7%
368 Meta Score Safety O Composite safety index ★★★★★ G Granite 3.3 8B Instruct 70.0%
369 Meta Score STEM O Composite capability index ★★★★★ 🇨🇳 Qwen3-VL-235B-A22B Instruct 100.0%
370 Meta Score Text O Composite capability index ★★★★★ 🇺🇸 Claude Opus 4.5 85.7%
371 Meta Score Visual O Composite capability index ★★★★★ 🇺🇸 Gemini 3 Pro 100.0%
372 Meta Score Writing O Composite capability index ★★★★★ 🇨🇳 Qwen3-235B-A22B-Instruct-2507 60.0%
373 FATE-H o Formal theorem proving (abstract algebra) FATE-H: graduate-level abstract algebra problems formalized in Lean 4 (100 problems). ★★★★★ G Leanstral 1.5 87.0%
374 FATE-X o Formal theorem proving (abstract algebra) FATE-X: PhD-level abstract algebra problems formalized in Lean 4 (100 problems). ★★★★★ G Leanstral 1.5 34.0%
375 FD-Bench V1.5 o Full-duplex speech interaction Full-duplex spoken dialogue benchmark v1.5 (interruption handling, backchannel, turn-taking quality); average score. ★★★★★ G TML-Interaction-Small 77.8%
376 FD-Bench V3 o Full-duplex speech + tool use Full-duplex spoken dialogue benchmark v3 with tool calling; response quality (%). ★★★★★ G TML-Interaction-Small 82.8%
377 FeatBench o Code (feature implementation) Benchmark of feature implementation in real repositories from natural-language requirements (vibe coding). ★★★★★ G openPangu-2.0-Pro 48.4%
378 FeTaQA o Table QA Free-form table QA (MMTab subset; BLEU). ★★★★★ G Yuan3.0 Ultra 39.2%
379 FigQA o Figure understanding and QA Figure question answering benchmark evaluating visual reasoning over scientific figures and diagrams. ★★★★★ 🇺🇸 Claude Mythos 5 88.9%
380 FinanceReasoning o Financial reasoning Financial reasoning benchmark evaluating quantitative and qualitative finance problem solving. ★★★★★ G Ling 1T 87.5%
381 FinanceAgent O Agentic finance tasks Interactive financial agent benchmark requiring multi-step tool use. ★★★★★ 🇺🇸 Claude Opus 4.6 60.7%
382 FinanceAgent v1.1 O Agentic finance tasks Finance Agent v1.1 benchmark for interactive financial agent evaluation. ★★★★★ 🇺🇸 Gemini 3.7 Flash 71.0%
383 FinanceAgent v2 O Agentic finance tasks Vals AI Finance Agent Benchmark v2: agentic financial-research tasks over SEC filings; new dataset, not comparable to v1.1. ★★★★★ 🇺🇸 Gemini 4 Argon 65.4%
384 FinanceBench (FullDoc) o Finance QA FinanceBench full-document question answering benchmark requiring long-context financial understanding. ★★★★★ G Jamba Mini 1.6 45.4%
385 FinancialPhraseBank o Financial sentiment Sentence-level financial news sentiment classification. ★★★★★ G pplx-decider-v1-27b 84.2%
386 FinEntity (decision) O Financial sentiment Entity-level financial sentiment; macro-F1. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. ★★★★★ G Clef-Flash 97.1
387 FinQA 128k o Long-context financial QA Long-context (128k) variant of FinQA: analytics questions over financial reports. ★★★★★ G AliceAI-Foundation-80B-A3B-Base 74.1%
388 FinSearchComp O Financial retrieval Financial search and comprehension benchmark measuring retrieval grounded reasoning over financial content. ★★★★★ G Seed2.0 Pro 70.2%
389 FinSearchComp-CN O Financial retrieval (Chinese) Chinese financial search and comprehension benchmark measuring retrieval-grounded reasoning over regional financial content. ★★★★★ G doubao-1-5-vision-pro 54.2%
390 FinSearchComp (T2&T3) o Finance search Finance search competition tasks (tracks T2 and T3). ★★★★★ 🇺🇸 GPT-5 (high) 64.5%
391 Flame-React-Eval O Frontend coding Front-end React coding tasks and evaluation. ★★★★★ 🇺🇸 Claude Opus 4.6 98.8%
392 FLEURS o Speech recognition FLEURS multilingual automatic speech recognition benchmark; word error rate, lower is better. ★★★★★ 🇺🇸 Gemma 4 E4B IT ↓ 0.1%
393 Flickr30K Captions o Image captioning Flickr30K image captioning (Karpathy test split, CIDEr). ★★★★★ 🇨🇳 Qwen-VL 85.8
394 Flores O Machine translation (multilingual) FLORES multilingual translation benchmark. ★★★★★ G EuroLLM-22B 88.9%
395 FLTEval o Formal proof engineering Mistral benchmark built from real pull requests to the Fermat's Last Theorem Lean repository: complete all formal proofs and new definitions per PR. ★★★★★ 🇺🇸 Claude Opus 4.6 39.6%
396 ForecastBench (decision) o Forecasting ForecastBench questions answered as probabilities; Brier score x100 (lower is better). Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. ★★★★★ G Clef-Flash ↓ 10.6
397 FORTE o General agent AGI-Eval FORTE general agentic task benchmark. ★★★★★ 🇺🇸 GPT-5.5 77.8%
398 FORTRESS (Adversarial) O Safety Scale AI FORTRESS adversarial prompts (CBRNE, political violence, crime); safety score. ★★★★★ 🇺🇸 Claude Fable 5 96.0%
399 FORTRESS (Benign) O Safety (over-refusal) Scale AI FORTRESS benign counterpart prompts; share answered without over-refusal. ★★★★★ 🇨🇳 DeepSeek V4-Flash-Thinking (max) 99.2%
400 Fox-Page-cn o Document OCR (Chinese) Fox Page benchmark evaluating OCR accuracy and layout understanding on Chinese document pages. ★★★★★ 🇺🇸 Dolphin 1.5 ↓ 0.8%
401 Fox-Page-en o Document OCR (English) Fox Page benchmark evaluating OCR accuracy and layout understanding on English document pages. ★★★★★ 🇺🇸 Dolphin 1.5 ↓ 0.7%
402 fr-bench-pdf2md o Document parsing French PDF-to-Markdown conversion benchmark (baseline, forms, graphics, handwritten, long tables, multi-column, tiny text); overall score. ★★★★★ G LightOnOCR-3-4B 74.1%
403 FRAMES O Interactive reasoning Frame-based interactive reasoning and dialogue benchmark. ★★★★★ 🇨🇳 Tongyi DeepResearch 90.6%
404 FreshQA o Recency QA Question answering benchmark emphasizing up-to-date knowledge and recency. ★★★★★ 🇨🇳 Qwen3-4B Thinking 2507 66.9%
405 Frontier-CS O Algorithmic / open-ended CS problems Frontier-CS agent track (188 tasks): open-ended computer-science optimization problems scored by verifiable feedback. ★★★★★ 🇺🇸 GPT-5.6 Sol 76.4%
406 FrontierBench O Agentic terminal tasks Successor to Terminal-Bench 2.1 (v0.1): 74 harder terminal tasks emphasizing science and engineering. ★★★★★ 🇺🇸 Claude Opus 5 43.3%
407 FrontierCode o Software engineering Cognition's agentic coding benchmark of 150 tasks from real PRs, graded on hidden tests plus code-quality rubric (Main set, v1.1). ★★★★★ 🇺🇸 Claude Opus 5.5 54.4%
408 FrontierCode (Diamond) o Software engineering Cognition's FrontierCode agentic coding benchmark, Diamond subset; patch correctness graded by held-out tests plus rubric. ★★★★★ 🇺🇸 Claude Fable 5 29.3%
409 FrontierCode Extended O Software engineering Cognition FrontierCode 1.1 Extended set (score). ★★★★★ 🇺🇸 Claude Opus 5.5 65.3%
410 FrontierCode Main O Software engineering Cognition FrontierCode 1.1 Main: coding-agent changes graded on correctness and mergeability (tests, scope, style, conventions). ★★★★★ 🇺🇸 Claude Opus 5.5 54.6%
411 FrontierFinance O Finance Professional finance benchmark combining quantitative reasoning with analysis of financial data and documents (reported in Step 5 Preview release). ★★★★★ 🇺🇸 Claude Opus 5 69.7%
412 FrontierMath o Math (frontier) Frontier mathematics benchmark. ★★★★★ 🇺🇸 GPT-5 pro 32.1%
413 FrontierScience O Science reasoning Frontier-level scientific reasoning and QA benchmark. ★★★★★ G MAI-Code-1-Flash 58.2%
414 FrontierScience Olympiad O Science reasoning (olympiad) Olympiad-level problems from the FrontierScience benchmark. ★★★★★ G Intern-S2-397B 87.0%
415 FrontierScience Research O Science reasoning (research) Research-level problems from the FrontierScience benchmark. ★★★★★ G Apodex 1.1 63.3%
416 FrontierSWE O Software engineering Proximal's FrontierSWE benchmark of frontier-difficulty software engineering tasks; dominance score (snapshot-dependent). ★★★★★ 🇺🇸 Claude Fable 5 88.8%
417 FSC-147 O Few-shot counting Few-shot counting benchmark across 147 categories. ★★★★★ G Glimpse 83.4%
418 FullStackBench O Full-stack development End-to-end web/app development tasks and evaluation. ★★★★★ 🇺🇸 Claude Opus 4.5 72.3%
419 FullStackBench (zh) o Full-stack development Chinese language full-stack development tasks and evaluation. ★★★★★ 🇨🇳 Qwen3-235B-A22B 63.1%
420 FutureOmni o Video forecasting FutureOmni: omni-modal (audio-visual) future event forecasting benchmark from video. ★★★★★ G MiMo V2 Omni 66.7%
421 GAIA O General AI tasks Comprehensive benchmark for agentic tasks. ★★★★★ 🇨🇳 DeepSeek V4-Pro-Thinking (max) 98.1%
422 GAIA (no file) o General AI tasks GAIA benchmark subset without file inputs. ★★★★★ G Step-3.5 Flash 20260204 84.5%
423 GAIA (Text Only) O General AI tasks GAIA benchmark subset restricted to text-only inputs. ★★★★★ G XYZ-Aquila-mini 97.1%
424 GAIA 2 O General agent tasks Grounded agentic intelligence benchmark version 2 covering multi-tool tasks. ★★★★★ G Ring-1T-2.5 75.0%
425 GaoKao 2023 En O Math (exam) English translation of 2023 Chinese college entrance exam (Gaokao) math questions, used in Qwen2.5-Math / DeepSeekMath style evaluations. ★★★★★ 🇨🇳 Qwen2.5-Math-72B-Instruct 71.9%
426 GAOKAO-Bench O Chinese exams GAOKAO benchmark measuring Chinese college entrance exam performance. ★★★★★ 🇨🇳 Qwen3-30B-A3B-Instruct-2507 94.5%
427 Gaokao Math 2026 o Math exams Five 2026 Chinese Gaokao math exams (National I, II, Beijing, Shanghai, Tianjin); average score out of 150. ★★★★★ 🇨🇳 Qwen3.5-9B 136
428 Gaokao-MathCloze o Chinese math (exam) Fill-in-the-blank math problems from Chinese Gaokao (AGIEval). ★★★★★ 🇨🇳 DeepSeek Math 7B 20.3%
429 Gaokao-MathQA o Chinese math (exam) Multiple-choice math problems from Chinese Gaokao (AGIEval). ★★★★★ 🇨🇳 DeepSeek Math 7B 35.3%
430 GDP.pdf O Document understanding Surge AI multimodal benchmark of 100 professional prompts over information-dense PDFs; strict pass rate. ★★★★★ 🇺🇸 GPT-5.6 Sol 40.0%
431 GDPVal O General capability GDPVal benchmark evaluating broad general capabilities of LLMs across diverse tasks. ★★★★★ 🇺🇸 Claude Opus 5 89.4%
432 GDPVal-AA Elo O Office tasks GDPVal Artificial Analysis Elo rating for office-style tasks. ★★★★★ 🇺🇸 Claude Fable 5 1932
433 GDPval-AA v2 Elo O Office tasks Artificial Analysis GDPval-AA v2 leaderboard Elo (re-based; not comparable to v1 GDPval-AA Elo). ★★★★★ 🇺🇸 Claude Opus 5 1861
434 GDPval (rubrics) O Professional knowledge work GDPval tasks scored pointwise against rubrics (not pairwise win rate), as reported by MiniMax. ★★★★★ 🇨🇳 Kimi-K3 91.6%
435 GeneBench o Scientific data analysis OpenAI eval of multi-step scientific data analysis in genetics and quantitative biology, with ambiguous or flawed data and hidden confounders. ★★★★★ 🇺🇸 GPT-5.5 Pro 33.2%
436 GeneBench Pro o Biology agent tasks Long-horizon genomics and quantitative-biology analysis tasks. ★★★★★ 🇺🇸 GPT-6 Astra 37.1%
437 General Tool Use O Tool use General tool-use benchmark covering web and API tasks. ★★★★★ 🇺🇸 Claude Opus 4.5 78.9%
438 GeoBench1 o Geospatial reasoning Geospatial visual QA and reasoning (set 1). ★★★★★ 🇨🇳 GLM-4.5V 79.7%
439 GlobalMGSM o Math (multilingual) Global multilingual grade school math word problems. ★★★★★ 🇨🇳 Qwen3-4B 60.9%
440 Global-MMLU O Multi-domain knowledge (global) Full Global-MMLU evaluation across diverse languages and regions. ★★★★★ 🇺🇸 Claude Opus 5.5 94.3%
441 Global-MMLU-Lite O Multi-domain knowledge (global) Lightweight global variant of MMLU covering diverse languages and regions. ★★★★★ 🇺🇸 Claude Fable 5 93.3%
442 Global PIQA O Commonsense reasoning across 100 Languages and Cultures Physical commonsense reasoning benchmark spanning 100 languages and diverse cultural contexts. ★★★★★ 🇺🇸 Gemini 3 Flash 95.6%
443 GLUE O Multi-task NLU General Language Understanding Evaluation. ★★★★★ 🇺🇸 Turing ULR v6 91.3%
444 Gorilla Benchmark API Bench o Tool use Gorilla API Bench tool-use evaluation. ★★★★★ 🇺🇸 Llama 3.1 405B 35.3%
445 GPQA O Graduate-level QA Graduate-level question answering evaluating advanced reasoning. ★★★★★ 406 🇺🇸 GPT-5.2 Thinking 92.4%
446 GPQA-diamond O Graduate-level QA Hard subset of GPQA (diamond level). ★★★★★ 🇺🇸 GPT-6 Astra 96.0%
447 GQA o Visual reasoning VQA Compositional question answering over real-world images (scene-graph based); test-dev balanced accuracy. ★★★★★ 🇨🇳 Qwen-VL 59.3%
448 GraphWalks BFS O Long-context reasoning Graph traversal/GraphWalks benchmark (BFS variant) for long-context reasoning. ★★★★★ 🇺🇸 Gemini 4 Argon 99.7%
449 GraphWalks BFS 1M O Long-context reasoning GraphWalks BFS long-context graph traversal at 1M context (F1). ★★★★★ 🇺🇸 Claude Mythos 5 79.4%
450 GraphWalks BFS 256K O Long-context reasoning GraphWalks BFS long-context graph traversal at 256K context (F1). ★★★★★ G Spark-X2.6-4B 92.2%
451 GraphWalks BFS 256K-1M o Long-context reasoning GraphWalks BFS long-context graph traversal at 256K-1M context (F1). ★★★★★ 🇺🇸 Gemini 4 Argon 84.2%
452 GraphWalks Parents O Long-context reasoning Graph traversal/GraphWalks benchmark (Parents variant) for long-context reasoning. ★★★★★ G Seed2.0 Lite 100.0%
453 GraphWalks Parents 1M o Long-context reasoning GraphWalks Parents long-context graph task at 1M context (F1). ★★★★★ 🇺🇸 Claude Opus 4.6 72.0%
454 GraphWalks Parents 256K o Long-context reasoning GraphWalks Parents long-context graph traversal at 256K context (F1). ★★★★★ 🇺🇸 Claude Mythos 5 100.0%
455 GRE Math maj@16 o Math (standardized tests) GRE quantitative section evaluated via majority voting over 16 samples. ★★★★★ 🇨🇳 Qwen2 7B 58.5%
456 Ground-UI-1K O GUI grounding Accuracy on the Ground-UI-1K grounding benchmark. ★★★★★ G Holo2-235B-A22B 85.5%
457 GSM-Infinite Hard (128K) o Math reasoning GSM-Infinite Hard benchmark at 128K context. ★★★★★ G MiMo V2 Flash Base 29.0%
458 GSM-Infinite Hard (16K) o Math reasoning GSM-Infinite Hard benchmark at 16K context. ★★★★★ 🇨🇳 DeepSeek V3.2-Exp 50.4%
459 GSM-Infinite Hard (32K) o Math reasoning GSM-Infinite Hard benchmark at 32K context. ★★★★★ 🇨🇳 DeepSeek V3.2-Exp 45.2%
460 GSM-Infinite Hard (64K) o Math reasoning GSM-Infinite Hard benchmark at 64K context. ★★★★★ 🇨🇳 DeepSeek V3.1 34.7%
461 GSM-Plus O Math (grade-school, enhanced) Enhanced GSM-style grade-school math benchmark variant. ★★★★★ 🇨🇳 Qwen3.5-9B 91.4%
462 GSM-Symbolic o Math reasoning Symbolic reasoning variant of GSM that tests algebraic manipulation and arithmetic with structured problems. ★★★★★ G Granite-4.0-H-Small 87.4%
463 GSM8K O Math (grade-school) Grade-school math word problems requiring multi-step reasoning. ★★★★★ 1322 G MiMo V2.5 Pro Base 99.6%
464 GSM8K (DE) o Math (grade-school, German) German translation of the GSM8K grade-school math word problems. ★★★★★ 🇺🇸 Llama-3_1-70B-TFree-HAT-SFT 0.630
465 GSM8K (decision) O Math (grade-school) GSM8K posed as choices among candidate answers; accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. ★★★★★ G vLLM-SR d3 85.2%
466 GSM8K (Japanese) O Math (grade-school, Japanese) Japanese translation of GSM8K grade-school math. ★★★★★ 🇺🇸 LFM2.5 1.2B JP 202606 62.2%
467 GSM8K-Ko o Math (grade-school, Korean) Korean translation of the GSM8K grade-school math word problems. ★★★★★ 🇨🇳 Qwen3-30B-A3B 88.1%
468 GSM8K Platinum o Math (grade-school, hard) Harder subset/setting of GSM8K grade-school math problems. ★★★★★ 🇨🇳 Kimi-Linear-Base 89.6%
469 GSO Benchmark O Code generation LiveCodeBench GSO benchmark. ★★★★★ 🇺🇸 Claude Fable 5.1 88.2%
470 Habermas Machine (decision) O Deliberation / preference Predict group preference among candidate consensus statements; accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. ★★★★★ G Clef-Flash 71.8%
471 HAE-RAE Bench O Korean language understanding Korean language understanding benchmark evaluating knowledge and reasoning. ★★★★★ G Kanana-1.5-32.5B-Base 90.7%
472 HallusionBench O Multimodal hallucination Benchmark for evaluating hallucination tendencies in multimodal LLMs. ★★★★★ 🇨🇳 Qwen3-VL 32B Thinking 76.6%
473 Harbor-Index o Agentic coding Aggregate index of agentic terminal/coding tasks run through the Harbor evaluation framework. ★★★★★ 🇺🇸 Claude Opus 5 56.9%
474 HardMultiQA o Russian factual knowledge Yandex 5-shot benchmark of hard multi-answer factual questions in Russian. ★★★★★ G AliceAI-Foundation-80B-A3B-Base 67.9%
475 HarmBench O Safety Harmfulness and safety compliance benchmark across a variety of risky prompts. ★★★★★ 🇺🇸 GPT-realtime-1.5 100.0%
476 HarmfulQA o Safety Harmful question set testing models' ability to avoid unsafe answers. ★★★★★ 104 G K2-THINK 99.0%
477 Harvey LAB-AA O Legal agent tasks Harvey's Legal Agent Benchmark as run by Artificial Analysis (LAB-AA): complex legal workflows score. ★★★★★ 🇨🇳 Kimi-K3 94.6%
478 HealthBench O Medical QA Comprehensive medical knowledge and clinical reasoning benchmark across specialties and tasks. ★★★★★ 🇺🇸 GPT-5 67.2%
479 HealthBench-Hard O Medical QA (hard) Challenging subset of HealthBench focusing on complex, ambiguous clinical cases. ★★★★★ 🇺🇸 GPT-5 46.2%
480 HealthBench-Hard Hallucinations o Medical hallucination safety Measures hallucination and unsafe medical advice under hard clinical scenarios. ★★★★★ 🇺🇸 GPT-5 ↓ 1.6%
481 HealthBench Professional O Medical QA 525 physician-authored clinical conversations (consults, documentation, research) graded against rubric criteria. ★★★★★ G OpenEvidence Darwin 82.7%
482 HellaSwag O Commonsense reasoning Adversarial commonsense sentence completion. ★★★★★ 220 G Clef-Omni 99.1%
483 HellaSwag (DE) o Commonsense reasoning (German) German translation of the HellaSwag commonsense benchmark. ★★★★★ 🇺🇸 Llama-3_1-70B-TFree-HAT-SFT 0.727
484 HELMET LongQA o Long-context QA Long-context subset of the HELMET benchmark focusing on grounded question answering. ★★★★★ G Jamba Mini 1.6 46.9%
485 HelpSteer2 o Response quality rating NVIDIA HelpSteer2 helpfulness/attribute rating, posed as decisions. ★★★★★ G Decider 4B (Mapika) 42.0%
486 HeroBench O Long-horizon planning Benchmark for Long-Horizon Planning and Structured Reasoning in Virtual Worlds. ★★★★★ 🇺🇸 Grok 4 91.7%
487 HHEM v2.1 O Hallucination detection Hughes Hallucination Evaluation Model (Vectara) — lower is better. ★★★★★ G AntGroup Finix_S1_32b ↓ 0.6%
488 HiddenMath O Math reasoning Mathematical reasoning benchmark referenced in recent model cards. ★★★★★ 🇺🇸 Gemini 2.0 Pro 65.2%
489 HiPhO O Physics olympiad (multimodal) High-school physics olympiad problems with diagrams for evaluating multimodal physics reasoning. ★★★★★ 🇨🇳 Qwen3.8-Max 90.0%
490 HiTab o Table QA Hierarchical table QA (MMTab subset). ★★★★★ 🇺🇸 GPT-5.1 77.8%
491 HLE O Multi-domain reasoning Challenging LLMs at the frontier of human knowledge. ★★★★★ 1085 🇺🇸 Claude Opus 5.5 64.4%
492 HLE-Diamond o Multidisciplinary reasoning Hard 'Diamond' subset of Humanity's Last Exam, evaluated without tools. ★★★★★ G Muse Spark Open 26.1%
493 HLE-MultiChoice O Advanced reasoning The 513 text-only original multiple-choice questions of Humanity's Last Exam, posed as one choice among the lettered answers. ★★★★★ 🇺🇸 Claude Opus 5.5 (max) 70.2%
494 HLE Overconfidence O Overconfidence / safety Overconfidence rate derived from Humanity's Last Exam evaluations. ★★★★★ 🇺🇸 GPT-5.2 ↓ 43.7%
495 HLE (Text Only) O Advanced reasoning Humanity's Last Exam benchmark restricted to text-only inputs. ★★★★★ 1085 🇺🇸 Claude Fable 5 53.3%
496 HLE-Verified O Multi-domain reasoning Verified and revised version of Humanity's Last Exam (HLE) with component-wise verification protocol. ★★★★★ 🇺🇸 Gemini 3.8 Flash 54.9%
497 HLE-VL o Holistic language evaluation (vision-language) Vision-language HLE benchmark. ★★★★★ 🇺🇸 Gemini 3 Pro 36.0%
498 HLE (With Tools) O Tool-augmented reasoning Humanity's Last Exam benchmark evaluated with tool access. ★★★★★ 1085 🇺🇸 Claude Opus 5.5 67.7%
499 HMMT o Math (competition) Harvard–MIT Mathematics Tournament problems. ★★★★★ 🇺🇸 GPT-5 pro 100.0%
500 HMMT 2025 O Math (competition) Harvard–MIT Mathematics Tournament 2025 problems. ★★★★★ 🇺🇸 Gemini 3 Pro 99.8%
501 HMMT Feb 2025 O Math (competition) Harvard–MIT Mathematics Tournament February 2025 problems. ★★★★★ 🇺🇸 GPT-5.2 (high) 100.0%
502 HMMT Feb 2026 O Math (competition) Harvard–MIT Mathematics Tournament February 2026 problems. ★★★★★ G Darwin-180B-RSI 100.0%
503 HMMT Nov 2025 O Math (competition) Harvard–MIT Mathematics Tournament November 2025 problems. ★★★★★ 🇺🇸 GPT-5.2 (high) 100.0%
504 Home appliance simulator (decision) O Device control Multi-step home appliance control decisions; case exact accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. ★★★★★ G Bespoke-Nimble-9B-v3 98.9%
505 HotpotQA o Multi-hop QA Explainable multi-hop QA with supporting facts. ★★★★★ 🇨🇳 Qwen3-0.6B 64.0%
506 HoVer O Claim verification Multi-hop claim verification over Wikipedia introductions; accuracy (Jev Decision Index suite, posed as typed decisions). ★★★★★ G AutoTrust JEV-35B 88.1%
507 HRBench 4K O Hallucination robustness Hallucination robustness benchmark with 4K token contexts. ★★★★★ G SenseNova-MARS-32B 90.2%
508 HRBench 8K O Hallucination robustness Hallucination robustness benchmark with 8K token contexts. ★★★★★ G SenseNova-MARS-32B 86.6%
509 HRM8K O Korean reasoning 8k-question Korean reasoning and knowledge benchmark. ★★★★★ 🇨🇳 DeepSeek V4-Flash-Thinking (max) 93.4%
510 HSS o Scene understanding HSS scene-understanding benchmark over images and video. ★★★★★ 🇺🇸 GPT-6 Astra 53.6%
511 HumanEval O Code generation Python synthesis problems evaluated by unit tests. ★★★★★ 2916 🇺🇸 Gemini 3 Pro Preview 100.0%
512 HumanEval-FIM o Code infilling Fill-in-the-middle variant of HumanEval measuring code infilling. ★★★★★ 🇫🇷 Codestral 22B 91.6%
513 HumanEval+ O Code generation Extended HumanEval with more tests. ★★★★★ 1577 🇨🇳 Qwen3.8-27B-UD-Q4_K_XL 95.7%
514 HumanEval-V o Code generation (vision) HumanEval variant with visual programming prompts. ★★★★★ G Step3-VL-10B 66.0%
515 HumanEval-X O Code generation (multilingual) Multilingual code generation benchmark extending HumanEval to multiple programming languages. ★★★★★ G TeleChat3-36B-Thinking 92.7%
516 Humicroedit (decision) O Humor judgement Pick the funnier micro-edited headline; accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. ★★★★★ G Clef-Flash 75.1%
517 Hungarian National HS Math Exam O Math (held-out exam) Hungarian national high-school mathematics finals (2023) graded by humans; held-out math reasoning test. ★★★★★ 🇺🇸 GPT-4 68.0%
518 HybriDial O Conversational QA Hybrid table+text grounded dialogue QA (ChatRAG-Bench subset). ★★★★★ G Yuan3.0 Ultra 72.9%
519 Hypersim O 3D scene understanding Hypersim benchmark for synthetic indoor scene understanding and reconstruction. ★★★★★ 🇺🇸 GPT-5 Mini (minimal) 39.3%
520 IFBench O Instruction following Instruction-following benchmark measuring compliance and adherence. ★★★★★ 70 G Flint 89.2%
521 IFEval O Instruction following Instruction following capability evaluation for LLMs. ★★★★★ 36312 G Jamba2 Mini 97.0%
522 IFEval-Code o Instruction following (code) Instruction following evaluation for code generation tasks. ★★★★★ 🇨🇳 Qwen3-32B 28.0%
523 IFEval (Strict Prompt) O Instruction following IFEval strict prompt-level accuracy. ★★★★★ 🇨🇳 Qwen2.5 Plus (11-27) 86.3%
524 IFStruct o Structured output Instruction-following benchmark for structured outputs (format/schema-constrained generation). ★★★★★ 🇺🇸 LFM2.5 2.6B 85.5%
525 ImageMining o Multimodal tool use Multimodal tool-use benchmark of mining information from images with tools. ★★★★★ 🇨🇳 GLM-5V-Turbo 30.7%
526 Image QA Average O Image QA (aggregate) Average of single-image visual question answering benchmarks. ★★★★★ 🇺🇸 Gemini 3 Pro 86.2%
527 IMO 2025 o Math olympiad (proofs) 2025 International Mathematical Olympiad (6 proof problems), graded with MathArena methodology. ★★★★★ 🇺🇸 GPT-5.5 76.1%
528 IMO AnswerBench O Math (competition) Evaluates free-form solutions to International Mathematical Olympiad problems using expert-style grading rubrics. ★★★★★ 🇨🇳 DeepSeek V4-Pro 93.0%
529 ImplicitMemBench o Long-term memory Tests implicit memory: whether agents apply previously seen preferences and procedures without being explicitly asked to recall them. ★★★★★ G MemT-1.0-Flash 49.2%
530 INCLUDE O Inclusiveness / bias Evaluates inclusive language use and bias mitigation in model outputs. ★★★★★ 🇺🇸 Gemini 3 Pro 90.5%
531 IndicIFEval (ground) o Indic languages Natively written Indic instruction-following prompts; prompt-level strict. ★★★★★ G Sarvam-30B 40.7%
532 IndicIFEval (translated) o Indic languages IFEval-style instruction following, translated into Indic languages; prompt-level strict. ★★★★★ 🇺🇸 Gemma 4 31B (IT) 81.1%
533 IndicMMLU-Pro o Indic languages MMLU-Pro translated/adapted to Indic languages. ★★★★★ 🇺🇸 Gemma 4 31B (IT) 79.0%
534 IndicSafe5 o Indic languages Safety benchmark of harmful/benign prompts across Indic languages; accuracy. ★★★★★ 🇺🇸 Gemma 4 31B (IT) 89.8%
535 IndoMMLU O Indonesian knowledge Indonesian multi-task exam questions from primary school to university entrance level. ★★★★★ 🇨🇳 Qwen2.5-72B-Instruct 69.3%
536 InfoQA O Information-seeking QA Information retrieval question answering benchmark evaluating factual responses. ★★★★★ 🇺🇸 Gemini 3 Pro 86.9%
537 Information Extraction o Information extraction Information extraction benchmark for economically valuable fields. ★★★★★ 🇺🇸 Claude Sonnet 4.5 46.9%
538 Information Processing o Information processing Information processing benchmark for economically valuable tasks. ★★★★★ 🇺🇸 Gemini 3 Pro 56.5%
539 InfoTabs o Table NLI Natural language inference over infobox tables (MMTab subset). ★★★★★ G Yuan3.0 Ultra 89.7%
540 InfoVQA O Infographic VQA Visual question answering over infographics requiring reading, counting, and reasoning. ★★★★★ 🇨🇳 Kimi-K2.5 Thinking 92.6%
541 INSCIT O Conversational QA Information-seeking conversations with mixed-initiative (ChatRAG-Bench subset). ★★★★★ G Yuan3.0 Ultra 40.0%
542 Intention Recognition o Intent recognition Intent recognition benchmark for practical applications. ★★★★★ 🇺🇸 Gemini 3 Pro 65.3%
543 IntPhys 2 O Intuitive physics Intuitive physics reasoning benchmark. ★★★★★ G Muse Spark Open 68.3%
544 Inverse IFEval O Instruction following (inverse) Inverse instruction-following evaluation. ★★★★★ 🇺🇸 Gemini 3 Flash 80.9%
545 IOI 2025 o Competitive programming International Olympiad in Informatics 2025 problem set; total score out of 600 points. ★★★★★ 🇨🇳 Kimi-K2.6 Thinking 585
546 IPhO 2025 (Theory) o Physics olympiad Theory problems of the 2025 International Physics Olympiad. ★★★★★ 🇺🇸 GPT-5.4 Pro 93.5%
547 iSarcasmEval O Sarcasm detection SemEval-2022 iSarcasmEval track A (English) sarcasm F1 (Jev Decision Index suite, posed as typed decisions). ★★★★★ G pplx-decider-v1.2 74.7
548 ISL/OSL 8k/16k o Throughput Relative throughput on ISL/OSL 8k/16k context workloads. ★★★★★ 🇺🇸 Nemotron-3-Nano-30B-A3B 3.30
549 JamC-QA O Japanese knowledge QA Japanese-specific knowledge multiple-choice QA (culture, customs, regional, law, medicine) from the Swallow evaluation suite. ★★★★★ 🇺🇸 GPT-5 mini 70.1%
550 JCulture O Japanese culture knowledge Japanese culture knowledge QA benchmark (as reported by Liquid AI). ★★★★★ 🇺🇸 LFM2.5 1.2B JP 202606 35.8%
551 JEC-QA O Legal QA (Chinese) Chinese National Judicial Examination multiple-choice QA. ★★★★★ 🇺🇸 GPT-4 59.3%
552 JevBench (Benchmark Heaven) O Typed decisions Benchmark Heaven JevBench for typed probabilistic decision models; score is the chance-corrected Intelligence (typed decision accuracy) axis. ★★★★★ 🇺🇸 GPT-6 Luna (medium) 96.9%
553 JevBench public set o Typed decisions Accuracy on JevBench's 231 public items (easy, original/standard and hard tiers). ★★★★★ 🇺🇸 strands-decider-26B-A4B-gemma4-v1-2610 89.6%
554 JevBench public (easy) O Typed decisions Accuracy on the 48-item easy tier of JevBench's public set. ★★★★★ G Jev 1.13 100.0%
555 JevBench public (hard) O Typed decisions Accuracy on the hard tier of JevBench's public set. ★★★★★ G H2O-Lightning-4B 76.6%
556 JevBench public (original/standard) O Typed decisions Accuracy on the 72-item original (standard) tier of JevBench's public set. ★★★★★ G Jev 1.13 98.6%
557 JFBench O Instruction following (Japanese) Japanese instruction-following benchmark. ★★★★★ 🇺🇸 LFM2.5 1.2B JP 202606 54.8%
558 JGPQA O Graduate-level QA (Japanese) Japanese translation of GPQA graduate-level science QA. ★★★★★ 🇺🇸 GPT-5 mini 75.0%
559 JHumanEval O Code (Japanese) Japanese-prompt translation of HumanEval code generation benchmark. ★★★★★ 🇺🇸 GPT-5 mini 94.5%
560 JHumanEval+ O Code (Japanese) Japanese-prompt HumanEval+ code generation benchmark. ★★★★★ G Granite-4.0-1B 51.2%
561 JMMLU O Multi-domain knowledge (Japanese) Japanese Massive Multitask Language Understanding benchmark (translated and culture-specific MMLU questions). ★★★★★ 🇨🇳 Qwen2.5-72B-Instruct 80.6%
562 JobBench O Professional tool use Agentic benchmark of professional, job-like tasks requiring tool use. ★★★★★ 🇺🇸 Claude Opus 5.5 75.4%
563 JobBench (Easy) O Professional job tasks JobBench easy split of realistic professional job tasks. ★★★★★ 🇨🇳 Qwen3.8-Max 82.8%
564 JudgeBench o LLM-as-judge Pairwise response-correctness judgement benchmark for LLM judges. ★★★★★ G Jev 1.13 78.6%
565 JudgeMark v2.1 O LLM judging ability A benchmark measuring LLM judging ability. ★★★★★ 🇺🇸 Claude Sonnet 4.6 94.1%
566 KernelBench Hard o GPU kernel optimization KernelBench Hard: agents write optimized GPU kernels; score relative to hardware peak throughput. ★★★★★ 🇺🇸 Claude Opus 4.7 30.7%
567 KernelGen 1P o AI research automation OpenAI internal eval (GPT-5.6 announcement): optimizing GPU kernels. ★★★★★ 🇺🇸 GPT-5.6 Sol 61.1%
568 KGC-Safety o Safety (Korean) Korean safety benchmark evaluating harmfulness and compliance. ★★★★★ G K-EXAONE 2.0 99.8%
569 Kimi Claw 24/7 Bench o Long-horizon agents Moonshot in-house benchmark of persistent multi-day coworking tasks (17 scenarios, 610 evaluation points) run through the OpenClaw harness; average pass rate over 3 runs. ★★★★★ 🇺🇸 GPT-5.5 52.8%
570 Kimi Code Bench v2 o Agentic coding Moonshot in-house coding-agent benchmark (Kimi Code Bench 2.0): realistic software engineering tasks across 10+ languages and production tech stacks. ★★★★★ 🇺🇸 Claude Fable 5 76.9%
571 KINA o Knowledge Knowledge benchmark of culturally grounded factual questions (reported in the Seed2.1 model card). ★★★★★ 🇺🇸 Gemini 3.1 Pro 53.2%
572 KK-4 People o Working memory (4 people) Keep/kill working-memory benchmark with 4 people entities. ★★★★★ G K2-V2 92.9%
573 KK-8 People o Working memory (8 people) Keep/kill working-memory benchmark with 8 people entities. ★★★★★ G K2-V2 82.8%
574 KMMLU O Korean knowledge Korean Massive Multitask Language Understanding benchmark. ★★★★★ 🇨🇳 DeepSeek V3.1 78.7%
575 KMMLU-Pro O Multilingual knowledge Korean Multilingual Massive Multitask Language Understanding Pro ★★★★★ 🇨🇳 DeepSeek V4-Pro-Thinking (max) 80.5%
576 KMMLU-Redux O Multilingual knowledge Redux variant of KMMLU benchmark ★★★★★ 🇺🇸 o1 81.1%
577 Ko-LongBench o Korean long-context Long-context understanding benchmark in Korean. ★★★★★ 🇨🇳 DeepSeek V4-Pro-Thinking (max) 91.4%
578 KoBALT O Korean knowledge Korean benchmark for knowledge and language understanding. ★★★★★ 🇨🇳 DeepSeek V4-Flash-Thinking (max) 75.3%
579 KoMT-Bench o Korean chat ability Korean multi-turn chat evaluation benchmark. ★★★★★ 🇨🇳 Qwen3-30B-A3B-Instruct-2507 8.49
580 KoNET o Korean exams (multimodal) Korean National Educational Test benchmark (elementary to college entrance exams) with images. ★★★★★ 🇺🇸 Gemini 2.5 Pro 96.2%
581 KOR-Bench O Reasoning Comprehensive reasoning benchmark spanning diverse domains and cognitive skills. ★★★★★ 🇺🇸 GPT-5 (high) 77.4%
582 KORBench o General reasoning Korean reasoning benchmark evaluating diverse reasoning capabilities. ★★★★★ 🇺🇸 GPT-5.2 (high) 79.2%
583 KoSimpleQA O Korean QA Korean simple question answering benchmark. ★★★★★ G Kanana-2-30B-A3B-Mid-2601 49.7%
584 KSM o Multilingual math Korean STEM and math benchmark ★★★★★ G EXAONE Deep 2.4B 60.9%
585 L-Eval O Long-context eval Long-context understanding and generation. ★★★★★ 389 🇨🇳 Qwen1.5-72B-Chat 78.1%
586 LAB-Bench 2 O Biology research LABBench2: biology real-world research tasks (macro-average over 11 sub-tasks: DbQA2, FigQA2-img/pdf, LitQA3, PatentQA, ProtocolQA2, SourceQuality, SuppQA2, TableQA2-img/pdf, TrialQA). ★★★★★ 🇺🇸 Gemini 4 Argon 88.8%
587 LAMBADA O Language modeling Word prediction requiring broad context understanding. ★★★★★ 🇺🇸 GPT-3 86.4%
588 LatentJailbreak o Safety / jailbreak Robustness to latent jailbreak adversarial techniques. ★★★★★ 39 🇺🇸 GPT-3.5-turbo 77.4%
589 LBV1-QA O Vision-language Vision-language QA benchmark v1. ★★★★★ 🇺🇸 GPT-5 73.7%
590 LBV2 O Vision-language Vision-language benchmark v2. ★★★★★ 🇺🇸 Gemini 2.5 Pro 65.7%
591 LeetCode Contest O Code generation (competition) LeetCode weekly/biweekly contest problems (Jul 2023 - Jan 2024) collected by DeepSeek; pass@1. ★★★★★ 🇺🇸 GPT-4 48.4%
592 Legal Agent Benchmark o Legal agents Harvey Legal Agent Benchmark, full public set. ★★★★★ 🇺🇸 Claude Mythos 5 16.9%
593 Harvey LAB-AA o Legal agents Artificial Analysis run of Harvey's Legal Agent Benchmark (legal work products from source documents). ★★★★★ G Muse Wonder Full 13.1%
594 Legal Agent Benchmark (Held-Out) O Legal agents Harvey Legal Agent Benchmark, Harvey's held-out set. ★★★★★ 🇺🇸 Gemini 4 Argon 19.6%
595 Legal Research Bench o Legal research Vals AI Legal Research benchmark of agentic legal research tasks. ★★★★★ 🇺🇸 Claude Fable 5 49.5%
596 LegalBench o Legal QA Suite of legal question answering and reasoning tasks. ★★★★★ 🇺🇸 Gemma 4 31B (IT) 83.1%
597 LEXam o Legal exams LEXam benchmark of law-school exam questions (open and multiple-choice) across jurisdictions. ★★★★★ G Darwin-180B-RSI 68.9%
598 LEXam (hard) o Legal exams Hard subset of the LEXam legal-exam benchmark. ★★★★★ G Darwin-180B-RSI 45.7%
599 LEXam MCQ (4-choice, EN) o Legal exams LEXam legal-exam benchmark, English 4-option multiple-choice subset. ★★★★★ 🇺🇸 Gemma 4 31B (IT) 87.4%
600 LIFEBench O Instruction following Length-based instruction-following evaluation benchmark. ★★★★★ G Ling 3.0 Flash 77.3%
601 LifeSciBench o Life sciences Life-sciences research workflow benchmark. ★★★★★ 🇺🇸 GPT-6 Astra 60.3%
602 LingoQA O Driving scene QA Question answering benchmark for autonomous driving scene understanding. ★★★★★ 🇨🇳 Qwen3.8-Max 84.8%
603 LiveBench O General capability Continually updated capability benchmark across diverse tasks. ★★★★★ 🇺🇸 Gemini 2.5 Pro 82.4%
604 LiveBrowseComp O Agentic web browsing Live (continuously refreshed) BrowseComp-style benchmark of hard-to-find web information for deep-search agents. ★★★★★ G XYZ-Aquila-pro 53.7%
605 LiveClaw o Agentic assistant tasks Live benchmark of OpenClaw agent tasks executed in real environments. ★★★★★ 🇺🇸 LFM2.5 24B A2B 48.1%
606 LiveCodeBench O Code generation Live coding and execution-based evaluation benchmark (v6 dataset). ★★★★★ 🇺🇸 GPT-5.5 96.2%
607 LiveCodeBench-Ko o Code generation (Korean) Korean translation of LiveCodeBench. ★★★★★ 🇨🇳 Qwen3-30B-A3B-Thinking-2507 66.3%
608 LiveCodeBench Pro O Competitive coding problems from Codeforces, ICPC, and IOI LiveCodeBench Pro evaluates competitive programming performance across Codeforces, ICPC, and IOI contests. Elo rating, higher is better. ★★★★★ 🇺🇸 Gemini 3 Deep Think 3298
609 LCB Pro 25Q2 (Easy) O Code generation LiveCodeBench Pro 2025 Q2 easy subset. ★★★★★ 🇺🇸 Gemini 3 Pro 94.8%
610 LCB Pro 25Q2 (Med) O Code generation LiveCodeBench Pro 2025 Q2 medium subset. ★★★★★ 🇺🇸 Gemini 3 Deep Think 86.0%
611 LiveCodeBench v3 O Code generation LiveCodeBench v3 snapshot measuring pass rates on streaming coding tasks. ★★★★★ 🇨🇳 Qwen3-32B 90.2%
612 LiveCodeBench v5 (2024.10-2025.02) O Code generation LiveCodeBench v5 snapshot covering Oct 2024-Feb 2025. ★★★★★ 🇺🇸 Gemini 3 Pro Preview 91.7%
613 LiveMathematicianBench o Research math Live benchmark of research-level mathematics problems. ★★★★★ 🇺🇸 GPT-5.5 50.8%
614 LiveMCP-101 O Agent real-time eval A novel real-time evaluation framework and benchmark to stress‑test agents on complex, real‑world tasks. ★★★★★ 🇺🇸 GPT-5 58.4%
615 LiveSports-3K O Sports video Live sports video understanding benchmark (3K). ★★★★★ 🇺🇸 Gemini 3.1 Pro 89.0%
616 LiveSQLBench o Text-to-SQL LiveSQLBench (Base-Full v1): contamination-free text-to-SQL / database tasks over PostgreSQL databases, solved agentically. ★★★★★ 🇺🇸 Claude Opus 4.7 41.0%
617 LLaVA-Bench (In-the-Wild) O Visual chat GPT-4-judged open-ended visual conversation, detailed description and complex reasoning on 60 in-the-wild images; relative score vs. GPT-4 reference. ★★★★★ 🇺🇸 GPT-4 Vision Preview (1106) 93.1%
618 LMArena Text O Crowd eval (text) Chatbot Arena text evaluation (average win rate). ★★★★★ 🇺🇸 Gemini 4 Argon (high) 1524
619 LMArena Vision O Crowd eval (vision) Chatbot Arena vision evaluation leaderboard (ELO ratings). ★★★★★ 🇺🇸 Claude Fable 5 (high) 1308
620 LOCA-Bench o Long-context agents LOCA-Bench long-context agent benchmark (environment description length 256K). ★★★★★ 🇺🇸 Claude Opus 4.7 57.0%
621 Local Agent Bench O Tool calling judgment Tests whether small open-weight models can reliably decide when to call tools and when not to. Agent Score = (Action x 0.4) + (Restraint x 0.3) + (Wrong-Tool-Avoidance x 0.3). ★★★★★ 🇺🇸 LFM2.5 1.2B Instruct 88.0%
622 LoCoMo o Long-term memory Very long-term conversational memory: QA over multi-session dialogues. ★★★★★ G MemT-1.0-Flash 65.1%
623 LogicVista O Visual logical reasoning Visual logic and pattern reasoning tasks requiring compositional and spatial understanding. ★★★★★ 🇨🇳 Qwen3.8-Max 91.9%
624 LogiQA o Logical reasoning Reading comprehension with logical reasoning. ★★★★★ 138 G Pythia 70M 23.5%
625 LongBench O Long-context eval Long-context understanding across tasks. ★★★★★ 957 🇨🇳 Qwen3.6-27B 60.9%
626 longbench-v2 O Long-context eval Next-generation LongBench v2 long-context evaluation benchmark. ★★★★★ 🇺🇸 Claude Opus 4.8 69.1%
627 LongCodeBench o Long-context code understanding Long-context code comprehension and repair benchmark (reported at 1M-token context). ★★★★★ G Nova 2 Lite 84.0%
628 LongFact-Concepts O Factuality on open-source prompts Long-context factuality eval focused on conceptual statements; scored as 100 - hallucination rate, higher is better. ★★★★★ 🇺🇸 GPT-5.2 (high) 99.7%
629 LongFact-Objects O Factuality on open-source prompts Long-context factuality eval focused on object/entity references; scored as 100 - hallucination rate, higher is better. ★★★★★ 🇺🇸 GPT-5 99.2%
630 LongMemEval O Long-term memory Evaluates LLMs' long-term memory and retrieval over extended contexts and multi-session interactions. ★★★★★ G GoLongRL-30B-A3B 75.2%
631 LongText-Bench EN o Text rendering LongText-Bench English subset score for text rendering. ★★★★★ G Seedream 4.5 0.989
632 LongText-Bench ZH o Text rendering LongText-Bench Chinese subset score for text rendering. ★★★★★ G Seedream 4.5 0.987
633 LongVideoBench O Long video QA Long video understanding and QA benchmark. ★★★★★ G Seed2.1 Turbo 80.6%
634 LPFQA O Finance QA Long-form financial question answering benchmark. ★★★★★ 🇺🇸 Claude Sonnet 4.5 54.9%
635 LVBench O Video understanding Long video understanding benchmark (LVBench). ★★★★★ 🇺🇸 Gemini 4 Argon 91.7%
636 M-IFEval (Japanese) O Instruction following (Japanese) Japanese split of M-IFEval multilingual instruction-following benchmark. ★★★★★ 🇺🇸 LFM2.5 1.2B JP 202606 79.1%
637 M3GIA (CN) o Chinese multimodal QA Chinese-language M3GIA benchmark covering grounded multimodal question answering. ★★★★★ 🇨🇳 Seed1.5-VL-Thinking 91.2%
638 Machiavelli O Deception / safety Benchmark for deceptive or manipulative behavior in social interactions. ★★★★★ 🇺🇸 Claude Haiku 4.5 ↓ 52.2%
639 MakeMeSay o Adversarial robustness Adversarial benchmark testing model robustness against manipulation attempts. Lower is better. ★★★★★ 🇺🇸 Grok 4.1 (Thinking)
640 Mantis O Multimodal reasoning Multimodal reasoning and instruction following benchmark (Mantis). ★★★★★ G dots.vlm1 86.2%
641 mArenaHard o Chat ability (multilingual) Multilingual variant of Arena-Hard evaluating chat quality across languages. ★★★★★ 🇨🇳 Qwen3-4B 70.1%
642 MARS-Bench O Instruction following Instruction-following benchmark with complex tasks. ★★★★★ 🇺🇸 GPT-5.2 (high) 87.9%
643 MASK O Safety / red teaming Model behavior safety assessment via red-teaming scenarios. ★★★★★ 🇺🇸 Claude Opus 4.6 96.3%
644 MASSIVE intent o Intent classification Amazon MASSIVE intent classification (English). ★★★★★ G Decider 4B (Mapika) 88.3%
645 MatBench o Materials property prediction Materials property prediction benchmark for scientific AI models. ★★★★★ G Intern-S1-Pro 72.8%
646 MATH O Math (competition) Competition-level mathematics across algebra, geometry, number theory, combinatorics. ★★★★★ 1185 🇺🇸 o3 mini 97.9%
647 MATH-100 (Japanese) O Math (Japanese) 100 MATH problems translated to Japanese (Swallow evaluation suite). ★★★★★ 🇺🇸 GPT-OSS 120B (medium) 97.0%
648 MATH-Ko o Math (Korean) Korean translation of the MATH competition benchmark. ★★★★★ 🇨🇳 Qwen3-30B-A3B 58.2%
649 MATH Level 5 O Math (competition) Level 5 subset of the MATH benchmark emphasizing the hardest competition-style problems. ★★★★★ 🇺🇸 o1 Mini (2024-09-12) 89.3%
650 MATH500 O Math reasoning 500-problem slice of the MATH benchmark for challenging math reasoning. ★★★★★ 🇨🇳 Qwen3.8-27B 99.8%
651 MATH500 (ES) o Math (multilingual) Spanish MATH500 benchmark ★★★★★ G EXAONE 4.0 1.2B 88.8%
652 MATH500 (Japanese) O Math (Japanese) Japanese translation of MATH500. ★★★★★ 🇺🇸 LFM2.5 1.2B JP 202606 62.8%
653 MathArena Apex O Challenging Math Contest problems Challenging math contest problems from MathArena Apex benchmark. ★★★★★ 🇺🇸 Claude Opus 5 91.4%
654 MathArena Apex Shortlist O Mathematics Shortlist of candidate hard competition problems considered for MathArena Apex. ★★★★★ 🇨🇳 Qwen3.8-Flash-Next 91.8%
655 MathBench-A O Math (multi-level) MathBench application split covering arithmetic, primary, middle, high school and college levels (overall score). ★★★★★ 🇺🇸 GPT-4o-2024-05-13 70.9%
656 MathBench (ZH) O Math (Chinese) Chinese portion of MathBench as reported in the InternLM-Math paper. ★★★★★ 🇨🇳 Qwen-72B-Chat 47.8%
657 MathOlympiadBench O Formal theorem proving Olympiad-level math problems formalized in Lean 4; score = share of theorems proved (Pass@32 in LongCat-Flash-Prover report). ★★★★★ 🇨🇳 LongCat-Flash-Prover 35.8%
658 MathVerse O Math reasoning (multimodal) Visual math reasoning benchmark combining images and text across diverse mathematical tasks. ★★★★★ 🇺🇸 Gemini 3 Pro 91.6%
659 MathVerse-mini O Math reasoning (multimodal) Compact MathVerse split focusing on single-image math puzzles and visual reasoning. ★★★★★ 🇨🇳 MiniCPM-V 4.7 35B-A3B 86.5%
660 MathVerse-Vision O Math reasoning (multimodal) Multi-image visual mathematical reasoning tasks from the MathVerse ecosystem. ★★★★★ G Seed2.1 Pro 89.7%
661 MathVision O Math reasoning (multimodal) Visual math reasoning benchmark with problems that combine images (charts, diagrams) and text. ★★★★★ 🇺🇸 Claude Fable 5 98.6%
662 MathVision (With Tools) O Math reasoning (multimodal) MathVision visual math benchmark evaluated with a Python code-interpreter tool. ★★★★★ 🇺🇸 Claude Fable 5 98.6%
663 MathVista O Multimodal math reasoning Visual math reasoning across diverse tasks. ★★★★★ G Seed2.1 Pro 90.7%
664 MathVista-Mini O Math reasoning (multimodal) Lightweight subset of MathVista for quick evaluation of visual mathematical reasoning. ★★★★★ 🇨🇳 Qwen3.5-397B-A17B 90.3%
665 MatTools O Materials science tool use Benchmark evaluating LLM agents on materials-science tool usage (pymatgen-based QA and code/tool generation tasks). ★★★★★ 🇺🇸 GPT-5.5 68.8%
666 MAVERIX o Audio-visual video understanding Benchmark requiring joint video and audio understanding (situational awareness, social sentiment). ★★★★★ 🇺🇸 Gemini 2.5 Flash 67.4%
667 MAXIFE O Instruction following (multilingual) Multilingual instruction-following evaluation across English and multilingual original prompts. ★★★★★ 🇨🇳 Qwen3.7-Max 89.2%
668 MBE Bar Exam o Legal exams Multistate Bar Examination (US bar exam) multiple-choice questions. ★★★★★ 🇺🇸 Gemma 4 31B (IT) 88.8%
669 MBPP O Code generation Short Python problems with hidden tests. ★★★★★ 36312 🇨🇳 Kimi-K2 Thinking 97.4%
670 MBPP-Ko o Code generation (Korean) Korean translation of MBPP code generation benchmark. ★★★★★ 🇨🇳 Qwen3-30B-A3B 66.8%
671 MBPP+ O Code generation Extended MBPP with more tests and stricter evaluation. ★★★★★ 🇨🇳 GLM 4.6 94.2%
672 MCP-Atlas O Agent evaluation Aggregate MCP agent benchmark covering tool-use and planning tasks. ★★★★★ G Muse Spark 1.2 90.3%
673 MCP Universe O Agent evaluation Benchmarks multi-step tool-use agents across diverse task suites with a unified overall success metric. ★★★★★ 🇺🇸 Gemini 3 Pro 50.7%
674 MCPMark O Agent tool-use (MCP) Benchmark for Model Context Protocol (MCP) agent tool-use. ★★★★★ 127 🇺🇸 GPT-5.6 Terra 74.0%
675 MCPMark-Verified O Agent tool-use (MCP) Human-verified edition of MCPMark covering Notion, GitHub, Filesystem, Postgres and Playwright MCP servers. ★★★★★ 🇨🇳 Kimi-K3 96.1%
676 mDolly o Instruction following (multilingual) Multilingual variant of the Dolly instruction-following benchmark. ★★★★★ G Tiny Aya Global 86.9%
677 MDPBench o Document parsing Document parsing benchmark (reported in OvisOCR3 release). ★★★★★ G OvisOCR3 88.8%
678 MeasureBench o Visual measurement reading Benchmark of reading measurement instruments (gauges, rulers, meters) from images. ★★★★★ G Seed2.1 Pro 62.9%
679 Medbullets (4 options) o Medical QA USMLE Step 2/3-style clinical questions, 4-option version. ★★★★★ G MedMO-8B 66.5%
680 Medbullets (5 options) o Medical QA USMLE Step 2/3-style clinical questions, 5-option version. ★★★★★ G MedMO-8B 60.2%
681 MedChemBench (internal) o Medicinal chemistry OpenAI internal eval reported in the GPT-6 Astra announcement: medicinal chemistry tasks. ★★★★★ 🇺🇸 GPT-6 Astra 49.3%
682 MedMCQA O Medical exams Indian medical entrance exam (AIIMS/NEET PG) multiple-choice QA. ★★★★★ 🇺🇸 GPT-4 72.5%
683 MedQA O Medical exams US medical licensing exam-style multiple choice questions. ★★★★★ G OpenEvidence Darwin 100.0%
684 MedQA (MCMLE) O Medical QA (Chinese) MedQA Mainland China Medical Licensing Exam subset (5 options). ★★★★★ 🇺🇸 GPT-4 74.6%
685 MedXpertQA-MM O Medical VQA Multimodal medical expert question answering benchmark. ★★★★★ 🇺🇸 GPT-5.6 Sol 81.5%
686 MedXpertQA (Text) O Medical QA Text-only subset of MedXpertQA expert-level medical multiple-choice questions. ★★★★★ G OpenEvidence Darwin 72.8%
687 MemBench o Long-term memory Benchmark of factual and reflective memory of LLM-based agents across participation and observation scenarios. ★★★★★ G MemT-1.0-Flash 73.5%
688 MemoryAgentBench o Long-term memory Evaluates memory agents on accurate retrieval, test-time learning, long-range understanding and conflict resolution. ★★★★★ G MemT-1.0-Flash 17.4%
689 MemoryBench o Long-term memory Meta-reported benchmark of remembering and retrieving user context across conversations. ★★★★★ G Muse Wonder 85.6%
690 METR O Long task benchmark METR evaluates AI agents on long-horizon coding and agentic tasks, measuring autonomous task completion time. ★★★★★ 🇺🇸 Claude Opus 4.5 4.8 h
691 MEWC o Web comprehension Multi-page End-to-end Web Comprehension benchmark. ★★★★★ 🇺🇸 Claude Opus 4.6 89.8%
692 MGSM O Math (multilingual) Multilingual grade school math word problems. ★★★★★ 🇺🇸 Claude Opus 4.1 (2025-08-05) Thinking 94.4%
693 MHPP O Code generation Mostly Hard Python Problems: 210 hand-written function-level Python problems probing harder code-generation skills; pass@1. ★★★★★ G Stable-DiffCoder-8B-Instruct 42.4%
694 MIABench O Multimodal instruction following Multimodal instruction-following benchmark evaluating accuracy on complex image-text tasks. ★★★★★ 🇺🇸 Gemini 2.5 Pro 96.0%
695 MicroVQA O Biological microscopy Visual question answering benchmark for biological microscopy images. ★★★★★ 🇺🇸 Gemini 3.1 Pro 71.0%
696 MILU o Indic languages Multi-task Indic Language Understanding benchmark (11 Indian languages + English), accuracy. ★★★★★ 🇺🇸 Gemma 4 31B (IT) 85.3%
697 MIMIC-CXR (classification) o Medical imaging Chest X-ray condition classification on MIMIC-CXR, macro F1 over top conditions. ★★★★★ 🇺🇸 MedGemma 27B 90.0%
698 NIAH-Multi 128K o Long-context QA Needle-in-a-haystack multi-query benchmark at 128K context. ★★★★★ 🇨🇳 Kimi-K2 Base 99.5%
699 NIAH-Multi 32K o Long-context QA Needle-in-a-haystack multi-query benchmark at 32K context. ★★★★★ 🇨🇳 Kimi-K2 Base 99.8%
700 NIAH-Multi 64K o Long-context QA Needle-in-a-haystack multi-query benchmark at 64K context. ★★★★★ 🇨🇳 Kimi-K2 Base 100.0%
701 MindCube O Spatial navigation Spatial navigation benchmark. ★★★★★ G ACE-Brain-0.5-8B 94.2%
702 MindCube-Tiny o Spatial reasoning Tiny evaluation subset of MindCube spatial mental-model benchmark. ★★★★★ G ZDTaichu5.0-9B 78.3%
703 Minerva Math O University-level math Advanced quantitative reasoning set inspired by the Minerva benchmark for STEM problem solving. ★★★★★ 🇨🇳 Qwen3 235B A22B Thinking 98.0%
704 Minerva (Video) O Video reasoning Minerva video question-answering benchmark with complex multi-step reasoning over videos (not the Minerva math set). ★★★★★ G Seed2.1 Pro 70.7%
705 MiniF2F pass@1 o Math competition MiniF2F competition benchmark pass@1 accuracy. ★★★★★ 🇺🇸 NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 50.0%
706 MiniF2F pass@32 O Math competition MiniF2F competition benchmark pass@32 accuracy. ★★★★★ 🇨🇳 LongCat-Flash-Prover 93.9%
707 MiniF2F (Test) O Math competition MiniF2F competition benchmark (test split). ★★★★★ G Leanstral 1.5 100.0%
708 MIRB O Multi-image reasoning Multi-Image Relational Benchmark (average over perception, visual world knowledge, reasoning, multi-hop). ★★★★★ 🇨🇳 InternVL2.5-38B 61.2%
709 MixEval O Multi-task reasoning Mixed-subject benchmark covering knowledge and reasoning tasks across domains. ★★★★★ 🇨🇳 Qwen2 72B Instruct 86.7%
710 MixEval Hard o Multi-task reasoning (hard) Hard subset of MixEval covering diverse reasoning tasks. ★★★★★ 🇨🇳 Qwen3.5-9B 50.3%
711 MLE-Bench o ML engineering OpenAI MLE-Bench: Kaggle-style machine-learning engineering competitions solved by agents. ★★★★★ 🇺🇸 Claude Sonnet 5 66.9%
712 MLE-bench-30 o ML engineering 30-competition subset of MLE-bench (Kaggle ML engineering tasks); average position score as used in the Gemini 3.6 Flash model card. ★★★★★ G Naive-N0.5-Flash 73.7%
713 MLE-Bench Lite O ML engineering MLE-Bench Lite: 22-competition low-complexity subset of OpenAI MLE-Bench; metric is medal rate. ★★★★★ 🇺🇸 GPT-5.6 Sol 88.9%
714 MLS-Bench-Lite O Agentic ML engineering Lite version of MLS-Bench, a long-horizon agentic machine-learning research/engineering benchmark with an official leaderboard. ★★★★★ 🇺🇸 Claude Fable 5 49.9%
715 MLVU O Large video understanding MLVU: Large-scale multi-task benchmark for video understanding. ★★★★★ 🇨🇳 Qwen3.8-Max 90.8%
716 MM-BrowseComp o Multimodal browsing Multimodal browsing comprehension benchmark. ★★★★★ 🇺🇸 Claude Opus 4.6 59.3%
717 MM-ClawBench o Personal agent tasks MiniMax MM-ClawBench (MM Claw): end-to-end OpenClaw-style personal-agent benchmark. ★★★★★ 🇺🇸 Claude Opus 4.6 75.4%
718 MM-IFEval O Multimodal instruction following Instruction-following benchmark assessing multimodal obedience to complex prompts. ★★★★★ 🇺🇸 GPT-5 83.1%
719 MM-MT-Bench O Multimodal instruction following Multi-turn multimodal instruction following benchmark evaluating dialogue quality and helpfulness. ★★★★★ 🇨🇳 Qwen3-VL-235B-A22B Thinking 8.50
720 MMAU-Pro o Audio reasoning MMAU-Pro: challenging audio understanding and reasoning benchmark (speech, sound, music). ★★★★★ 🇨🇳 Qwen3.5-Omni-Plus 70.0%
721 MMBench (EN) O Multimodal understanding (English) MMBench v1.0 English test split. ★★★★★ 🇨🇳 InternVL2.5-78B 88.3%
722 MMBench (CN) O Multimodal understanding (Chinese) MMBench v1.0 Chinese test split. ★★★★★ 🇨🇳 InternVL2.5-78B 88.5%
723 MMBench-GUI L2 O GUI grounding MMBench-GUI level-2 element grounding across Windows, macOS, Linux, iOS, Android and Web. ★★★★★ G MAI-UI-8B 88.8%
724 MMBench v1.1 (CN) O Multimodal understanding (Chinese) MMBench v1.1 Chinese subset for evaluating multimodal LLMs. ★★★★★ 🇺🇸 Gemini 3 Pro 91.3%
725 MMBench v1.1 (EN) O Multimodal understanding (English) MMBench v1.1 English subset for evaluating multimodal LLMs. ★★★★★ 🇺🇸 Gemini 3 Pro 93.3%
726 MMBench v1.1 (EN dev) O General VQA English dev split of MMBench v1.1 measuring multimodal question answering. ★★★★★ 🇨🇳 Kimi-K2.5 94.2%
727 MMBench-Video O Video QA Long-form multi-shot video understanding with GPT-4-judged free-form answers; overall mean score on a 0-3 scale. ★★★★★ 🇨🇳 InternVL2.5-78B 1.97
728 MMC-Benchmark O Chart understanding MMC multimodal chart understanding benchmark. ★★★★★ 🇨🇳 Qwen-VL-Max 76.4%
729 MMDU o Multi-turn multi-image dialog Multi-turn, multi-image dialog understanding benchmark with GPT-4o-judged scores. ★★★★★ 🇺🇸 GPT-4V(ision) 66.3%
730 MME-CC o Multimodal evaluation MME-CC multimodal evaluation suite. ★★★★★ 🇺🇸 Gemini 3 Pro 56.9%
731 MME Elo O Multimodal perception Elo-style scoring for the MME multimodal evaluation benchmark. ★★★★★ 🇨🇳 InternVL2.5-78B 2495
732 MME-RealWorld (cn) O Real-world perception (CN) MME-RealWorld Chinese split. ★★★★★ 🇺🇸 Gemini 3 Flash 72.6%
733 MME-RealWorld (en) O Real-world perception (EN) MME-RealWorld English split. ★★★★★ 🇺🇸 Gemini 3 Flash 74.9%
734 MMHal-Bench (Score) O Multimodal hallucination MMHal-Bench GPT-judged response quality score (0-6) for hallucination in vision-language models. ★★★★★ 🇨🇳 Qwen3.8-27B 5.03
735 MMHal-Bench (Hallucination Rate) O Multimodal hallucination MMHal-Bench hallucination rate of vision-language model responses; lower is better. ★★★★★ 🇨🇳 MiniCPM-V 4.7 35B-A3B ↓ 21.9%
736 MMIU O Multi-image understanding Multi-image understanding benchmark evaluating cross-image reasoning. ★★★★★ 🇺🇸 Gemini 3 Pro 72.1%
737 MMLB-NIAH (128k) o Multimodal long-context MMLB-NIAH 128k long-context multimodal benchmark. ★★★★★ G Seed1.8 72.2%
738 MMLB-VRAG (128k) o Multimodal long-context MMLB-VRAG 128k long-context multimodal benchmark. ★★★★★ 🇺🇸 Gemini 3 Pro 88.9%
739 MMLongBench-128K o Long-context multimodal 128K-context variant of MMLongBench evaluating multimodal long-context understanding. ★★★★★ G Seed2.1 Pro 78.3%
740 MMLongBench-Doc O Long-context multimodal documents Evaluates long-context document understanding with mixed text, tables, and figures across multiple pages. ★★★★★ 🇨🇳 Qwen3.8-Flash-Next 67.9%
741 MMLU O Multi-domain knowledge 57 tasks spanning STEM, humanities, social sciences; broad knowledge and reasoning. ★★★★★ 1488 🇺🇸 Gemini 4 Argon 95.3%
742 MMLU Arabic O Arabic knowledge and reasoning Arabic-language variant of MMLU evaluating knowledge and reasoning. ★★★★★ 🇨🇳 Qwen 2.5 72B 74.1%
743 MMLU (cloze) o Multi-domain knowledge (cloze) Cloze-form MMLU evaluation variant. ★★★★★ 🇺🇸 SmolLM2 135M Base 31.5%
744 Full Text MMLU o Multi-domain knowledge (long-form) Full-context MMLU variant evaluating reasoning over long passages. ★★★★★ 🇺🇸 Llama 3.3 70B Instruct 83.0
745 MMLU Med O Medical knowledge Medical subset of MMLU (clinical knowledge, anatomy, college medicine, medical genetics, professional medicine, college biology). ★★★★★ 🇺🇸 MedGemma 27B 86.2%
746 MMLU-Pro O Multi-domain knowledge Harder successor to MMLU with more challenging questions. ★★★★★ 286 🇺🇸 Gemini 3.1 Pro 91.2%
747 MMLU Pro MCF o Multi-domain knowledge (few-shot) MMLU-Pro common format (MCF) few-shot evaluation. ★★★★★ 🇨🇳 Qwen3-4B-Base 41.1%
748 MMLU-ProX O Multi-domain knowledge Cross-lingual and robust variant of MMLU-Pro. ★★★★★ 🇺🇸 Gemini 3 Pro 87.7%
749 MMLU-ProX (Japanese) O Multi-domain knowledge (Japanese) Japanese subset of MMLU-ProX (JMMLU-ProX). ★★★★★ 🇨🇳 Qwen3-235B-A22B-Thinking-2507 81.9%
750 MMLU-Redux O Multi-domain knowledge Updated MMLU-style evaluation with revised questions and scoring. ★★★★★ G GRM-3.2-Sky 96.9%
751 MMLU-STEM O STEM knowledge STEM subset of MMLU. ★★★★★ 1488 🇨🇳 Qwen3-8B 92.5%
752 MMMB O Multilingual MMBench Multilingual Multimodal Benchmark (MMMB) average score. ★★★★★ 🇨🇳 Qwen3.5-4B 83.4%
753 MMMLU O Multi-domain knowledge (multilingual) Massively multilingual MMLU-style evaluation across many languages. ★★★★★ 🇺🇸 Claude Mythos Preview 92.7%
754 MMMLU (ES) o Multilingual knowledge Spanish MMMLU benchmark ★★★★★ 🇺🇸 SmolLM3-3B 64.7%
755 MMMU O Multimodal understanding Multi-discipline multimodal understanding benchmark. ★★★★★ 🇺🇸 Gemini 3 Pro 87.2%
756 MMMU-Med o Medical QA Medical/health subset of MMMU (validation). ★★★★★ G MedMO-8B-Next 69.3%
757 MMMU PRO O Multimodal understanding (hard) Professional/advanced subset of MMMU for multimodal reasoning. ★★★★★ 🇺🇸 Gemini 3.5 Flash 84.6%
758 MMMU-Pro (vision) O Multimodal understanding (vision) MMMU-Pro vision-only setting. ★★★★★ 🇺🇸 GPT-5 (high) 66.9%
759 MMMU Pro (with tools) O Multimodal understanding (with tools) MMMU-Pro benchmark evaluated with tool access. ★★★★★ 🇺🇸 Claude Fable 5 86.5%
760 MMMU (test) O Multimodal understanding MMMU test-set accuracy (multi-discipline college-level multimodal questions). ★★★★★ 🇨🇳 InternVL2.5-78B 61.8%
761 MMSearch O Multimodal search Benchmark for multimodal search engines / LMMs answering queries via image+web search. ★★★★★ 🇨🇳 Kimi-K2.6 Thinking 80.0%
762 MMSearch-Plus o Multimodal search Harder MMSearch variant requiring fine-grained visual reasoning plus multi-step search. ★★★★★ 🇨🇳 GLM-5V-Turbo 30.0%
763 MMSI-Bench O Spatial reasoning Multi-image spatial intelligence benchmark (standard, non-circular evaluation). ★★★★★ 🇺🇸 GPT-6 Astra 57.9%
764 MMSIBench (circular) O Spatial understanding MMSIBench circular subset for spatial reasoning. ★★★★★ 🇺🇸 GPT-5.5 36.0%
765 MMStar O Multimodal reasoning Broad evaluation of multimodal LLMs across diverse tasks. ★★★★★ 🇨🇳 Qwen3.8-Max 85.9%
766 MMT-Bench O Multimodal multitask understanding Large-scale multimodal multitask benchmark covering diverse meta-tasks for LVLMs. ★★★★★ 🇺🇸 Gemma 4 31B (IT) 75.7%
767 MMVet O Multimodal evaluation Comprehensive evaluation suite for assessing multimodal LLM capabilities. ★★★★★ 🇨🇳 R-4B-Base 85.9%
768 MM-Vet v2 O Multimodal evaluation MM-Vet v2: integrated capabilities incl. image-text sequence understanding; GPT-4 graded. ★★★★★ 🇺🇸 Claude 3.5 Sonnet 20240620 71.8%
769 MMVP O Multimodal video perception Benchmark for multimodal video understanding and perception. ★★★★★ G Seed1.8 91.6%
770 MMVU O Video understanding Multimodal video understanding benchmark (MMVU). ★★★★★ 🇺🇸 GPT-5 (high) 87.3%
771 Mobile Actions o Tool use Google Mobile Actions dataset: function calling for on-device Android actions (FunctionGemma eval). ★★★★★ 🇨🇳 Qwen3.5-9B 91.8%
772 MobileWorld O Mobile GUI agents Benchmark of agents completing tasks in mobile (Android) app environments. ★★★★★ 🇺🇸 Claude Fable 5 85.5%
773 Mol-Instructions O Bio-molecular instruction following Instruction-following benchmark for bio-molecular understanding and generation. ★★★★★ G Intern-S2-397B 54.0%
774 MolBench-bind o Molecular science agents MolBench binding-affinity subset: scientific agentic tasks on molecular binding prediction. ★★★★★ 🇺🇸 GPT-5.5 62.2%
775 MolecularIQ O Molecular structure reasoning Benchmark of molecular structure reasoning for chemistry-capable LLMs. ★★★★★ 🇺🇸 GPT-5.5 76.4%
776 MotionBench O Video motion understanding Video motion and temporal reasoning benchmark. ★★★★★ G Seed2.0 Pro 75.2%
777 MPDocBench-Parse Truncated Table TEDS o Cross-page tables MPDocBench-Parse: TEDS on truncated (page-split) tables. ★★★★★ G OvisOCR3 92.0%
778 OpenAI-MRCR (128k) O Long-context reasoning OpenAI Multi-Round Chain Reasoning benchmark with 128k context window. ★★★★★ G K-EXAONE 2.0 94.4%
779 MRCR 128K-2N o Long-context reasoning Multi-Round Coreference Resolution benchmark at 128k context with 2 needles. ★★★★★ 🇫🇷 Ministral-3-R 8B 50.3%
780 MRCR 128K-4N o Long-context reasoning Multi-Round Coreference Resolution benchmark at 128k context with 4 needles. ★★★★★ 🇫🇷 Ministral-3-R 8B 22.7%
781 MRCR 128K-8N O Long-context reasoning Multi-Round Coreference Resolution benchmark at 128k context with 8 needles. ★★★★★ 🇺🇸 Gemini 3.7 Flash 97.0%
782 MRCR 16K-8N o Long-context reasoning OpenAI MRCR v2 8-needle, 8K-16K context bucket. ★★★★★ 🇺🇸 GPT-5.5 93.0%
783 OpenAI-MRCR (1M) O Long-context reasoning OpenAI Multi-Round Chain Reasoning benchmark with 1M context window. ★★★★★ 🇺🇸 Claude Opus 4.6 92.9%
784 MRCR 1M-8N o Long-context reasoning OpenAI MRCR v2, 8 needles, 512K-1M context bucket. ★★★★★ G Muse Spark 1.3 98.1%
785 MRCR 256K-8N O Long-context reasoning OpenAI MRCR v2 8-needle, 128K-256K context bucket. ★★★★★ 🇺🇸 GPT-5.6 Sol 93.8%
786 MRCR 32K-8N o Long-context reasoning OpenAI MRCR v2 8-needle, 16K-32K context bucket. ★★★★★ 🇺🇸 GPT-5.4 97.2%
787 MRCR 512K-8N o Long-context reasoning OpenAI MRCR v2, 8 needles, 256K-512K context bucket. ★★★★★ 🇺🇸 GPT-6 Astra 100.0%
788 MRCR 64K-2N o Long-context reasoning Multi-Round Coreference Resolution benchmark at 64k context with 2 needles. ★★★★★ 🇫🇷 Ministral-3-R 8B 44.0%
789 MRCR 64K-4N o Long-context reasoning Multi-Round Coreference Resolution benchmark at 64k context with 4 needles. ★★★★★ 🇫🇷 Ministral-3-R 8B 35.8%
790 MRCR 64K-8N o Long-context reasoning Multi-Round Coreference Resolution benchmark at 64k context with 8 needles. ★★★★★ 🇺🇸 GPT-5.2 92.0%
791 MRCR 8K-8N o Long-context reasoning OpenAI MRCR v2 8-needle, 4K-8K context bucket. ★★★★★ 🇺🇸 GPT-5.2 98.2%
792 MRCR v2 O Multimodal reasoning Multi-round multimodal chain-of-reasoning evaluation (v2). ★★★★★ 🇺🇸 GPT-5.5 94.8%
793 MS-CXR-T o Medical imaging Temporal chest X-ray disease progression classification, macro accuracy. ★★★★★ 🇺🇸 MedGemma 1.5 4B 65.7%
794 MSEarth-MCQ o Earth science Earth science multiple-choice question benchmark for scientific AI models. ★★★★★ 🇺🇸 Gemini 3 Pro 65.8%
795 MSVD-QA o Video QA (open-ended) Open-ended QA on MSVD video clips; GPT-judged accuracy. ★★★★★ G Baichuan-Omni 72.2%
796 MT-Bench O Chat ability Multi-turn chat evaluation via GPT-4 grading. ★★★★★ 39074 🇺🇸 Apriel Nemotron 15B Thinker 85.7%
797 Japanese MT-Bench O Multi-turn chat (Japanese) Japanese adaptation of MT-Bench multi-turn conversational benchmark, LLM-as-judge score on a 1-10 scale. ★★★★★ 🇺🇸 GPT-5.4 8.87
798 MTOB (full book) o Long-form reasoning Long-context book understanding benchmark (full-book setting). ★★★★★ 🇺🇸 Llama 4 Maverick 50.8%
799 MTOB (half book) o Long-form reasoning Long-context book understanding benchmark (half-book setting). ★★★★★ 🇺🇸 Llama 4 Maverick 54.0%
800 MTVQA O Multilingual text-centric VQA Multilingual text-centric visual question answering across 9 languages (average accuracy). ★★★★★ 🇨🇳 InternVL2.5-78B 31.9%
801 MUIRBENCH O Multimodal robustness Evaluates multimodal understanding robustness and reliability. ★★★★★ 🇺🇸 Gemini 3 Pro 86.1%
802 Multi-IF O Instruction following (multi-task) Composite instruction-following evaluation across multiple tasks. ★★★★★ 🇺🇸 Gemma 4 31B (IT) 89.7%
803 Multi-IFEval O Instruction following (multi-task) Multi-task variant of instruction-following evaluation. ★★★★★ 🇺🇸 Llama 3.3 70B 88.7%
804 Multi-SWE-Bench O Code repair (multi-repo) Multi-repository SWE-Bench variant. ★★★★★ 246 G MiniMax M2.7 52.7%
805 MultiChallenge O Instruction following Multi-domain instruction-following benchmark. ★★★★★ G Nova 2 Pro 77.7%
806 Multi-Image QA Average O Multi-image QA (aggregate) Aggregate score over multi-image visual question answering tasks. ★★★★★ 🇺🇸 Gemini 3 Pro 81.9%
807 Multilingual MMBench O Multilingual vision benchmark Multilingual MMBench average score across languages. ★★★★★ 🇺🇸 LFM2.5-VL-3B 79.5%
808 Multilingual MMLU O Multi-domain knowledge (multilingual) Multilingual variant of MMLU across many languages. ★★★★★ 🇺🇸 GPT-4.1 87.3%
809 MultiLoKo o Multilingual knowledge Multilingual local-knowledge QA benchmark covering 31 languages with locally sourced questions. ★★★★★ 🇨🇳 DeepSeek V4-Pro Base 51.1%
810 MultiPL-E O Code generation (multilingual) Multilingual code generation and execution benchmark across many programming languages. ★★★★★ 269 🇺🇸 Claude Opus 4 89.6%
811 MultiPL-E HumanEval O Code generation (multilingual) MultiPL-E variant of HumanEval tasks. ★★★★★ 🇺🇸 Llama 3.1 405B 75.2%
812 MultiPL-E MBPP O Code generation (multilingual) MultiPL-E variant of MBPP tasks. ★★★★★ 🇺🇸 Llama 3.1 405B 65.7%
813 MuSR O Reasoning Multistep Soft Reasoning. ★★★★★ G Clef-Flash 86.0%
814 MVBench O Video QA Multi-view or multi-video QA benchmark (MVBench). ★★★★★ 🇺🇸 Gemini 3.7 Flash 82.2%
815 NanoGPT o AI research automation OpenAI internal eval (GPT-5.6 announcement): improving a NanoGPT training recipe (speed-up score). ★★★★★ 🇺🇸 GPT-5.6 Terra 14.5%
816 Nanonets KIE o Key information extraction Nanonets key information extraction benchmark (F1). ★★★★★ G Qianfan-OCR 86.5%
817 Natural2Code o Code generation Natural language to code benchmark for instruction-following synthesis. ★★★★★ 🇺🇸 Gemini 2.0 Flash 92.9%
818 NaturalCodeBench O Code generation NaturalCodeBench (NCB): real-world coding prompts in Python and Java, English and Chinese. ★★★★★ 🇺🇸 GPT-4 Turbo (2024-04-09) 53.8%
819 NaturalQuestions O Open-domain QA Google NQ; real user questions with long/short answers. ★★★★★ 🇫🇷 Mixtral-8x22B-Instruct-v0.1 54.9%
820 NAVSIM (PDMS) o Driving planning NAVSIM v1 non-reactive simulation benchmark for end-to-end driving planners; score = PDM Score (PDMS, 0-100). ★★★★★ 🇨🇳 Qwen-Drive-1.0-4B RL 90.7
821 New Yorker caption matching O Humor understanding Pick the caption written for a described New Yorker cartoon from five; accuracy (Jev Decision Index suite, posed as typed decisions). ★★★★★ G AutoTrust GEV-26B-Decide 82.0%
822 NExT-QA O Video understanding Video question answering benchmark on causal and temporal action reasoning (multiple-choice). ★★★★★ 🇺🇸 GPT-5 86.3%
823 Nexus (0-shot) O Tool use Nexus tool-use benchmark, zero-shot setting. ★★★★★ 🇺🇸 Llama 3.1 405B 58.7%
824 Needle In A Haystack o Long-context retrieval Needle In A Haystack test for locating hidden facts in long contexts. ★★★★★ G MobileLLM P1 Base 100.0%
825 NL2Repo O Software engineering Long-horizon repository generation: agents build a complete Python repository from a natural-language requirements document; scored by the repo's test suite. ★★★★★ 🇺🇸 Claude Opus 5 75.3%
826 NLI4CT O Clinical NLI NLI over clinical trial reports (SemEval NLI4CT); macro-F1 (Jev Decision Index suite, posed as typed decisions). ★★★★★ G vLLM-SR d3 86.0
827 NoCaps o Image captioning Novel object captioning benchmark (CIDEr on val). ★★★★★ G InstructBLIP (Vicuna-13B) 121.9
828 NoLiMa 128K o Long-context eval NoLiMa (No Literal Match) long-context benchmark at 128k context window. ★★★★★ 🇨🇳 MiniCPM-SALA 23.9%
829 NoLiMa 32K o Long-context eval NoLiMa (No Literal Match) long-context benchmark at 32k context window. ★★★★★ 🇨🇳 MiniCPM-SALA 54.5%
830 NoLiMa 64K o Long-context eval NoLiMa (No Literal Match) long-context benchmark at 64k context window. ★★★★★ 🇨🇳 MiniCPM-SALA 43.0%
831 NOVA-63 O Multilingual evaluation Multilingual evaluation benchmark covering 63 languages. ★★★★★ 🇨🇳 Qwen3.5-397B-A17B 59.1%
832 NuScenes O 3D scene understanding 3D scene understanding and perception benchmark for autonomous driving. ★★★★★ 🇨🇳 Qwen3.5-397B-A17B 16.0%
833 Objectron o Object detection Objectron benchmark for 3D object detection in video captures. ★★★★★ 🇨🇳 Qwen3-VL-235B-A22B Thinking 71.2%
834 OBQA o Open book QA OpenBookQA science question answering benchmark. ★★★★★ 🇨🇳 Qwen2.5-Omni-3B 76.3%
835 OCNLI O Natural language inference (Chinese) Original Chinese Natural Language Inference benchmark. ★★★★★ G Aquila2-34B 79.9%
836 OCRBench V2 O OCR (vision text extraction) OCRBench v2 evaluating text extraction from images and documents. ★★★★★ 🇨🇳 Qwen3.5-397B-A17B 93.1%
837 OCRBench-ELO O OCR (ELO ranking) OCR benchmark using ELO rating system to rank model performance on text extraction tasks. ★★★★★ 🇺🇸 Gemini 3 Pro 909
838 OCRBench (KIE) o Key information extraction Key information extraction subset of OCRBench (v1). ★★★★★ 🇺🇸 Gemini 3.1 Pro 96.0%
839 OCRBenchV2 (CN) O OCR (Chinese) OCRBenchV2 Chinese subset assessing OCR performance on Chinese-language documents. ★★★★★ 🇨🇳 Qwen3.8-Max 68.3%
840 OCRBenchV2 (EN) O OCR (English) OCRBenchV2 English subset evaluating OCR accuracy on English documents and layouts. ★★★★★ 🇨🇳 Qwen3.8-Max 74.2%
841 OCRBenchV2 KIE o Key information extraction Key information extraction subset of OCRBench v2 as reported without a language split (OvisOCR3 release). ★★★★★ G OvisOCR3 83.0%
842 OCRBenchV2 KIE (EN) o Key information extraction English key information extraction subset of OCRBench v2. ★★★★★ 🇺🇸 Gemini 3.1 Pro 87.8%
843 OCRBenchV2 KIE (ZH) o Key information extraction (Chinese) Chinese key information extraction subset of OCRBench v2. ★★★★★ G Qianfan-OCR 82.3%
844 OCRReasoning O OCR reasoning OCR reasoning benchmark combining text extraction with multi-step reasoning over documents. ★★★★★ 🇺🇸 Gemini 2.5 Pro 70.8%
845 OCR-VQA o Text-based VQA Visual question answering by reading text in book cover images. ★★★★★ 🇨🇳 Qwen-VL 75.7%
846 OctoCodingBench o Code generation Coding benchmark across multi-language programming tasks. ★★★★★ 🇺🇸 Claude Opus 4.5 36.2%
847 OCW Courses o STEM math reasoning MIT OpenCourseWare undergraduate STEM problems (from Minerva). ★★★★★ G Minerva 540B 17.6%
848 ODinW-13 O Object detection (in the wild) Object Detection in the Wild benchmark covering 13 real-world domains. ★★★★★ 🇨🇳 Qwen3.6-Plus 51.8%
849 ODVBench o Streaming video understanding Online/streaming video understanding benchmark (accuracy). ★★★★★ G JoyAI-VL-Interaction 68.6%
850 Odyssey Math O Math reasoning Odyssey multi-step math benchmark. ★★★★★ 🇨🇳 DeepSeek-Coder V2 Instruct 53.7%
851 OfficeQA o Enterprise document QA Grounded question answering over office and enterprise documents, as reported by OpenAI for GPT-5.4. ★★★★★ 🇺🇸 Claude Opus 4.7 86.3%
852 OfficeQA Pro O Enterprise document QA Harder OfficeQA variant. Models must use files, tools and context together instead of answering from memory. ★★★★★ 🇺🇸 Claude Opus 4.7 80.6%
853 OIBench EN o Code generation English subset of OIBench for code generation. ★★★★★ 🇺🇸 Gemini 3 Pro 58.2%
854 OJBench O Code generation (online judge) Programming problems evaluated via online judge-style execution. ★★★★★ 🇨🇳 Keye-VL-2.0-30B-A3B 71.5%
855 OK-VQA O Knowledge-based VQA Visual question answering requiring outside knowledge. ★★★★★ G Emu2-Chat 64.8%
856 olmOCR-Bench O Document OCR olmOCR benchmark assessing OCR fidelity and structured extraction on complex document pages. ★★★★★ G Infinity-Parser2-Pro 87.6%
857 OlympiadBench O Math (olympiad) Advanced mathematics olympiad-style problem benchmark. ★★★★★ 🇨🇳 Qwen3-30B-A3B-Instruct-2507 77.6%
858 OlympicArena o Math (competition) Olympiad-style mathematics reasoning benchmark. ★★★★★ 🇨🇳 DeepSeek V3 76.2%
859 OMEGA O Math (advanced) OMEGA olympiad-grade mathematics reasoning benchmark. ★★★★★ 🇺🇸 OLMo-3.1-Think-32B 53.4%
860 Omni-MATH O Math reasoning Omni-MATH benchmark covering diverse math reasoning tasks across difficulty levels. ★★★★★ G Ling 1T 74.5%
861 Omni-MATH-HARD O Math Challenging math benchmark (Omni-MATH-HARD). ★★★★★ 🇺🇸 GPT-5 (high) 73.6%
862 Omni3D o 3D detection Omni3D monocular 3D object detection benchmark across indoor/outdoor datasets (AP3D). ★★★★★ 🇨🇳 Qwen3.5-4B 47.4%
863 OmniDocBench O Document understanding Document understanding benchmark covering multi-page layouts, tables, and charts for robust question answering. Overall score, higher is better (v1.0 edit-distance results are in omnidocbench-edit). ★★★★★ G PaddleOCR-VL-1.5 94.5%
864 OmniDocBench 1.5 O OCR Document understanding benchmark v1.5 with OCR evaluation. Overall Edit Distance metric, lower is better. ★★★★★ G PaddleOCR-VL ↓ 0.024
865 OmniDocBench-CN O Document understanding (Chinese) Chinese subset of OmniDocBench focusing on OCR-grounded document comprehension and reasoning. ★★★★★ G PP-StructureV3 ↓ 13.6%
866 OmniDocBench (Edit Distance) O Document understanding OmniDocBench v1.0 overall normalized edit distance (x100), lower is better. ★★★★★ G Gundam-M ↓ 12.3%
867 OmniDocBench v1.6 O Document parsing OmniDocBench v1.6 end-to-end document parsing; Overall score (higher is better) combining text edit, formula CDM and table TEDS. ★★★★★ G Youtu-Parsing-Omni 97.0%
868 OmniGAIA o Multimodal agent OmniGAIA: GAIA-style omni-modal (image/video/audio) agentic perception and reasoning benchmark. ★★★★★ 🇺🇸 Gemini 3 Pro 62.5%
869 OmniMedVQA o Medical QA Large-scale medical VQA across 12 imaging modalities. ★★★★★ G MedMO-8B-Next 93.3%
870 OmniMMI o Multimodal interaction OmniMMI benchmark for multimodal interaction across video streams. ★★★★★ G Seed1.8 53.0%
871 OmniSpatial o Spatial reasoning Spatial understanding and reasoning benchmark (OmniSpatial). ★★★★★ 🇨🇳 GLM-4.6V 52.0%
872 OneIG-Bench EN O Text-to-image OneIG-Bench English subset score for text-to-image generation. ★★★★★ G Nano Banana 2.0 0.578
873 OneIG-Bench ZH O Text-to-image OneIG-Bench Chinese subset score for text-to-image generation. ★★★★★ G Nano Banana 2.0 0.567
874 OneMillionBench O Agentic search / expert tasks OneMillion Bench: expert-level, economically valuable tasks evaluated with tools (search). ★★★★★ 🇺🇸 Claude Opus 4.7 73.0%
875 Online-Mind2web O Web automation Online web automation and task execution benchmark. ★★★★★ 🇺🇸 GPT-5.4 92.8%
876 Open Rewrite o Instruction following Rewrite benchmark assessing open-ended editing and directive-following quality. ★★★★★ G MobileLLM P1 51.0%
877 OpenAI Internal Advanced Cybersecurity Completion Rate o Cybersecurity OpenAI internal eval (Daybreak announcement): share of advanced, dual-use cybersecurity requests (exploit chains, auth bypass, privilege escalation) the model completes. ★★★★★ 🇺🇸 GPT-5.6 Cyber 95.0%
878 OpenAI Internal Broken Search Tool Non-Disclosure o Safety OpenAI internal alignment eval: rate of failing to disclose a broken search tool (lower is better). ★★★★★ 🇺🇸 GPT-6 Astra ↓ 1.5%
879 OpenAI Internal Circumvention o Safety OpenAI internal eval reported in the GPT-6 Astra announcement: rate of circumventing safeguards (lower is better). ★★★★★ 🇺🇸 GPT-6 Astra
880 OpenAI Internal Coding Deception o Safety OpenAI internal alignment eval: fraction of coding-agent answers with detected deception (lower is better). ★★★★★ 🇺🇸 GPT-6 Astra ↓ 0.5%
881 OpenAI Internal Computer-Use Safety o Safety OpenAI internal eval reported in the GPT-6 Astra announcement: rate of unsafe actions during computer use (lower is better). ★★★★★ 🇺🇸 GPT-6 Astra ↓ 2.4%
882 OpenAI Internal Computer-Use Safety (AutoReview) o Safety OpenAI internal eval reported in the GPT-6 Astra announcement: unsafe computer-use action rate with AutoReview enabled (lower is better). ★★★★★ 🇺🇸 GPT-6 Astra ↓ 1.8%
883 OpenAI Internal Data Science Tasks o Data science OpenAI internal eval reported in the GPT-6 Astra announcement: data-science tasks. ★★★★★ 🇺🇸 GPT-6 Astra 40.9%
884 OpenAI Internal Database Migration Tasks o Software engineering OpenAI internal eval reported in the GPT-6 Astra announcement: database migration coding tasks. ★★★★★ 🇺🇸 GPT-6 Astra 63.9%
885 OpenAI Internal Design Tasks o Design OpenAI internal eval reported in the GPT-6 Astra announcement: professional design tasks. ★★★★★ 🇺🇸 GPT-6 Astra 50.0%
886 OpenAI Internal Expert-SWE o Software engineering OpenAI internal eval (GPT-5.5 announcement): expert-level, long-horizon software-engineering tasks. ★★★★★ 🇺🇸 GPT-5.5 73.1%
887 OpenAI Internal Factual Error Rate (Difficult Prompts) o Factuality OpenAI internal factuality eval on de-identified ChatGPT conversations where users flagged factual errors; error rate (lower is better). ★★★★★ 🇺🇸 GPT-6 Astra ↓ 3.9%
888 OpenAI Internal Hallucination o Hallucination OpenAI internal eval reported in the GPT-6 Astra announcement: hallucination rate (lower is better). ★★★★★ 🇺🇸 GPT-6 Astra ↓ 4.2%
889 OpenAI Internal Investment Banking Modeling Tasks o Finance OpenAI internal eval (GPT-5.4/GPT-5.5 announcements): investment-banking spreadsheet modeling tasks. ★★★★★ 🇺🇸 GPT-5.5 Pro 88.6%
890 OpenAI Internal Management Consulting Tasks o Professional work OpenAI internal eval (GPT-5.6 announcement): management-consulting tasks. ★★★★★ 🇺🇸 GPT-5.6 Sol 43.2%
891 OpenAI Internal Research Debugging Evaluation o AI research automation OpenAI internal eval (GPT-5.6 announcement): debugging real AI research systems. ★★★★★ 🇺🇸 GPT-5.6 Sol 68.3%
892 OpenAI Internal Reviewer Bypass Attempts o Safety OpenAI internal alignment eval: rate of attempts to bypass a reviewer (lower is better). ★★★★★ 🇺🇸 GPT-6 Astra
893 OpenAI Internal Unauthorized Agent Interaction o Safety OpenAI internal alignment eval: rate of unauthorized interaction with other agents (lower is better). ★★★★★ 🇺🇸 GPT-6 Astra
894 OpenAI Internal Vulnerability Discovery and Report Writing o Cybersecurity OpenAI internal eval (Daybreak announcement): open-ended vulnerability finding in a repo with a known vulnerability, PoC and report quality. ★★★★★ 🇺🇸 GPT-5.6 Sol 17.9%
895 OpenAI Internal Warning Circumvention o Safety OpenAI internal alignment eval: rate of circumventing warnings (lower is better). ★★★★★ 🇺🇸 GPT-6 Astra ↓ 17.4%
896 OpenAI Internal Zero-Day Discovery Eval o Cybersecurity OpenAI internal eval (Daybreak announcement): find zero-day vulnerabilities in current open-source releases, build max-impact PoCs and write-ups; graded on severity, impact and calibration. ★★★★★ 🇺🇸 GPT-5.6 Cyber 24.5%
897 OpenBookQA O Science QA Open-book multiple choice science questions with supporting facts. ★★★★★ 128 🇺🇸 Hermes 4.3 36B Pyche 96.6%
898 OpenEQA O Embodied QA Open-vocabulary embodied question answering over episodic memory of indoor scenes. ★★★★★ 🇺🇸 GPT-5 64.4%
899 OpenRewrite-Eval o Rewrite quality OpenRewrite evaluation; micro-averaged RougeL. ★★★★★ 🇨🇳 Qwen2.5 1.5B Instruct 46.9
900 OpenScore String Quartets o Optical music recognition Optical music recognition on OpenScore string quartet scores; score is 1 - OMR normalized edit distance. ★★★★★ 🇺🇸 GPT-6 Astra 0.840
901 OptMATH o Math optimization reasoning OptMATH benchmark targeting challenging math optimization and problem-solving tasks. ★★★★★ G Ling 1T 57.7%
902 Order 15 Items o List ordering Ordering benchmark requiring models to sequence 15 items correctly. ★★★★★ G K2-V2 87.6%
903 Order 30 Items o List ordering (long) Ordering benchmark requiring models to sequence 30 items correctly. ★★★★★ G K2-V2 40.3%
904 OSWorld O GUI agents Agentic GUI task completion and grounding on desktop environments. ★★★★★ 🇺🇸 Claude Opus 4.6 72.7%
905 OSWorld 2.0 O GUI agents OSWorld 2.0 long-horizon computer-use benchmark on a live Ubuntu VM; partial-credit score. ★★★★★ 🇺🇸 Claude Opus 5.5 81.8%
906 OSWorld 2.0 (Offline) O GUI agents OSWorld 2.0 offline set (v2026.08.08), partial-credit score on long-horizon computer-use workflows without internet access. ★★★★★ 🇺🇸 GPT-6 Astra 73.5%
907 OSWorld 2.0 (Strict) O GUI agents OSWorld 2.0 strict pass rate (full task success). ★★★★★ 🇺🇸 Claude Opus 5.5 48.7%
908 OSWorld 2.1 o GUI agents OSWorld 2.1 long-horizon computer-use benchmark; partial-credit score. ★★★★★ 🇺🇸 Claude Opus 5.5 81.8%
909 OSWorld 2.1 (Offline) o GUI agents OSWorld 2.1 offline subset (82 tasks), partial-credit score on long-horizon computer-use workflows without internet access. ★★★★★ 🇺🇸 Claude Sonnet 5.5 83.9%
910 OSWorld 2.1 (Strict) o GUI agents OSWorld 2.1 strict pass rate (full task success). ★★★★★ 🇺🇸 Claude Opus 5.5 48.7%
911 OSWorld-G O GUI agents OSWorld-G center accuracy (no_refusal). ★★★★★ G Nex-N2.5-Pro 87.4%
912 OSWorld Verified O GUI agents Verified subset of OSWorld GUI agent benchmark. ★★★★★ 🇨🇳 Qwen-CUA 86.2%
913 OSWorld2 o GUI agents Second-generation OSWorld GUI agent benchmark. ★★★★★ 🇨🇳 GLM-4.5V 35.8%
914 OVBench O Open-vocabulary streaming Open-vocabulary benchmark for streaming video understanding. ★★★★★ G Seed2.1 Pro 70.0%
915 OvisOCR In-house KIE Bench o Key information extraction OvisOCR team in-house key information extraction benchmark. ★★★★★ G OvisOCR3 77.7%
916 OVOBench O Streaming video QA Streaming video QA benchmark with open-vocabulary queries. ★★★★★ G Seed2.1 Pro 80.7%
917 PAgentBench o Personal assistant agents iFlytek in-house Personal Assistant Benchmark: task management, file operations and system interactions. ★★★★★ G Spark-X2.6-4B 74.1%
918 PaperBench Code-Dev O Code understanding PaperBench developer subset measuring code reasoning accuracy. ★★★★★ 🇨🇳 Qwen3.8-Max 93.0%
919 PaperBench O Research paper understanding Benchmark for understanding and reasoning over research papers. ★★★★★ 🇺🇸 Claude Opus 4.5 Thinking 72.9%
920 ParseBench O Document parsing Document-parsing benchmark covering tables, charts, content faithfulness, semantic formatting and visual grounding; overall over 5 categories. ★★★★★ G LightOnOCR-3-4B 75.1%
921 Part-Affordance O Embodied reasoning Part-level affordance grounding benchmark. ★★★★★ G PhysBrain1.5-8B 84.0%
922 PathVQA o Medical QA Pathology visual question answering. ★★★★★ G MedMO-4B-Next 73.3%
923 PAWS-X o Paraphrase identification Cross-lingual adversarial paraphrase identification. ★★★★★ G d1-omni-600M 79.5%
924 PenguinScrolls o Long-context eval Long-context reading comprehension and reasoning benchmark. ★★★★★ 🇨🇳 Hunyuan-Large-Instruct 85.2%
925 PerceptionBench O Visual perception Moonshot benchmark of atomic visual perception capabilities in multimodal LLMs. ★★★★★ 🇺🇸 Gemini 3.5 Flash 60.9%
926 Perception Test O Video understanding DeepMind Perception Test: multimodal video benchmark of perception and reasoning skills (MC video QA). ★★★★★ G PLM-8B 82.7%
927 PerMemSafe o Memory safety Personal-memory safety benchmark: appropriate use and protection of stored user memories. ★★★★★ G MemT-1.0-Nano 38.7%
928 PersonaMem o Personalization Evaluates whether models track evolving user personas and preferences across long interaction histories. ★★★★★ G MemT-1.0-Flash 61.6%
929 Perspective Bench o Value alignment Re-alignment score over 80 prompts measuring adherence to the Public AI Constitution (balanced perspectives). ★★★★★ G Snowdon1.1-Small 93.0%
930 PhishNChips O Phishing detection Phishing vs legitimate email-and-link decisions; accuracy (Jev Decision Index suite, posed as typed decisions). ★★★★★ 🇺🇸 DiffusionGemma Jev 85.4%
931 PHYBench O Physics reasoning Physics reasoning and calculation benchmark. ★★★★★ 🇺🇸 Gemini 3.1 Pro 82.0%
932 PhyX O Physics reasoning (multimodal) Multimodal physics reasoning benchmark (PhyX). ★★★★★ 🇨🇳 Qwen3.8-Max 83.5%
933 PinchBench O Agentic assistant tasks Benchmark of OpenClaw agent task success across real-world personal-assistant tasks. ★★★★★ G KAT-Coder-V2.5-Dev 93.4%
934 PinchBench (Avg) O Agent tasks (OpenClaw) PinchBench OpenClaw-style agent task benchmark, average score across runs. ★★★★★ G KAT-Coder-Pro V2.5 94.9%
935 PinchBench (Best) O Agent tasks (OpenClaw) PinchBench OpenClaw-style agent task benchmark, best-of-runs score. ★★★★★ 🇺🇸 Claude Opus 4.6 93.3%
936 PIOBench O Embodied reasoning Pointing / interaction-object grounding benchmark for embodied VLMs. ★★★★★ 🇺🇸 Gemini 3.6 Flash 80.9%
937 PIQA O Physical commonsense Physical commonsense about everyday tasks and object affordances. ★★★★★ G LLaDA2.0 Flash 96.5%
938 PixMo-Points O Embodied reasoning Pointing accuracy on the PixMo-Points evaluation set. ★★★★★ 🇺🇸 GPT-6 Astra 75.2%
939 PixmoCount O Visual counting Counting objects/instances in images (PixmoCount). ★★★★★ G Eagle2.5-8B 90.2%
940 PLawBench o Legal reasoning Legal-practice benchmark; Qwen evaluates it with gemini-3.1-pro-preview as judge. ★★★★★ 🇨🇳 Qwen3.8-Max 73.2%
941 PMC-VQA O Medical VQA PubMed Central visual question answering benchmark for biomedical images. ★★★★★ G MedMO-4B-Next 75.7%
942 Point-Bench O Pointing and counting Benchmark for pointing and counting objects in images. ★★★★★ 🇺🇸 Gemini 2.5 Pro 85.5%
943 PointArena o Visual grounding Language-guided pointing benchmark for multimodal models. ★★★★★ 🇺🇸 GPT-6 Astra 75.4%
944 PolyMATH O Math reasoning Polyglot mathematics benchmark assessing cross-topic math reasoning. ★★★★★ 🇨🇳 Qwen3.7-Max 86.5%
945 POP909-CL (decision) O Music understanding Chord-label decisions over POP909 symbolic music; accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. ★★★★★ G Bespoke-Nimble-9B-v3 74.6%
946 POPE O Hallucination detection Vision-language hallucination benchmark focusing on object existence verification. ★★★★★ 🇨🇳 InternVL2.5-4B 90.9%
947 PopQA O Knowledge / QA Open-domain popular culture question answering benchmark testing long-tail factual recall. ★★★★★ 🇺🇸 Llama 3.1 Tulu 3 405B SFT 55.7%
948 PostTrainBench O Post-training automation Measures how well AI agents can post-train base LLMs under fixed compute/time constraints; average score across AIME 2025, BFCL, GPQA Main, GSM8K, and HumanEval. ★★★★★ 🇺🇸 Claude Opus 5.5 49.3%
949 PostTrainBench Lite o Post-training automation Lite variant of PostTrainBench (agents post-training base LLMs under fixed compute). ★★★★★ 🇺🇸 GPT-5.6 Terra 51.5%
950 PRBench Finance o Professional reasoning (finance) PRBench finance subset: rubric-graded professional reasoning tasks in finance. ★★★★★ 🇨🇳 Qwen3.8-Max 58.3%
951 PRBench Legal O Professional reasoning (legal) PRBench legal subset: rubric-graded professional reasoning tasks in law. ★★★★★ 🇺🇸 Claude Fable 5 57.6%
952 PRDBench o Agentic coding Product Requirements Document benchmark for evaluating agentic coding capabilities. ★★★★★ 🇨🇳 LongCat-Flash-Lite 39.6%
953 PresentBench o Agentic office work (slides) Benchmark for generating presentation decks from source materials, scored with rubrics. ★★★★★ 🇺🇸 GPT-5.5 68.9%
954 ProcBench o Procedural reasoning Procedural reasoning benchmark evaluating step-by-step logical reasoning. ★★★★★ G Seed2.0 Pro 96.6%
955 ProgramBench O Software engineering ProgramBench (200 instances): agents re-implement complete programs from scratch; score as reported in GLM-5.2 release. ★★★★★ 🇺🇸 GPT-6 Astra 85.4%
956 ProgramBench (Almost Solved) O Software engineering ProgramBench share of instances that are almost solved (near-complete test pass rate). ★★★★★ 🇺🇸 Claude Opus 5 37.0%
957 PrOntoQA O Logical reasoning Probing ontological reasoning via question answering. ★★★★★ G Ling Flash 2.0 97.9%
958 ProofBench Advanced O Mathematical proofs (advanced) Advanced mathematical proof benchmark covering complex theorem proving tasks. ★★★★★ G Aletheia 91.9%
959 ProofBench Basic o Mathematical proofs Entry-level mathematical proof benchmarking set. ★★★★★ 🇨🇳 DeepSeekMath-V2-Heavy 99.0%
960 ProofNet (Test) O Formal theorem proving ProofNet test split: undergraduate-level mathematics theorems formalized in Lean 4; score = share proved. ★★★★★ 🇨🇳 LongCat-Flash-Prover 47.3%
961 ProtocolQA o Protocol understanding and QA Protocol question answering benchmark evaluating understanding of scientific protocols and procedures. ★★★★★ 🇺🇸 Grok 4.1 (Thinking) 79.0%
962 ProverBench O Formal theorem proving ProverBench (DeepSeek-Prover-V2): 325 formalized problems incl. AIME and textbook problems in Lean 4; score = share proved. ★★★★★ 🇨🇳 LongCat-Flash-Prover 66.5%
963 PubMedQA O Biomedical QA Yes/no/maybe question answering over PubMed abstracts. ★★★★★ G MedMO-4B-Next 78.2%
964 PubTabNet o Table recognition Image-based table recognition benchmark from PubMed Central articles; TEDS score on validation set. ★★★★★ G Infinity-Parser2-Pro 94.8%
965 PureDocBench o Document parsing Document parsing benchmark (reported in OvisOCR3 release). ★★★★★ G OvisOCR3 82.3%
966 Putnam 2025 o Math (competition, proof writing) Natural-language proof solutions to the 2025 William Lowell Putnam Mathematical Competition (12 problems, 10 points each); score = points out of 120. ★★★★★ G Nomos 1 87
967 PutnamBench O Formal theorem proving 672 Putnam competition problems formalized in Lean 4; score = share of problems proved. ★★★★★ G Aleph Prover 99.4%
968 Q-Bench O Low-level visual perception Q-Bench (A1 perception, test split) evaluating MLLMs on low-level visual attributes and image quality questions. ★★★★★ 🇺🇸 GPT-4V(ision) 74.1%
969 Q-Spatial-Bench O Embodied reasoning Quantitative spatial reasoning (metric distances/sizes) benchmark for VLMs. ★★★★★ 🇺🇸 Gemini 3.6 Flash 87.1%
970 QIVD o Audio-visual QA Qualcomm Interactive Video Dataset: situated audio-visual question answering on live-style video with speech questions; accuracy. ★★★★★ 🇨🇳 Qwen3.5-Omni-Plus-Realtime 59.0%
971 QReCC O Conversational QA Open-domain conversational QA with question rewriting (ChatRAG-Bench subset). ★★★★★ G Yuan3.0 Ultra 57.3%
972 QuAC O Conversational QA Question answering in context. ★★★★★ G Yuan3.0 Ultra 54.5%
973 QuALITY o Long-context reading comprehension Long-document multiple-choice reading comprehension benchmark. ★★★★★ 🇺🇸 Llama-3_1-70B-TFree-HAT-SFT 48.8
974 Query Sufficiency o Legal query triage Judging whether a legal research query contains sufficient information to answer (Thomson Reuters eval). ★★★★★ G Thomson-1.0-Small 59.4%
975 QVHighlights o Video temporal grounding Video moment retrieval / highlight detection benchmark; R1@0.5. ★★★★★ G Nova 2 Lite 77.2%
976 QwenClawBench O Agent tasks (OpenClaw) Real-user-distribution Claw (OpenClaw-style) agent benchmark released by Qwen. ★★★★★ 🇺🇸 Claude Opus 4.6 65.5%
977 R-Bench O Robustness (multimodal) R-Bench: robustness of LMMs to real-world image corruptions (discriminative subset). ★★★★★ 🇺🇸 GPT-4o-2024-05-13 77.7%
978 RACE O Reading comprehension English exams for middle and high school. ★★★★★ 🇫🇷 Mistral Large 3 Base 93.3%
979 RACE-High O Reading comprehension RACE reading-comprehension questions from Chinese high-school English exams. ★★★★★ G Orion-14B-Base 91.3%
980 RACE-Middle O Reading comprehension RACE reading-comprehension questions from Chinese middle-school English exams. ★★★★★ 🇺🇸 GPT-4 93.7%
981 RAGTruth O Hallucination detection Response-level hallucination detection for RAG answers; F1 on the hallucinated class. ★★★★★ G pplx-decider-v1-27b 88.8
982 Random Complex Tasks o Agentic tasks (random) Randomly constructed complex task environments for agent generalization. ★★★★★ 🇨🇳 LongCat-Flash-Thinking-2601 35.8%
983 Real5-OmniDocBench O Document parsing (real-world distortions) OmniDocBench v1.5 pages re-captured under five real-world conditions (scanning, warping, screen photography, illumination, skew); Overall score. ★★★★★ G OvisOCR3 93.6%
984 Realbench o Web browsing Real-world browsing and QA benchmark. ★★★★★ G Seed1.8 49.1%
985 RealKIE-FCC Verified O Key information extraction Human-verified version of the RealKIE FCC document key-information-extraction benchmark (ALNS). ★★★★★ 🇺🇸 Gemini 3 Pro Preview 71.7%
986 RealWorldQA O Real-world visual QA Visual question answering with real-world images and scenarios. ★★★★★ 🇨🇳 Qwen3.8-Flash-Next 88.5%
987 Recruit-Bench o Recruitment / HR tasks Benchmark of recruiting-domain tasks (resume screening, job matching, candidate communication) used by Nanbeige. ★★★★★ 🇺🇸 Gemma 4 12B IT 69.4%
988 Ref-L4 (test) o Referring expressions Ref-L4 referring expression comprehension on the test split. ★★★★★ 🇨🇳 GLM-4.6V 88.9%
989 RefCOCO O Referring expressions RefCOCO average accuracy at IoU 0.5 (val). ★★★★★ 🇨🇳 DeepSeek-VL2 95.1%
990 RefCOCOg O Referring expressions RefCOCOg average accuracy at IoU 0.5 (val). ★★★★★ 🇨🇳 DeepSeek-VL2 92.8%
991 RefCOCO+ O Referring expressions RefCOCO+ accuracy at IoU 0.5 on the val split. ★★★★★ 🇨🇳 DeepSeek-VL2 91.2%
992 RefSpatialBench O Spatial reasoning Reference spatial understanding benchmark covering spatial grounding tasks. ★★★★★ 🇺🇸 GPT-6 Astra 80.1%
993 RefusalBench o Safety / refusal Safety-oriented refusal and policy adherence benchmark. ★★★★★ 🇺🇸 Hermes 4.3 36B Pyche 72.3%
994 ReMI o Multimodal reasoning Reasoning over multimodal inputs (ReMI). ★★★★★ G Step3-VL-10B 67.3%
995 RepoBench O Code understanding Repository-level code comprehension and reasoning benchmark. ★★★★★ 🇺🇸 Claude Sonnet 4.5 83.8%
996 ResearchRubrics O Research evaluation Benchmark evaluating model ability to conduct research and synthesize findings. ★★★★★ 🇨🇳 Kimi-K3 76.2%
997 RewardBench O Reward modeling Preference/reward model evaluation benchmark. ★★★★★ 🇺🇸 INF-ORM-Llama3.1-70B 95.1%
998 RiemannBench o Research math Research-level mathematics benchmark (reported in Anthropic Fable 5/Mythos 5 system card). ★★★★★ 🇺🇸 Gemini 4 Argon 76.0%
999 RoboAfford O Embodied reasoning Robotic affordance (object & spatial) grounding benchmark. ★★★★★ G ACE-Brain-0.6-9B 88.1%
1000 RoboRefIt O Embodied reasoning Referring-expression grounding in cluttered robotic scenes. ★★★★★ G PhysBrain1.5-8B 89.6%
1001 RoboSpatialHome O Embodied spatial understanding RoboSpatialHome benchmark for embodied spatial reasoning in domestic environments. ★★★★★ 🇺🇸 Gemini 4 Argon 80.9%
1002 RobotGrounding (in-house) o Embodied grounding ACE Robotics in-house robot grounding benchmark. ★★★★★ G ACE-Brain-0.6-9B 93.4%
1003 RoboVQA O Embodied reasoning Robotics video question answering benchmark. ★★★★★ G PhysBrain1.5-2B 62.9%
1004 Roo Code Evals O Code assistant eval Community-maintained coding evals and leaderboard by Roo Code. ★★★★★ 🇺🇸 Claude Opus 4.6 100.0%
1005 RouterBench (decision) O Model routing Choose which LLM to route a query to; selected quality (quality objective). Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. ★★★★★ G Kev 9B 80.0
1006 RULER-100 @1M o Long-context eval RULER-100 evaluation at a 1M context window. ★★★★★ 🇺🇸 NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 86.3%
1007 RULER-100 @256k o Long-context eval RULER-100 evaluation at a 256k context window. ★★★★★ 🇺🇸 NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 92.9%
1008 RULER-100 @512k o Long-context eval RULER-100 evaluation at a 512k context window. ★★★★★ 🇺🇸 NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 91.3%
1009 Ruler 128k O Long-context eval RULER benchmark at 128k context window. ★★★★★ 🇨🇳 Qwen3-Next-80B-A3B-Instruct 96.0%
1010 Ruler 16k O Long-context eval RULER benchmark at 16k context window. ★★★★★ 🇨🇳 Qwen2.5-72B-Instruct 97.7%
1011 Ruler 1M O Long-context eval RULER benchmark at 1M context window. ★★★★★ 🇨🇳 Kimi-Linear-Instruct 94.8%
1012 Ruler 256k O Long-context eval RULER benchmark at 256k context window. ★★★★★ 🇨🇳 Qwen3.5-122B-A10B 96.7%
1013 Ruler 32k O Long-context eval RULER benchmark at 32k context window. ★★★★★ 🇨🇳 Qwen2.5-72B-Instruct 96.5%
1014 Ruler 4k O Long-context eval RULER benchmark at 4k context window. ★★★★★ 🇨🇳 Qwen2.5 14B Instruct 97.7%
1015 Ruler 512k O Long-context eval RULER benchmark at 512k context window. ★★★★★ 🇨🇳 Qwen3.5-122B-A10B 96.0%
1016 Ruler 64k O Long-context eval RULER benchmark at 64k context window. ★★★★★ 🇺🇸 Nemotron 3 Ultra Base 95.3%
1017 Ruler 8k O Long-context eval RULER benchmark at 8k context window. ★★★★★ 🇨🇳 Qwen2.5-72B-Instruct 97.2%
1018 RW Search O Agentic search Real-world search benchmark evaluating retrieval and reasoning. ★★★★★ 🇺🇸 GPT-5.5 85.3%
1019 SaaS-Bench o Agentic SaaS tasks Benchmark of agents completing tasks in SaaS application environments (as reported in the Kimi K3 release). ★★★★★ 🇺🇸 GPT-5.6 Sol 61.4%
1020 SafetyBench o Safety Multiple-choice safety evaluation across 7 categories in Chinese and English. ★★★★★ 🇺🇸 GPT 4 0613 89.7%
1021 SALAD-Bench o Safety alignment Safety Alignment and Dangerous-behavior benchmark evaluating harmful assistance and refusal consistency. ★★★★★ G Granite-4.1-3B ↓ 94.0%
1022 SArena (Icon) o SVG generation SVG Arena benchmark for icon generation evaluation. ★★★★★ G Intern-S1-Pro 83.5%
1023 SAT O Spatial reasoning Spatial Aptitude Training benchmark of static and dynamic spatial reasoning questions for VLMs. ★★★★★ 🇺🇸 GPT-6 Astra 96.7%
1024 SAT Math o Math (exam) SAT mathematics questions (DeepSeekMath / Minerva-style evaluation). ★★★★★ 🇨🇳 DeepSeek Math 7B 84.4%
1025 SATA-Bench O Multiple-answer QA Select-all-that-apply multiple-choice questions; case exact accuracy (Jev Decision Index suite, posed as typed decisions). ★★★★★ G vLLM-SR d3 38.4%
1026 Scale AI Multi Challenge O Chat & instruction following Scale AI Multi Challenge crowd-evaluated instruction following benchmark. ★★★★★ 🇨🇳 Qwen3.5-122B-A10B 61.5%
1027 SciCode (sub) O Code SciCode subset score (sub). ★★★★★ 🇺🇸 GPT-5.5 95.1%
1028 SciCode (main) O Code SciCode main score. ★★★★★ 🇺🇸 GPT-5.5 90.6%
1029 ScienceQA O Science QA (multimodal) Multiple-choice science questions with images, diagrams, and text context. ★★★★★ G Youtu-VL-4B-Instruct 97.0%
1030 SciQ O Science QA Multiple choice science questions. ★★★★★ 🇨🇳 Qwen3 8B Base 96.1%
1031 SciReasoner O Scientific reasoning Scientific reasoning benchmark evaluating multimodal AI models on scientific tasks. ★★★★★ G Intern-S2-Preview-397B 64.0%
1032 SciRes FrontierMath Tier 1-3 O Math (frontier) SciRes FrontierMath benchmark covering tiers 1-3. ★★★★★ 🇺🇸 GPT-5.6 Sol 89.0%
1033 SciRes FrontierMath Tier 4 O Math (frontier) SciRes FrontierMath benchmark covering tier 4. ★★★★★ 🇺🇸 GPT-6 Astra 97.6%
1034 ScreenQA Complex O GUI QA Complex ScreenQA benchmark accuracy. ★★★★★ 🇺🇸 Holo1.5-72B 87.1%
1035 ScreenQA Short O GUI QA Short-form ScreenQA benchmark accuracy. ★★★★★ 🇺🇸 Holo1.5-72B 91.9%
1036 ScreenSpot O Screen UI locators Center accuracy on ScreenSpot. ★★★★★ 🇨🇳 Qwen3-VL 32B Instruct 95.8%
1037 ScreenSpot-Pro O Screen UI locators Average center accuracy on ScreenSpot-Pro. ★★★★★ 🇺🇸 GPT-6 Astra 92.7%
1038 ScreenSpot-Pro (with tools) o Screen UI locators (with tools) ScreenSpot-Pro evaluated with tool access (e.g. zoom/crop). ★★★★★ 🇺🇸 Claude Opus 4.8 87.9%
1039 ScreenSpot-v2 O Screen UI locators Center accuracy on ScreenSpot-v2. ★★★★★ G MAI-UI-32B 96.5%
1040 SEAL-0 O Agentic web search Evaluation of multi-step browsing agents on search, evidence gathering, and synthesis tasks. ★★★★★ G MiroThinker-H1 61.3%
1041 SEC-Bench Pro O Cybersecurity Proof-of-concept exploit generation for vulnerabilities in complex software. ★★★★★ 🇺🇸 GPT-6 Astra 85.4%
1042 SecCodeBench o Secure code generation Benchmark evaluating secure code generation capabilities. ★★★★★ 🇺🇸 GPT-5.2 68.7%
1043 SEED-Bench-2-Plus O Multimodal evaluation SEED-Bench-2-Plus overall accuracy. ★★★★★ 🇺🇸 Claude 3.7 Sonnet 72.9%
1044 SEED-Bench-Img O Multimodal image understanding SEED-Bench image-only subset (SEED-Bench-Img). ★★★★★ G Bagel 14B 78.5%
1045 SEED-Bench O Multimodal evaluation SEED-Bench comprehensive multimodal understanding benchmark evaluating generative comprehension across multiple dimensions. ★★★★★ 🇨🇳 Qwen3-VL-4B-Instruct 77.0%
1046 SFE O Multimodal reasoning Structured factual evaluation for multimodal models. ★★★★★ 🇺🇸 Gemma 4 26B A4B IT 63.1%
1047 SGD/SGD-X (decision) O Dialogue state tracking Schema-Guided Dialogue (SGD-X) intent/slot decisions; macro-F1. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. ★★★★★ G Bespoke-Nimble-9B-v3 71.8
1048 SGI-Bench O Scientific agents Scientific general intelligence benchmark of agentic scientific research workflows. ★★★★★ G Intern-S2-Preview 52.5%
1049 ShareRobot-Affordance o Embodied affordance ShareRobot affordance prediction benchmark. ★★★★★ G ACE-Brain-0.6-9B 48.8%
1050 ShareRobot-Trajectory O Embodied reasoning ShareRobot trajectory prediction benchmark. ★★★★★ G RoboBrain2.5-8B 85.2%
1051 Showdown O GUI agents Success rate on the Showdown UI interaction benchmark. ★★★★★ G Holo2-235B-A22B 80.4%
1052 SIFO o Instruction following Single-turn instruction following benchmark. ★★★★★ 🇨🇳 Qwen3-VL-30B-A3B Thinking 66.9%
1053 SIFO Multiturn o Instruction following Multi-turn SIFO benchmark for sustained instruction adherence. ★★★★★ 🇨🇳 Qwen3-VL-30B-A3B Thinking 60.3%
1054 SimpleQA O QA Simple question answering benchmark. ★★★★★ 🇨🇳 ERNIE 5.0 74.0%
1055 SimpleQA (with search) O QA SimpleQA answered with web search / tool use enabled (agentic setting), not closed-book. ★★★★★ 🇨🇳 DeepSeek V3.2-Exp 97.1%
1056 SimpleQA Verified O QA Verified SimpleQA variant for parametric knowledge accuracy. ★★★★★ 🇺🇸 Gemini 3.1 Pro 77.3%
1057 SimpleVQA O General VQA Lightweight visual question answering set with everyday scenes. ★★★★★ 🇨🇳 Qwen3.7-Plus 81.7%
1058 SimpleVQA-DS o General VQA SimpleVQA variant curated by DeepSeek with everyday image question answering tasks. ★★★★★ 🇨🇳 Seed1.5-VL-Thinking 61.3%
1059 SimpleVQA (With Tools) O Visual factuality SimpleVQA visual factuality benchmark evaluated with tool access. ★★★★★ G Step 3.7 Flash 79.2%
1060 Social Interaction QA (SIQA) O Social commonsense QA Social Interaction QA benchmark evaluating social commonsense and situational reasoning. ★★★★★ 🇨🇳 Qwen 14B 78.1%
1061 SkillsBench O Agent skills SkillsBench agentic tasks; Qwen reports avg of 5 runs on the 78-task self-contained subset (API-dependent tasks excluded). ★★★★★ 🇺🇸 GPT-5.6 Sol 73.5%
1062 SkillsBench v1.1 o Agent skills Public SkillsBench v1.1 benchmark (87 tasks), average score over three runs per task. ★★★★★ 🇺🇸 GPT-5.6 Sol 73.5%
1063 SLAKE O Medical VQA Semantically-Labeled Knowledge-Enhanced medical visual question answering benchmark. ★★★★★ 🇨🇳 Qwen3.8-Max 90.8%
1064 SmolInstruct o Small molecule understanding Small molecule instruction-following and understanding benchmark. ★★★★★ G Intern-S1-Pro 74.8%
1065 SOB (Structured Output Benchmark) o Structured output Structured Output Benchmark: extract values into a JSON schema from text, images and audio (5,324 records); value accuracy over exact field matches. ★★★★★ G Interfaze 1.1 Lite 81.5%
1066 SocialIQA O Social commonsense Social interaction commonsense QA. ★★★★★ 🇨🇳 Qwen3 8B Base 83.3%
1067 SPAR-Bench o Spatial reasoning Spatial perception and reasoning benchmark derived from the SPAR dataset (single- and multi-view). ★★★★★ 🇺🇸 GPT-5.2 55.1%
1068 SpatialSense VQA o Spatial reasoning SpatialSense spatial-relation recognition posed as VQA (as used in Gemma/PaliGemma evaluations). ★★★★★ 🇺🇸 T5Gemma 2 4B-4B 51.7%
1069 SpatialViz O Mental visualization Mental visualization benchmark. ★★★★★ 🇺🇸 GPT-5.2 65.8%
1070 Spider O Text-to-SQL Complex text-to-SQL benchmark over cross-domain databases. ★★★★★ G Yuan3.0 Ultra 83.9%
1071 Spider 2.0-Lite o Text-to-SQL Spider 2.0-Lite enterprise text-to-SQL over real schemas, scored by executing the query (Interfaze reports the 135-task SQLite subset). ★★★★★ G Interfaze 52.9%
1072 Spiral-Bench O Safety / sycophancy A LLM-judged benchmark measuring sycophancy and delusion reinforcement. ★★★★★ 🇺🇸 GPT-5 87.0%
1073 SpreadsheetBench 2 O Spreadsheet manipulation Second version of SpreadsheetBench real-world spreadsheet manipulation tasks. ★★★★★ 🇨🇳 Kimi-K3 34.8%
1074 SpreadSheetBench v1 O Spreadsheet manipulation Real-world spreadsheet manipulation tasks (v1). ★★★★★ G MiniMax M3 89.4%
1075 SpreadsheetBench-Verified o Spreadsheet manipulation Verified subset of SpreadsheetBench real-world spreadsheet manipulation tasks. ★★★★★ 🇺🇸 Claude Opus 4.6 82.4%
1076 SQA O Table QA Sequential QA over tables (ChatRAG-Bench subset). ★★★★★ G Yuan3.0 Ultra 91.0%
1077 SQuAD v1.1 O Reading comprehension Extractive QA from Wikipedia articles. ★★★★★ 566 🇺🇸 OLMo 3 7B 93.5%
1078 SQuAD v2.0 O Reading comprehension Like v1.1 with unanswerable questions. ★★★★★ 566 G Ling 2.6 1T Base 93.0%
1079 SRE-Bench o Cybersecurity Software reverse engineering of binaries without source code (single-attempt solve rate). ★★★★★ 🇺🇸 GPT-6 Astra 88.0%
1080 StreamingBench O Streaming video Streaming video understanding benchmark. ★★★★★ G Seed1.8 84.4%
1081 StrongREJECT O Safety (jailbreak refusal) StrongREJECT jailbreak benchmark; score = share of forbidden prompts safely handled. ★★★★★ 🇨🇳 Kimi-K2.6 Thinking 99.8%
1082 SUNRGBD O 3D scene understanding SUN RGB-D benchmark for indoor scene understanding from RGB-D imagery. ★★★★★ 🇺🇸 GPT-5 Mini (minimal) 45.8%
1083 SuperChem O Chemistry reasoning Chemistry reasoning benchmark evaluating text-based chemical knowledge and problem solving. ★★★★★ 🇺🇸 Claude Opus 5 76.7%
1084 SuperGLUE O Multi-task NLU Suite of harder NLU tasks beyond GLUE. ★★★★★ G Hairuo 91.4%
1085 SuperGPQA O Graduate-level QA Harder GPQA variant assessing advanced graduate-level reasoning. ★★★★★ 🇺🇸 Gemini 3.1 Pro 76.6%
1086 SURDS O Driving scene spatial reasoning Spatial understanding and reasoning benchmark on autonomous-driving scenes. ★★★★★ 🇺🇸 Claude Fable 5 79.4%
1087 SVG-Bench o SVG generation MiniMax internal SVG generation benchmark (build from scratch or edit from text/image inputs), VLM-verified rendering accuracy. ★★★★★ 🇺🇸 Claude Sonnet 4.6 64.1%
1088 SWE-Atlas Codebase QnA O Codebase understanding SWE-Atlas Codebase QnA: agentic question answering over real codebases. ★★★★★ 🇨🇳 Kimi-K3 68.0%
1089 SWE-Atlas Refactoring O Software engineering SWE-Atlas refactoring track: agentic refactoring tasks over real codebases. ★★★★★ 🇺🇸 Claude Opus 5 60.0%
1090 SWE-Atlas Test Writing O Software testing SWE-Atlas Test Writing: agentic test-writing tasks over real codebases (Scale AI SWE Atlas). ★★★★★ 🇺🇸 Claude Opus 5 68.7%
1091 SWE-Bench O Code repair Supervised software engineering benchmark across many repos and issues. ★★★★★ 3442 🇺🇸 GPT-5 Codex 74.5%
1092 SWE-Bench Multilingual O Code repair (multilingual) Multilingual variant of SWE-Bench for issue fixing. ★★★★★ 🇺🇸 Claude Opus 5.5 93.9%
1093 SWE-bench Multimodal O Software engineering SWE-bench variant with visual context (screenshots, design mockups) in issue descriptions. ★★★★★ 🇺🇸 Claude Opus 5.5 61.4%
1094 SWE-Bench (OpenHands) O Code repair SWE-Bench results using the OpenHands autonomous coding agent. ★★★★★ 3442 🇨🇳 Qwen3.5-35B-A3B 69.2%
1095 SWE-Bench Pro O Software engineering Full SWE-Bench Pro benchmark for software-engineering agents. ★★★★★ 🇺🇸 Claude Opus 5.5 89.9%
1096 SWE-Bench Pro (Public) O Software engineering Public subset of the SWE-Bench Pro benchmark for software-engineering agents. ★★★★★ 🇺🇸 Claude Fable 5 80.4%
1097 SWE-Bench Pro v2 o Software engineering SWE-Bench Pro v2 (Scale AI) agentic software-engineering tasks; full set. ★★★★★ 🇨🇳 Qwen3.8-27B 94.9%
1098 SWE-Bench Pro v2 (Hard) o Software engineering Hard split of SWE-Bench Pro v2 (Scale AI) for software-engineering agents. ★★★★★ 🇨🇳 Kimi-K3 88.2%
1099 SWE-Bench Verified O Code repair Verified subset of SWE-Bench for issue fixing. ★★★★★ 🇺🇸 Claude Opus 5 96.0%
1100 SWE-Dev o Code repair Software engineering development and bug fixing benchmark. ★★★★★ 🇺🇸 Claude Sonnet 4 67.1%
1101 SWE-fficiency o Code performance optimization SWE-fficiency: repository-level performance-optimization tasks; agents must speed up real codebases while keeping tests passing. ★★★★★ 🇺🇸 GPT-5.5 46.6%
1102 SWE-Interact o Interactive software engineering Interactive SWE benchmark (75 tasks) where the agent must clarify underspecified issues with a simulated user before fixing them. ★★★★★ 🇨🇳 Kimi-K3 21.3%
1103 SWE-Lancer o Code repair (freelance tasks) Software engineering benchmark using real freelance-style issues. ★★★★★ 🇺🇸 GPT-5.1 Codex-Max 79.9%
1104 SWE-Lancer Diamond o Code repair (freelance) Diamond subset of SWE-Lancer focusing on the hardest freelance-style issues. ★★★★★ 🇺🇸 GPT-5.3 Codex 81.4%
1105 SWE-Lancer IC Diamond o Code repair (freelance) Individual Contributor Diamond subset of SWE-Lancer. ★★★★★ 🇺🇸 GPT-5.3 Codex 81.4%
1106 SWE-Marathon O Software engineering Abundant AI's SWE-Marathon: very long-horizon software engineering tasks (1M context, max effort). ★★★★★ G Muse Wonder Full 57.5%
1107 SWE-Marathon v1.1 o Software engineering SWE-Marathon v1.1: very long-horizon software engineering tasks (Abundant AI). ★★★★★ 🇺🇸 Claude Opus 4.8 48.8%
1108 SWE-Perf o Code repair Software engineering benchmark focused on performance-oriented fixes. ★★★★★ 🇺🇸 Gemini 3 Pro 6.5%
1109 SWE-Review o Code review Software engineering review benchmark for assessing code review quality. ★★★★★ 🇺🇸 Claude Opus 4.5 16.2%
1110 SWebBench o Web research agents iFlytek in-house benchmark for web browsing, information retrieval and research (reported with Spark-X2.6). ★★★★★ G Spark-X2.6-4B 71.7%
1111 SWT-Bench o Code repair Software tool-use benchmark for code tasks. ★★★★★ 🇺🇸 GPT-5.2 Thinking 80.7%
1112 SysBench O System prompts System prompt understanding and adherence benchmark. ★★★★★ 🇺🇸 Claude Sonnet 4.6 94.9%
1113 T-Eval O Tool use Step-by-step evaluation of LLM tool-utilization (instruct, plan, reason, retrieve, understand, review); EN/ZH average. ★★★★★ 🇨🇳 Qwen 14B Chat 66.2%
1114 TabFact o Table fact verification Table-based fact verification (MMTab subset). ★★★★★ 🇨🇳 Kimi-K2.5 90.6%
1115 TabMWP O Table math Tabular math word problems (MMTab subset, image tables). ★★★★★ 🇨🇳 Kimi-K2.5 95.9%
1116 TallyQA o Visual counting TallyQA open-ended counting questions over images (simple and complex). ★★★★★ 🇺🇸 T5Gemma 2 4B-4B 39.6%
1117 TAT-QA o Table QA Hybrid tabular and textual financial QA (MMTab subset). ★★★★★ G Yuan3.0 Ultra 74.9%
1118 TAU1-Airline O Agent tasks (airline) Tool-augmented agent evaluation in airline scenarios (TAU1). ★★★★★ G openPangu-R-72B-2512 Slow Thinking 56.0%
1119 TAU1-Retail O Agent tasks (retail) Tool-augmented agent evaluation in retail scenarios (TAU1). ★★★★★ G openPangu-R-72B-2512 Slow Thinking 73.0%
1120 TAU2-Airline O Agent tasks (airline) Tool-augmented agent evaluation in airline scenarios (TAU2). ★★★★★ 🇺🇸 Claude Opus 4.7 88.5%
1121 TAU2-Bench Airline (Verified) O Agent tasks Human-validated revision of the tau2-bench airline domain (amazon-agi/tau2-bench-verified). ★★★★★ 🇺🇸 GPT-5.1 72.4%
1122 TAU2-Bench o Agent tasks Aggregate tool-augmented agent evaluation across airline, retail, and telecom scenarios (TAU2). ★★★★★ 🇺🇸 Claude Opus 4.5 91.6%
1123 TAU2-Retail O Agent tasks (retail) Tool-augmented agent evaluation in retail scenarios (TAU2). ★★★★★ 🇺🇸 Claude Opus 4.6 91.9%
1124 TAU2-Bench Retail (Verified) O Agent tasks Human-validated revision of the tau2-bench retail domain (amazon-agi/tau2-bench-verified). ★★★★★ 🇺🇸 GPT-5 78.3%
1125 TAU2-Telecom O Agent tasks (telecom) Tool-augmented agent evaluation in telecom scenarios (TAU2). ★★★★★ 🇨🇳 Qwen3.6-35B-A3B 100.0%
1126 τ³-Bench Airline O Agent tasks (airline) τ³-Bench airline domain: tool-augmented customer-service agent with simulated user. ★★★★★ 🇨🇳 Kimi-K2.6 Thinking 85.8%
1127 τ³-Bench Banking O Agent tasks (banking) τ³-Bench banking domain: tool-augmented customer-service agent with knowledge retrieval and simulated user. ★★★★★ 🇨🇳 Qwen3.8-Max 55.2%
1128 τ³-Bench Retail O Agent tasks (retail) τ³-Bench retail domain: tool-augmented customer-service agent with simulated user. ★★★★★ 🇨🇳 DeepSeek V4-Flash 89.1%
1129 τ³-Bench Telecom O Agent tasks (telecom) τ³-Bench telecom domain: tool-augmented customer-service agent with simulated user (dual-control). ★★★★★ 🇨🇳 GLM-5.1 98.7%
1130 TempCompass O Temporal reasoning Temporal reasoning benchmark evaluating understanding of time-related concepts in videos and images. ★★★★★ 🇺🇸 Gemini 3 Pro 88.0%
1131 Terminal-Bench O Agent terminal tasks Command-line task completion benchmark for agents. ★★★★★ 637 🇺🇸 Claude Sonnet 4.5 64.5%
1132 Terminal-Bench 2.0 O Agent terminal tasks Second-generation Terminal-Bench leaderboard for end-to-end terminal agents. ★★★★★ 🇺🇸 GPT-5.5 84.7%
1133 Terminal-Bench 2.1 O Agent terminal tasks Updated Terminal-Bench 2.x task set (89 tasks) for end-to-end terminal/command-line agents; successor to Terminal-Bench 2.0. ★★★★★ G SWE-2 92.8%
1134 Terminal-Bench 3.0 O Agentic terminal tasks Terminal-Bench 3.0: harder long-horizon command-line agent tasks scored by official verifiers. ★★★★★ 🇺🇸 Claude Opus 5 53.0%
1135 Terminal-Bench 4.0 O Agentic terminal tasks Terminal-Bench 4.0: complex multi-step professional tasks in a command-line interface. ★★★★★ 🇺🇸 Claude Sonnet 5.5 70.6%
1136 Terminal-Bench 4.0-AA o Agentic terminal coding Artificial Analysis run of Terminal-Bench 4.0. ★★★★★ 🇺🇸 Claude Opus 5.5 59.6%
1137 Terminal-Bench Hard O Agent terminal tasks Hard subset of Terminal-Bench command-line agent tasks. ★★★★★ 🇺🇸 GPT-5.4 57.6%
1138 Terminal-Bench Lite O Agent terminal tasks OpenThoughts TBLite: lighter, faster subset-style variant of Terminal-Bench 2.0 for evaluating terminal agents. ★★★★★ 🇨🇳 Qwen3.6-27B 70.8%
1139 Terminal-Bench-Science O Agentic scientific research Terminal-Bench-Science 0.1: agentic scientific research tasks in terminal environments. ★★★★★ 🇺🇸 GPT-6 Astra 68.1%
1140 Terminal-Bench Terminus O Agent terminal tasks Terminal-Bench Terminus track assessing end-to-end terminal tool use. ★★★★★ 🇺🇸 Gemini 3.1 Pro 68.5%
1141 TextQuests O Text-based video games Text-based video game benchmark. ★★★★★ 🇺🇸 Gemini 3 Pro 41.0%
1142 TextQuests Harm O Harmful propensities Harmfulness evaluation on TextQuests scenarios. ★★★★★ 🇺🇸 Grok 4.1 Fast ↓ 9.1%
1143 TextVQA O Text-based VQA Visual question answering that requires reading text in images. ★★★★★ G Ovis2.6-80B-A3B 91.2%
1144 TheoremQA O Theorem-driven STEM QA Theorem-driven question answering in math, physics, EE/CS and finance. ★★★★★ 🇨🇳 Qwen2 72B Instruct 44.4%
1145 TIIF-Bench Long O Text-to-image TIIF-Bench long prompt score for text-to-image generation. ★★★★★ G Seedream 4.5 88.5
1146 TIIF-Bench Short O Text-to-image TIIF-Bench short prompt score for text-to-image generation. ★★★★★ G Nano Banana 2.0 91.0
1147 ActivityNet-TimeLens o Video temporal grounding TimeLens-Bench re-annotated ActivityNet Captions split for video temporal grounding (mIoU). ★★★★★ 🇨🇳 Keye-VL-2.0-30B-A3B 58.5%
1148 Charades-TimeLens o Video temporal grounding TimeLens-Bench re-annotated Charades-STA split for video temporal grounding (mIoU). ★★★★★ 🇨🇳 Keye-VL-2.0-30B-A3B 58.4%
1149 QVHighlights-TimeLens o Video temporal grounding TimeLens-Bench re-annotated QVHighlights split for video temporal grounding (mIoU). ★★★★★ 🇨🇳 Keye-VL-2.0-30B-A3B 70.1%
1150 TIR-Bench O Tool-integrated reasoning Benchmark for tool-integrated reasoning with visual models. ★★★★★ 🇨🇳 Qwen3.5-397B-A17B 62.5%
1151 TLDR9+ o Summarization Long-form summarization benchmark with nine-domain TLDR prompts plus extended variations. ★★★★★ G MobileLLM P1 16.8%
1152 TOMATO O Temporal understanding Temporal ordering and motion analysis benchmark (TOMATO). ★★★★★ 🇺🇸 GPT-6 Astra 96.0%
1153 TOMG-Bench o Molecule generation Text-based open molecule generation benchmark (editing, optimization, customized generation). ★★★★★ 🇺🇸 GPT-5.5 69.9%
1154 Tool-Decathlon O Agent tool-use Composite tool-use suite measuring multi-domain tool invocation success (Pass@1). ★★★★★ 🇺🇸 GPT-5.2 43.8%
1155 ToolACE o Tool calling ToolACE function-calling evaluation, posed as decisions. ★★★★★ G Intern-Decision-2B 96.5%
1156 Toolathlon O Agentic software tasks Long-horizon, real-world software tool-use tasks. ★★★★★ 🇺🇸 Claude Opus 5 80.6%
1157 Toolathlon Verified O Tool use Verified version of Toolathlon (The Tool Decathlon), run via the official evaluation service; pass@1 averaged over 3 runs. ★★★★★ 🇺🇸 Claude Opus 5 80.6%
1158 ToolHop o Multi-hop tool use Query-driven benchmark of 995 multi-hop tool-use queries with 3,912 locally executable tools; answer accuracy. ★★★★★ 🇨🇳 Qwen3.5-9B 52.0%
1159 ToolRet (decision) O Tool retrieval Rate usefulness of 32 retrieved tools per request; nDCG@10. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. ★★★★★ G Clef 69.2
1160 ToolSandbox O Tool use Apple's stateful, conversational, interactive tool-use benchmark with simulated user. ★★★★★ 🇺🇸 LFM2.5 24B A2B 80.7%
1161 TopiOCQA O Conversational QA Topic-switching open-domain conversational QA (ChatRAG-Bench subset). ★★★★★ G Yuan3.0 Ultra 72.4%
1162 TreeBench O Reasoning with tree structures Evaluates hierarchical/tree-structured reasoning and planning capabilities in LLMs/VLMs. ★★★★★ G Seed2.1 Turbo 71.1%
1163 TriQA o Knowledge QA Triadic question answering benchmark evaluating world knowledge and reasoning. ★★★★★ 🇫🇷 Mixtral 8x22B 82.2%
1164 TriviaQA O Open-domain QA Open-domain question answering benchmark built from trivia and web evidence. ★★★★★ 🇺🇸 Nemotron-3-Super-120B-A12B-Base 89.8%
1165 TriviaQA-Wiki o Open-domain QA TriviaQA subset answering using Wikipedia evidence. ★★★★★ 🇺🇸 Llama 3.1 405B Base 91.8%
1166 TrustLLM o Safety / reliability TrustLLM benchmark for trustworthiness and safety behaviors. ★★★★★ 🇨🇳 Qwen3-Coder-480B-A35B-Instruct 88.4%
1167 TruthfulQA O Truthfulness / hallucination Measures whether a model imitates human falsehoods (truthfulness). ★★★★★ G SOLAR-10.7B-Instruct-v1.0 71.4%
1168 TruthfulQA (binary) o Truthfulness Binary-choice variant of TruthfulQA. ★★★★★ G Jev 1.13 92.0%
1169 TruthfulQA (DE) o Truthfulness / hallucination (German) German translation of the TruthfulQA benchmark. ★★★★★ 🇺🇸 Llama 3.3 70B Instruct 0.174
1170 TurkishMMLU O Turkish knowledge Native Turkish multiple-choice QA benchmark from Turkish curriculum exams. ★★★★★ 🇨🇳 Qwen2.5-72B-Instruct 76.1%
1171 TVBench O TV comprehension Benchmark for TV show video comprehension and QA. ★★★★★ 🇺🇸 GPT-5.6 Sol 83.2%
1172 TydiQA O Cross-lingual QA Typologically diverse QA across languages. ★★★★★ 313 G InternLM-20B 43.2%
1173 Typed Decision o Typed decisions Typed-decision evaluation set used in Intern-Decision model card (Laya Typed Decisions). ★★★★★ G Intern-Decision-4B 80.5%
1174 U-Artifacts o Agentic coding artifacts Benchmark focusing on generated code artifacts quality. ★★★★★ 🇺🇸 Gemini 3 Pro 57.8%
1175 UGC-VideoCap o Video captioning Short-form user-generated video captioning benchmark. ★★★★★ G Glimpse 82.7%
1176 UI-Vision O GUI grounding Desktop GUI grounding benchmark (basic, functional, spatial) for computer-use agents. ★★★★★ G POINTS-GUI-G-8B 49.9%
1177 UniMERNet o Formula recognition UniMER-Test benchmark for universal mathematical expression recognition from images. ★★★★★ G Infinity-Parser2-Pro 97.7%
1178 USACO O Code generation (olympiad) USA Computing Olympiad problems benchmark (307 problems); pass@1. ★★★★★ 🇺🇸 GPT-4o-2024-05-13 18.8%
1179 USAMO 2026 O Math olympiad (proofs) 2026 USA Mathematical Olympiad (6 proof problems), graded with MathArena methodology. ★★★★★ 🇺🇸 Claude Mythos 5 99.8%
1180 V* O Multimodal reasoning V* benchmark accuracy. ★★★★★ 🇨🇳 Qwen3.6-Plus 96.9%
1181 V* (With Tools) o Multimodal reasoning V* visual search benchmark evaluated with a Python code-interpreter tool. ★★★★★ 🇺🇸 GPT-5.4 98.4%
1182 VABench-Point O Embodied reasoning VABench visual-action pointing subset. ★★★★★ G Embodied-R1.5-8B 75.7%
1183 VABench-VisualTrace O Embodied reasoning VABench visual-trace (trajectory) subset. ★★★★★ G Embodied-R1.5-8B 92.3%
1184 Vals Index O Knowledge work Vals AI composite index of real-world professional/enterprise tasks (finance, legal, coding, etc.). ★★★★★ 🇺🇸 Gemini 4 Argon 68.9%
1185 VAST O Stance detection VAST zero-shot stance detection; macro-F1 (Jev Decision Index suite, posed as typed decisions). ★★★★★ G pplx-decider-v1.2 80.2
1186 VCR (EN, easy) O Visual caption restoration Visual Caption Restoration, English easy split; exact-match accuracy of restoring partially occluded text. ★★★★★ 🇨🇳 InternVL2.5-78B 95.7%
1187 VCRBench o Visual commonsense reasoning Visual commonsense reasoning benchmark. ★★★★★ G Seed1.8 59.8%
1188 VCT O Virology capability (protocol troubleshooting) Virology Capabilities Test: a benchmark that measures an LLM's ability to troubleshoot complex virology laboratory protocols. ★★★★★ 🇺🇸 Grok 4.1 (Thinking) 61.0%
1189 Vending-Bench 2 O Long-horizon agentic tasks Long-horizon agentic task benchmark evaluating sustained goal completion. ★★★★★ 🇺🇸 Claude Opus 4.7 $10,937
1190 Vibe Android o Vibe evaluation (Android) Vibe evaluation on Android tasks. ★★★★★ 🇺🇸 Claude Opus 4.5 92.2%
1191 Vibe Average o Vibe evaluation Aggregate Vibe evaluation score. ★★★★★ G MiniMax M2.1 88.6%
1192 Vibe Backend o Vibe evaluation (backend) Vibe evaluation on backend tasks. ★★★★★ 🇺🇸 Claude Opus 4.5 98.0%
1193 Vibe Code Bench o App generation Vals AI Vibe Code Bench: building complete web applications from natural-language specs. ★★★★★ 🇺🇸 Gemini 4 Argon 91.9%
1194 Vibe Code Bench v1.1 o Application building Vals AI Vibe Code Bench v1.1: build working web applications from natural-language specs. ★★★★★ G MiniMax M3.1 Flash (max) 73.9%
1195 Vibe iOS o Vibe evaluation (iOS) Vibe evaluation on iOS tasks. ★★★★★ 🇺🇸 Claude Opus 4.5 90.0%
1196 Vibe Simulation o Vibe evaluation (simulation) Vibe evaluation on simulation tasks. ★★★★★ 🇺🇸 Gemini 3 Pro 89.2%
1197 Vibe Web o Vibe evaluation (web) Vibe evaluation on web tasks. ★★★★★ G MiniMax M2.1 91.5%
1198 VibeEval O Aesthetic/visual quality VLM aesthetic evaluation with GPT scores. ★★★★★ 🇺🇸 Gemini 2.5 Pro 76.4%
1199 Video-MME O Video understanding (multimodal) Multimodal evaluation of video understanding and reasoning. ★★★★★ 🇺🇸 Gemini 3 Pro 88.4%
1200 VideoEval-Pro O Video understanding Long video understanding benchmark with open-ended short-answer questions. ★★★★★ 🇺🇸 Gemini 2.5 Pro 78.4%
1201 VideoHallucer o Video hallucination Benchmark for detecting intrinsic and extrinsic hallucinations in video-language models. ★★★★★ 🇺🇸 Gemini 3 Pro 66.8%
1202 VideoHolmes O Video QA Video question answering benchmark focused on detective-style clues. ★★★★★ G Seed2.1 Pro 68.2%
1203 VideoMathQA o Video math reasoning Benchmark for mathematical reasoning grounded in video content. ★★★★★ G JoyAI-VL-Interaction 56.2%
1204 VideoMME O Multimodal video evaluation Video multimodal evaluation suite (VideoMME). ★★★★★ G Seed2.0 Pro 89.5%
1205 VideoMME (w/o sub) O Video understanding Video understanding benchmark without subtitles. ★★★★★ 🇺🇸 Gemini 3 Pro 88.6%
1206 VideoMME (w/sub) O Video understanding Video understanding benchmark with subtitles. ★★★★★ 🇨🇳 Qwen3.8-Max 90.4%
1207 Video-MME v2 O Video understanding Second version of the Video-MME video multimodal evaluation suite. ★★★★★ 🇺🇸 Gemini 3 Pro 66.1%
1208 VideoMMMU O Multimodal video understanding Video-based extension of MMMU evaluating temporal multimodal reasoning and perception across disciplines. ★★★★★ 🇨🇳 Qwen3.8-Max 88.7%
1209 VideoReasonBench o Video reasoning Video reasoning benchmark assessing temporal and causal understanding. ★★★★★ G Seed2.0 Pro 77.8%
1210 VideoSimpleQA O Video QA Simple question answering over short videos. ★★★★★ G Seed2.1 Pro 76.4%
1211 ViewSpatial-Bench O Spatial reasoning Multi-perspective spatial localization benchmark (camera vs. human viewpoints). ★★★★★ G ZDTaichu5.0-9B 62.5%
1212 VisFactor O Visual perception Benchmark of fundamental visual-cognition factors adapted from human psychometric perception tests. ★★★★★ 🇺🇸 GPT-5.6 Sol 62.8%
1213 Vision2Web O Coding (UI) Visual website generation benchmark: build working websites from visual designs/prototypes, judged on rendered result. ★★★★★ 🇺🇸 GPT-5.6 Sol 79.8%
1214 ViSpeak o Video dialogue Video-grounded dialogue and description benchmark. ★★★★★ 🇺🇸 Gemini 3 Pro 89.0%
1215 VisualProbe o Visual search / fine-grained perception High-resolution visual search benchmark requiring iterative zoom/crop tool use (introduced with Mini-o3). ★★★★★ 🇺🇸 Gemini 3 Flash 69.9%
1216 VisualPuzzle o Visual reasoning Visual puzzle solving benchmark evaluating reasoning and pattern recognition capabilities. ★★★★★ 🇺🇸 Gemini 3 Pro 71.5%
1217 VisualWebBench O Web UI understanding Average accuracy on VisualWebBench. ★★★★★ 🇺🇸 Holo1.5-72B 83.8%
1218 VisuLogic O Visual logical reasoning Logical reasoning and compositionality benchmark for visual-language models. ★★★★★ G Seed2.1 Pro 54.3%
1219 VitaBench O Industry QA Industry-focused benchmark evaluating domain QA performance. ★★★★★ 🇺🇸 Claude Opus 4.5 56.3%
1220 VitaBench 2.0 o Agentic real-world tasks VitaBench 2.0: interactive agent tasks in real-world service scenarios (v2.0, not comparable to v1). ★★★★★ G Spark-X2.6-4B 30.8%
1221 VizWiz-VQA o Accessibility VQA Visual question answering on photos taken by blind people (VQA score). ★★★★★ 🇨🇳 Qwen-VL-Chat 38.9%
1222 VL-RewardBench O Reward modeling (VL) Reward alignment benchmark for VLMs. ★★★★★ 🇺🇸 Claude 3.7 Sonnet 67.4%
1223 VLABench O Embodied reasoning Language-conditioned robot manipulation reasoning benchmark (VLM track). ★★★★★ G PhysBrain1.5-8B 76.4%
1224 VLAD o Driving scene QA Vision-language autonomous-driving QA benchmark (as reported in the Qwen-Drive-1.0 evaluation). ★★★★★ 🇨🇳 Qwen-Drive-1.0-4B 66.5%
1225 VLMs are Biased O Multimodal bias Evaluates whether VLMs truly 'see' vs. relying on memorized knowledge; measures bias toward non-visual priors. ★★★★★ 90 🇺🇸 GPT-6 Astra 84.1%
1226 VLMs are Blind O Visual grounding robustness Evaluates failure modes of VLMs in grounding and perception tasks. ★★★★★ 🇨🇳 ERNIE 5.0 91.4%
1227 VLMsAreBiased O Multimodal bias Benchmark evaluating biases in vision-language models. ★★★★★ 🇨🇳 Qwen3.8-Max 88.3%
1228 VLMsAreBlind O Multimodal robustness Benchmark probing robustness of vision-language models to visual perturbations. ★★★★★ 🇺🇸 Gemini 3 Pro 97.5%
1229 VoiceBench AdvBench O VoiceBench VoiceBench adversarial safety evaluation. ★★★★★ 🇨🇳 Qwen3-Omni-30B-A3B-Thinking 99.4%
1230 VoiceBench AlpacaEval O VoiceBench VoiceBench evaluation on AlpacaEval instructions. ★★★★★ 🇨🇳 LongCat-Flash-Omni 98.8%
1231 VoiceBench BBH O VoiceBench VoiceBench evaluation on Big-Bench Hard prompts. ★★★★★ 🇺🇸 Gemini 2.5 Pro 92.6%
1232 VoiceBench CommonEval O VoiceBench VoiceBench evaluation on CommonEval. ★★★★★ 🇨🇳 Qwen3-Omni-Flash-Instruct 91.0%
1233 VoiceBench IFEval O VoiceBench VoiceBench instruction-following evaluation (IFEval). ★★★★★ 🇺🇸 Gemini 2.5 Pro 85.7%
1234 VoiceBench-ja Elyza o Spoken QA (Japanese) ELYZA-tasks subset of VoiceBench-ja (spoken input), GPT-4o judge score 0-5. ★★★★★ 🇨🇳 Qwen2.5-Omni-3B 2.37
1235 VoiceBench-ja M-IFEval o Spoken instruction following (Japanese) M-IFEval subset of VoiceBench-ja (spoken input), scored 0-1. ★★★★★ G J-Moshi 0.151
1236 VoiceBench-ja spoken_Elyza o Spoken QA (Japanese) spoken_Elyza subset of VoiceBench-ja, GPT-4o judge score 0-5. ★★★★★ 🇺🇸 LFM2.5-Audio-1.5B-JP 2.12
1237 MMAU v05.15.25 O Audio reasoning Audio reasoning benchmark MMAU v05.15.25. ★★★★★ 🇺🇸 Gemini 3.1 Pro 82.5%
1238 VoiceBench MMSU O VoiceBench VoiceBench MMSU benchmark (voice modality). ★★★★★ 🇨🇳 Qwen3-Omni-Flash-Thinking 84.3%
1239 VoiceBench MMSU (Audio) O Audio reasoning Audio reasoning MMSU results. ★★★★★ 🇺🇸 Gemini 2.5 Pro 77.7%
1240 VoiceBench OpenBookQA o VoiceBench VoiceBench results on OpenBookQA prompts. ★★★★★ 🇨🇳 Qwen3-Omni-Flash-Thinking 95.0%
1241 VoiceBench SD-QA O VoiceBench VoiceBench Spoken Dialogue QA results. ★★★★★ 🇺🇸 Gemini 2.5 Pro 90.1%
1242 VoiceBench WildVoice O VoiceBench VoiceBench evaluation on WildVoice dataset. ★★★★★ 🇺🇸 Gemini 2.5 Pro 93.4%
1243 VPCT o Multimodal reasoning Visual perception and comprehension test. ★★★★★ 🇺🇸 Gemini 3 Pro 90.0%
1244 VQA-RAD O Medical VQA Radiology visual question answering (closed-question accuracy). ★★★★★ G MedMO-8B-Next 86.4%
1245 VQAv2 O Visual question answering Standard Visual Question Answering v2 benchmark on natural images. ★★★★★ 🇺🇸 Gemini 3 Pro 89.8%
1246 VSI-Bench O Spatial intelligence Visual spatial intelligence benchmark covering 3D reasoning and spatial inference tasks. ★★★★★ 🇺🇸 Molmo2-ER 74.5%
1247 WaymoQA o Driving scene QA Driving-scene multiple-choice QA benchmark built on Waymo data; overall accuracy. ★★★★★ 🇨🇳 Qwen-Drive-1.0-4B 74.5%
1248 WaymoQA (Safety) o Driving scene QA Safety-critical subset of WaymoQA; accuracy. ★★★★★ 🇨🇳 Qwen-Drive-1.0-4B 70.7%
1249 WebArena o Web agents Realistic self-hosted web environments (shopping, forums, GitLab, CMS, maps) for autonomous web agents; task success rate. ★★★★★ 🇺🇸 Claude Sonnet 4.6 65.6%
1250 WebArena-Verified O Web automation Verified version of the WebArena browser-use benchmark. ★★★★★ 🇨🇳 Kimi-K3 71.6%
1251 WebClick O GUI agents Task success on the WebClick UI agent benchmark. ★★★★★ G Holo2-235B-A22B 94.3%
1252 WebDev Arena O Web development agents Arena evaluation for autonomous web development agents. ★★★★★ 🇺🇸 Gemini 3.7 Flash 1588
1253 WebQuest-MultiQA o Web agents Multi-question web search and interaction tasks. ★★★★★ 🇨🇳 GLM-4.5V 60.6%
1254 WebQuest-SingleQA o Web agents Single-question web search and interaction tasks. ★★★★★ 🇨🇳 GLM-4.6V 79.5%
1255 WebSrc O Web QA Webpage question answering (SQuAD F1). ★★★★★ 🇺🇸 Holo1.5-72B 97.2%
1256 WebTailBench o Web agents Microsoft long-tail web task benchmark (shopping, travel, comparison and other real-world tasks) introduced with Fara-7B. ★★★★★ 🇺🇸 MolmoWeb-8B 49.5%
1257 WebTestBench o Web testing agents Benchmark of web-app testing agents that detect defects against a checklist in live web applications. ★★★★★ 🇺🇸 GPT-5.6 Sol 54.0%
1258 WebVoyager O Web agents Web navigation and interaction tasks for LLM agents. ★★★★★ G UI-Venus-2 27B 93.4%
1259 WebVoyager2 o Web agents Web navigation and interaction tasks for LLM agents (v2). ★★★★★ 🇨🇳 GLM-4.5V 84.4%
1260 WebWalkerQA O Web agents WebWalker tasks evaluating autonomous browsing question answering performance. ★★★★★ G SMTL-30B 76.5%
1261 WeMath O Math reasoning Math reasoning benchmark spanning diverse curricula and difficulty levels. ★★★★★ 🇨🇳 Qwen3.6-Plus 89.0%
1262 WGO Bench o Embodied manipulation WGO Bench: end-to-end robot manipulation event understanding (Elorian-reported). ★★★★★ G Glimpse 42.7%
1263 When2Call O Tool calling NVIDIA's When2Call multiple-choice test set: choose whether to answer directly, call a tool, ask for more information, or say it cannot answer. ★★★★★ G d1-ultra 92.6%
1264 Where2Place O Embodied spatial understanding Spatial free-space pointing benchmark: predict points for placement locations described in language. ★★★★★ G ACE-Brain-0.6-9B 81.4%
1265 WideSearch O Web search Wide web search and QA benchmark. ★★★★★ 🇺🇸 GPT-5.6 Sol 86.3%
1266 WikiWebFacts o Russian factual knowledge Yandex 5-shot benchmark of factual knowledge in Russian. ★★★★★ G AliceAI-Foundation-80B-A3B-Base 86.5%
1267 Wild-Jailbreak O Safety / jailbreak Adversarial jailbreak benchmark evaluating refusal robustness. ★★★★★ 🇺🇸 GPT-OSS 120B (high) 98.2%
1268 WildBench V2 O Instruction following WildBench V2 human preference benchmark for instruction following and helpfulness. ★★★★★ 🇫🇷 Ministral 3 14B Instruct 68.5%
1269 WildClawBench O Agentic tasks Agentic benchmark of in-the-wild tasks run in OpenClaw-style agent scaffolds. ★★★★★ 🇺🇸 Claude Opus 4.8 68.0%
1270 WildGuardTest O Safety WildGuardTest safety benchmark. ★★★★★ G d1-ultra 95.4%
1271 WildVision-Bench O Multimodal chat In-the-wild multimodal chat benchmark judged against reference model (WildVision Arena). ★★★★★ 🇺🇸 Gemini 2.5 Pro 90.1%
1272 Windows Agent Arena o GUI agents Windows OS computer-use agent benchmark (WAA), task success rate. ★★★★★ G EvoCUA-32B-20260105 56.5%
1273 Winogender o Gender bias (coreference) Coreference resolution dataset for measuring gender bias. ★★★★★ 🇺🇸 Llama 3.3 70B Instruct 84.3%
1274 WinoGrande O Coreference reasoning Large-scale adversarial Winograd Schema-style pronoun resolution. ★★★★★ 99 G Clef-Flash 97.5%
1275 WinoGrande (DE) o Coreference reasoning (German) German translation of the WinoGrande pronoun resolution benchmark. ★★★★★ 🇺🇸 Llama-3_1-70B-TFree-HAT-SFT 0.793
1276 WMDP Bio o Biosecurity knowledge Weapons of Mass Destruction Proxy benchmark for biosecurity, measuring hazardous biological knowledge without info hazards. ★★★★★ 🇺🇸 Zephyr 7B ↓ 63.7%
1277 WMDP Chem o Chemical security knowledge WMDP benchmark for chemical security, evaluating knowledge relevant to chemical weapons development. ★★★★★ 🇺🇸 Zephyr 7B ↓ 45.8%
1278 WMDP Cyber o Cybersecurity knowledge WMDP benchmark for cybersecurity, assessing knowledge that could aid in cyber weapons development. ★★★★★ 🇺🇸 Zephyr 7B ↓ 44.0%
1279 WMT16 En–De o Machine translation WMT16 English–German translation benchmark (news). ★★★★★ 🇺🇸 Llama 3.3 70B Instruct 38.8
1280 WMT16 En–De (Instruct) o Machine translation Instruction-tuned evaluation on the WMT16 English–German translation set. ★★★★★ 🇺🇸 Llama 3.3 70B Instruct 37.9
1281 WMT20 En-Ja O Machine translation WMT20 English-to-Japanese news translation (Swallow eval, BLEU). ★★★★★ 🇺🇸 GPT-OSS-Swallow-120B-SFT-v0.1 27.0
1282 WMT20 Ja-En O Machine translation WMT20 Japanese-to-English news translation (Swallow eval, BLEU). ★★★★★ 🇺🇸 GPT-5 mini 23.9
1283 WMT24++ O Machine translation Extended WMT 2024 evaluation across multiple language pairs. ★★★★★ 🇨🇳 Qwen3-235B-A22B-Thinking-2507 94.7%
1284 WOD-E2E (RFS) o Driving planning Waymo Open Dataset end-to-end driving challenge on long-tail scenarios; score = Rater Feedback Score (RFS, 0-10) on the test split. ★★★★★ 🇨🇳 Qwen-Drive-1.0-4B RL 7.91
1285 WorkBench o Agentic tool use Workplace agent benchmark: tasks over simulated email, calendar, CRM, analytics and project-management tools; scored by task completion (outcome-based). ★★★★★ 🇺🇸 Gemma 4 E4B IT 46.1%
1286 WorkBuddy Bench O Office assistant agents Agentic benchmark of workplace-assistant (WorkBuddy) tasks such as documents, spreadsheets and messaging. ★★★★★ G Muse Glimmer 30B 45.9%
1287 Workspace Bench O Agentic office work Workspace Bench: OpenClaw-style workspace tasks with large heterogeneous file dependencies (100-task setting). ★★★★★ 🇺🇸 Claude Opus 5 75.0%
1288 WorldBench o Visual knowledge Benchmark of real-world visual knowledge and recognition. ★★★★★ G Seed2.1 Pro 67.6%
1289 WorldTravel2 (multi-modal) o Travel planning (multimodal) WorldTravel2 benchmark multimodal track. ★★★★★ 🇺🇸 Gemini 3 Pro 47.2%
1290 WorldTravel2 (text) o Travel planning (text) WorldTravel2 benchmark text-only track. ★★★★★ 🇺🇸 GPT-5 (high) 56.4%
1291 WorldVQA O World knowledge VQA Visual question answering requiring world knowledge and commonsense reasoning. ★★★★★ 🇺🇸 Claude Opus 4.6 65.4%
1292 WorldVQA (With Tools) o Visual world knowledge WorldVQA evaluated with tool access in an agent harness. ★★★★★ G Step 3.7 Flash 58.1%
1293 WritingBench O Writing quality General-purpose writing quality benchmark. ★★★★★ 🇨🇳 Qwen3-235B-A22B-Thinking-2507 88.3%
1294 WSC O Coreference reasoning Classic Winograd Schema Challenge measuring commonsense coreference. ★★★★★ G Hairuo 100.0%
1295 WikiTableQuestions o Table QA WikiTableQuestions (MMTab subset, image tables). ★★★★★ 🇺🇸 Gemini 3.1 Pro 79.6%
1296 xBench-DeepSearch O Agentic research Evaluates multi-hop deep research workflows on xBench DeepSearch tasks. ★★★★★ 🇺🇸 GPT-5 (high) 77.9%
1297 xBench-DeepSearch (2025.05) O Agentic research xBench DeepSearch benchmark May 2025 snapshot. ★★★★★ G Step-3.5 Flash 20260204 83.7%
1298 xBench-DeepSearch (2025.10) O Agentic research xBench DeepSearch benchmark October 2025 snapshot. ★★★★★ 🇨🇳 Kimi-K2.6 Thinking 90.0%
1299 XCOPA O Cross-lingual commonsense causal reasoning Multilingual extension of COPA causal commonsense reasoning in 11 languages. ★★★★★ 🇨🇳 Zhipu GLM 4-9B-Chat 80.1%
1300 XLRS-Bench O Remote sensing Remote sensing benchmark for evaluating multimodal AI on satellite and aerial imagery. ★★★★★ 🇺🇸 Gemini 3.1 Pro 54.3%
1301 XNLI O Cross-lingual NLI Cross-lingual natural language inference benchmark (MultiNLI translated into 15 languages). ★★★★★ G Decider 4B (Mapika) 88.6%
1302 XorQA-IN o Indic languages Cross-lingual open-retrieval QA in Indian languages (IndicXTREME), F1. ★★★★★ G gnani-evon-v3.3-30B-A3B 41.8%
1303 XpertBench (Edu) o Economics/education XpertBench education domain subset. ★★★★★ 🇺🇸 GPT-5 (high) 56.9%
1304 XpertBench (Fin) o Economics/finance XpertBench finance domain subset. ★★★★★ 🇺🇸 GPT-5 (high) 64.5%
1305 XpertBench (Humanities) o Economics/humanities XpertBench humanities domain subset. ★★★★★ 🇺🇸 GPT-5 (high) 68.5%
1306 XpertBench (Law) o Economics/legal XpertBench legal domain subset. ★★★★★ 🇺🇸 Claude Sonnet 4.5 58.7%
1307 XpertBench (Research) o Economics/research XpertBench research domain subset. ★★★★★ 🇺🇸 GPT-5 (high) 48.2%
1308 XQuAD o Multilingual QA Cross-lingual Question Answering Dataset: SQuAD v1.1 subset professionally translated into 10 languages. ★★★★★ 🇺🇸 T5Gemma 2 4B-4B 70.6%
1309 XQuAD-IN o Indic languages Extractive QA (XQuAD) in Indian languages (IndicXTREME), F1. ★★★★★ G gnani-evon-v3.3-30B-A3B 64.6%
1310 XSTest O Safety XSTest safety benchmark. ★★★★★ 🇨🇳 Qwen3.5-9B 97.6%
1311 XStoryCloze o Commonsense reasoning (multilingual) Multilingual StoryCloze: choose the correct story ending in 11 languages. ★★★★★ 🇨🇳 Zhipu GLM 4-9B-Chat 90.7%
1312 XWinograd o Coreference reasoning (multilingual) Multilingual Winograd schema benchmark (en, fr, jp, pt, ru, zh). ★★★★★ 🇨🇳 Zhipu GLM 4-9B-Chat 73.1%
1313 YC-Bench o Long-horizon business simulation YC-Bench: long-horizon startup-management simulation; metric is final assets of the fund in millions of USD. ★★★★★ 🇺🇸 Claude Opus 4.7 $2.2M
1314 ZClawBench o Agent tasks (OpenClaw) Z.ai end-to-end benchmark for OpenClaw agent workflows (environment setup, software dev, retrieval, data analysis, content creation). ★★★★★ 🇺🇸 Claude Opus 4.6 62.3%
1315 ZebraLogic O Logical reasoning Logical reasoning benchmark assessing complex pattern and rule inference. ★★★★★ 🇺🇸 GPT-5 (high) 98.8%
1316 ZeroBench O Zero-shot generalization Evaluates zero-shot performance across diverse tasks without task-specific finetuning. ★★★★★ 🇨🇳 GLM-4.5V 23.4%
1317 ZeroBench (sub) O Zero-shot generalization Subset of ZeroBench targeting harder zero-shot reasoning cases. ★★★★★ G Seed2.1 Pro 49.4%
1318 ZeroBench (With Tools) O Visual reasoning ZeroBench main set evaluated with tool access in an agent harness (pass@5). ★★★★★ 🇺🇸 GPT-5.6 Sol 54.6%
1319 ZeroSCROLLS BookSumSort o Long-context summarization ZeroSCROLLS split based on BookSumSort long-form summarization. ★★★★★ 🇺🇸 GPT-4 60.5
1320 ZeroSCROLLS GovReport o Long-context summarization ZeroSCROLLS split based on the GovReport summarization benchmark. ★★★★★ G CoLT5 41.0
1321 ZeroSCROLLS MuSiQue o Long-context reasoning ZeroSCROLLS split derived from MuSiQue multi-hop QA. ★★★★★ 🇺🇸 Llama 3.3 70B Instruct 52.2
1322 ZeroSCROLLS NarrativeQA o Long-context QA ZeroSCROLLS split based on the NarrativeQA reading comprehension benchmark. ★★★★★ 🇺🇸 Claude v1.3 32.6
1323 ZeroSCROLLS Qasper o Long-context QA ZeroSCROLLS split based on the Qasper paper QA benchmark. ★★★★★ G FLAN-UL2 56.9
1324 ZeroSCROLLS QMSum o Long-context summarization ZeroSCROLLS split based on the QMSum meeting summarization benchmark. ★★★★★ G CoLT5 22.5
1325 ZeroSCROLLS QuALITY o Long-context QA ZeroSCROLLS split based on the QuALITY reading comprehension benchmark. ★★★★★ 🇺🇸 GPT-4 89.2%
1326 ZeroSCROLLS SpaceDigest o Long-context summarization ZeroSCROLLS SpaceDigest extractive summarization task. ★★★★★ 🇺🇸 Llama-3_1-70B-TFree-HAT-SFT 77.9
1327 ZeroSCROLLS SQuALITY o Long-context summarization ZeroSCROLLS split based on the SQuALITY long-form summarization benchmark. ★★★★★ 🇺🇸 GPT-4 22.6
1328 ZeroSCROLLS SummScreenFD o Long-context summarization ZeroSCROLLS split based on the SummScreenFD summarization benchmark. ★★★★★ G CoLT5 20.0

Model Size

Release Date