Benchmarks
1.392 Benchmarks mit Schwierigkeit, Sättigung und führendem Modell. Suche nach Name, Thema oder Modell.
1.392 Benchmarks
| # | Benchmark | Thema | Schwierigkeit | Sättigung | Trennsch. | Spitzenreiter | Top-Score | Modelle |
|---|---|---|---|---|---|---|---|---|
| 1 | Terminal-Bench 4.0 O Terminal-Bench 4.0: complex multi-step professional tasks in a command-line interface. | Agentic terminal tasks | 0,1 % | 64,6 % | 3,28 |  🇺🇸 Claude Sonnet 5.5 | 70,6 % | 47 |
| 2 | FrontierBench O Successor to Terminal-Bench 2.1 (v0.1): 74 harder terminal tasks emphasizing science and engineering. | Agentic terminal tasks | 0,1 % | 31,1 % | 2,61 | 🇺🇸 Claude Opus 5 | 43,3 % | 13 |
| 3 | MathArena Apex O Challenging math contest problems from MathArena Apex benchmark. | Challenging Math Contest problems | 0,2 % | 87,7 % | 3,51 | 🇺🇸 Claude Opus 5 | 91,4 % | 51 |
| 4 | Complex Research using Integrated Thinking – Physics Test O CritPt (Complex Research using Integrated Thinking – Physics Test) benchmark. | Reasoning | 0,3 % | 29,1 % | 2,16 |  🇺🇸 GPT-5.6 Sol | 32,3 % | 95 |
| 5 | ExploitGym O UC Berkeley-led benchmark: agents turn real known vulnerabilities into working exploits achieving arbitrary code execution. | Cybersecurity | 0,3 % | 33,4 % | 2,02 | 🇺🇸 GPT-6 Astra | 42,4 % | 17 |
| 6 | SciRes FrontierMath Tier 4 O SciRes FrontierMath benchmark covering tier 4. | Math (frontier) | 0,5 % | 86,4 % | 2,83 | 🇺🇸 GPT-6 Astra | 97,6 % | 21 |
| 7 | ARC-AGI-2 O ARC-AGI Phase 2 aggregate accuracy. | General reasoning | 0,5 % | 94,2 %gesättigt | 3,32 | 🇺🇸 GPT-6 Astra | 95,0 % | 340 |
| 8 | DeepSWE O 113 long-horizon software engineering tasks written from scratch to avoid contamination (v1.1). | Software engineering | 0,5 % | 72,9 % | 2,79 | 🇺🇸 Claude Opus 5.5 | 74,2 % | 41 |
| 9 | SWE-Marathon O Abundant AI's SWE-Marathon: very long-horizon software engineering tasks (1M context, max effort). | Software engineering | 0,6 % | 50,8 % | 2,11 | G Muse Wonder Full | 57,5 % | 22 |
| 10 | Terminal-Bench 3.0 O Terminal-Bench 3.0: harder long-horizon command-line agent tasks scored by official verifiers. | Agentic terminal tasks | 0,7 % | 39,0 % | 1,85 | 🇺🇸 Claude Opus 5 | 53,0 % | 19 |
| 11 | BioMysteryBench (Hard) O BioMysteryBench hard subset (problems not solved by human experts). | Biology research | 0,8 % | 49,6 % | 2,15 |  🇺🇸 Gemini 3.8 Flash | 56,5 % | 21 |
| 12 | DeepSWE v1.1 O Original long-horizon software-engineering tasks in real codebases (Datacurve). | Software engineering | 0,8 % | 77,6 % | 2,67 | G Muse Wonder Full | 79,0 % | 80 |
| 13 | OSWorld 2.0 O OSWorld 2.0 long-horizon computer-use benchmark on a live Ubuntu VM; partial-credit score. | GUI agents | 0,9 % | 76,3 % | 2,46 | 🇺🇸 Claude Opus 5.5 | 81,8 % | 35 |
| 14 | Legal Agent Benchmark (Held-Out) O Harvey Legal Agent Benchmark, Harvey's held-out set. | Legal agents | 0,9 % | 16,2 % | 1,06 | 🇺🇸 Gemini 4 Argon | 19,6 % | 19 |
| 15 | GSO Benchmark O LiveCodeBench GSO benchmark. | Code generation | 0,9 % | 73,9 % | 2,37 | 🇺🇸 Claude Fable 5.1 | 88,2 % | 31 |
| 16 | GDP.pdf O Surge AI multimodal benchmark of 100 professional prompts over information-dense PDFs; strict pass rate. | Document understanding | 1,0 % | 36,2 % | 1,68 | 🇺🇸 GPT-5.6 Sol | 40,0 % | 33 |
| 17 | JobBench O Agentic benchmark of professional, job-like tasks requiring tool use. | Professional tool use | 1,2 % | 72,5 % | 2,29 | 🇺🇸 Claude Opus 5.5 | 75,4 % | 47 |
| 18 | DeepSWE 1.0 O Datacurve DeepSWE v1.0 long-horizon software-engineering tasks, run by Artificial Analysis with each provider's own harness (pass@1). | Software engineering | 1,2 % | 62,4 % | 2,26 | 🇺🇸 GPT-5.5 | 70,1 % | 26 |
| 19 | APEX-Agents O APEX benchmark evaluating agents on long-horizon professional tasks. | Long horizon professional tasks | 1,2 % | 52,8 % | 2,06 | 🇺🇸 Claude Fable 5 | 59,2 % | 76 |
| 20 | OSWorld 2.0 (Strict) O OSWorld 2.0 strict pass rate (full task success). | GUI agents | 1,2 % | 41,5 % | 1,62 | 🇺🇸 Claude Opus 5.5 | 48,7 % | 15 |
| 21 | EnigmaEval O Challenging puzzle benchmark. | Challenging puzzles | 1,3 % | 14,5 % | 1,76 | 🇺🇸 Gemini 3 Pro | 17,8 % | 15 |
| 22 | Terminal-Bench-Science O Terminal-Bench-Science 0.1: agentic scientific research tasks in terminal environments. | Agentic scientific research | 1,5 % | 61,2 % | 1,64 | 🇺🇸 GPT-6 Astra | 68,1 % | 14 |
| 23 | ProgramBench (Almost Solved) O ProgramBench share of instances that are almost solved (near-complete test pass rate). | Software engineering | 1,5 % | 26,5 % | 1,34 | 🇺🇸 Claude Opus 5 | 37,0 % | 11 |
| 24 | ZeroBench O Evaluates zero-shot performance across diverse tasks without task-specific finetuning. | Zero-shot generalization | 1,6 % | 22,9 % | 1,24 |  🇨🇳 GLM-4.5V | 23,4 % | 48 |
| 25 | ExploitBench O Developing V8 vulnerabilities into increasingly capable exploits (capability coverage). | Cybersecurity | 1,8 % | 81,7 % | 2,01 | 🇺🇸 GPT-6 Astra | 100,0 % | 19 |
| 26 | LCB Pro 25Q2 (Med) O LiveCodeBench Pro 2025 Q2 medium subset. | Code generation | 1,9 % | 61,4 % | 2,58 | 🇺🇸 Gemini 3 Deep Think | 86,0 % | 39 |
| 27 | AA-Omniscience-Decision Non-Hallucination Rate O Share of AA-Omniscience-Decision questions not answered correctly where the model chose "I don't know". | Knowledge and hallucination | 1,9 % | 62,5 % | 0,60 | G d1-ultra | 84,5 % | 28 |
| 28 | FrontierScience Research O Research-level problems from the FrontierScience benchmark. | Science reasoning (research) | 1,9 % | 57,8 % | 1,91 | G Apodex 1.1 | 63,3 % | 55 |
| 29 | ProofBench Advanced O Advanced mathematical proof benchmark covering complex theorem proving tasks. | Mathematical proofs (advanced) | 1,9 % | 78,9 % | 2,08 | G Aletheia | 91,9 % | 28 |
| 30 | AutomationBench O Zapier benchmark of end-to-end business workflows via REST APIs across 47 simulated apps; pass/fail on deterministic state assertions. | Agentic business workflows | 2,1 % | 59,8 % | 1,71 | G MiniMax M3.1 Flash (max) | 62,8 % | 70 |
| 31 | EEBench O atopile's electrical-engineering agent benchmark: circuit/PCB design tasks expressed as declarative atopile code and verified automatically. | Electrical engineering | 2,2 % | 68,5 % | 1,90 | 🇺🇸 Claude Opus 5.5 | 75,0 % | 26 |
| 32 | Chartography O Surge AI benchmark for professional chart understanding (no tools). | Chart understanding | 2,3 % | 64,8 % | 1,74 | 🇺🇸 Gemini 4 Argon | 71,6 % | 11 |
| 33 | SciCode (main) O SciCode main score. | Code | 2,5 % | 77,1 % | 2,46 | 🇺🇸 GPT-5.5 | 90,6 % | 31 |
| 34 | Agents' Last Exam O Long-horizon agentic workflows across 55 professional sub-industries performed on a computer. | Agentic professional work | 2,6 % | 55,5 % | 1,47 | 🇺🇸 GPT-6 Astra | 59,3 % | 114 |
| 35 | FrontierCode Main O Cognition FrontierCode 1.1 Main: coding-agent changes graded on correctness and mergeability (tests, scope, style, conventions). | Software engineering | 2,6 % | 53,4 % | 1,61 | 🇺🇸 Claude Opus 5.5 | 54,6 % | 46 |
| 36 | FrontierScience O Frontier-level scientific reasoning and QA benchmark. | Science reasoning | 2,8 % | 31,8 % | 1,52 | G MAI-Code-1-Flash | 58,2 % | 10 |
| 37 | SWE-Atlas Test Writing O SWE-Atlas Test Writing: agentic test-writing tasks over real codebases (Scale AI SWE Atlas). | Software testing | 3,2 % | 55,7 % | 1,68 | 🇺🇸 Claude Opus 5 | 68,7 % | 23 |
| 38 | Biology-Instruction O Multi-omics sequence reasoning benchmark for biological data understanding. | Biology multi-omics | 3,3 % | 52,1 % | 0,78 | G Intern-S2-Preview-397B | 56,9 % | 23 |
| 39 | SEC-Bench Pro O Proof-of-concept exploit generation for vulnerabilities in complex software. | Cybersecurity | 3,3 % | 73,6 % | 1,83 | 🇺🇸 GPT-6 Astra | 85,4 % | 12 |
| 40 | SciReasoner O Scientific reasoning benchmark evaluating multimodal AI models on scientific tasks. | Scientific reasoning | 3,4 % | 59,8 % | 1,93 | G Intern-S2-Preview-397B | 64,0 % | 20 |
| 41 | SWE-Atlas Refactoring O SWE-Atlas refactoring track: agentic refactoring tasks over real codebases. | Software engineering | 3,4 % | 53,7 % | 1,64 | 🇺🇸 Claude Opus 5 | 60,0 % | 20 |
| 42 | Harbor-Index O Aggregate index of agentic terminal/coding tasks run through the Harbor evaluation framework. | Agentic coding | 3,5 % | 44,8 % | 1,32 | 🇺🇸 Claude Opus 5 | 56,9 % | 10 |
| 43 | MCPMark O Benchmark for Model Context Protocol (MCP) agent tool-use. | Agent tool-use (MCP) | 3,6 % | 70,4 % | 2,29 | 🇺🇸 GPT-5.6 Terra | 74,0 % | 93 |
| 44 | SciRes FrontierMath Tier 1-3 O SciRes FrontierMath benchmark covering tiers 1-3. | Math (frontier) | 3,6 % | 85,2 % | 2,12 | 🇺🇸 GPT-5.6 Sol | 89,0 % | 17 |
| 45 | PaperBench O Benchmark for understanding and reasoning over research papers. | Research paper understanding | 3,7 % | 90,6 %gesättigt | 2,32 |  🇨🇳 Qwen3.8-Max | 94,1 % | 32 |
| 46 | SWE-Atlas Codebase QnA O SWE-Atlas Codebase QnA: agentic question answering over real codebases. | Codebase understanding | 3,9 % | 64,1 % | 1,72 |  🇨🇳 Kimi-K3 | 68,0 % | 44 |
| 47 | Mol-Instructions O Instruction-following benchmark for bio-molecular understanding and generation. | Bio-molecular instruction following | 4,0 % | 49,1 % | 1,40 | G Intern-S2-397B | 54,0 % | 23 |
| 48 | PostTrainBench O Measures how well AI agents can post-train base LLMs under fixed compute/time constraints; average score across AIME 2025, BFCL, GPQA Main, GSM8K, and HumanEval. | Post-training automation | 4,0 % | 45,2 % | 1,25 | 🇺🇸 Claude Opus 5.5 | 49,3 % | 32 |
| 49 | CyberGym O Benchmark for cybersecurity-related coding and reasoning tasks. | Cybersecurity tasks | 4,1 % | 90,4 %gesättigt | 2,31 | G MiMo V2.6 Flash RL | 95,1 % | 47 |
| 50 | ZeroBench (With Tools) O ZeroBench main set evaluated with tool access in an agent harness (pass@5). | Visual reasoning | 4,2 % | 59,1 % | 1,44 | 🇨🇳 Kimi-K3.1 Pro | 63,0 % | 23 |
| 51 | τ³-Bench Banking O τ³-Bench banking domain: tool-augmented customer-service agent with knowledge retrieval and simulated user. | Agent tasks (banking) | 4,3 % | 50,4 % | 1,32 | 🇨🇳 Qwen3.8-Max | 55,2 % | 121 |
| 52 | ARC-AGI-1 O ARC-AGI Phase 1 aggregate accuracy. | General reasoning | 4,4 % | 98,5 %gesättigt | 3,15 | 🇺🇸 GPT-6 Astra | 98,5 % | 325 |
| 53 | CL-bench Life O Tencent Hy context-learning benchmark built from real-life scenarios (companion to CL-bench); models must learn and apply rules given in long, messy contexts. | Context learning | 4,5 % | 19,0 % | 0,85 | 🇺🇸 GPT-5.5 | 21,1 % | 15 |
| 54 | BabyVision (With Tools) O BabyVision visual perception benchmark evaluated with tool access in an agent harness. | Visual reasoning | 4,8 % | 95,4 %gesättigt | 2,38 | 🇺🇸 GPT-6 Astra | 97,7 % | 20 |
| 55 | Terminal-Bench 2.1 O Updated Terminal-Bench 2.x task set (89 tasks) for end-to-end terminal/command-line agents; successor to Terminal-Bench 2.0. | Agent terminal tasks | 4,9 % | 91,4 %gesättigt | 2,45 | G SWE-2 | 92,8 % | 194 |
| 56 | APEX-Agents 1.1 O Mercor APEX-Agents 1.1 revision: 240 long-horizon cross-application professional-services tasks (investment banking, consulting, corporate law) with updated task specs and judge; Pass@1. Not comparable to APEX-Agents v1.0. | Long horizon professional tasks | 4,9 % | 74,4 % | 1,62 | 🇺🇸 Gemini 4 Argon | 82,4 % | 54 |
| 57 | HLE (Text Only) O Humanity's Last Exam benchmark restricted to text-only inputs. | Advanced reasoning | 5,0 % | 48,8 % | 1,46 | 🇺🇸 Claude Fable 5 | 53,3 % | 206 |
| 58 | SpreadsheetBench 2 O Second version of SpreadsheetBench real-world spreadsheet manipulation tasks. | Spreadsheet manipulation | 5,1 % | 34,4 % | 1,05 | 🇨🇳 Kimi-K3 | 35,8 % | 16 |
| 59 | HeroBench O Benchmark for Long-Horizon Planning and Structured Reasoning in Virtual Worlds. | Long-horizon planning | 5,1 % | 68,7 % | 2,39 |  🇺🇸 Grok 4 | 91,7 % | 24 |
| 60 | Terminal-Bench Hard O Hard subset of Terminal-Bench command-line agent tasks. | Agent terminal tasks | 5,3 % | 49,5 % | 1,85 | 🇺🇸 GPT-5.4 | 57,6 % | 78 |
| 61 | Deep Planning O Benchmark evaluating deep planning and multi-step reasoning capabilities. | Planning and reasoning | 5,3 % | 51,0 % | 1,60 | 🇨🇳 Qwen3.7-Plus | 62,3 % | 27 |
| 62 | HLE O Challenging LLMs at the frontier of human knowledge. | Multi-domain reasoning | 5,3 % | 59,8 % | 1,41 | 🇺🇸 Claude Opus 5.5 | 64,4 % | 258 |
| 63 | BrowseComp-Plus O BrowseComp-Plus: BrowseComp questions over a fixed, curated document corpus for reproducible deep-research agent evaluation. | Deep research | 5,4 % | 66,8 % | 2,15 | G Jais-3-32B | 71,9 % | 34 |
| 64 | USAMO 2026 O 2026 USA Mathematical Olympiad (6 proof problems), graded with MathArena methodology. | Math olympiad (proofs) | 5,5 % | 97,5 %gesättigt | 2,24 | 🇺🇸 Claude Mythos 5 | 99,8 % | 17 |
| 65 | NL2Repo O Long-horizon repository generation: agents build a complete Python repository from a natural-language requirements document; scored by the repo's test suite. | Software engineering | 5,7 % | 71,0 % | 1,61 | 🇺🇸 Claude Opus 5 | 79,9 % | 73 |
| 66 | OfficeQA Pro O Harder OfficeQA variant. Models must use files, tools and context together instead of answering from memory. | Enterprise document QA | 5,9 % | 73,7 % | 1,78 | 🇺🇸 Claude Opus 4.7 | 80,6 % | 34 |
| 67 | CommerceAgentBench O Agentic benchmark of B2B/e-commerce sourcing and commerce workflows (Accio-Lab). | E-commerce agents | 6,0 % | 43,2 % | 1,29 | 🇺🇸 GPT-5.6 Sol | 49,5 % | 10 |
| 68 | GraphWalks BFS 1M O GraphWalks BFS long-context graph traversal at 1M context (F1). | Long-context reasoning | 6,1 % | 74,0 % | 1,65 | 🇺🇸 Claude Mythos 5 | 79,4 % | 15 |
| 69 | MobileWorld O Benchmark of agents completing tasks in mobile (Android) app environments. | Mobile GUI agents | 6,1 % | 79,7 % | 1,87 | 🇺🇸 Claude Fable 5 | 85,5 % | 28 |
| 70 | SWE-bench Multimodal O SWE-bench variant with visual context (screenshots, design mockups) in issue descriptions. | Software engineering | 6,2 % | 57,8 % | 1,19 | 🇺🇸 Claude Opus 5.5 | 61,4 % | 19 |
| 71 | Vals Index O Vals AI composite index of real-world professional/enterprise tasks (finance, legal, coding, etc.). | Knowledge work | 6,6 % | 66,7 % | 1,38 | 🇺🇸 Gemini 4 Argon | 68,9 % | 50 |
| 72 | MLE-Bench Lite O MLE-Bench Lite: 22-competition low-complexity subset of OpenAI MLE-Bench; metric is medal rate. | ML engineering | 6,7 % | 87,6 % | 2,02 | 🇺🇸 GPT-5.6 Sol | 88,9 % | 40 |
| 73 | FSC-147 O Few-shot counting benchmark across 147 categories. | Few-shot counting | 7,0 % | 77,8 % | 1,57 | G Glimpse | 83,4 % | 13 |
| 74 | MMSIBench (circular) O MMSIBench circular subset for spatial reasoning. | Spatial understanding | 7,0 % | 31,2 % | 1,00 | 🇺🇸 GPT-5.5 | 36,0 % | 10 |
| 75 | Tool-Decathlon O Composite tool-use suite measuring multi-domain tool invocation success (Pass@1). | Agent tool-use | 7,2 % | 40,8 % | 1,41 | 🇺🇸 GPT-5.2 | 43,8 % | 18 |
| 76 | OmniDocBench 1.5 O Document understanding benchmark v1.5 with OCR evaluation. Overall Edit Distance metric, lower is better. | OCR | 7,2 % | — | 0,50 | G PaddleOCR-VL | ↓ 0,024 | 61 |
| 77 | NuScenes O 3D scene understanding and perception benchmark for autonomous driving. | 3D scene understanding | 7,3 % | 15,0 % | 0,65 | 🇨🇳 Qwen3.5-397B-A17B | 16,0 % | 12 |
| 78 | FrontierCode Extended O Cognition FrontierCode 1.1 Extended set (score). | Software engineering | 7,6 % | 64,5 % | 1,34 | 🇺🇸 Claude Opus 5.5 | 65,3 % | 46 |
| 79 | TextQuests O Text-based video game benchmark. | Text-based video games | 7,7 % | 37,5 % | 1,36 | 🇺🇸 Gemini 3 Pro | 41,0 % | 18 |
| 80 | GAIA 2 O Grounded agentic intelligence benchmark version 2 covering multi-tool tasks. | General agent tasks | 7,8 % | 56,9 % | 1,76 | G Ring-1T-2.5 | 75,0 % | 17 |
| 81 | AstaBench O Evaluates science agents across literature understanding, data analysis, planning, tool use, coding, and search. | Agent evaluation | 7,9 % | 54,7 % | 1,48 | 🇺🇸 Claude Opus 4.7 | 58,0 % | 27 |
| 82 | Frontier-CS O Frontier-CS agent track (188 tasks): open-ended computer-science optimization problems scored by verifiable feedback. | Algorithmic / open-ended CS problems | 8,3 % | 72,5 % | 1,58 | 🇺🇸 GPT-5.6 Sol | 76,4 % | 12 |
| 83 | OpenAI-MRCR (1M) O OpenAI Multi-Round Chain Reasoning benchmark with 1M context window. | Long-context reasoning | 8,4 % | 83,1 % | 1,53 | 🇺🇸 Claude Opus 4.6 | 92,9 % | 21 |
| 84 | MatTools O Benchmark evaluating LLM agents on materials-science tool usage (pymatgen-based QA and code/tool generation tasks). | Materials science tool use | 8,5 % | 55,2 % | 1,55 | 🇺🇸 GPT-5.5 | 68,8 % | 10 |
| 85 | MLS-Bench-Lite O Lite version of MLS-Bench, a long-horizon agentic machine-learning research/engineering benchmark with an official leaderboard. | Agentic ML engineering | 8,6 % | 45,6 % | 1,03 | 🇺🇸 Claude Fable 5 | 49,9 % | 10 |
| 86 | BenchCAD O CAD modelling benchmark for agents producing 3D CAD artifacts. | CAD generation | 8,6 % | 83,7 % | 1,39 | 🇺🇸 GPT-6 Astra | 95,9 % | 14 |
| 87 | BabyVision O Visual reasoning benchmark testing basic visual perception and understanding. | Visual reasoning | 8,6 % | 88,0 % | 1,49 | 🇨🇳 Qwen3.8-Max | 91,3 % | 71 |
| 88 | BrowseComp O Web browsing comprehension and competence benchmark. | Web browsing | 8,8 % | 92,2 %gesättigt | 2,28 | G Nex-N2.5-Max | 92,6 % | 212 |
| 89 | LiveMCP-101 O A novel real-time evaluation framework and benchmark to stress‑test agents on complex, real‑world tasks. | Agent real-time eval | 9,1 % | 46,7 % | 2,09 | 🇺🇸 GPT-5 | 58,4 % | 18 |
| 90 | USACO O USA Computing Olympiad problems benchmark (307 problems); pass@1. | Code generation (olympiad) | 9,1 % | 12,4 % | 1,27 | 🇺🇸 GPT-4o-2024-05-13 | 18,8 % | 10 |
| 91 | Terminal-Bench O Command-line task completion benchmark for agents. | Agent terminal tasks | 9,3 % | 60,6 % | 1,96 | 🇺🇸 Claude Sonnet 4.5 | 64,5 % | 90 |
| 92 | PaperBench Code-Dev O PaperBench developer subset measuring code reasoning accuracy. | Code understanding | 9,3 % | 83,5 % | 1,95 | 🇨🇳 Qwen3.8-Max | 93,0 % | 11 |
| 93 | MRCR 256K-8N O OpenAI MRCR v2 8-needle, 128K-256K context bucket. | Long-context reasoning | 9,4 % | 88,8 % | 2,09 | 🇺🇸 GPT-5.6 Sol | 93,8 % | 11 |
| 94 | AutomationBench (partial credit) O AutomationBench end-to-end business workflows scored with partial credit per task. | Business workflows | 9,6 % | 72,2 % | 1,63 | 🇺🇸 GPT-5.6 Sol | 81,2 % | 10 |
| 95 | CL-Bench O Comprehensive long-context benchmark evaluating reasoning over extended contexts. | Long-context reasoning | 9,7 % | 25,7 % | 0,60 | 🇺🇸 GPT-5.5 | 27,8 % | 36 |
| 96 | OJBench O Programming problems evaluated via online judge-style execution. | Code generation (online judge) | 9,8 % | 68,3 % | 1,77 |  🇨🇳 Keye-VL-2.0-30B-A3B | 71,5 % | 71 |
| 97 | SkillsBench O SkillsBench agentic tasks; Qwen reports avg of 5 runs on the 78-task self-contained subset (API-dependent tasks excluded). | Agent skills | 9,9 % | 69,6 % | 1,41 | 🇺🇸 GPT-5.6 Sol | 73,5 % | 59 |
| 98 | AA-Omniscience Accuracy O Share of AA-Omniscience questions answered correctly (accuracy component of the AA-Omniscience Index). | Knowledge and hallucination | 10,0 % | 43,8 % | 0,91 | 🇺🇸 Gemini 3.5 Flash | 51,4 % | 42 |
| 99 | VisFactor O Benchmark of fundamental visual-cognition factors adapted from human psychometric perception tests. | Visual perception | 10,1 % | 57,7 % | 1,12 | 🇺🇸 GPT-5.6 Sol | 62,8 % | 13 |
| 100 | AetherCode O Code generation benchmark for diverse coding tasks. | Code generation | 10,2 % | 72,8 % | 1,64 | 🇺🇸 GPT-5.5 | 81,6 % | 17 |
| 101 | AgentWorldBench (Search) O AgentWorldBench Search domain: open-ended evaluation of language world models predicting next environment state in Search agent interactions; 5-dimension rubric mean normalized 0-100. | Language world models | 10,6 % | 35,5 % | 0,81 | 🇨🇳 Qwen-AgentWorld-397B-A17B | 37,8 % | 14 |
| 102 | BenchCAD (python tool) O BenchCAD with a Python tool available. | CAD | 10,7 % | 92,2 %gesättigt | 1,72 | 🇺🇸 Claude Opus 5.5 | 96,2 % | 12 |
| 103 | Toolathlon O Long-horizon, real-world software tool-use tasks. | Agentic software tasks | 10,7 % | 75,9 % | 1,32 | 🇺🇸 Claude Opus 5 | 80,6 % | 61 |
| 104 | Terminal-Bench Terminus O Terminal-Bench Terminus track assessing end-to-end terminal tool use. | Agent terminal tasks | 10,7 % | 62,9 % | 1,60 | 🇺🇸 Gemini 3.1 Pro | 68,5 % | 11 |
| 105 | CMT-Benchmark O Condensed Matter Theory benchmark of expert-written research-level theoretical physics problems. | Physics reasoning (condensed matter) | 10,8 % | 58,6 % | 0,92 | G Sage-A1.5 | 80,0 % | 17 |
| 106 | ProgramBench O ProgramBench (200 instances): agents re-implement complete programs from scratch; score as reported in GLM-5.2 release. | Software engineering | 10,8 % | 80,7 % | 1,17 | 🇺🇸 GPT-6 Astra | 85,4 % | 28 |
| 107 | AMO-Bench O Advanced math olympiad-style benchmark. | Math (competition) | 11,2 % | 65,8 % | 1,85 | 🇺🇸 Gemini 3 Pro | 72,5 % | 25 |
| 108 | Terminal-Bench 2.0 O Second-generation Terminal-Bench leaderboard for end-to-end terminal agents. | Agent terminal tasks | 11,2 % | 81,8 % | 1,77 | 🇺🇸 GPT-5.5 | 84,7 % | 144 |
| 109 | Agents' Last Exam (average score) O Agents' Last Exam (ALE-CLI, 105-task Linux split) average rubric score (as opposed to pass rate). | Agentic professional work | 11,3 % | 61,9 % | 1,02 | 🇺🇸 Claude Opus 5.5 (max) | 63,7 % | 76 |
| 110 | SWE-Bench Pro O Full SWE-Bench Pro benchmark for software-engineering agents. | Software engineering | 11,4 % | 82,5 % | 1,49 | 🇺🇸 Claude Opus 5.5 | 89,9 % | 139 |
| 111 | HealthBench-Hard O Challenging subset of HealthBench focusing on complex, ambiguous clinical cases. | Medical QA (hard) | 11,5 % | 43,1 % | 0,69 | 🇺🇸 GPT-5 | 46,2 % | 24 |
| 112 | CWE-Bench O CWE-Bench v1 vulnerability detection benchmark (cwe-bench.com, Collinear AI); pass@1. | Cybersecurity | 11,8 % | 65,8 % | 1,11 | 🇺🇸 Gemini 4 Argon | 68,0 % | 11 |
| 113 | Hypersim O Hypersim benchmark for synthetic indoor scene understanding and reconstruction. | 3D scene understanding | 11,8 % | 28,2 % | 0,20 | 🇺🇸 GPT-5 Mini (minimal) | 39,3 % | 22 |
| 114 | Vision2Web O Visual website generation benchmark: build working websites from visual designs/prototypes, judged on rendered result. | Coding (UI) | 11,9 % | 77,1 % | 1,43 | 🇺🇸 GPT-5.6 Sol | 79,8 % | 22 |
| 115 | τ-Voice Telecom O τ-Voice (Sierra): τ²-bench telecom tasks over a live speech-to-speech connection; pass^1 (%). | Voice agents | 12,0 % | 76,3 % | 2,06 | 🇺🇸 Gemini 3.8 Live Extended Thinking (high) | 83,9 % | 29 |
| 116 | MCP Universe O Benchmarks multi-step tool-use agents across diverse task suites with a unified overall success metric. | Agent evaluation | 12,0 % | 47,1 % | 1,31 | 🇺🇸 Gemini 3 Pro | 50,7 % | 39 |
| 117 | WildClawBench O Agentic benchmark of in-the-wild tasks run in OpenClaw-style agent scaffolds. | Agentic tasks | 12,3 % | 66,8 % | 1,30 | 🇺🇸 Claude Opus 4.8 | 68,0 % | 53 |
| 118 | Toolathlon Verified O Verified version of Toolathlon (The Tool Decathlon), run via the official evaluation service; pass@1 averaged over 3 runs. | Tool use | 12,3 % | 78,1 % | 1,48 | 🇺🇸 Claude Opus 5 | 80,6 % | 59 |
| 119 | MathArena Apex Shortlist O Shortlist of candidate hard competition problems considered for MathArena Apex. | Mathematics | 12,4 % | 89,7 % | 2,19 | 🇨🇳 Qwen3.8-Flash-Next | 91,8 % | 27 |
| 120 | SuperChem O Chemistry reasoning benchmark evaluating text-based chemical knowledge and problem solving. | Chemistry reasoning | 12,6 % | 71,1 % | 1,34 | 🇺🇸 Claude Opus 5 | 76,7 % | 31 |
| 121 | BioMysteryBench O Anthropic biology research benchmark of open-ended data-analysis 'mysteries'; human-solved subset. | Biology research | 12,8 % | 88,8 % | 1,60 | 🇺🇸 Claude Opus 5 | 90,1 % | 30 |
| 122 | Multi-SWE-Bench O Multi-repository SWE-Bench variant. | Code repair (multi-repo) | 13,2 % | 51,1 % | 1,28 | G MiniMax M2.7 | 52,7 % | 32 |
| 123 | GDPVal O GDPVal benchmark evaluating broad general capabilities of LLMs across diverse tasks. | General capability | 13,2 % | 85,6 % | 1,68 | 🇺🇸 Claude Opus 5 | 89,4 % | 57 |
| 124 | Agentic Coding O Agentic coding benchmark for autonomous software tasks. | Agentic coding | 13,3 % | 51,2 % | 1,45 | 🇺🇸 Gemini 3 Flash Preview | 53,8 % | 22 |
| 125 | HLE (With Tools) O Humanity's Last Exam benchmark evaluated with tool access. | Tool-augmented reasoning | 13,3 % | 66,5 % | 1,15 | 🇨🇳 Kimi-K3.1 Pro | 70,0 % | 153 |
| 126 | FinSearchComp O Financial search and comprehension benchmark measuring retrieval grounded reasoning over financial content. | Financial retrieval | 13,4 % | 67,7 % | 1,66 | G Seed2.0 Pro | 70,2 % | 32 |
| 127 | Chartography (With Tools) O Surge AI Chartography evaluated with tool (code execution) access. | Chart understanding | 13,6 % | 89,8 % | 1,57 | 🇺🇸 GPT-6 Astra | 91,0 % | 16 |
| 128 | VitaBench O Industry-focused benchmark evaluating domain QA performance. | Industry QA | 13,7 % | 53,1 % | 1,00 | 🇺🇸 Claude Opus 4.5 | 56,3 % | 66 |
| 129 | HLE-MultiChoice O The 513 text-only original multiple-choice questions of Humanity's Last Exam, posed as one choice among the lettered answers. | Advanced reasoning | 13,9 % | 56,3 % | 0,20 | 🇺🇸 Claude Opus 5.5 (max) | 70,2 % | 58 |
| 130 | MMSearch O Benchmark for multimodal search engines / LMMs answering queries via image+web search. | Multimodal search | 13,9 % | 78,3 % | 1,77 | 🇨🇳 Kimi-K2.6 Thinking | 80,0 % | 24 |
| 131 | SimpleQA Verified O Verified SimpleQA variant for parametric knowledge accuracy. | QA | 14,2 % | 71,9 % | 0,91 | 🇺🇸 Gemini 3.1 Pro | 77,3 % | 82 |
| 132 | Terminal-Bench Lite O OpenThoughts TBLite: lighter, faster subset-style variant of Terminal-Bench 2.0 for evaluating terminal agents. | Agent terminal tasks | 14,5 % | 68,1 % | 1,77 | 🇨🇳 Qwen3.6-27B | 70,8 % | 15 |
| 133 | VLMs are Biased O Evaluates whether VLMs truly 'see' vs. relying on memorized knowledge; measures bias toward non-visual priors. | Multimodal bias | 14,5 % | 42,5 % | 0,68 | 🇺🇸 GPT-6 Astra | 84,1 % | 13 |
| 134 | ART Agent Red Teaming O Evaluation suite for adversarial red-teaming of autonomous AI agents. | Agent robustness | 14,7 % | — | 1,13 | 🇺🇸 Claude Opus 4.5 | ↓ 33,6 % | 38 |
| 135 | PHYBench O Physics reasoning and calculation benchmark. | Physics reasoning | 15,0 % | 79,3 % | 1,78 | 🇺🇸 Gemini 3.1 Pro | 82,0 % | 33 |
| 136 | ArxivMath O Math problems derived from recent arXiv papers (reported in Anthropic Fable 5/Mythos 5 system card). | Research math | 15,2 % | 74,6 % | 1,23 | 🇺🇸 GPT-5.6 Sol | 79,5 % | 18 |
| 137 | Workspace Bench O Workspace Bench: OpenClaw-style workspace tasks with large heterogeneous file dependencies (100-task setting). | Agentic office work | 15,3 % | 68,5 % | 1,14 | 🇺🇸 Claude Opus 5 | 75,0 % | 15 |
| 138 | SWE-Bench Pro (Public) O Public subset of the SWE-Bench Pro benchmark for software-engineering agents. | Software engineering | 15,3 % | 71,4 % | 1,29 | 🇺🇸 Claude Fable 5 | 80,4 % | 100 |
| 139 | ChessBench (decision) O Choose the best move among candidates (ChessBench); accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Chess | 15,3 % | 23,7 % | 0,46 | G Clef | 24,7 % | 38 |
| 140 | WorkBuddy Bench O Agentic benchmark of workplace-assistant (WorkBuddy) tasks such as documents, spreadsheets and messaging. | Office assistant agents | 15,5 % | 39,0 % | 1,07 | G Muse Glimmer 30B | 45,9 % | 12 |
| 141 | AA-Omniscience Non-Hallucination Rate O Share of AA-Omniscience questions not answered incorrectly (abstains instead of hallucinating) among non-correct answers. | Knowledge and hallucination | 15,6 % | 72,1 % | 1,11 |  🇺🇸 Nemotron 3 Ultra | 78,7 % | 24 |
| 142 | FrontierSWE O Proximal's FrontierSWE benchmark of frontier-difficulty software engineering tasks; dominance score (snapshot-dependent). | Software engineering | 15,7 % | 80,3 % | 1,16 | 🇺🇸 Claude Fable 5 | 88,8 % | 19 |
| 143 | BrowseComp_zh O Chinese variant of the BrowseComp web browsing benchmark. | Web browsing (Chinese) | 16,0 % | 84,7 % | 1,80 | G XYZ-Aquila-pro | 85,1 % | 101 |
| 144 | MolecularIQ O Benchmark of molecular structure reasoning for chemistry-capable LLMs. | Molecular structure reasoning | 16,1 % | 65,6 % | 1,17 | 🇺🇸 GPT-5.5 | 76,4 % | 19 |
| 145 | HLE Overconfidence O Overconfidence rate derived from Humanity's Last Exam evaluations. | Overconfidence / safety | 16,1 % | — | 1,12 | 🇺🇸 GPT-5.2 | ↓ 43,7 % | 15 |
| 146 | FrontierFinance O Professional finance benchmark combining quantitative reasoning with analysis of financial data and documents (reported in Step 5 Preview release). | Finance | 16,4 % | 64,7 % | 0,86 | 🇺🇸 Claude Opus 5 | 69,7 % | 18 |
| 147 | MRCR 128K-8N O Multi-Round Coreference Resolution benchmark at 128k context with 8 needles. | Long-context reasoning | 16,5 % | 93,8 %gesättigt | 1,91 | 🇺🇸 Gemini 3.7 Flash | 97,0 % | 44 |
| 148 | MCP-Atlas O Aggregate MCP agent benchmark covering tool-use and planning tasks. | Agent evaluation | 16,5 % | 87,6 % | 1,66 | G Muse Spark 1.2 | 90,3 % | 130 |
| 149 | SGI-Bench O Scientific general intelligence benchmark of agentic scientific research workflows. | Scientific agents | 16,5 % | 51,4 % | 0,91 | G Intern-S2-Preview | 52,5 % | 17 |
| 150 | WebArena-Verified O Verified version of the WebArena browser-use benchmark. | Web automation | 16,5 % | 69,6 % | 1,23 | 🇨🇳 Kimi-K3 | 71,6 % | 19 |
| 151 | POP909-CL (decision) O Chord-label decisions over POP909 symbolic music; accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Music understanding | 16,6 % | 62,0 % | 1,49 | G Bespoke-Nimble-9B-v3 | 74,6 % | 36 |
| 152 | VLMsAreBiased O Benchmark evaluating biases in vision-language models. | Multimodal bias | 16,7 % | 80,9 % | 1,07 | 🇨🇳 Qwen3.8-Max | 88,3 % | 21 |
| 153 | MedXpertQA (Text) O Text-only subset of MedXpertQA expert-level medical multiple-choice questions. | Medical QA | 17,1 % | 61,7 % | 1,14 | G OpenEvidence Darwin | 72,8 % | 17 |
| 154 | HLE-Verified O Verified and revised version of Humanity's Last Exam (HLE) with component-wise verification protocol. | Multi-domain reasoning | 17,1 % | 53,7 % | 0,82 | 🇺🇸 Gemini 3.8 Flash | 54,9 % | 17 |
| 155 | Aider-Polyglot (Diff) O Aider polyglot leaderboard using diff mode (pass@2). | Code assistant eval | 17,1 % | 88,2 % | 2,87 | 🇺🇸 Gemini 3 Pro Preview | 91,9 % | 65 |
| 156 | OSWorld 2.1 O OSWorld 2.1 long-horizon computer-use benchmark; partial-credit score. | GUI agents | 17,1 % | 80,1 % | 1,19 | 🇺🇸 Claude Opus 5.5 | 81,8 % | 10 |
| 157 | SimpleQA O Simple question answering benchmark. | QA | 17,2 % | 63,6 % | 1,20 |  🇨🇳 ERNIE 5.0 | 74,0 % | 133 |
| 158 | Aider-Polyglot O Aider polyglot coding leaderboard. | Code assistant eval | 17,3 % | 88,5 % | 2,85 | 🇺🇸 Gemini 3 Pro Preview | 92,9 % | 133 |
| 159 | HealthBench Professional O 525 physician-authored clinical conversations (consults, documentation, research) graded against rubric criteria. | Medical QA | 17,9 % | 69,8 % | 0,89 | G OpenEvidence Darwin | 82,7 % | 28 |
| 160 | TIR-Bench O Benchmark for tool-integrated reasoning with visual models. | Tool-integrated reasoning | 17,9 % | 58,5 % | 1,13 | 🇨🇳 Qwen3.5-397B-A17B | 62,5 % | 16 |
| 161 | DeepSearchQA O Multi-step web search and question answering benchmark. | Deep web search QA | 18,4 % | 95,7 %gesättigt | 2,09 | G Atria-Dawn | 96,5 % | 75 |
| 162 | τ-Voice Retail O τ-Voice (Sierra): τ-bench retail customer-service tasks over a live speech-to-speech connection; pass^1 (%). | Voice agents | 18,4 % | 61,7 % | 1,46 | 🇺🇸 GPT-Live-1 (medium) | 72,5 % | 29 |
| 163 | SWE-Bench O Supervised software engineering benchmark across many repos and issues. | Code repair | 18,8 % | 73,5 % | 1,79 | 🇺🇸 GPT-5 Codex | 74,5 % | 43 |
| 164 | ZeroBench (sub) O Subset of ZeroBench targeting harder zero-shot reasoning cases. | Zero-shot generalization | 19,0 % | 47,1 % | 0,61 | G Seed2.1 Pro | 49,4 % | 52 |
| 165 | JobBench (Easy) O JobBench easy split of realistic professional job tasks. | Professional job tasks | 19,1 % | 77,5 % | 1,38 | 🇨🇳 Qwen3.8-Max | 82,8 % | 13 |
| 166 | AutomationBench-AA O Artificial Analysis run of AutomationBench end-to-end business workflows. | Business workflows | 19,2 % | 58,6 % | 0,55 | 🇨🇳 Qwen3.8-Flash-Next | 72,1 % | 16 |
| 167 | FACTS Benchmark Suite O Comprehensive factuality benchmark suite covering held-out internal grounding, parametric knowledge, multimodal understanding, and search retrieval benchmarks. | Held out internal grounding, parametric, MM, and search retrieval benchmarks | 19,2 % | 61,6 % | 1,28 | 🇺🇸 Gemini 3 Pro | 70,5 % | 14 |
| 168 | AA-Omniscience-Decision Accuracy O Accuracy on 500 private AA-Omniscience questions posed as five options (four answers plus "I don't know") for decision models. | Knowledge and hallucination | 19,3 % | 88,9 % | 1,57 | 🇺🇸 Claude Opus 5.5 (max) | 93,4 % | 28 |
| 169 | DoQA O Domain-specific FAQ conversational QA (ChatRAG-Bench subset). | Conversational QA | 19,4 % | 39,9 % | 0,51 | G Yuan3.0 Ultra | 63,4 % | 10 |
| 170 | Audio MultiChallenge O Scale AI Audio MultiChallenge: multi-turn spoken conversation benchmark; average pass rate (APR). | Spoken multi-turn conversation | 19,8 % | 66,8 % | 1,21 | 🇺🇸 Gemini 3.8 Flash | 71,9 % | 31 |
| 171 | WorldVQA O Visual question answering requiring world knowledge and commonsense reasoning. | World knowledge VQA | 19,9 % | 58,5 % | 0,72 | 🇺🇸 Claude Opus 4.6 | 65,4 % | 30 |
| 172 | PerceptionBench O Moonshot benchmark of atomic visual perception capabilities in multimodal LLMs. | Visual perception | 20,3 % | 58,4 % | 0,83 | 🇺🇸 Gemini 3.5 Flash | 60,9 % | 11 |
| 173 | VCT O Virology Capabilities Test: a benchmark that measures an LLM's ability to troubleshoot complex virology laboratory protocols. | Virology capability (protocol troubleshooting) | 20,6 % | 48,8 % | 1,11 | 🇺🇸 Grok 4.1 (Thinking) | 61,0 % | 13 |
| 174 | OSWorld Verified O Verified subset of OSWorld GUI agent benchmark. | GUI agents | 20,7 % | 85,8 % | 1,44 | 🇨🇳 Qwen-CUA | 86,2 % | 87 |
| 175 | BALROG O Benchmark for assessing LLM agents under adversarial and out-of-distribution tool-use scenarios. | Agent robustness | 20,8 % | 37,6 % | 0,94 | 🇺🇸 Grok 4 | 43,6 % | 27 |
| 176 | QwenClawBench O Real-user-distribution Claw (OpenClaw-style) agent benchmark released by Qwen. | Agent tasks (OpenClaw) | 21,3 % | 62,0 % | 1,12 | 🇺🇸 Claude Opus 4.6 | 65,5 % | 37 |
| 177 | BFCL v3 (Multi-Turn) O BFCL v3 Multi-Turn subset for multi-turn tool calling evaluation. | Tool calling | 21,9 % | 66,1 % | 1,24 | G MiniMax M2.5 | 76,8 % | 37 |
| 178 | Defects4J O Real Java bugs from Defects4J used for automated program repair. | Program repair | 21,9 % | 23,9 % | 1,20 | 🇺🇸 GPT-4o-2024-05-13 | 26,1 % | 10 |
| 179 | BigCodeBench Hard O Harder variant of BigCodeBench testing complex programming and library tasks with function-level code generation. | Code generation (hard) | 22,0 % | 35,6 % | 0,90 | 🇺🇸 Claude 3.7 Sonnet (2025-02-19) | 35,8 % | 212 |
| 180 | BIG-Bench Extra Hard O Extra hard subset of BIG-bench tasks. | Multi-task reasoning | 22,2 % | 72,1 % | 1,41 | 🇺🇸 Gemini 3 Pro | 78,8 % | 47 |
| 181 | CountQA O Visual question answering benchmark on object counting in real-world images. | Visual counting | 22,3 % | 84,0 % | 1,11 | 🇺🇸 GPT-6 Astra | 86,4 % | 41 |
| 182 | Doc2Dial O Document-grounded dialogue QA (ChatRAG-Bench subset). | Conversational QA | 22,3 % | 38,3 % | 0,45 | G Yuan3.0 Ultra | 55,8 % | 10 |
| 183 | HiPhO O High-school physics olympiad problems with diagrams for evaluating multimodal physics reasoning. | Physics olympiad (multimodal) | 22,4 % | 86,9 % | 1,38 | 🇨🇳 Qwen3.8-Max | 90,0 % | 26 |
| 184 | AgentWorldBench (SWE) O AgentWorldBench SWE domain: open-ended evaluation of language world models predicting next environment state in SWE agent interactions; 5-dimension rubric mean normalized 0-100. | Language world models | 22,7 % | 65,9 % | 1,04 | 🇨🇳 Qwen-AgentWorld-397B-A17B | 68,5 % | 14 |
| 185 | CNMO 2024 O China National Mathematical Olympiad 2024 evaluation set. | Math (competition) | 22,8 % | 77,8 % | 2,71 | G openPangu-R-72B-2512 Slow Thinking | 82,8 % | 18 |
| 186 | xBench-DeepSearch (2025.10) O xBench DeepSearch benchmark October 2025 snapshot. | Agentic research | 23,0 % | 89,2 % | 1,45 | 🇨🇳 Kimi-K2.6 Thinking | 90,0 % | 37 |
| 187 | AgentWorldBench (Terminal) O AgentWorldBench Terminal domain: open-ended evaluation of language world models predicting next environment state in Terminal agent interactions; 5-dimension rubric mean normalized 0-100. | Language world models | 23,0 % | 57,3 % | 0,86 | 🇺🇸 Claude Opus 4.8 | 59,2 % | 14 |
| 188 | AICodeKing Non-Agentic O Non-agentic code generation benchmark from AICodeKing. | Code generation (non-agentic) | 23,0 % | 87,8 % | 1,15 | 🇺🇸 Claude Opus 4.6 | 100,0 % | 20 |
| 189 | INSCIT O Information-seeking conversations with mixed-initiative (ChatRAG-Bench subset). | Conversational QA | 23,1 % | 34,1 % | 0,39 | G Yuan3.0 Ultra | 40,0 % | 10 |
| 190 | BrowseComp-VL O Vision-language variant of BrowseComp requiring multimodal web browsing. | Multimodal browsing | 23,2 % | 55,8 % | 0,86 | 🇺🇸 GPT-5.5 | 65,7 % | 11 |
| 191 | SWE-Bench Multilingual O Multilingual variant of SWE-Bench for issue fixing. | Code repair (multilingual) | 23,5 % | 91,5 %gesättigt | 1,50 | 🇺🇸 Claude Opus 5.5 | 93,9 % | 167 |
| 192 | DeepResearchBench O Research-oriented agentic writing and planning benchmark. | Agentic research writing | 23,8 % | 48,9 % | 0,84 | G AgentCPM-Report | 50,1 % | 16 |
| 193 | VisuLogic O Logical reasoning and compositionality benchmark for visual-language models. | Visual logical reasoning | 23,8 % | 52,0 % | 0,51 | G Seed2.1 Pro | 54,3 % | 47 |
| 194 | DeepResearch Bench II O Second-generation DeepResearch Bench evaluating long-form deep-research reports with fine-grained rubrics. | Deep research reports | 24,1 % | 55,7 % | 0,61 | G Xing4.0-29B-A4B | 60,8 % | 14 |
| 195 | SciCode (sub) O SciCode subset score (sub). | Code | 24,1 % | 89,1 % | 0,75 | 🇺🇸 GPT-5.5 | 95,1 % | 213 |
| 196 | Video-MME v2 O Second version of the Video-MME video multimodal evaluation suite. | Video understanding | 24,3 % | 63,8 % | 0,61 | 🇺🇸 Gemini 3 Pro | 66,1 % | 27 |
| 197 | SWE-Bench Verified O Verified subset of SWE-Bench for issue fixing. | Code repair | 24,3 % | 94,7 %gesättigt | 1,75 | 🇺🇸 Claude Opus 5 | 96,0 % | 340 |
| 198 | BABE O STEM reasoning benchmark evaluating broad applied and basic engineering knowledge. | STEM reasoning | 24,4 % | 55,2 % | 0,90 | 🇺🇸 GPT-5.2 (high) | 58,1 % | 11 |
| 199 | SATA-Bench O Select-all-that-apply multiple-choice questions; case exact accuracy (Jev Decision Index suite, posed as typed decisions). | Multiple-answer QA | 24,5 % | 35,4 % | 0,79 | G vLLM-SR d3 | 38,4 % | 38 |
| 200 | FinanceAgent v2 O Vals AI Finance Agent Benchmark v2: agentic financial-research tasks over SEC filings; new dataset, not comparable to v1.1. | Agentic finance tasks | 25,1 % | 63,1 % | 0,68 | 🇺🇸 Gemini 4 Argon | 65,4 % | 32 |
| 201 | HMMT 2025 O Harvard–MIT Mathematics Tournament 2025 problems. | Math (competition) | 25,3 % | 99,1 %gesättigt | 2,93 | 🇺🇸 Gemini 3 Pro | 99,8 % | 103 |
| 202 | ClawEval (Pass^3) O ClawEval OpenClaw agent benchmark, pass^3 (all 3 trials succeed). | Agent tasks (OpenClaw) | 25,8 % | 75,0 % | 1,02 |  🇨🇳 DeepSeek V4-Flash-0731 | 78,9 % | 70 |
| 203 | AgentWorldBench (Web) O AgentWorldBench Web domain: open-ended evaluation of language world models predicting next environment state in Web agent interactions; 5-dimension rubric mean normalized 0-100. | Language world models | 26,8 % | 52,4 % | 0,69 | 🇺🇸 Claude Opus 4.8 | 54,7 % | 14 |
| 204 | GAIA O Comprehensive benchmark for agentic tasks. | General AI tasks | 26,9 % | 92,7 %gesättigt | 1,50 | 🇨🇳 DeepSeek V4-Pro-Thinking (max) | 98,1 % | 93 |
| 205 | MedXpertQA-MM O Multimodal medical expert question answering benchmark. | Medical VQA | 27,2 % | 80,6 % | 1,20 | 🇺🇸 GPT-5.6 Sol | 81,5 % | 58 |
| 206 | OMEGA O OMEGA olympiad-grade mathematics reasoning benchmark. | Math (advanced) | 27,3 % | 50,1 % | 1,58 |  🇺🇸 OLMo-3.1-Think-32B | 53,4 % | 38 |
| 207 | SpatialViz O Mental visualization benchmark. | Mental visualization | 28,0 % | 59,6 % | 0,86 | 🇺🇸 GPT-5.2 | 65,8 % | 15 |
| 208 | ResearchRubrics O Benchmark evaluating model ability to conduct research and synthesize findings. | Research evaluation | 28,3 % | 73,8 % | 0,97 | 🇨🇳 Kimi-K3 | 76,2 % | 27 |
| 209 | OSWorld O Agentic GUI task completion and grounding on desktop environments. | GUI agents | 28,4 % | 65,5 % | 0,98 | 🇺🇸 Claude Opus 4.6 | 72,7 % | 26 |
| 210 | TAU1-Airline O Tool-augmented agent evaluation in airline scenarios (TAU1). | Agent tasks (airline) | 28,4 % | 51,4 % | 1,19 | G openPangu-R-72B-2512 Slow Thinking | 56,0 % | 13 |
| 211 | MMSI-Bench O Multi-image spatial intelligence benchmark (standard, non-circular evaluation). | Spatial reasoning | 28,7 % | 49,8 % | 0,45 | 🇺🇸 GPT-6 Astra | 57,9 % | 20 |
| 212 | PopQA O Open-domain popular culture question answering benchmark testing long-tail factual recall. | Knowledge / QA | 28,8 % | 54,9 % | 0,31 |  🇺🇸 Llama 3.1 Tulu 3 405B SFT | 55,7 % | 66 |
| 213 | BrowseComp (With Content Manager) O BrowseComp benchmark evaluated with content manager assistance. | Web browsing | 28,8 % | 86,5 % | 1,40 | 🇨🇳 Kimi-K3 | 91,2 % | 35 |
| 214 | MTVQA O Multilingual text-centric visual question answering across 9 languages (average accuracy). | Multilingual text-centric VQA | 28,9 % | 30,8 % | 0,67 |  🇨🇳 InternVL2.5-78B | 31,9 % | 24 |
| 215 | SURDS O Spatial understanding and reasoning benchmark on autonomous-driving scenes. | Driving scene spatial reasoning | 29,1 % | 74,7 % | 1,00 | 🇺🇸 Claude Fable 5 | 79,4 % | 18 |
| 216 | OpenAI-MRCR (128k) O OpenAI Multi-Round Chain Reasoning benchmark with 128k context window. | Long-context reasoning | 29,2 % | 92,7 %gesättigt | 1,45 | G K-EXAONE 2.0 | 94,4 % | 44 |
| 217 | Omni-MATH-HARD O Challenging math benchmark (Omni-MATH-HARD). | Math | 29,4 % | 64,4 % | 1,50 | 🇺🇸 GPT-5 (high) | 73,6 % | 20 |
| 218 | LCB Pro 25Q2 (Easy) O LiveCodeBench Pro 2025 Q2 easy subset. | Code generation | 29,8 % | 93,8 %gesättigt | 2,27 | 🇺🇸 Gemini 3 Pro | 94,8 % | 39 |
| 219 | FinanceAgent O Interactive financial agent benchmark requiring multi-step tool use. | Agentic finance tasks | 29,9 % | 56,6 % | 0,78 | 🇺🇸 Claude Opus 4.6 | 60,7 % | 10 |
| 220 | AgentWorldBench (Android) O AgentWorldBench Android domain: open-ended evaluation of language world models predicting next environment state in Android agent interactions; 5-dimension rubric mean normalized 0-100. | Language world models | 30,0 % | 61,0 % | 0,78 | 🇺🇸 Claude Opus 4.6 | 61,7 % | 14 |
| 221 | OneMillionBench O OneMillion Bench: expert-level, economically valuable tasks evaluated with tools (search). | Agentic search / expert tasks | 30,6 % | 69,3 % | 0,76 | 🇺🇸 Claude Opus 4.7 | 73,0 % | 13 |
| 222 | EMMA O EMMA benchmark for multimodal reasoning. | Multimodal reasoning | 30,9 % | 75,5 % | 1,08 | G Seed2.1 Pro | 79,3 % | 11 |
| 223 | WideSearch O Wide web search and QA benchmark. | Web search | 30,9 % | 84,4 % | 1,18 | 🇺🇸 GPT-5.6 Sol | 86,3 % | 90 |
| 224 | TOMATO O Temporal ordering and motion analysis benchmark (TOMATO). | Temporal understanding | 31,0 % | 88,2 % | 0,95 | 🇺🇸 GPT-6 Astra | 96,0 % | 42 |
| 225 | SEAL-0 O Evaluation of multi-step browsing agents on search, evidence gathering, and synthesis tasks. | Agentic web search | 31,4 % | 57,8 % | 0,57 | G MiroThinker-H1 | 61,3 % | 44 |
| 226 | AgentWorldBench (OS) O AgentWorldBench OS domain: open-ended evaluation of language world models predicting next environment state in OS agent interactions; 5-dimension rubric mean normalized 0-100. | Language world models | 32,0 % | 68,0 % | 0,86 | 🇺🇸 Claude Opus 4.6 | 70,2 % | 14 |
| 227 | AutoCodeBench O End-to-end autonomous coding benchmark with unit-test based execution across diverse repositories and tasks. | Autonomous coding | 32,0 % | 51,3 % | 0,86 | 🇺🇸 Claude Opus 4 (Thinking) | 52,4 % | 45 |
| 228 | LPFQA O Long-form financial question answering benchmark. | Finance QA | 32,2 % | 53,8 % | 0,65 | 🇺🇸 Claude Sonnet 4.5 | 54,9 % | 12 |
| 229 | SUNRGBD O SUN RGB-D benchmark for indoor scene understanding from RGB-D imagery. | 3D scene understanding | 32,5 % | 41,1 % | 0,25 | 🇺🇸 GPT-5 Mini (minimal) | 45,8 % | 19 |
| 230 | ScreenSpot-Pro O Average center accuracy on ScreenSpot-Pro. | Screen UI locators | 32,6 % | 87,8 % | 1,26 | 🇺🇸 GPT-6 Astra | 92,7 % | 164 |
| 231 | Spiral-Bench O A LLM-judged benchmark measuring sycophancy and delusion reinforcement. | Safety / sycophancy | 32,6 % | 80,2 % | 0,93 | 🇺🇸 GPT-5 | 87,0 % | 24 |
| 232 | HMMT Feb 2025 O Harvard–MIT Mathematics Tournament February 2025 problems. | Math (competition) | 32,7 % | 99,6 %gesättigt | 2,86 | 🇺🇸 GPT-5.2 (high) | 100,0 % | 96 |
| 233 | EgoPlan-Bench2 O Egocentric embodied task-planning benchmark (v2). | Embodied reasoning | 32,7 % | 58,7 % | 0,56 | 🇺🇸 GPT-6 Astra | 69,3 % | 13 |
| 234 | KoBALT O Korean benchmark for knowledge and language understanding. | Korean knowledge | 32,9 % | 71,2 % | 1,02 | 🇨🇳 DeepSeek V4-Flash-Thinking (max) | 75,3 % | 26 |
| 235 | KoSimpleQA O Korean simple question answering benchmark. | Korean QA | 33,1 % | 45,3 % | 0,56 | G Kanana-2-30B-A3B-Mid-2601 | 49,7 % | 13 |
| 236 | IMO AnswerBench O Evaluates free-form solutions to International Mathematical Olympiad problems using expert-style grading rubrics. | Math (competition) | 33,2 % | 92,0 %gesättigt | 1,71 | 🇨🇳 DeepSeek V4-Pro | 93,0 % | 150 |
| 237 | PolyMATH O Polyglot mathematics benchmark assessing cross-topic math reasoning. | Math reasoning | 33,2 % | 83,1 % | 1,26 | 🇨🇳 Qwen3.7-Max | 86,5 % | 69 |
| 238 | BLEnD O Benchmark assessing cultural knowledge, sensitivity, and language use across contexts. | Cultural understanding | 33,5 % | 40,7 % | 0,90 | 🇨🇳 Qwen2.5-72B-Instruct | 75,4 % | 12 |
| 239 | HMMT Feb 2026 O Harvard–MIT Mathematics Tournament February 2026 problems. | Math (competition) | 33,9 % | 98,1 %gesättigt | 2,02 | G Darwin-180B-RSI | 100,0 % | 120 |
| 240 | AA-LCR O A challenging benchmark measuring language models' ability to extract, reason about, and synthesize information from long-form documents ranging from 10k to 100k tokens. | Long-context reasoning | 34,1 % | 83,9 % | 1,11 | 🇨🇳 Kimi-K3 | 88,7 % | 209 |
| 241 | Machiavelli O Benchmark for deceptive or manipulative behavior in social interactions. | Deception / safety | 34,1 % | — | 0,36 | 🇺🇸 Claude Haiku 4.5 | ↓ 52,2 % | 15 |
| 242 | MultiChallenge O Multi-domain instruction-following benchmark. | Instruction following | 34,2 % | 75,1 % | 0,83 | G Nova 2 Pro | 77,7 % | 129 |
| 243 | TAU2-Telecom O Tool-augmented agent evaluation in telecom scenarios (TAU2). | Agent tasks (telecom) | 34,2 % | 99,4 %gesättigt | 2,15 | 🇨🇳 Qwen3.6-35B-A3B | 100,0 % | 203 |
| 244 | QuAC O Question answering in context. | Conversational QA | 34,6 % | 51,9 % | 0,20 | G Yuan3.0 Ultra | 54,5 % | 23 |
| 245 | FinanceAgent v1.1 O Finance Agent v1.1 benchmark for interactive financial agent evaluation. | Agentic finance tasks | 34,6 % | 68,8 % | 0,66 | 🇺🇸 Gemini 3.7 Flash | 71,0 % | 30 |
| 246 | SFE O Structured factual evaluation for multimodal models. | Multimodal reasoning | 34,7 % | 62,1 % | 0,59 | 🇺🇸 Gemma 4 26B A4B IT | 63,1 % | 23 |
| 247 | HybriDial O Hybrid table+text grounded dialogue QA (ChatRAG-Bench subset). | Conversational QA | 35,0 % | 53,7 % | 0,48 | G Yuan3.0 Ultra | 72,9 % | 10 |
| 248 | BeyondAIME O Advanced math problems exceeding AIME difficulty. | Math (beyond AIME) | 35,1 % | 88,5 % | 1,38 | 🇺🇸 GPT-5.5 | 91,0 % | 35 |
| 249 | Enamel O Composite capability benchmark capturing broad model performance (Enamel score). | Composite capability | 35,1 % | 42,6 % | 1,01 | G Rnj-1.5 | 50,7 % | 10 |
| 250 | Home appliance simulator (decision) O Multi-step home appliance control decisions; case exact accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Device control | 35,2 % | 92,5 %gesättigt | 2,59 | G Bespoke-Nimble-9B-v3 | 98,9 % | 38 |
| 251 | ContPhy O Continuum Physical Concept Learning and Reasoning from videos. | Video physical reasoning | 35,3 % | 64,3 % | 0,74 | G Seed2.0 Pro | 67,4 % | 10 |
| 252 | VLABench O Language-conditioned robot manipulation reasoning benchmark (VLM track). | Embodied reasoning | 35,4 % | 68,2 % | 0,63 | G PhysBrain1.5-8B | 76,4 % | 13 |
| 253 | SpreadSheetBench v1 O Real-world spreadsheet manipulation tasks (v1). | Spreadsheet manipulation | 35,5 % | 88,6 % | 1,32 | G MiniMax M3 | 89,4 % | 24 |
| 254 | CrossVid O Benchmark for cross-video (multi-video) understanding and reasoning. | Multi-video reasoning | 35,5 % | 62,3 % | 0,63 | G Seed2.1 Pro | 65,0 % | 10 |
| 255 | AutoCodeBench-Lite O Lite version of AutoCodeBench focusing on smaller tasks with the same end-to-end, unit-test-based evaluation. | Autonomous coding | 35,7 % | 63,4 % | 1,18 | 🇺🇸 Claude Opus 4 | 64,5 % | 46 |
| 256 | XLRS-Bench O Remote sensing benchmark for evaluating multimodal AI on satellite and aerial imagery. | Remote sensing | 35,8 % | 52,5 % | 0,43 | 🇺🇸 Gemini 3.1 Pro | 54,3 % | 18 |
| 257 | ChartMuseum O Large-scale curated collection of charts for evaluating parsing, grounding, and reasoning. | Chart understanding | 35,8 % | 79,6 % | 1,10 | 🇺🇸 Claude Mythos 5 | 85,9 % | 50 |
| 258 | MiniF2F (Test) O MiniF2F competition benchmark (test split). | Math competition | 35,8 % | 91,3 %gesättigt | 0,48 | G Leanstral 1.5 | 100,0 % | 26 |
| 259 | MHPP O Mostly Hard Python Problems: 210 hand-written function-level Python problems probing harder code-generation skills; pass@1. | Code generation | 36,0 % | 33,7 % | 1,31 | G Stable-DiffCoder-8B-Instruct | 42,4 % | 10 |
| 260 | Minerva (Video) O Minerva video question-answering benchmark with complex multi-step reasoning over videos (not the Minerva math set). | Video reasoning | 36,1 % | 68,0 % | 0,82 | G Seed2.1 Pro | 70,7 % | 10 |
| 261 | AndroidWorld O Benchmark for agents operating Android apps via UI automation. | Mobile agents | 36,2 % | 86,2 % | 1,06 | 🇺🇸 Claude Fable 5 | 88,8 % | 70 |
| 262 | FrontierScience Olympiad O Olympiad-level problems from the FrontierScience benchmark. | Science reasoning (olympiad) | 36,3 % | 81,1 % | 0,96 | G Intern-S2-397B | 87,0 % | 49 |
| 263 | LiveCodeBench v5 (2024.10-2025.02) O LiveCodeBench v5 snapshot covering Oct 2024-Feb 2025. | Code generation | 36,5 % | 87,5 % | 1,91 | 🇺🇸 Gemini 3 Pro Preview | 91,7 % | 147 |
| 264 | ArtifactsBench O Artifacts-focused coding and tool-use benchmark evaluating generated code artifacts. | Agentic coding | 36,5 % | 77,1 % | 0,88 | G TeleChat3-Coder-36B-Thinking | 82,8 % | 55 |
| 265 | TAU2-Airline O Tool-augmented agent evaluation in airline scenarios (TAU2). | Agent tasks (airline) | 36,6 % | 85,2 % | 1,24 | 🇺🇸 Claude Opus 4.7 | 88,5 % | 101 |
| 266 | TopiOCQA O Topic-switching open-domain conversational QA (ChatRAG-Bench subset). | Conversational QA | 36,6 % | 54,0 % | 0,37 | G Yuan3.0 Ultra | 72,4 % | 10 |
| 267 | Omni-MATH O Omni-MATH benchmark covering diverse math reasoning tasks across difficulty levels. | Math reasoning | 37,2 % | 67,1 % | 1,29 | G Ling 1T | 74,5 % | 36 |
| 268 | AIME 2025 O American Invitational Mathematics Examination 2025 problems. | Math (competition) | 37,2 % | 100 %gesättigt | 2,84 | 🇺🇸 Claude Sonnet 4.5 | 100,0 % | 498 |
| 269 | AgentWorldBench (MCP) O AgentWorldBench MCP domain: open-ended evaluation of language world models predicting next environment state in MCP agent interactions; 5-dimension rubric mean normalized 0-100. | Language world models | 37,7 % | 69,3 % | 0,71 | 🇺🇸 GPT-5.4 | 70,1 % | 14 |
| 270 | TreeBench O Evaluates hierarchical/tree-structured reasoning and planning capabilities in LLMs/VLMs. | Reasoning with tree structures | 37,7 % | 63,8 % | 0,61 | G Seed2.1 Turbo | 71,1 % | 17 |
| 271 | TAU2-Bench Airline (Verified) O Human-validated revision of the tau2-bench airline domain (amazon-agi/tau2-bench-verified). | Agent tasks | 38,0 % | 70,6 % | 1,13 | 🇺🇸 GPT-5.1 | 72,4 % | 11 |
| 272 | GDPval (rubrics) O GDPval tasks scored pointwise against rubrics (not pairwise win rate), as reported by MiniMax. | Professional knowledge work | 38,1 % | 90,2 %gesättigt | 1,28 | 🇨🇳 Kimi-K3 | 91,6 % | 26 |
| 273 | VABench-Point O VABench visual-action pointing subset. | Embodied reasoning | 38,2 % | 66,3 % | 0,56 | G Embodied-R1.5-8B | 75,7 % | 13 |
| 274 | MMMU-Pro (vision) O MMMU-Pro vision-only setting. | Multimodal understanding (vision) | 38,3 % | 61,9 % | 0,98 | 🇺🇸 GPT-5 (high) | 66,9 % | 49 |
| 275 | τ³-Bench Airline O τ³-Bench airline domain: tool-augmented customer-service agent with simulated user. | Agent tasks (airline) | 38,4 % | 83,4 % | 1,37 | 🇨🇳 Kimi-K2.6 Thinking | 85,8 % | 17 |
| 276 | longbench-v2 O Next-generation LongBench v2 long-context evaluation benchmark. | Long-context eval | 38,5 % | 68,2 % | 0,69 | 🇺🇸 Claude Opus 4.8 | 69,1 % | 136 |
| 277 | LiveCodeBench O Live coding and execution-based evaluation benchmark (v6 dataset). | Code generation | 38,7 % | 94,4 %gesättigt | 1,77 | 🇺🇸 GPT-5.5 | 96,2 % | 663 |
| 278 | MRCR v2 O Multi-round multimodal chain-of-reasoning evaluation (v2). | Multimodal reasoning | 38,7 % | 90,5 %gesättigt | 1,22 | 🇺🇸 GPT-5.5 | 94,8 % | 22 |
| 279 | FD-Bench V1.5 O Full-duplex spoken dialogue benchmark v1.5 (interruption handling, backchannel, turn-taking quality); average score. | Full-duplex speech interaction | 38,8 % | 61,7 % | 0,86 | 🇺🇸 GPT-Live-1 (low) | 80,1 % | 10 |
| 280 | ODinW-13 O Object Detection in the Wild benchmark covering 13 real-world domains. | Object detection (in the wild) | 38,9 % | 50,9 % | 0,29 | 🇨🇳 Qwen3.6-Plus | 51,8 % | 33 |
| 281 | RepoBench O Repository-level code comprehension and reasoning benchmark. | Code understanding | 39,0 % | 69,9 % | 1,04 | 🇺🇸 Claude Sonnet 4.5 | 83,8 % | 16 |
| 282 | τ³-Bench Telecom O τ³-Bench telecom domain: tool-augmented customer-service agent with simulated user (dual-control). | Agent tasks (telecom) | 39,0 % | 97,8 %gesättigt | 2,37 | 🇨🇳 GLM-5.1 | 98,7 % | 19 |
| 283 | Longform Creative Writing O Longform creative writing evaluation (EQ-Bench). | Creative writing | 39,2 % | 84,4 % | 0,91 | 🇺🇸 Claude Opus 5 | 86,3 % | 145 |
| 284 | Charades-STA O Charades-STA temporal grounding (mIoU). | Video grounding | 39,3 % | 62,1 % | 1,18 |  🇨🇳 Seed1.5-VL-Thinking | 64,0 % | 25 |
| 285 | HealthBench O Comprehensive medical knowledge and clinical reasoning benchmark across specialties and tasks. | Medical QA | 39,4 % | 66,7 % | 0,45 | 🇺🇸 GPT-5 | 67,2 % | 45 |
| 286 | QReCC O Open-domain conversational QA with question rewriting (ChatRAG-Bench subset). | Conversational QA | 39,4 % | 51,0 % | 0,32 | G Yuan3.0 Ultra | 57,3 % | 10 |
| 287 | Arena-Hard V2 Hard Prompt O Chatbot Arena Hard V2 benchmark using the hard prompt win-rate subset. | Chat ability | 39,5 % | 75,7 % | 1,00 | 🇺🇸 Nemotron-Cascade-2-30B-A3B | 88,2 % | 19 |
| 288 | LeetCode Contest O LeetCode weekly/biweekly contest problems (Jul 2023 - Jan 2024) collected by DeepSeek; pass@1. | Code generation (competition) | 39,5 % | 31,9 % | 2,08 | 🇺🇸 GPT-4 | 48,4 % | 15 |
| 289 | SuperGPQA O Harder GPQA variant assessing advanced graduate-level reasoning. | Graduate-level QA | 39,6 % | 74,4 % | 0,90 | 🇺🇸 Gemini 3.1 Pro | 76,6 % | 149 |
| 290 | DUDE O Long-context multimodal understanding benchmark. | Multimodal long-context | 39,8 % | 82,8 % | 1,07 | 🇺🇸 Claude Opus 4.7 | 84,5 % | 11 |
| 291 | Part-Affordance O Part-level affordance grounding benchmark. | Embodied reasoning | 39,9 % | 78,8 % | 0,72 | G PhysBrain1.5-8B | 84,0 % | 13 |
| 292 | MindCube O Spatial navigation benchmark. | Spatial navigation | 39,9 % | 83,9 % | 0,81 | G ACE-Brain-0.5-8B | 94,2 % | 51 |
| 293 | TAU2-Retail O Tool-augmented agent evaluation in retail scenarios (TAU2). | Agent tasks (retail) | 40,1 % | 90,4 %gesättigt | 1,51 | 🇺🇸 Claude Opus 4.6 | 91,9 % | 114 |
| 294 | Harvey LAB-AA O Harvey's Legal Agent Benchmark as run by Artificial Analysis (LAB-AA): complex legal workflows score. | Legal agent tasks | 40,1 % | 92,2 %gesättigt | 1,12 | 🇨🇳 Kimi-K3 | 94,6 % | 20 |
| 295 | Nexus (0-shot) O Nexus tool-use benchmark, zero-shot setting. | Tool use | 40,1 % | 52,7 % | 0,74 | 🇺🇸 Llama 3.1 405B | 58,7 % | 15 |
| 296 | JevBench (Benchmark Heaven) O Benchmark Heaven JevBench for typed probabilistic decision models; score is the chance-corrected Intelligence (typed decision accuracy) axis. | Typed decisions | 40,1 % | 83,3 % | 1,56 | 🇺🇸 GPT-6 Luna (medium) | 96,9 % | 25 |
| 297 | CGBench O Cartoon/CG long video question answering benchmark. | Long video QA | 40,3 % | 65,2 % | 0,75 | G Seed2.0 Lite (0428) | 65,5 % | 12 |
| 298 | ChartQA-Pro O Professional-grade chart question answering with diverse chart types and complex reasoning. | Chart understanding (VQA) | 40,4 % | 71,1 % | 0,80 | 🇺🇸 Claude Mythos 5 | 71,6 % | 46 |
| 299 | MathVision O Visual math reasoning benchmark with problems that combine images (charts, diagrams) and text. | Math reasoning (multimodal) | 40,5 % | 97,5 %gesättigt | 1,51 | 🇺🇸 Claude Fable 5 | 98,6 % | 246 |
| 300 | Bird-SQL O Natural language to SQL generation benchmark. | Text-to-SQL | 41,2 % | 56,2 % | 0,24 | 🇺🇸 Gemini 2.0 Pro | 59,3 % | 34 |
| 301 | MMLongBench-Doc O Evaluates long-context document understanding with mixed text, tables, and figures across multiple pages. | Long-context multimodal documents | 41,4 % | 64,4 % | 0,58 | 🇨🇳 Qwen3.8-Flash-Next | 67,9 % | 52 |
| 302 | DRACO O DRACO deep-research benchmark scored with per-question rubrics. | Deep research | 41,4 % | 83,9 % | 0,85 | 🇺🇸 Claude Opus 5 | 88,6 % | 20 |
| 303 | RefSpatialBench O Reference spatial understanding benchmark covering spatial grounding tasks. | Spatial reasoning | 41,5 % | 74,8 % | 0,55 | 🇺🇸 GPT-6 Astra | 80,1 % | 71 |
| 304 | IFBench O Instruction-following benchmark measuring compliance and adherence. | Instruction following | 41,6 % | 85,8 % | 0,91 | G Flint | 89,2 % | 372 |
| 305 | AIME 2024 O American Invitational Mathematics Examination 2024 problems. | Math (competition) | 41,7 % | 95,5 %gesättigt | 3,19 | 🇺🇸 GPT-OSS 120B | 96,6 % | 266 |
| 306 | τ-Voice Airline O τ-Voice (Sierra): τ-bench airline customer-service tasks over a live speech-to-speech connection; pass^1 (%). | Voice agents | 41,8 % | 72,0 % | 0,76 | 🇺🇸 GPT-Live-1 (medium) | 76,7 % | 29 |
| 307 | BFCL v4 O BFCL v4 benchmark for functional code correctness and logic. | Code reasoning | 42,0 % | 78,3 % | 0,83 | 🇨🇳 Qwen3.6-27B | 80,2 % | 161 |
| 308 | ZebraLogic O Logical reasoning benchmark assessing complex pattern and rule inference. | Logical reasoning | 42,0 % | 97,0 %gesättigt | 2,84 | 🇺🇸 GPT-5 (high) | 98,8 % | 147 |
| 309 | RW Search O Real-world search benchmark evaluating retrieval and reasoning. | Agentic search | 42,2 % | 81,5 % | 0,99 | 🇺🇸 GPT-5.5 | 85,3 % | 15 |
| 310 | ViewSpatial-Bench O Multi-perspective spatial localization benchmark (camera vs. human viewpoints). | Spatial reasoning | 42,5 % | 59,2 % | 0,30 | G ZDTaichu5.0-9B | 62,5 % | 20 |
| 311 | MARS-Bench O Instruction-following benchmark with complex tasks. | Instruction following | 42,6 % | 86,3 % | 1,32 | 🇺🇸 GPT-5.2 (high) | 87,9 % | 12 |
| 312 | ERQA O Spatial recognition and reasoning QA benchmark (ERQA). | Spatial reasoning | 42,6 % | 82,7 % | 0,57 | G Glimpse | 83,8 % | 114 |
| 313 | AlpacaEval O Automatic eval using GPT-4 as a judge. | Instruction following | 42,7 % | 77,1 % | 1,31 | 🇺🇸 Llama-3_1-70B-TFree-HAT-SFT | 99,4 % | 51 |
| 314 | BankerToolBench O BankerToolBench: tool-using agent tasks modeled on investment-banking workflows, scored with dataset rubrics. | Investment banking agent tasks | 42,8 % | 79,7 % | 0,74 | 🇺🇸 Claude Opus 5 | 81,9 % | 14 |
| 315 | JCulture O Japanese culture knowledge QA benchmark (as reported by Liquid AI). | Japanese culture knowledge | 42,9 % | 32,9 % | 0,59 |  🇺🇸 LFM2.5 1.2B JP 202606 | 35,8 % | 10 |
| 316 | Arena-Hard V2 O Updated Arena-Hard v2 prompts on Chatbot Arena. | Chat ability | 43,0 % | 88,4 % | 1,28 | 🇺🇸 GPT-OSS 120B | 90,3 % | 104 |
| 317 | Arena-Hard V2 Creative Writing O Chatbot Arena Hard V2 creative writing win-rate subset. | Creative writing | 43,0 % | 89,2 % | 1,65 | 🇺🇸 Gemini 3 Pro | 93,6 % | 18 |
| 318 | JamC-QA O Japanese-specific knowledge multiple-choice QA (culture, customs, regional, law, medicine) from the Swallow evaluation suite. | Japanese knowledge QA | 43,3 % | 63,9 % | 0,69 | 🇺🇸 GPT-5 mini | 70,1 % | 31 |
| 319 | SimpleVQA O Lightweight visual question answering set with everyday scenes. | General VQA | 43,4 % | 78,3 % | 0,67 | 🇨🇳 Qwen3.7-Plus | 81,7 % | 100 |
| 320 | AdvancedIF O Meta's AdvancedIF rubric-based benchmark for complex, multi-turn and system-prompt instruction following. | Instruction following | 44,1 % | 77,5 % | 0,79 | 🇺🇸 Gemini 3.1 Pro | 83,9 % | 12 |
| 321 | Codeforces (Percentile) O Competitive programming performance on Codeforces problems, reported as the percentile of human competitors outperformed. | Competitive programming | 44,1 % | 92,3 %gesättigt | 2,70 | G Ring-1T-preview | 94,7 % | 27 |
| 322 | OCRReasoning O OCR reasoning benchmark combining text extraction with multi-step reasoning over documents. | OCR reasoning | 44,1 % | 67,1 % | 0,60 | 🇺🇸 Gemini 2.5 Pro | 70,8 % | 16 |
| 323 | OpenEQA O Open-vocabulary embodied question answering over episodic memory of indoor scenes. | Embodied QA | 44,2 % | 56,8 % | 0,66 | 🇺🇸 GPT-5 | 64,4 % | 12 |
| 324 | COLLIE O Comprehensive instruction-following evaluation suite. | Instruction following | 44,7 % | 97,6 %gesättigt | 1,96 | 🇺🇸 GPT-5 | 99,0 % | 24 |
| 325 | xBench-DeepSearch O Evaluates multi-hop deep research workflows on xBench DeepSearch tasks. | Agentic research | 44,7 % | 77,1 % | 0,77 | 🇺🇸 GPT-5 (high) | 77,9 % | 42 |
| 326 | MicroVQA O Visual question answering benchmark for biological microscopy images. | Biological microscopy | 44,9 % | 69,4 % | 0,55 | 🇺🇸 Gemini 3.1 Pro | 71,0 % | 18 |
| 327 | OlympiadBench O Advanced mathematics olympiad-style problem benchmark. | Math (olympiad) | 45,4 % | 76,5 % | 2,57 | 🇨🇳 Qwen3-30B-A3B-Instruct-2507 | 77,6 % | 80 |
| 328 | VideoHolmes O Video question answering benchmark focused on detective-style clues. | Video QA | 45,5 % | 67,5 % | 0,59 | G Seed2.1 Pro | 68,2 % | 22 |
| 329 | BigCodeBench O BigCodeBench evaluates large language models on practical code generation tasks with unit-test verification. | Code Generation | 45,5 % | 64,3 % | 0,54 | G MiMo V2 Flash Base | 70,1 % | 233 |
| 330 | CharXiv (RQ) O Scientific chart/table reasoning questions from arXiv PDFs. | Chart reasoning (PDF) | 45,7 % | 91,8 %gesättigt | 1,07 | 🇺🇸 Claude Fable 5 | 93,5 % | 274 |
| 331 | LAB-Bench 2 O LABBench2: biology real-world research tasks (macro-average over 11 sub-tasks: DbQA2, FigQA2-img/pdf, LitQA3, PatentQA, ProtocolQA2, SourceQuality, SuppQA2, TableQA2-img/pdf, TrialQA). | Biology research | 45,7 % | 85,3 % | 0,79 | 🇺🇸 Gemini 4 Argon | 88,8 % | 11 |
| 332 | LongBench O Long-context understanding across tasks. | Long-context eval | 45,8 % | 57,7 % | 0,51 | 🇨🇳 Qwen3.6-27B | 60,9 % | 37 |
| 333 | AA-Index O Comprehensive QA index across diverse domains. | Multi-domain QA | 46,2 % | 62,5 % | 0,31 | 🇺🇸 Grok 4 | 73,2 % | 11 |
| 334 | HMMT Nov 2025 O Harvard–MIT Mathematics Tournament November 2025 problems. | Math (competition) | 46,3 % | 98,5 %gesättigt | 1,99 | 🇺🇸 GPT-5.2 (high) | 100,0 % | 81 |
| 335 | All-Angles Bench O All-Angles benchmark for spatial recognition and 3D perception. | Spatial perception | 46,5 % | 59,3 % | 0,56 | 🇨🇳 Qwen3.6-35B-A3B | 64,0 % | 17 |
| 336 | JGPQA O Japanese translation of GPQA graduate-level science QA. | Graduate-level QA (Japanese) | 46,6 % | 71,3 % | 0,91 | 🇺🇸 GPT-5 mini | 75,0 % | 41 |
| 337 | Odyssey Math O Odyssey multi-step math benchmark. | Math reasoning | 46,6 % | 49,6 % | 1,16 | 🇨🇳 DeepSeek-Coder V2 Instruct | 53,7 % | 14 |
| 338 | NOVA-63 O Multilingual evaluation benchmark covering 63 languages. | Multilingual evaluation | 46,6 % | 58,9 % | 0,31 | 🇨🇳 Qwen3.5-397B-A17B | 59,1 % | 29 |
| 339 | VSI-Bench O Visual spatial intelligence benchmark covering 3D reasoning and spatial inference tasks. | Spatial intelligence | 46,6 % | 72,8 % | 0,32 | 🇺🇸 Molmo2-ER | 74,5 % | 55 |
| 340 | OSWorld-G O OSWorld-G center accuracy (no_refusal). | GUI agents | 46,7 % | 83,6 % | 0,86 | G Nex-N2.5-Pro | 87,4 % | 86 |
| 341 | RoboVQA O Robotics video question answering benchmark. | Embodied reasoning | 46,9 % | 60,9 % | 0,20 | G PhysBrain1.5-2B | 62,9 % | 13 |
| 342 | MMMU PRO O Professional/advanced subset of MMMU for multimodal reasoning. | Multimodal understanding (hard) | 47,0 % | 83,7 % | 0,95 | 🇺🇸 Gemini 3.5 Flash | 84,6 % | 226 |
| 343 | Habermas Machine (decision) O Predict group preference among candidate consensus statements; accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Deliberation / preference | 47,0 % | 66,1 % | 0,28 | G Clef-Flash | 71,8 % | 38 |
| 344 | VoiceBench CommonEval O VoiceBench evaluation on CommonEval. | VoiceBench | 47,2 % | 90,6 %gesättigt | 3,21 | 🇨🇳 Qwen3-Omni-Flash-Instruct | 91,0 % | 22 |
| 345 | RealKIE-FCC Verified O Human-verified version of the RealKIE FCC document key-information-extraction benchmark (ALNS). | Key information extraction | 47,4 % | 64,6 % | 0,51 | 🇺🇸 Gemini 3 Pro Preview | 71,7 % | 14 |
| 346 | LVBench O Long video understanding benchmark (LVBench). | Video understanding | 47,5 % | 87,2 % | 0,77 | 🇺🇸 Gemini 4 Argon | 91,7 % | 101 |
| 347 | BuzzBench O A humour analysis benchmark. | Humor analysis | 47,6 % | 69,7 % | 1,17 | 🇺🇸 Gemini 2.5 Pro | 71,1 % | 28 |
| 348 | Q-Spatial-Bench O Quantitative spatial reasoning (metric distances/sizes) benchmark for VLMs. | Embodied reasoning | 47,6 % | 79,0 % | 0,69 | 🇺🇸 Gemini 3.6 Flash | 87,1 % | 13 |
| 349 | QIVD O Qualcomm Interactive Video Dataset: situated audio-visual question answering on live-style video with speech questions; accuracy. | Audio-visual QA | 47,8 % | 65,1 % | 0,47 | 🇺🇸 Gemini 3.8 Flash | 69,1 % | 12 |
| 350 | IntPhys 2 O Intuitive physics reasoning benchmark. | Intuitive physics | 47,9 % | 61,9 % | 0,35 | G Muse Spark Open | 68,3 % | 18 |
| 351 | FD-Bench V3 Pass@1 O Full-Duplex-Bench v3: tool-calling task success (pass@1, %) in full-duplex spoken dialogue. | Full-duplex speech + tool use | 48,0 % | 66,6 % | 0,75 | 🇺🇸 GPT-Live-1 (medium) | 87,0 % | 13 |
| 352 | GAIA (Text Only) O GAIA benchmark subset restricted to text-only inputs. | General AI tasks | 48,1 % | 93,8 %gesättigt | 1,06 | G XYZ-Aquila-mini | 97,1 % | 66 |
| 353 | VideoEval-Pro O Long video understanding benchmark with open-ended short-answer questions. | Video understanding | 48,1 % | 71,0 % | 0,85 | 🇺🇸 Gemini 2.5 Pro | 78,4 % | 26 |
| 354 | WeMath O Math reasoning benchmark spanning diverse curricula and difficulty levels. | Math reasoning | 48,6 % | 86,5 % | 1,28 | 🇨🇳 Qwen3.6-Plus | 89,0 % | 49 |
| 355 | ARKitScenes O ARKitScenes benchmark for assessing 3D scene reconstruction and understanding from mixed reality captures. | 3D scene understanding | 48,6 % | 57,4 % | 0,20 | 🇨🇳 Qwen2.5-VL 72B Instruct | 61,5 % | 11 |
| 356 | OCRBenchV2 (CN) O OCRBenchV2 Chinese subset assessing OCR performance on Chinese-language documents. | OCR (Chinese) | 48,9 % | 67,6 % | 0,30 | 🇨🇳 Qwen3.8-Max | 68,3 % | 61 |
| 357 | PhyX O Multimodal physics reasoning benchmark (PhyX). | Physics reasoning (multimodal) | 49,0 % | 78,7 % | 0,64 | 🇨🇳 Qwen3.8-Max | 83,5 % | 11 |
| 358 | VibeEval O VLM aesthetic evaluation with GPT scores. | Aesthetic/visual quality | 49,1 % | 67,8 % | 0,98 | 🇺🇸 Gemini 2.5 Pro | 76,4 % | 11 |
| 359 | LIFEBench O Length-based instruction-following evaluation benchmark. | Instruction following | 49,2 % | 73,4 % | 0,61 | G Ling 3.0 Flash | 77,3 % | 21 |
| 360 | AlpacaEval 2.0 O Updated AlpacaEval with improved prompts and judging. | Instruction following | 49,3 % | 82,7 % | 1,42 | 🇨🇳 DeepSeek R1 | 87,6 % | 121 |
| 361 | Design2Code O Translating UI designs into code. | Coding (UI) | 49,4 % | 92,2 %gesättigt | 1,98 | 🇨🇳 GLM-5V-Turbo | 94,8 % | 14 |
| 362 | VideoSimpleQA O Simple question answering over short videos. | Video QA | 49,5 % | 73,7 % | 0,66 | G Seed2.1 Pro | 76,4 % | 13 |
| 363 | OVBench O Open-vocabulary benchmark for streaming video understanding. | Open-vocabulary streaming | 49,6 % | 67,9 % | 0,43 | G Seed2.1 Pro | 70,0 % | 17 |
| 364 | FORTRESS (Adversarial) O Scale AI FORTRESS adversarial prompts (CBRNE, political violence, crime); safety score. | Safety | 49,7 % | 88,0 % | 0,70 | 🇺🇸 Claude Fable 5 | 96,0 % | 17 |
| 365 | TAU1-Retail O Tool-augmented agent evaluation in retail scenarios (TAU1). | Agent tasks (retail) | 49,9 % | 70,6 % | 1,08 | G openPangu-R-72B-2512 Slow Thinking | 73,0 % | 13 |
| 366 | KMMLU-Pro O Korean Multilingual Massive Multitask Language Understanding Pro | Multilingual knowledge | 50,1 % | 79,5 % | 0,96 | 🇨🇳 DeepSeek V4-Pro-Thinking (max) | 80,5 % | 84 |
| 367 | LiveBench O Continually updated capability benchmark across diverse tasks. | General capability | 50,2 % | 79,2 % | 1,23 | 🇺🇸 Gemini 2.5 Pro | 82,4 % | 164 |
| 368 | CollegeMath O College-level mathematics problems (MathScale) covering algebra, calculus, linear algebra, probability, etc. | College-level math | 50,4 % | 48,3 % | 1,14 | 🇨🇳 Qwen2.5-Math-72B-Instruct | 49,5 % | 24 |
| 369 | LogicVista O Visual logic and pattern reasoning tasks requiring compositional and spatial understanding. | Visual logical reasoning | 50,5 % | 86,8 % | 0,88 | 🇨🇳 Qwen3.8-Max | 91,9 % | 65 |
| 370 | DocMath O Math reasoning on document-based problems. | Document math | 50,5 % | 66,9 % | 1,01 | 🇨🇳 Qwen3.5-35B-A3B | 68,9 % | 48 |
| 371 | 3DSRBench O 3D spatial reasoning benchmark for large multimodal models (height, location, orientation, multi-object relations). | Spatial reasoning | 50,6 % | 65,0 % | 0,32 | 🇺🇸 Gemini 3 Pro | 68,9 % | 20 |
| 372 | XNLI O Cross-lingual natural language inference benchmark (MultiNLI translated into 15 languages). | Cross-lingual NLI | 50,7 % | 75,9 % | 0,31 | G Decider 4B (Mapika) | 88,6 % | 24 |
| 373 | MATH Level 5 O Level 5 subset of the MATH benchmark emphasizing the hardest competition-style problems. | Math (competition) | 50,8 % | 77,1 % | 2,10 | 🇺🇸 o1 Mini (2024-09-12) | 89,3 % | 51 |
| 374 | Multi-Image QA Average O Aggregate score over multi-image visual question answering tasks. | Multi-image QA (aggregate) | 50,9 % | 72,6 % | 0,88 | 🇺🇸 Gemini 3 Pro | 81,9 % | 19 |
| 375 | NaturalCodeBench O NaturalCodeBench (NCB): real-world coding prompts in Python and Java, English and Chinese. | Code generation | 51,1 % | 50,7 % | 1,36 | 🇺🇸 GPT-4 Turbo (2024-04-09) | 53,8 % | 15 |
| 376 | FullStackBench O End-to-end web/app development tasks and evaluation. | Full-stack development | 51,1 % | 70,9 % | 0,62 | 🇺🇸 Claude Opus 4.5 | 72,3 % | 56 |
| 377 | MMIU O Multi-image understanding benchmark evaluating cross-image reasoning. | Multi-image understanding | 51,1 % | 67,8 % | 0,51 | 🇺🇸 Gemini 3 Pro | 72,1 % | 40 |
| 378 | KMMLU-Redux O Redux variant of KMMLU benchmark | Multilingual knowledge | 51,2 % | 79,3 % | 1,51 | 🇺🇸 o1 | 81,1 % | 53 |
| 379 | OCRBenchV2 (EN) O OCRBenchV2 English subset evaluating OCR accuracy on English documents and layouts. | OCR (English) | 51,2 % | 72,5 % | 0,36 | 🇨🇳 Qwen3.8-Max | 74,2 % | 78 |
| 380 | PMC-VQA O PubMed Central visual question answering benchmark for biomedical images. | Medical VQA | 51,5 % | 68,9 % | 0,28 | G MedMO-4B-Next | 75,7 % | 35 |
| 381 | TAU2-Bench Retail (Verified) O Human-validated revision of the tau2-bench retail domain (amazon-agi/tau2-bench-verified). | Agent tasks | 51,6 % | 77,9 % | 1,04 | 🇺🇸 GPT-5 | 78,3 % | 11 |
| 382 | KOR-Bench O Comprehensive reasoning benchmark spanning diverse domains and cognitive skills. | Reasoning | 51,7 % | 75,8 % | 1,00 | 🇺🇸 GPT-5 (high) | 77,4 % | 26 |
| 383 | JudgeMark v2.1 O A benchmark measuring LLM judging ability. | LLM judging ability | 51,8 % | 88,9 % | 1,16 | 🇺🇸 Claude Sonnet 4.6 | 94,1 % | 49 |
| 384 | GPQA O Graduate-level question answering evaluating advanced reasoning. | Graduate-level QA | 51,9 % | 91,2 %gesättigt | 1,12 | 🇺🇸 GPT-5.2 Thinking | 92,4 % | 323 |
| 385 | MASK O Model behavior safety assessment via red-teaming scenarios. | Safety / red teaming | 52,0 % | 95,7 %gesättigt | 1,05 | 🇺🇸 Claude Opus 4.6 | 96,3 % | 79 |
| 386 | Claw-Eval O Benchmark of end-to-end OpenClaw-style personal-agent tasks; overall (EN) score. | Agentic assistant tasks | 52,2 % | 83,0 % | 0,65 | 🇨🇳 GLM-5.1 | 84,4 % | 59 |
| 387 | MathVerse-Vision O Multi-image visual mathematical reasoning tasks from the MathVerse ecosystem. | Math reasoning (multimodal) | 52,2 % | 87,1 % | 1,13 | G Seed2.1 Pro | 89,7 % | 21 |
| 388 | TheoremQA O Theorem-driven question answering in math, physics, EE/CS and finance. | Theorem-driven STEM QA | 52,2 % | 43,5 % | 1,77 | 🇨🇳 Qwen2 72B Instruct | 44,4 % | 87 |
| 389 | VLMs are Blind O Evaluates failure modes of VLMs in grounding and perception tasks. | Visual grounding robustness | 52,2 % | 81,4 % | 1,19 | 🇨🇳 ERNIE 5.0 | 91,4 % | 16 |
| 390 | Minerva Math O Advanced quantitative reasoning set inspired by the Minerva benchmark for STEM problem solving. | University-level math | 52,4 % | 96,3 %gesättigt | 1,38 | 🇨🇳 Qwen3 235B A22B Thinking | 98,0 % | 67 |
| 391 | NaturalQuestions O Google NQ; real user questions with long/short answers. | Open-domain QA | 52,5 % | 52,1 % | 1,20 |  🇫🇷 Mixtral-8x22B-Instruct-v0.1 | 54,9 % | 124 |
| 392 | MultiPL-E MBPP O MultiPL-E variant of MBPP tasks. | Code generation (multilingual) | 52,6 % | 60,2 % | 0,79 | 🇺🇸 Llama 3.1 405B | 65,7 % | 22 |
| 393 | xBench-DeepSearch (2025.05) O xBench DeepSearch benchmark May 2025 snapshot. | Agentic research | 52,8 % | 80,7 % | 0,89 | G Step-3.5 Flash 20260204 | 83,7 % | 31 |
| 394 | MathVerse-mini O Compact MathVerse split focusing on single-image math puzzles and visual reasoning. | Math reasoning (multimodal) | 53,3 % | 84,8 % | 1,15 |  🇨🇳 MiniCPM-V 4.7 35B-A3B | 86,5 % | 69 |
| 395 | PIOBench O Pointing / interaction-object grounding benchmark for embodied VLMs. | Embodied reasoning | 53,3 % | 73,1 % | 0,48 | 🇺🇸 Gemini 3.6 Flash | 80,9 % | 13 |
| 396 | JFBench O Japanese instruction-following benchmark. | Instruction following (Japanese) | 53,4 % | 38,7 % | 0,84 | 🇺🇸 LFM2.5 1.2B JP 202606 | 54,8 % | 10 |
| 397 | MMMU Pro (with tools) O MMMU-Pro benchmark evaluated with tool access. | Multimodal understanding (with tools) | 53,4 % | 84,6 % | 0,77 | 🇺🇸 Claude Fable 5 | 86,5 % | 19 |
| 398 | GraphWalks BFS O Graph traversal/GraphWalks benchmark (BFS variant) for long-context reasoning. | Long-context reasoning | 53,5 % | 96,7 %gesättigt | 1,23 | 🇺🇸 Gemini 4 Argon | 99,7 % | 18 |
| 399 | LiveCodeBench v3 O LiveCodeBench v3 snapshot measuring pass rates on streaming coding tasks. | Code generation | 53,5 % | 87,4 % | 2,43 | 🇨🇳 Qwen3-32B | 90,2 % | 45 |
| 400 | CharXiv (RQ, With Tools) O CharXiv reasoning questions evaluated with tool access. | Chart understanding | 53,6 % | 93,3 %gesättigt | 1,03 | 🇺🇸 Claude Mythos 5 | 93,5 % | 33 |
| 401 | WebWalkerQA O WebWalker tasks evaluating autonomous browsing question answering performance. | Web agents | 53,9 % | 71,6 % | 0,68 | G SMTL-30B | 76,5 % | 15 |
| 402 | MultiPL-E HumanEval O MultiPL-E variant of HumanEval tasks. | Code generation (multilingual) | 54,2 % | 66,4 % | 1,36 | 🇺🇸 Llama 3.1 405B | 75,2 % | 22 |
| 403 | AIME 2026 I O American Invitational Mathematics Examination 2026 I problems. | Math (competition) | 54,3 % | 99,8 %gesättigt | 2,13 | 🇺🇸 GPT-5.5 | 100,0 % | 170 |
| 404 | MathVision (With Tools) O MathVision visual math benchmark evaluated with a Python code-interpreter tool. | Math reasoning (multimodal) | 54,3 % | 97,6 %gesättigt | 1,65 | 🇺🇸 Claude Fable 5 | 98,6 % | 12 |
| 405 | Flame-React-Eval O Front-end React coding tasks and evaluation. | Frontend coding | 54,3 % | 90,0 %gesättigt | 1,84 | 🇺🇸 Claude Opus 4.6 | 98,8 % | 10 |
| 406 | JHumanEval+ O Japanese-prompt HumanEval+ code generation benchmark. | Code (Japanese) | 54,4 % | 42,4 % | 0,91 | G Granite-4.0-1B | 51,2 % | 10 |
| 407 | DiscoX O DiscoX benchmark for agentic writing and reasoning. | Agentic writing | 54,5 % | 78,8 % | 0,76 | G Seed2.0 Pro | 82,0 % | 12 |
| 408 | DynaMath O Dynamic/video-based mathematical reasoning evaluating temporal and visual understanding. | Math reasoning (video) | 54,5 % | 86,9 % | 0,90 | 🇨🇳 Qwen3.6-Plus | 88,0 % | 68 |
| 409 | Where2Place O Spatial free-space pointing benchmark: predict points for placement locations described in language. | Embodied spatial understanding | 54,6 % | 76,2 % | 0,29 | G ACE-Brain-0.6-9B | 81,4 % | 29 |
| 410 | PixMo-Points O Pointing accuracy on the PixMo-Points evaluation set. | Embodied reasoning | 54,6 % | 74,2 % | 0,27 | 🇺🇸 GPT-6 Astra | 75,2 % | 17 |
| 411 | HiddenMath O Mathematical reasoning benchmark referenced in recent model cards. | Math reasoning | 54,8 % | 59,1 % | 1,42 | 🇺🇸 Gemini 2.0 Pro | 65,2 % | 11 |
| 412 | GPQA-diamond O Hard subset of GPQA (diamond level). | Graduate-level QA | 54,8 % | 95,4 %gesättigt | 1,25 | 🇺🇸 GPT-6 Astra | 96,0 % | 631 |
| 413 | COSMOS (Cosmos-Reason benchmark) O NVIDIA Cosmos-Reason embodied physical-reasoning benchmark. | Embodied reasoning | 54,9 % | 71,6 % | 0,40 | 🇺🇸 GPT-6 Astra | 75,7 % | 13 |
| 414 | OVOBench O Streaming video QA benchmark with open-vocabulary queries. | Streaming video QA | 55,0 % | 77,8 % | 0,59 | G Seed2.1 Pro | 80,7 % | 25 |
| 415 | MMVU O Multimodal video understanding benchmark (MMVU). | Video understanding | 55,5 % | 83,5 % | 0,68 | 🇺🇸 GPT-5 (high) | 87,3 % | 69 |
| 416 | MMLU-ProX (Japanese) O Japanese subset of MMLU-ProX (JMMLU-ProX). | Multi-domain knowledge (Japanese) | 56,0 % | 79,0 % | 1,41 | 🇨🇳 Qwen3-235B-A22B-Thinking-2507 | 81,9 % | 41 |
| 417 | TVBench O Benchmark for TV show video comprehension and QA. | TV comprehension | 56,1 % | 81,7 % | 0,57 | 🇺🇸 GPT-5.6 Sol | 83,2 % | 25 |
| 418 | Showdown O Success rate on the Showdown UI interaction benchmark. | GUI agents | 56,2 % | 77,8 % | 1,00 | G Holo2-235B-A22B | 80,4 % | 22 |
| 419 | MathVerse O Visual math reasoning benchmark combining images and text across diverse mathematical tasks. | Math reasoning (multimodal) | 56,2 % | 86,2 % | 1,33 | 🇺🇸 Gemini 3 Pro | 91,6 % | 19 |
| 420 | VL-RewardBench O Reward alignment benchmark for VLMs. | Reward modeling (VL) | 56,5 % | 63,9 % | 0,46 | 🇺🇸 Claude 3.7 Sonnet | 67,4 % | 14 |
| 421 | MMMU (test) O MMMU test-set accuracy (multi-discipline college-level multimodal questions). | Multimodal understanding | 56,7 % | 57,0 % | 0,62 | 🇨🇳 InternVL2.5-78B | 61,8 % | 30 |
| 422 | HallusionBench O Benchmark for evaluating hallucination tendencies in multimodal LLMs. | Multimodal hallucination | 57,0 % | 74,4 % | 0,35 | 🇨🇳 Qwen3-VL 32B Thinking | 76,6 % | 163 |
| 423 | Inverse IFEval O Inverse instruction-following evaluation. | Instruction following (inverse) | 57,1 % | 79,9 % | 0,71 | 🇺🇸 Gemini 3 Flash | 80,9 % | 12 |
| 424 | RoboSpatialHome O RoboSpatialHome benchmark for embodied spatial reasoning in domestic environments. | Embodied spatial understanding | 57,4 % | 75,8 % | 0,28 | 🇺🇸 Gemini 4 Argon | 80,9 % | 49 |
| 425 | MME-RealWorld (cn) O MME-RealWorld Chinese split. | Real-world perception (CN) | 57,5 % | 70,4 % | 0,46 | 🇺🇸 Gemini 3 Flash | 72,6 % | 25 |
| 426 | MotionBench O Video motion and temporal reasoning benchmark. | Video motion understanding | 57,5 % | 73,6 % | 0,37 | G Seed2.0 Pro | 75,2 % | 55 |
| 427 | GraphWalks BFS 256K O GraphWalks BFS long-context graph traversal at 256K context (F1). | Long-context reasoning | 57,6 % | 89,7 % | 0,68 | G Spark-X2.6-4B | 92,2 % | 17 |
| 428 | SimpleVQA (With Tools) O SimpleVQA visual factuality benchmark evaluated with tool access. | Visual factuality | 57,6 % | 78,5 % | 0,40 | G Step 3.7 Flash | 79,2 % | 12 |
| 429 | DeepSearchQA (F1) O DeepSearchQA multi-step web search benchmark, F1-score metric (the existing deepsearchqa entry is used for the accuracy metric). | Deep web search QA | 58,0 % | 93,8 %gesättigt | 1,11 | 🇨🇳 Kimi-K3 | 95,0 % | 22 |
| 430 | MM-IFEval O Instruction-following benchmark assessing multimodal obedience to complex prompts. | Multimodal instruction following | 58,1 % | 76,0 % | 0,69 | 🇺🇸 GPT-5 | 83,1 % | 42 |
| 431 | WebVoyager O Web navigation and interaction tasks for LLM agents. | Web agents | 58,2 % | 91,2 %gesättigt | 0,98 | G UI-Venus-2 27B | 93,4 % | 36 |
| 432 | LongMemEval O Evaluates LLMs' long-term memory and retrieval over extended contexts and multi-session interactions. | Long-term memory | 58,2 % | 70,3 % | 0,71 | G GoLongRL-30B-A3B | 75,2 % | 11 |
| 433 | BFCL v3 (Japanese) O Japanese translation of BFCL v3 function-calling benchmark. | Function calling (Japanese) | 58,3 % | 55,3 % | 1,39 | 🇺🇸 NVIDIA-Nemotron-Nano-9B-v2-Japanese | 64,9 % | 12 |
| 434 | CaseLaw v2 O U.S. case law benchmark evaluating legal reasoning and judgment over court opinions. | Legal reasoning | 58,4 % | 77,3 % | 0,20 | 🇺🇸 Grok 4.3 | 79,3 % | 78 |
| 435 | CCBench O MMBench-family benchmark on Chinese culture (painting, calligraphy, cuisine, history...). | Chinese culture VQA | 58,7 % | 63,8 % | 0,35 | G Step-1V 32K | 71,2 % | 11 |
| 436 | GSM8K (decision) O GSM8K posed as choices among candidate answers; accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Math (grade-school) | 58,7 % | 81,7 % | 1,11 | G vLLM-SR d3 | 85,2 % | 38 |
| 437 | τ³-Bench Retail O τ³-Bench retail domain: tool-augmented customer-service agent with simulated user. | Agent tasks (retail) | 58,7 % | 87,6 % | 0,91 | 🇨🇳 DeepSeek V4-Flash | 89,1 % | 17 |
| 438 | FACTS Grounding O Grounded factuality benchmark evaluating model alignment with source facts. | Grounding / factuality | 58,8 % | 86,5 % | 0,29 | 🇨🇳 Kimi K2 Instruct | 88,5 % | 81 |
| 439 | Point-Bench O Benchmark for pointing and counting objects in images. | Pointing and counting | 59,0 % | 78,2 % | 0,24 | 🇺🇸 Gemini 2.5 Pro | 85,5 % | 49 |
| 440 | MME-RealWorld (en) O MME-RealWorld English split. | Real-world perception (EN) | 59,2 % | 72,4 % | 0,45 | 🇺🇸 Gemini 3 Flash | 74,9 % | 48 |
| 441 | BLINK O Evaluates visual-language grounding and reference resolution to reduce hallucinations. | Multimodal grounding | 59,2 % | 87,9 % | 0,53 | 🇺🇸 Gemini 3.6 Flash | 89,2 % | 169 |
| 442 | General Tool Use O General tool-use benchmark covering web and API tasks. | Tool use | 59,4 % | 76,1 % | 0,57 | 🇺🇸 Claude Opus 4.5 | 78,9 % | 18 |
| 443 | CSimpleQA O Chinese SimpleQA benchmark variant (short factual questions). | QA | 59,5 % | 77,3 % | 0,56 | G Ling 2.5 1T | 79,0 % | 36 |
| 444 | BFCL v3 O Benchmark for functional code correctness and logic (v3). | Code reasoning | 59,7 % | 76,3 % | 0,64 | 🇨🇳 GLM 4.5 | 77,8 % | 129 |
| 445 | WildBench V2 O WildBench V2 human preference benchmark for instruction following and helpfulness. | Instruction following | 59,9 % | 66,4 % | 0,66 | 🇫🇷 Ministral 3 14B Instruct | 68,5 % | 20 |
| 446 | M-IFEval (Japanese) O Japanese split of M-IFEval multilingual instruction-following benchmark. | Instruction following (Japanese) | 59,9 % | 60,8 % | 1,02 | 🇺🇸 LFM2.5 1.2B JP 202606 | 79,1 % | 15 |
| 447 | MUIRBENCH O Evaluates multimodal understanding robustness and reliability. | Multimodal robustness | 60,1 % | 80,7 % | 0,89 | 🇺🇸 Gemini 3 Pro | 86,1 % | 80 |
| 448 | LiveSports-3K O Live sports video understanding benchmark (3K). | Sports video | 60,3 % | 81,9 % | 0,65 | 🇺🇸 Gemini 3.1 Pro | 89,0 % | 13 |
| 449 | VideoMMMU O Video-based extension of MMMU evaluating temporal multimodal reasoning and perception across disciplines. | Multimodal video understanding | 60,3 % | 88,2 % | 0,89 | 🇨🇳 Qwen3.8-Max | 88,7 % | 93 |
| 450 | TruthfulQA O Measures whether a model imitates human falsehoods (truthfulness). | Truthfulness / hallucination | 60,7 % | 70,3 % | 0,43 | G SOLAR-10.7B-Instruct-v1.0 | 71,4 % | 169 |
| 451 | OpenBookQA O Open-book multiple choice science questions with supporting facts. | Science QA | 60,7 % | 94,9 %gesättigt | 0,20 |  🇺🇸 Hermes 4.3 36B Pyche | 96,6 % | 108 |
| 452 | RACE-High O RACE reading-comprehension questions from Chinese high-school English exams. | Reading comprehension | 60,8 % | 83,2 % | 0,20 | G Orion-14B-Base | 91,3 % | 21 |
| 453 | PinchBench O Benchmark of OpenClaw agent task success across real-world personal-assistant tasks. | Agentic assistant tasks | 60,9 % | 91,9 %gesättigt | 0,97 | G KAT-Coder-V2.5-Dev | 93,4 % | 47 |
| 454 | EvalPerf O Measures performance of LLM code evaluation, including runtime, memory, and efficiency metrics. | Code evaluation performance | 61,2 % | 93,9 %gesättigt | 2,42 | 🇺🇸 GPT-4o (2024-08-06) | 100,0 % | 34 |
| 455 | ToolSandbox O Apple's stateful, conversational, interactive tool-use benchmark with simulated user. | Tool use | 61,2 % | 77,1 % | 0,97 | 🇺🇸 LFM2.5 24B A2B | 80,7 % | 12 |
| 456 | MedMCQA O Indian medical entrance exam (AIIMS/NEET PG) multiple-choice QA. | Medical exams | 61,2 % | 66,0 % | 0,56 | 🇺🇸 GPT-4 | 72,5 % | 27 |
| 457 | Humicroedit (decision) O Pick the funnier micro-edited headline; accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Humor judgement | 61,4 % | 68,3 % | 0,28 | G Clef-Flash | 75,1 % | 38 |
| 458 | SysBench O System prompt understanding and adherence benchmark. | System prompts | 61,4 % | 93,4 %gesättigt | 1,46 | 🇺🇸 Claude Sonnet 4.6 | 94,9 % | 17 |
| 459 | LongVideoBench O Long video understanding and QA benchmark. | Long video QA | 61,5 % | 80,3 % | 0,47 | G Seed2.1 Turbo | 80,6 % | 87 |
| 460 | MIRB O Multi-Image Relational Benchmark (average over perception, visual world knowledge, reasoning, multi-hop). | Multi-image reasoning | 61,6 % | 58,6 % | 0,84 | 🇨🇳 InternVL2.5-38B | 61,2 % | 19 |
| 461 | CorpusQA O Question answering over large text corpora. | Long-context QA | 61,6 % | 80,1 % | 0,20 | 🇺🇸 GPT-5 | 81,6 % | 19 |
| 462 | cfcolor (decision) O Color-name / color judgement decisions; accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Color perception | 61,7 % | 66,5 % | 0,25 | G pplx-decider-v1.2 | 68,2 % | 38 |
| 463 | MuSR O Multistep Soft Reasoning. | Reasoning | 61,9 % | 83,7 % | 0,63 | G Clef-Flash | 86,0 % | 94 |
| 464 | MMLU-ProX O Cross-lingual and robust variant of MMLU-Pro. | Multi-domain knowledge | 62,0 % | 86,6 % | 0,82 | 🇺🇸 Gemini 3 Pro | 87,7 % | 58 |
| 465 | Multi-IFEval O Multi-task variant of instruction-following evaluation. | Instruction following (multi-task) | 62,1 % | 80,2 % | 1,03 | 🇺🇸 Llama 3.3 70B | 88,7 % | 15 |
| 466 | KMMLU O Korean Massive Multitask Language Understanding benchmark. | Korean knowledge | 62,2 % | 73,0 % | 0,78 | 🇨🇳 DeepSeek V3.1 | 78,7 % | 29 |
| 467 | LingoQA O Question answering benchmark for autonomous driving scene understanding. | Driving scene QA | 62,3 % | 82,5 % | 0,42 | 🇨🇳 Qwen3.8-Max | 84,8 % | 38 |
| 468 | SAT O Spatial Aptitude Training benchmark of static and dynamic spatial reasoning questions for VLMs. | Spatial reasoning | 62,4 % | 88,4 % | 0,85 | 🇺🇸 GPT-6 Astra | 96,7 % | 24 |
| 469 | AceBench O Industry-focused benchmark assessing domain QA and reasoning. | Industry QA | 62,8 % | 82,6 % | 0,90 | 🇨🇳 ERNIE 5.0 | 87,7 % | 17 |
| 470 | VoiceBench IFEval O VoiceBench instruction-following evaluation (IFEval). | VoiceBench | 62,9 % | 83,5 % | 1,36 | 🇺🇸 Gemini 2.5 Pro | 85,7 % | 24 |
| 471 | BRuMo25 O BruMo 2025 olympiad-style mathematics benchmark. | Math competition | 63,1 % | 96,8 %gesättigt | 2,12 | 🇨🇳 Kimi-K2.5 Thinking | 98,3 % | 28 |
| 472 | PixmoCount O Counting objects/instances in images (PixmoCount). | Visual counting | 63,4 % | 89,5 % | 0,59 | G Eagle2.5-8B | 90,2 % | 71 |
| 473 | MMMU O Multi-discipline multimodal understanding benchmark. | Multimodal understanding | 63,5 % | 86,1 % | 0,76 | 🇺🇸 Gemini 3 Pro | 87,2 % | 317 |
| 474 | EgoSchema O EgoSchema validation accuracy. | Egocentric video QA | 63,6 % | 78,4 % | 0,54 | 🇺🇸 GPT-5.2 | 81,2 % | 70 |
| 475 | INCLUDE O Evaluates inclusive language use and bias mitigation in model outputs. | Inclusiveness / bias | 63,7 % | 87,6 % | 0,88 | 🇺🇸 Gemini 3 Pro | 90,5 % | 67 |
| 476 | VoiceBench MMSU (Audio) O Audio reasoning MMSU results. | Audio reasoning | 63,8 % | 84,5 % | 0,56 | G StepAudio 3 Realtime | 90,6 % | 26 |
| 477 | AgentDojo O Interactive evaluation suite for autonomous agents across tools and tasks. | Agent evaluation | 63,8 % | 81,0 % | 1,96 | 🇺🇸 Claude 3.7 Sonnet | 88,7 % | 21 |
| 478 | MMLU-Pro O Harder successor to MMLU with more challenging questions. | Multi-domain knowledge | 63,9 % | 90,2 %gesättigt | 1,14 | 🇺🇸 Gemini 3.1 Pro | 91,2 % | 779 |
| 479 | Social Interaction QA (SIQA) O Social Interaction QA benchmark evaluating social commonsense and situational reasoning. | Social commonsense QA | 64,1 % | 74,5 % | 0,33 | 🇨🇳 Qwen 14B | 78,1 % | 58 |
| 480 | JEC-QA O Chinese National Judicial Examination multiple-choice QA. | Legal QA (Chinese) | 64,2 % | 48,0 % | 0,69 | 🇺🇸 GPT-4 | 59,3 % | 16 |
| 481 | OCNLI O Original Chinese Natural Language Inference benchmark. | Natural language inference (Chinese) | 64,5 % | 73,6 % | 0,67 | G Aquila2-34B | 79,9 % | 37 |
| 482 | Aider Code Editing O Measures interactive code editing quality within the Aider assistant workflow. | Code editing | 64,9 % | 87,6 % | 1,85 | 🇺🇸 Gemini 2.5 Pro | 89,9 % | 51 |
| 483 | CRUXEval (decision) O CRUXEval output prediction posed as choices among candidate outputs; accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Code reasoning | 65,4 % | 87,1 % | 0,90 | G Clef-Omni | 88,8 % | 38 |
| 484 | FD-Bench V3 O Full-duplex spoken dialogue benchmark v3 with tool calling; response quality (%). | Full-duplex speech + tool use | 65,4 % | 84,6 % | 1,21 | 🇺🇸 GPT-Live-1 (medium) | 90,0 % | 10 |
| 485 | MMT-Bench O Large-scale multimodal multitask benchmark covering diverse meta-tasks for LVLMs. | Multimodal multitask understanding | 65,5 % | 74,1 % | 0,41 | 🇺🇸 Gemma 4 31B (IT) | 75,7 % | 63 |
| 486 | MMStar O Broad evaluation of multimodal LLMs across diverse tasks. | Multimodal reasoning | 65,7 % | 84,2 % | 0,59 | 🇨🇳 Qwen3.8-Max | 85,9 % | 211 |
| 487 | GSM8K (Japanese) O Japanese translation of GSM8K grade-school math. | Math (grade-school, Japanese) | 65,8 % | 53,7 % | 0,93 | 🇺🇸 LFM2.5 1.2B JP 202606 | 62,2 % | 13 |
| 488 | New Yorker caption matching O Pick the caption written for a described New Yorker cartoon from five; accuracy (Jev Decision Index suite, posed as typed decisions). | Humor understanding | 65,8 % | 80,6 % | 0,75 | G AutoTrust GEV-26B-Decide | 82,0 % | 38 |
| 489 | CoQA O Conversational question answering (ChatRAG-Bench subset). | Conversational QA | 66,3 % | 81,3 % | 0,37 | G Yuan3.0 Ultra | 94,6 % | 10 |
| 490 | PhishNChips O Phishing vs legitimate email-and-link decisions; accuracy (Jev Decision Index suite, posed as typed decisions). | Phishing detection | 66,4 % | 82,1 % | 0,47 | 🇺🇸 DiffusionGemma Jev | 85,4 % | 38 |
| 491 | VoiceBench MMSU O VoiceBench MMSU benchmark (voice modality). | VoiceBench | 66,5 % | 83,6 % | 0,98 | 🇺🇸 Gemini 2.5 Pro | 88,3 % | 18 |
| 492 | Multi-IF O Composite instruction-following evaluation across multiple tasks. | Instruction following (multi-task) | 66,5 % | 88,6 % | 0,83 | 🇺🇸 Gemma 4 31B (IT) | 89,7 % | 129 |
| 493 | Chinese SimpleQA O Chinese variant of the SimpleQA benchmark. | QA (Chinese) | 66,5 % | 83,9 % | 0,40 | 🇨🇳 ERNIE 5.0 | 86,0 % | 28 |
| 494 | VideoMME (w/o sub) O Video understanding benchmark without subtitles. | Video understanding | 66,5 % | 86,2 % | 0,60 | 🇺🇸 Gemini 3 Pro | 88,6 % | 122 |
| 495 | VisualWebBench O Average accuracy on VisualWebBench. | Web UI understanding | 66,6 % | 81,4 % | 0,99 | 🇺🇸 Holo1.5-72B | 83,8 % | 28 |
| 496 | MM-Vet v2 O MM-Vet v2: integrated capabilities incl. image-text sequence understanding; GPT-4 graded. | Multimodal evaluation | 66,8 % | 69,0 % | 0,81 | 🇺🇸 Claude 3.5 Sonnet 20240620 | 71,8 % | 21 |
| 497 | VoiceBench SD-QA O VoiceBench Spoken Dialogue QA results. | VoiceBench | 67,0 % | 85,1 % | 1,28 | 🇺🇸 Gemini 2.5 Pro | 90,1 % | 18 |
| 498 | MVBench O Multi-view or multi-video QA benchmark (MVBench). | Video QA | 67,0 % | 79,7 % | 0,33 | 🇺🇸 Gemini 3.7 Flash | 82,2 % | 128 |
| 499 | MAXIFE O Multilingual instruction-following evaluation across English and multilingual original prompts. | Instruction following (multilingual) | 67,2 % | 88,7 % | 0,86 | 🇨🇳 Qwen3.7-Max | 89,2 % | 29 |
| 500 | IndoMMLU O Indonesian multi-task exam questions from primary school to university entrance level. | Indonesian knowledge | 67,2 % | 67,4 % | 0,89 | 🇨🇳 Qwen2.5-72B-Instruct | 69,3 % | 12 |
| 501 | MATH500 (Japanese) O Japanese translation of MATH500. | Math (Japanese) | 67,4 % | 43,9 % | 1,67 | 🇺🇸 LFM2.5 1.2B JP 202606 | 62,8 % | 10 |
| 502 | HRBench 8K O Hallucination robustness benchmark with 8K token contexts. | Hallucination robustness | 67,6 % | 84,6 % | 0,64 | G SenseNova-MARS-32B | 86,6 % | 32 |
| 503 | TabMWP O Tabular math word problems (MMTab subset, image tables). | Table math | 67,7 % | 84,9 % | 0,36 | 🇨🇳 Kimi-K2.5 | 95,9 % | 17 |
| 504 | Video-MME O Multimodal evaluation of video understanding and reasoning. | Video understanding (multimodal) | 67,8 % | 83,2 % | 0,65 | 🇺🇸 Gemini 3 Pro | 88,4 % | 26 |
| 505 | GaoKao 2023 En O English translation of 2023 Chinese college entrance exam (Gaokao) math questions, used in Qwen2.5-Math / DeepSeekMath style evaluations. | Math (exam) | 67,9 % | 68,6 % | 1,41 | 🇨🇳 Qwen2.5-Math-72B-Instruct | 71,9 % | 24 |
| 506 | olmOCR-Bench O olmOCR benchmark assessing OCR fidelity and structured extraction on complex document pages. | Document OCR | 68,2 % | 86,3 % | 0,25 | G Infinity-Parser2-Pro | 87,6 % | 61 |
| 507 | ACP-Bench MCQ O Safety and behavior evaluation with multiple-choice questions. | Safety evaluation (MCQ) | 68,2 % | 75,1 % | 1,28 | 🇺🇸 Llama 3.3 70B | 82,1 % | 23 |
| 508 | MathVista-Mini O Lightweight subset of MathVista for quick evaluation of visual mathematical reasoning. | Math reasoning (multimodal) | 68,3 % | 88,7 % | 0,89 | 🇨🇳 Qwen3.5-397B-A17B | 90,3 % | 212 |
| 509 | MultiPL-E O Multilingual code generation and execution benchmark across many programming languages. | Code generation (multilingual) | 68,4 % | 88,1 % | 1,28 | 🇺🇸 Claude Opus 4 | 89,6 % | 130 |
| 510 | Creative Writing v3 O A LLM-judged creative writing benchmark. | Creative writing | 68,4 % | — | 1,25 | 🇺🇸 GPT-6 Astra | 2173 | 190 |
| 511 | CyBench O Framework with 40 professional-level CTF tasks evaluating LLMs' practical cybersecurity capabilities. | Cybersecurity CTF | 68,6 % | — | 0,20 | 🇺🇸 Llama 3 70B | ↓ 5,0 % | 24 |
| 512 | TurkishMMLU O Native Turkish multiple-choice QA benchmark from Turkish curriculum exams. | Turkish knowledge | 68,6 % | 71,2 % | 1,32 | 🇨🇳 Qwen2.5-72B-Instruct | 76,1 % | 12 |
| 513 | Ruler 1M O RULER benchmark at 1M context window. | Long-context eval | 68,7 % | 94,0 %gesättigt | 1,24 | 🇨🇳 Kimi-Linear-Instruct | 94,8 % | 21 |
| 514 | Global-MMLU O Full Global-MMLU evaluation across diverse languages and regions. | Multi-domain knowledge (global) | 68,9 % | 92,8 %gesättigt | 0,80 | 🇺🇸 Claude Opus 5.5 | 94,3 % | 46 |
| 515 | MMVet O Comprehensive evaluation suite for assessing multimodal LLM capabilities. | Multimodal evaluation | 69,0 % | 84,0 % | 0,76 |  🇨🇳 R-4B-Base | 85,9 % | 107 |
| 516 | FRAMES O Frame-based interactive reasoning and dialogue benchmark. | Interactive reasoning | 69,0 % | 88,1 % | 0,67 | 🇨🇳 Tongyi DeepResearch | 90,6 % | 55 |
| 517 | MLVU O MLVU: Large-scale multi-task benchmark for video understanding. | Large video understanding | 69,1 % | 88,0 % | 0,64 | 🇨🇳 Qwen3.8-Max | 90,8 % | 105 |
| 518 | VideoMME O Video multimodal evaluation suite (VideoMME). | Multimodal video evaluation | 69,2 % | 89,0 % | 0,78 | G Seed2.0 Pro | 89,5 % | 41 |
| 519 | DeepMind Mathematics O Synthetic math problem sets from DeepMind covering arithmetic, algebra, calculus, and more. | Math reasoning | 69,2 % | 66,7 % | 1,98 | G Granite-4.1-30B | 81,9 % | 18 |
| 520 | PinchBench (Avg) O PinchBench OpenClaw-style agent task benchmark, average score across runs. | Agent tasks (OpenClaw) | 69,3 % | 90,4 %gesättigt | 0,48 | G KAT-Coder-Pro V2.5 | 94,9 % | 37 |
| 521 | CLIcK O Korean long-form instruction-following benchmark. | Korean instruction following | 69,4 % | 90,6 %gesättigt | 0,80 | 🇨🇳 DeepSeek V4-Pro-Thinking (max) | 91,6 % | 23 |
| 522 | MBPP+ O Extended MBPP with more tests and stricter evaluation. | Code generation | 69,8 % | 90,0 %gesättigt | 0,81 | 🇨🇳 GLM 4.6 | 94,2 % | 379 |
| 523 | CRUX-I O Code Reasoning and Understanding eXam – Interactive. | Code reasoning | 70,0 % | 98,0 %gesättigt | 1,14 | 🇺🇸 Gemini 3 Pro Preview | 98,8 % | 132 |
| 524 | SEED-Bench-2-Plus O SEED-Bench-2-Plus overall accuracy. | Multimodal evaluation | 70,5 % | 72,2 % | 0,43 | 🇺🇸 Claude 3.7 Sonnet | 72,9 % | 35 |
| 525 | VoiceBench AlpacaEval O VoiceBench evaluation on AlpacaEval instructions. | VoiceBench | 70,5 % | 96,7 %gesättigt | 3,44 |  🇨🇳 LongCat-Flash-Omni | 98,8 % | 18 |
| 526 | ArabicMMLU O Native Arabic multi-task knowledge benchmark (school exams across MENA countries), 40 tasks. | Arabic knowledge | 70,6 % | 72,6 % | 0,87 | G Fanar-2-27B-Instruct | 74,7 % | 17 |
| 527 | MathVista O Visual math reasoning across diverse tasks. | Multimodal math reasoning | 70,6 % | 90,3 %gesättigt | 0,73 | G Seed2.1 Pro | 90,7 % | 111 |
| 528 | MMMLU O Massively multilingual MMLU-style evaluation across many languages. | Multi-domain knowledge (multilingual) | 70,7 % | 92,1 %gesättigt | 0,95 | 🇺🇸 Claude Mythos Preview | 92,7 % | 114 |
| 529 | RoboAfford O Robotic affordance (object & spatial) grounding benchmark. | Embodied reasoning | 70,7 % | 84,0 % | 0,20 | G ACE-Brain-0.6-9B | 88,1 % | 18 |
| 530 | RealWorldQA O Visual question answering with real-world images and scenarios. | Real-world visual QA | 70,8 % | 87,6 % | 0,46 | 🇨🇳 Qwen3.8-Flash-Next | 88,5 % | 228 |
| 531 | MMHal-Bench (Hallucination Rate) O MMHal-Bench hallucination rate of vision-language model responses; lower is better. | Multimodal hallucination | 70,8 % | — | 0,30 | 🇨🇳 MiniCPM-V 4.7 35B-A3B | ↓ 21,9 % | 18 |
| 532 | CC-OCR O Cross-lingual OCR benchmark evaluating character recognition across mixed-language documents. | OCR (cross-lingual) | 70,8 % | 82,8 % | 0,40 | 🇨🇳 Qwen3.6-Plus | 83,4 % | 43 |
| 533 | When2Call O NVIDIA's When2Call multiple-choice test set: choose whether to answer directly, call a tool, ask for more information, or say it cannot answer. | Tool calling | 70,9 % | 90,5 %gesättigt | 0,60 | G d1-ultra | 92,6 % | 61 |
| 534 | MMAU v05.15.25 O Audio reasoning benchmark MMAU v05.15.25. | Audio reasoning | 71,0 % | 81,2 % | 0,26 | 🇺🇸 Gemini 3.1 Pro | 82,5 % | 30 |
| 535 | XCOPA O Multilingual extension of COPA causal commonsense reasoning in 11 languages. | Cross-lingual commonsense causal reasoning | 71,0 % | 71,4 % | 0,46 | 🇨🇳 Zhipu GLM 4-9B-Chat | 80,1 % | 22 |
| 536 | Perception Test O DeepMind Perception Test: multimodal video benchmark of perception and reasoning skills (MC video QA). | Video understanding | 71,1 % | 81,9 % | 0,47 | G PLM-8B | 82,7 % | 37 |
| 537 | BBH (decision) O BIG-Bench Hard fixed-option tasks answered directly with no working; accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Multi-task reasoning | 71,1 % | 88,3 % | 0,87 | G Jev 1.14 | 93,2 % | 39 |
| 538 | SLAKE O Semantically-Labeled Knowledge-Enhanced medical visual question answering benchmark. | Medical VQA | 71,7 % | 87,6 % | 0,37 | 🇨🇳 Qwen3.8-Max | 90,8 % | 39 |
| 539 | HRBench 4K O Hallucination robustness benchmark with 4K token contexts. | Hallucination robustness | 71,7 % | 88,7 % | 0,69 | G SenseNova-MARS-32B | 90,2 % | 35 |
| 540 | AMC-23 O American Mathematics Competition 2023 evaluation. | Math (competition) | 71,8 % | 94,9 %gesättigt | 2,02 | G QwQ-32B | 98,5 % | 64 |
| 541 | Arena-Hard O Hard prompts on Chatbot Arena. | Chat ability | 71,8 % | 96,0 %gesättigt | 2,47 | 🇫🇷 Mistral Medium 3 | 97,1 % | 89 |
| 542 | TempCompass O Temporal reasoning benchmark evaluating understanding of time-related concepts in videos and images. | Temporal reasoning | 71,9 % | 85,0 % | 0,59 | 🇺🇸 Gemini 3 Pro | 88,0 % | 31 |
| 543 | VideoMME (w/sub) O Video understanding benchmark with subtitles. | Video understanding | 71,9 % | 89,8 % | 0,62 | 🇨🇳 Qwen3.8-Max | 90,4 % | 94 |
| 544 | WildVision-Bench O In-the-wild multimodal chat benchmark judged against reference model (WildVision Arena). | Multimodal chat | 72,1 % | 79,7 % | 1,26 | 🇺🇸 Gemini 2.5 Pro | 90,1 % | 23 |
| 545 | VQA-RAD O Radiology visual question answering (closed-question accuracy). | Medical VQA | 72,2 % | 79,4 % | 1,22 | G MedMO-8B-Next | 86,4 % | 11 |
| 546 | VLMsAreBlind O Benchmark probing robustness of vision-language models to visual perturbations. | Multimodal robustness | 72,2 % | 97,0 %gesättigt | 1,45 | 🇺🇸 Gemini 3 Pro | 97,5 % | 26 |
| 547 | CRUX-O O Code Reasoning and Understanding eXam – Offline. | Code reasoning | 72,4 % | 98,6 %gesättigt | 1,20 | G IQuest-Coder-V1-40B-Loop-Thinking | 99,4 % | 166 |
| 548 | CruxEval O Mathematical coding challenge set from the CruxEval benchmark. | Code reasoning | 72,4 % | 87,9 % | 1,62 | 🇨🇳 Qwen3-32B Thinking | 95,0 % | 56 |
| 549 | WMT24++ O Extended WMT 2024 evaluation across multiple language pairs. | Machine translation | 72,5 % | 91,5 %gesättigt | 0,56 | 🇨🇳 Qwen3-235B-A22B-Thinking-2507 | 94,7 % | 70 |
| 550 | BFCL v3 (Live) O BFCL v3 Live subset for real-time tool calling evaluation. | Tool calling | 72,6 % | 81,1 % | 0,34 | 🇺🇸 Gemini 3.1 Pro | 84,9 % | 19 |
| 551 | Spider O Complex text-to-SQL benchmark over cross-domain databases. | Text-to-SQL | 72,8 % | 82,7 % | 0,69 | G Yuan3.0 Ultra | 83,9 % | 25 |
| 552 | DA-2K O 2D/3D spatial reasoning benchmark. | Spatial reasoning | 73,0 % | 84,9 % | 0,40 | G HY-Embodied-0.5 MoT-2B | 92,3 % | 13 |
| 553 | JMMLU O Japanese Massive Multitask Language Understanding benchmark (translated and culture-specific MMLU questions). | Multi-domain knowledge (Japanese) | 73,1 % | 76,8 % | 1,19 | 🇨🇳 Qwen2.5-72B-Instruct | 80,6 % | 25 |
| 554 | CLadder O Causal reasoning questions across the ladder of causation; accuracy (Jev Decision Index suite, posed as typed decisions). | Causal reasoning | 73,1 % | 93,7 %gesättigt | 0,70 | G Clef-Flash | 97,7 % | 38 |
| 555 | L-Eval O Long-context understanding and generation. | Long-context eval | 73,2 % | 73,3 % | 0,99 | 🇨🇳 Qwen1.5-72B-Chat | 78,1 % | 21 |
| 556 | AGIEval (English) O English subset of AGIEval; academic and professional exam questions. | Exams | 73,3 % | 90,5 %gesättigt | 1,01 | 🇨🇳 Qwen3-VL 32B Thinking | 92,2 % | 228 |
| 557 | Mantis O Multimodal reasoning and instruction following benchmark (Mantis). | Multimodal reasoning | 73,8 % | 82,4 % | 0,79 | G dots.vlm1 | 86,2 % | 42 |
| 558 | ScreenQA Complex O Complex ScreenQA benchmark accuracy. | GUI QA | 74,1 % | 83,8 % | 0,81 | 🇺🇸 Holo1.5-72B | 87,1 % | 12 |
| 559 | EmbSpatialBench O Embodied spatial understanding benchmark evaluating navigation and localization. | Spatial understanding | 74,4 % | 84,7 % | 0,32 | 🇨🇳 Qwen3.5-Omni-Plus | 85,4 % | 71 |
| 560 | Big Bench Audio O Artificial Analysis Big Bench Audio: spoken-question reasoning benchmark adapted from BIG-Bench Hard. | Speech reasoning | 74,5 % | 99,3 %gesättigt | 2,15 | G StepAudio 3 Realtime | 99,7 % | 70 |
| 561 | InfoQA O Information retrieval question answering benchmark evaluating factual responses. | Information-seeking QA | 74,9 % | 84,1 % | 0,97 | 🇺🇸 Gemini 3 Pro | 86,9 % | 49 |
| 562 | PubMedQA O Yes/no/maybe question answering over PubMed abstracts. | Biomedical QA | 74,9 % | 77,4 % | 0,55 | G MedMO-4B-Next | 78,2 % | 15 |
| 563 | Multilingual MMBench O Multilingual MMBench average score across languages. | Multilingual vision benchmark | 75,1 % | 76,1 % | 0,68 | 🇺🇸 LFM2.5-VL-3B | 79,5 % | 12 |
| 564 | V* O V* benchmark accuracy. | Multimodal reasoning | 75,2 % | 95,3 %gesättigt | 0,73 | 🇨🇳 Qwen3.6-Plus | 96,9 % | 72 |
| 565 | MedQA O US medical licensing exam-style multiple choice questions. | Medical exams | 75,2 % | 89,5 % | 1,14 | G OpenEvidence Darwin | 100,0 % | 40 |
| 566 | ConvFinQA O Conversational numerical reasoning over financial reports (ChatRAG-Bench subset). | Conversational QA | 75,2 % | 83,1 % | 0,20 | 🇺🇸 Claude Opus 4.6 | 86,5 % | 10 |
| 567 | R-Bench O R-Bench: robustness of LMMs to real-world image corruptions (discriminative subset). | Robustness (multimodal) | 75,2 % | 75,0 % | 0,66 | 🇺🇸 GPT-4o-2024-05-13 | 77,7 % | 19 |
| 568 | MixEval O Mixed-subject benchmark covering knowledge and reasoning tasks across domains. | Multi-task reasoning | 75,3 % | 84,7 % | 0,20 | 🇨🇳 Qwen2 72B Instruct | 86,7 % | 33 |
| 569 | VoiceBench BBH O VoiceBench evaluation on Big-Bench Hard prompts. | VoiceBench | 75,4 % | 87,6 % | 1,09 | 🇺🇸 Gemini 2.5 Pro | 92,6 % | 13 |
| 570 | HumanEval-X O Multilingual code generation benchmark extending HumanEval to multiple programming languages. | Code generation (multilingual) | 75,4 % | 90,4 %gesättigt | 1,58 | G TeleChat3-36B-Thinking | 92,7 % | 58 |
| 571 | HAE-RAE Bench O Korean language understanding benchmark evaluating knowledge and reasoning. | Korean language understanding | 75,5 % | 87,5 % | 0,96 | G Kanana-1.5-32.5B-Base | 90,7 % | 21 |
| 572 | BFCL O Benchmark for functional code correctness and logic. | Code reasoning | 75,5 % | 91,5 %gesättigt | 0,26 | 🇨🇳 Qwen3-4B | 95,0 % | 24 |
| 573 | HoVer O Multi-hop claim verification over Wikipedia introductions; accuracy (Jev Decision Index suite, posed as typed decisions). | Claim verification | 75,5 % | 86,7 % | 0,49 | G AutoTrust JEV-35B | 88,1 % | 38 |
| 574 | Global PIQA O Physical commonsense reasoning benchmark spanning 100 languages and diverse cultural contexts. | Commonsense reasoning across 100 Languages and Cultures | 75,7 % | 94,3 %gesättigt | 0,86 | 🇺🇸 Gemini 3 Flash | 95,6 % | 55 |
| 575 | Ground-UI-1K O Accuracy on the Ground-UI-1K grounding benchmark. | GUI grounding | 75,7 % | 85,2 % | 0,85 | G Holo2-235B-A22B | 85,5 % | 22 |
| 576 | MMLU Med O Medical subset of MMLU (clinical knowledge, anatomy, college medicine, medical genetics, professional medicine, college biology). | Medical knowledge | 76,1 % | 80,4 % | 1,00 | 🇺🇸 MedGemma 27B | 86,2 % | 11 |
| 577 | Global-MMLU-Lite O Lightweight global variant of MMLU covering diverse languages and regions. | Multi-domain knowledge (global) | 76,4 % | 92,4 %gesättigt | 0,72 | 🇺🇸 Claude Fable 5 | 93,3 % | 49 |
| 578 | InfoVQA O Visual question answering over infographics requiring reading, counting, and reasoning. | Infographic VQA | 76,8 % | 91,1 %gesättigt | 0,88 | 🇨🇳 Kimi-K2.5 Thinking | 92,6 % | 166 |
| 579 | CIBench O Evaluation of LLMs using a code interpreter for data-science tasks; EN/ZH average. | Code interpreter | 76,8 % | 50,5 % | 1,46 | G InternLM2-Chat-20B | 55,4 % | 13 |
| 580 | SQA O Sequential QA over tables (ChatRAG-Bench subset). | Table QA | 76,9 % | 86,5 % | 0,44 | G Yuan3.0 Ultra | 91,0 % | 10 |
| 581 | LAMBADA O Word prediction requiring broad context understanding. | Language modeling | 76,9 % | 80,5 % | 0,20 | 🇺🇸 GPT-3 | 86,4 % | 60 |
| 582 | ACP-Bench Bool O Safety and behavior evaluation with yes/no questions. | Safety evaluation (boolean) | 77,0 % | 80,8 % | 0,95 | 🇨🇳 Qwen3-32B | 85,1 % | 23 |
| 583 | CharXiv (DQ) O Scientific chart/table descriptive questions from arXiv PDFs. | Chart description (PDF) | 77,0 % | 95,3 %gesättigt | 1,20 | 🇺🇸 Gemini 3 Pro | 96,0 % | 171 |
| 584 | MATH O Competition-level mathematics across algebra, geometry, number theory, combinatorics. | Math (competition) | 77,2 % | 96,9 %gesättigt | 2,19 | 🇺🇸 o3 mini | 97,9 % | 526 |
| 585 | Roo Code Evals O Community-maintained coding evals and leaderboard by Roo Code. | Code assistant eval | 77,3 % | 100 %gesättigt | 1,96 | 🇺🇸 Claude Opus 4.6 | 100,0 % | 40 |
| 586 | Ruler 128k O RULER benchmark at 128k context window. | Long-context eval | 77,3 % | 94,7 %gesättigt | 0,81 | 🇨🇳 Qwen3-Next-80B-A3B-Instruct | 96,0 % | 44 |
| 587 | Multilingual MMLU O Multilingual variant of MMLU across many languages. | Multi-domain knowledge (multilingual) | 77,4 % | 85,5 % | 1,07 | 🇺🇸 GPT-4.1 | 87,3 % | 34 |
| 588 | WebClick O Task success on the WebClick UI agent benchmark. | GUI agents | 77,7 % | 92,6 %gesättigt | 1,31 | G Holo2-235B-A22B | 94,3 % | 22 |
| 589 | AA Speech Agent Task Success O Artificial Analysis Speech Agent Arena task completion rate (%). | Voice agents | 77,8 % | 93,0 %gesättigt | 0,88 | 🇺🇸 Grok Voice 3.0 | 94,8 % | 29 |
| 590 | SEED-Bench-Img O SEED-Bench image-only subset (SEED-Bench-Img). | Multimodal image understanding | 77,9 % | 77,7 % | 0,40 | G Bagel 14B | 78,5 % | 75 |
| 591 | API-Bank (decision) O API-Bank cases answered as option choices; accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Tool use | 77,9 % | 90,7 %gesättigt | 1,33 | G Clef-Flash | 93,1 % | 36 |
| 592 | ShareRobot-Trajectory O ShareRobot trajectory prediction benchmark. | Embodied reasoning | 77,9 % | 84,7 % | 0,24 | G RoboBrain2.5-8B | 85,2 % | 13 |
| 593 | TriviaQA O Open-domain question answering benchmark built from trivia and web evidence. | Open-domain QA | 78,0 % | 88,1 % | 0,90 | 🇺🇸 Nemotron-3-Super-120B-A12B-Base | 89,8 % | 169 |
| 594 | MedQA (MCMLE) O MedQA Mainland China Medical Licensing Exam subset (5 options). | Medical QA (Chinese) | 78,0 % | 60,6 % | 1,08 | 🇺🇸 GPT-4 | 74,6 % | 16 |
| 595 | Image QA Average O Average of single-image visual question answering benchmarks. | Image QA (aggregate) | 78,0 % | 83,0 % | 0,35 | 🇺🇸 Gemini 3 Pro | 86,2 % | 22 |
| 596 | SEED-Bench O SEED-Bench comprehensive multimodal understanding benchmark evaluating generative comprehension across multiple dimensions. | Multimodal evaluation | 78,0 % | 76,5 % | 0,37 | 🇨🇳 Qwen3-VL-4B-Instruct | 77,0 % | 24 |
| 597 | ERQA-PLUS O Extended Embodied Reasoning QA benchmark. | Embodied reasoning | 78,6 % | 87,1 % | 0,34 | 🇺🇸 Gemini 3.6 Flash | 92,2 % | 13 |
| 598 | WritingBench O General-purpose writing quality benchmark. | Writing quality | 78,6 % | 86,7 % | 0,34 | 🇨🇳 Qwen3-235B-A22B-Thinking-2507 | 88,3 % | 43 |
| 599 | IFEval (Strict Prompt) O IFEval strict prompt-level accuracy. | Instruction following | 78,6 % | 85,2 % | 1,46 | 🇨🇳 Qwen2.5 Plus (11-27) | 86,3 % | 53 |
| 600 | CMRC 2018 O Span-extraction Chinese machine reading comprehension (CMRC 2018). | Chinese reading comprehension | 78,7 % | 81,1 % | 0,20 | G TigerBot-70B-Base | 85,9 % | 16 |
| 601 | TextQuests Harm O Harmfulness evaluation on TextQuests scenarios. | Harmful propensities | 78,7 % | — | 0,23 | 🇺🇸 Grok 4.1 Fast | ↓ 9,1 % | 15 |
| 602 | Ruler 512k O RULER benchmark at 512k context window. | Long-context eval | 78,7 % | 92,8 %gesättigt | 1,34 | 🇨🇳 Qwen3.5-122B-A10B | 96,0 % | 10 |
| 603 | EvalPlus O Aggregated code evaluation suite from EvalPlus. | Code generation | 79,0 % | 88,6 % | 1,47 | 🇺🇸 o1 Mini | 89,0 % | 173 |
| 604 | BBH-ZH O Chinese translation of BIG-Bench Hard reasoning tasks. | Multi-task reasoning (Chinese) | 79,1 % | 86,6 % | 0,80 | G LLaDA2.0 Flash | 87,5 % | 14 |
| 605 | OmniDocBench O Document understanding benchmark covering multi-page layouts, tables, and charts for robust question answering. Overall score, higher is better (v1.0 edit-distance results are in omnidocbench-edit). | Document understanding | 79,2 % | 93,3 %gesättigt | 0,50 | G PaddleOCR-VL-1.5 | 94,5 % | 57 |
| 606 | Ruler 256k O RULER benchmark at 256k context window. | Long-context eval | 79,5 % | 92,2 %gesättigt | 1,45 | 🇨🇳 Qwen3.5-122B-A10B | 96,7 % | 11 |
| 607 | DROP O Discrete reasoning over paragraphs (addition, counting, comparisons). | Reading + reasoning | 79,9 % | 92,1 %gesättigt | 1,28 | 🇨🇳 Kimi K2 Instruct | 93,5 % | 151 |
| 608 | FActScore O Measures factuality on an open-source prompt suite; scored as 100 - hallucination rate, higher is better. | Factuality on open-source prompts | 79,9 % | 94,8 %gesättigt | 0,73 | 🇺🇸 GPT-5 | 99,0 % | 11 |
| 609 | CRPE O Circular-based Relation Probing Evaluation (relation split) for object-relation hallucination. | Relation hallucination | 80,1 % | 78,4 % | 0,62 | 🇨🇳 InternVL2.5-26B | 79,1 % | 18 |
| 610 | HumanEval+ O Extended HumanEval with more tests. | Code generation | 80,2 % | 95,3 %gesättigt | 1,52 | 🇨🇳 Qwen3.8-27B-UD-Q4_K_XL | 95,7 % | 439 |
| 611 | C-Eval O Chinese college-level exam benchmark. | Chinese exams | 80,6 % | 93,5 %gesättigt | 0,92 | 🇨🇳 Kimi-K2.5 | 94,0 % | 330 |
| 612 | RoboRefIt O Referring-expression grounding in cluttered robotic scenes. | Embodied reasoning | 80,6 % | 86,9 % | 0,24 | G PhysBrain1.5-8B | 89,6 % | 13 |
| 613 | MMLU-STEM O STEM subset of MMLU. | STEM knowledge | 80,7 % | 87,1 % | 1,27 | 🇨🇳 Qwen3-8B | 92,5 % | 106 |
| 614 | VQAv2 O Standard Visual Question Answering v2 benchmark on natural images. | Visual question answering | 80,8 % | 87,8 % | 0,20 | 🇺🇸 Gemini 3 Pro | 89,8 % | 83 |
| 615 | TextVQA O Visual question answering that requires reading text in images. | Text-based VQA | 80,8 % | 89,8 % | 0,49 | G Ovis2.6-80B-A3B | 91,2 % | 194 |
| 616 | Q-Bench O Q-Bench (A1 perception, test split) evaluating MLLMs on low-level visual attributes and image quality questions. | Low-level visual perception | 80,8 % | 71,6 % | 0,60 | 🇺🇸 GPT-4V(ision) | 74,1 % | 10 |
| 617 | CMMLU O Chinese counterpart to MMLU. | Chinese multi-domain | 80,9 % | 90,9 %gesättigt | 0,93 | 🇨🇳 Qwen2.5 Max | 91,9 % | 281 |
| 618 | GSM-Plus O Enhanced GSM-style grade-school math benchmark variant. | Math (grade-school, enhanced) | 80,9 % | 90,0 % | 1,68 | 🇨🇳 Qwen3.5-9B | 91,4 % | 47 |
| 619 | CV-Bench O Diverse CV tasks for VLMs. | Computer vision QA | 81,1 % | 90,3 %gesättigt | 0,44 | 🇺🇸 Gemini 3 Pro | 92,0 % | 72 |
| 620 | T-Eval O Step-by-step evaluation of LLM tool-utilization (instruct, plan, reason, retrieve, understand, review); EN/ZH average. | Tool use | 81,1 % | 65,5 % | 0,97 | 🇨🇳 Qwen 14B Chat | 66,2 % | 13 |
| 621 | MMBench v1.1 (CN) O MMBench v1.1 Chinese subset for evaluating multimodal LLMs. | Multimodal understanding (Chinese) | 81,1 % | 90,2 %gesättigt | 0,74 | 🇺🇸 Gemini 3 Pro | 91,3 % | 345 |
| 622 | OCRBench V2 O OCRBench v2 evaluating text extraction from images and documents. | OCR (vision text extraction) | 81,2 % | 92,2 %gesättigt | 0,65 | 🇨🇳 Qwen3.5-397B-A17B | 93,1 % | 239 |
| 623 | MMLU O 57 tasks spanning STEM, humanities, social sciences; broad knowledge and reasoning. | Multi-domain knowledge | 81,3 % | 94,6 %gesättigt | 0,92 | 🇺🇸 Gemini 4 Argon | 95,3 % | 760 |
| 624 | IFEval O Instruction following capability evaluation for LLMs. | Instruction following | 81,4 % | 96,2 %gesättigt | 0,80 | G Jamba2 Mini | 97,0 % | 482 |
| 625 | EQ-Bench O General reasoning benchmark assessing equation/logic capabilities. | Reasoning | 81,4 % | 85,9 % | 0,87 | 🇺🇸 Claude 3.5 Sonnet 20240620 | 86,4 % | 186 |
| 626 | Flores O FLORES multilingual translation benchmark. | Machine translation (multilingual) | 81,6 % | 88,1 % | 1,08 | G EuroLLM-22B | 88,9 % | 17 |
| 627 | RewardBench O Preference/reward model evaluation benchmark. | Reward modeling | 81,7 % | 94,6 %gesättigt | 0,60 | 🇺🇸 INF-ORM-Llama3.1-70B | 95,1 % | 175 |
| 628 | ScreenSpot O Center accuracy on ScreenSpot. | Screen UI locators | 81,7 % | 95,1 %gesättigt | 0,48 | 🇨🇳 Qwen3-VL 32B Instruct | 95,8 % | 40 |
| 629 | BBH O Hard subset of BIG-bench with diverse reasoning tasks. | Multi-task reasoning | 81,9 % | 93,0 %gesättigt | 1,21 | 🇨🇳 ERNIE 4.5 424B A47B | 94,3 % | 457 |
| 630 | NExT-QA O Video question answering benchmark on causal and temporal action reasoning (multiple-choice). | Video understanding | 82,1 % | 86,0 % | 0,25 | 🇺🇸 GPT-5 | 86,3 % | 22 |
| 631 | MMLU-Redux O Updated MMLU-style evaluation with revised questions and scoring. | Multi-domain knowledge | 82,1 % | 95,9 %gesättigt | 1,12 | G GRM-3.2-Sky | 96,9 % | 272 |
| 632 | MMMB O Multilingual Multimodal Benchmark (MMMB) average score. | Multilingual MMBench | 82,1 % | 82,1 % | 0,72 | 🇨🇳 Qwen3.5-4B | 83,4 % | 20 |
| 633 | MBPP O Short Python problems with hidden tests. | Code generation | 82,2 % | 96,3 %gesättigt | 1,34 | 🇨🇳 Kimi-K2 Thinking | 97,4 % | 522 |
| 634 | MMC-Benchmark O MMC multimodal chart understanding benchmark. | Chart understanding | 82,5 % | 73,5 % | 0,72 | 🇨🇳 Qwen-VL-Max | 76,4 % | 14 |
| 635 | HRM8K O 8k-question Korean reasoning and knowledge benchmark. | Korean reasoning | 82,7 % | 92,1 %gesättigt | 0,75 | 🇨🇳 DeepSeek V4-Flash-Thinking (max) | 93,4 % | 11 |
| 636 | MMBench v1.1 (EN) O MMBench v1.1 English subset for evaluating multimodal LLMs. | Multimodal understanding (English) | 82,9 % | 92,9 %gesättigt | 0,69 | 🇺🇸 Gemini 3 Pro | 93,3 % | 367 |
| 637 | MMBench v1.1 (EN dev) O English dev split of MMBench v1.1 measuring multimodal question answering. | General VQA | 83,1 % | 93,4 %gesättigt | 0,63 | 🇨🇳 Kimi-K2.5 | 94,2 % | 80 |
| 638 | Local Agent Bench O Tests whether small open-weight models can reliably decide when to call tools and when not to. Agent Score = (Action x 0.4) + (Restraint x 0.3) + (Wrong-Tool-Avoidance x 0.3). | Tool calling judgment | 83,2 % | 87,2 % | 0,79 | 🇺🇸 LFM2.5 1.2B Instruct | 88,0 % | 21 |
| 639 | Wild-Jailbreak O Adversarial jailbreak benchmark evaluating refusal robustness. | Safety / jailbreak | 83,5 % | 93,5 %gesättigt | 0,65 | 🇺🇸 GPT-OSS 120B (high) | 98,2 % | 12 |
| 640 | VABench-VisualTrace O VABench visual-trace (trajectory) subset. | Embodied reasoning | 83,5 % | 90,3 %gesättigt | 0,30 | G Embodied-R1.5-8B | 92,3 % | 13 |
| 641 | JHumanEval O Japanese-prompt translation of HumanEval code generation benchmark. | Code (Japanese) | 83,5 % | 93,5 %gesättigt | 1,33 | 🇺🇸 GPT-5 mini | 94,5 % | 31 |
| 642 | WinoGrande O Large-scale adversarial Winograd Schema-style pronoun resolution. | Coreference reasoning | 83,6 % | 95,4 %gesättigt | 0,61 | G Clef-Flash | 97,5 % | 325 |
| 643 | WildGuardTest O WildGuardTest safety benchmark. | Safety | 83,7 % | 94,3 %gesättigt | 0,48 | G d1-ultra | 95,4 % | 32 |
| 644 | Belebele O Parallel multiple-choice reading-comprehension benchmark over FLORES passages in 122 language variants. | Multilingual reading comprehension | 83,8 % | 95,4 %gesättigt | 0,87 | 🇺🇸 Gemma 4 31B (IT) | 95,9 % | 34 |
| 645 | MGSM O Multilingual grade school math word problems. | Math (multilingual) | 83,9 % | 93,8 %gesättigt | 1,44 | 🇺🇸 Claude Opus 4.1 (2025-08-05) Thinking | 94,4 % | 154 |
| 646 | ChartQA O Visual question answering over charts and plots. | Chart understanding (VQA) | 84,3 % | 92,9 %gesättigt | 0,62 | 🇨🇳 Keye-VL-1.5-8B | 94,1 % | 223 |
| 647 | MMBench (CN) O MMBench v1.0 Chinese test split. | Multimodal understanding (Chinese) | 84,4 % | 86,9 % | 0,75 | 🇨🇳 InternVL2.5-78B | 88,5 % | 78 |
| 648 | ScreenQA Short O Short-form ScreenQA benchmark accuracy. | GUI QA | 84,5 % | 89,6 % | 0,71 | 🇺🇸 Holo1.5-72B | 91,9 % | 12 |
| 649 | GAOKAO-Bench O GAOKAO benchmark measuring Chinese college entrance exam performance. | Chinese exams | 84,7 % | 90,9 %gesättigt | 1,27 | 🇨🇳 Qwen3-30B-A3B-Instruct-2507 | 94,5 % | 93 |
| 650 | CountBenchQA O Visual question answering benchmark focused on counting objects across varied scenes. | Visual counting QA | 84,8 % | 92,9 %gesättigt | 0,62 | G dots.mocr | 94,5 % | 50 |
| 651 | SQuAD v2.0 O Like v1.1 with unanswerable questions. | Reading comprehension | 84,9 % | 91,1 %gesättigt | 1,32 | G Ling 2.6 1T Base | 93,0 % | 35 |
| 652 | AA Conversational Dynamics: Backchannel O Artificial Analysis Conversational Dynamics (Full-Duplex-Bench subset): backchannelling while the user speaks (%). | Full-duplex speech interaction | 84,9 % | 98,4 %gesättigt | 1,09 | 🇨🇳 Qwen Audio 3.0 Realtime Plus | 100,0 % | 37 |
| 653 | AI2D O Visual question answering over science and diagram images. | Diagram understanding (VQA) | 85,5 % | 97,4 %gesättigt | 0,72 | 🇺🇸 Gemini 3 Pro | 98,7 % | 258 |
| 654 | RefCOCO+ O RefCOCO+ accuracy at IoU 0.5 on the val split. | Referring expressions | 85,5 % | 90,5 %gesättigt | 0,26 | 🇨🇳 DeepSeek-VL2 | 91,2 % | 33 |
| 655 | MMBench (EN) O MMBench v1.0 English test split. | Multimodal understanding (English) | 85,7 % | 86,9 % | 0,72 | 🇨🇳 InternVL2.5-78B | 88,3 % | 100 |
| 656 | VoiceBench OpenBookQA O VoiceBench results on OpenBookQA prompts. | VoiceBench | 85,7 % | 93,9 %gesättigt | 1,13 | 🇺🇸 Gemini 2.5 Pro | 95,2 % | 13 |
| 657 | RACE-Middle O RACE reading-comprehension questions from Chinese middle-school English exams. | Reading comprehension | 85,8 % | 92,2 %gesättigt | 0,60 | 🇺🇸 GPT-4 | 93,7 % | 36 |
| 658 | ScreenSpot-v2 O Center accuracy on ScreenSpot-v2. | Screen UI locators | 85,8 % | 96,1 %gesättigt | 1,04 | G MAI-UI-32B | 96,5 % | 88 |
| 659 | AttaQ O Adversarial jailbreak suite measuring refusal robustness against targeted attack prompts. | Safety / jailbreak | 85,9 % | 87,0 % | 0,30 | G Granite 3.3 8B Instruct | 88,5 % | 18 |
| 660 | HumanEval O Python synthesis problems evaluated by unit tests. | Code generation | 86,0 % | 99,3 %gesättigt | 1,86 | 🇺🇸 Gemini 3 Pro Preview | 100,0 % | 627 |
| 661 | MATH-100 (Japanese) O 100 MATH problems translated to Japanese (Swallow evaluation suite). | Math (Japanese) | 86,0 % | 96,8 %gesättigt | 1,96 | 🇺🇸 GPT-OSS 120B (medium) | 97,0 % | 31 |
| 662 | HellaSwag O Adversarial commonsense sentence completion. | Commonsense reasoning | 86,5 % | 98,3 %gesättigt | 0,67 | G Clef-Omni | 99,1 % | 415 |
| 663 | RefCOCOg O RefCOCOg average accuracy at IoU 0.5 (val). | Referring expressions | 86,6 % | 91,9 %gesättigt | 0,20 | 🇨🇳 DeepSeek-VL2 | 92,8 % | 35 |
| 664 | PIQA O Physical commonsense about everyday tasks and object affordances. | Physical commonsense | 86,7 % | 93,6 %gesättigt | 0,42 | G LLaDA2.0 Flash | 96,5 % | 207 |
| 665 | RefCOCO O RefCOCO average accuracy at IoU 0.5 (val). | Referring expressions | 86,9 % | 94,3 %gesättigt | 0,31 | 🇨🇳 DeepSeek-VL2 | 95,1 % | 97 |
| 666 | Ruler 64k O RULER benchmark at 64k context window. | Long-context eval | 86,9 % | 93,7 %gesättigt | 0,68 | 🇺🇸 Nemotron 3 Ultra Base | 95,3 % | 38 |
| 667 | BPoMP (decision) O BPoMP preference decisions; accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Preference prediction | 87,1 % | 96,0 %gesättigt | 1,24 | G Clef | 96,9 % | 38 |
| 668 | CLUEWSC O Chinese Winograd Schema-style coreference benchmark from CLUE. | Coreference reasoning (Chinese) | 87,2 % | 91,1 %gesättigt | 0,73 | 🇨🇳 DeepSeek R1 | 92,8 % | 49 |
| 669 | CommonsenseQA O Multiple-choice QA requiring commonsense knowledge. | Commonsense QA | 87,3 % | 90,9 %gesättigt | 0,79 | 🇨🇳 Hunyuan-Large | 92,9 % | 114 |
| 670 | MATH500 O 500-problem slice of the MATH benchmark for challenging math reasoning. | Math reasoning | 87,3 % | 99,3 %gesättigt | 2,21 | 🇨🇳 Qwen3.8-27B | 99,8 % | 266 |
| 671 | GraphWalks Parents O Graph traversal/GraphWalks benchmark (Parents variant) for long-context reasoning. | Long-context reasoning | 87,3 % | 99,6 %gesättigt | 1,23 | G Seed2.0 Lite | 100,0 % | 13 |
| 672 | SocialIQA O Social interaction commonsense QA. | Social commonsense | 87,4 % | 82,2 % | 0,97 | 🇨🇳 Qwen3 8B Base | 83,3 % | 21 |
| 673 | FORTRESS (Benign) O Scale AI FORTRESS benign counterpart prompts; share answered without over-refusal. | Safety (over-refusal) | 87,7 % | 98,4 %gesättigt | 0,91 | 🇨🇳 DeepSeek V4-Flash-Thinking (max) | 99,2 % | 17 |
| 674 | CountBench O Object counting and numeracy benchmark for visual-language models across varied scenes. | Visual counting | 87,7 % | 97,7 %gesättigt | 0,89 | 🇨🇳 Qwen3.5-27B | 97,8 % | 96 |
| 675 | BoolQ O Yes/no QA from naturally occurring questions. | Reading comprehension | 88,8 % | 91,9 %gesättigt | 0,59 | G ST-MoE-32B | 92,4 % | 154 |
| 676 | C3 O Multiple-choice Chinese machine reading comprehension (CLUE C3). | Chinese reading comprehension | 88,8 % | 94,4 %gesättigt | 0,92 | 🇨🇳 InternVL2.5-26B | 95,8 % | 60 |
| 677 | POPE O Vision-language hallucination benchmark focusing on object existence verification. | Hallucination detection | 89,1 % | 90,7 %gesättigt | 0,20 | 🇨🇳 InternVL2.5-4B | 90,9 % | 58 |
| 678 | CCPM O Chinese Classical Poetry Matching: match modern Chinese to classical poetry lines. | Chinese classical poetry | 89,7 % | 90,0 %gesättigt | 0,85 | 🇨🇳 DeepSeek-V2 | 93,1 % | 13 |
| 679 | VCR (EN, easy) O Visual Caption Restoration, English easy split; exact-match accuracy of restoring partially occluded text. | Visual caption restoration | 89,8 % | 94,3 %gesättigt | 1,82 | 🇨🇳 InternVL2.5-78B | 95,7 % | 27 |
| 680 | LLaVA-Bench (In-the-Wild) O GPT-4-judged open-ended visual conversation, detailed description and complex reasoning on 60 in-the-wild images; relative score vs. GPT-4 reference. | Visual chat | 90,0 % | 87,6 % | 1,01 | 🇺🇸 GPT-4 Vision Preview (1106) | 93,1 % | 38 |
| 681 | WSC O Classic Winograd Schema Challenge measuring commonsense coreference. | Coreference reasoning | 90,2 % | 98,0 %gesättigt | 1,00 | G Hairuo | 100,0 % | 54 |
| 682 | OmniDocBench v1.6 O OmniDocBench v1.6 end-to-end document parsing; Overall score (higher is better) combining text edit, formula CDM and table TEDS. | Document parsing | 90,2 % | 96,8 %gesättigt | 0,23 | G Youtu-Parsing-Omni | 97,0 % | 42 |
| 683 | MIABench O Multimodal instruction-following benchmark evaluating accuracy on complex image-text tasks. | Multimodal instruction following | 90,2 % | 94,7 %gesättigt | 0,53 | 🇺🇸 Gemini 2.5 Pro | 96,0 % | 23 |
| 684 | AA Conversational Dynamics: Interruption O Artificial Analysis Conversational Dynamics (Full-Duplex-Bench subset): handling user interruptions (%). | Full-duplex speech interaction | 90,2 % | 98,2 %gesättigt | 1,07 | G Eleven Realtime v1 Max | 99,0 % | 37 |
| 685 | XSTest O XSTest safety benchmark. | Safety | 90,8 % | 96,6 %gesättigt | 0,81 | 🇨🇳 Qwen3.5-9B | 97,6 % | 17 |
| 686 | CMath O Chinese mathematics benchmark. | Math (Chinese) | 91,6 % | 96,7 %gesättigt | 1,68 | G LLaDA2.0 Flash | 96,9 % | 93 |
| 687 | SQuAD v1.1 O Extractive QA from Wikipedia articles. | Reading comprehension | 92,0 % | 92,2 %gesättigt | 0,39 | 🇺🇸 OLMo 3 7B | 93,5 % | 15 |
| 688 | Ruler 32k O RULER benchmark at 32k context window. | Long-context eval | 92,1 % | 95,5 %gesättigt | 1,01 | 🇨🇳 Qwen2.5-72B-Instruct | 96,5 % | 35 |
| 689 | AA Conversational Dynamics: Pause handling O Artificial Analysis Conversational Dynamics (Full-Duplex-Bench subset): not barging in during user pauses (%). | Full-duplex speech interaction | 92,2 % | 99,6 %gesättigt | 1,27 | 🇺🇸 Grok Voice 1.0 | 100,0 % | 37 |
| 690 | ARC-Challenge O Hard subset of AI2 Reasoning Challenge; grade-school science. | Science QA | 92,5 % | 97,9 %gesättigt | 1,52 | G Clef-Flash | 98,3 % | 378 |
| 691 | CHID O Chinese IDiom cloze-test dataset (CLUE). | Chinese idiom cloze | 92,6 % | 90,7 %gesättigt | 1,05 | 🇨🇳 DeepSeek-V2 | 92,7 % | 28 |
| 692 | LongFact-Objects O Long-context factuality eval focused on object/entity references; scored as 100 - hallucination rate, higher is better. | Factuality on open-source prompts | 92,6 % | 99,1 %gesättigt | 1,06 | 🇺🇸 GPT-5 | 99,2 % | 11 |
| 693 | PrOntoQA O Probing ontological reasoning via question answering. | Logical reasoning | 92,6 % | 96,9 %gesättigt | 1,49 | G Ling Flash 2.0 | 97,9 % | 10 |
| 694 | RACE O English exams for middle and high school. | Reading comprehension | 92,8 % | 92,6 %gesättigt | 1,26 | 🇫🇷 Mistral Large 3 Base | 93,3 % | 67 |
| 695 | DocVQA O Visual question answering over scanned documents. | Document understanding (VQA) | 92,8 % | 96,7 %gesättigt | 0,91 | 🇨🇳 Seed1.5-VL-Thinking | 96,9 % | 245 |
| 696 | GSM8K O Grade-school math word problems requiring multi-step reasoning. | Math (grade-school) | 93,0 % | 98,2 %gesättigt | 1,98 | G MiMo V2.5 Pro Base | 99,6 % | 705 |
| 697 | LongFact-Concepts O Long-context factuality eval focused on conceptual statements; scored as 100 - hallucination rate, higher is better. | Factuality on open-source prompts | 93,1 % | 99,2 %gesättigt | 1,08 | 🇺🇸 GPT-5.2 (high) | 99,7 % | 11 |
| 698 | WebSrc O Webpage question answering (SQuAD F1). | Web QA | 93,6 % | 96,8 %gesättigt | 0,81 | 🇺🇸 Holo1.5-72B | 97,2 % | 18 |
| 699 | ScienceQA O Multiple-choice science questions with images, diagrams, and text context. | Science QA (multimodal) | 94,0 % | 95,8 %gesättigt | 1,07 | G Youtu-VL-4B-Instruct | 97,0 % | 34 |
| 700 | BFCL (decision) O Yes/no per supplied function on BFCL cases; case exact accuracy. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Function calling | 94,3 % | 98,2 %gesättigt | 1,53 | G Clef-Flash | 98,8 % | 38 |
| 701 | Ruler 16k O RULER benchmark at 16k context window. | Long-context eval | 94,4 % | 96,0 %gesättigt | 1,21 | 🇨🇳 Qwen2.5-72B-Instruct | 97,7 % | 28 |
| 702 | Do-Not-Answer O Evaluates a model's ability to refuse unsafe or disallowed requests. | Safety / refusal | 94,4 % | 97,9 %gesättigt | 0,20 | 🇺🇸 Llama 2 7B Chat | 99,4 % | 12 |
| 703 | COPA O Choice of Plausible Alternatives. | Causal reasoning | 95,2 % | 99,0 %gesättigt | 1,02 | G Vega v2 | 99,4 % | 41 |
| 704 | Ruler 8k O RULER benchmark at 8k context window. | Long-context eval | 95,3 % | 96,6 %gesättigt | 1,04 | 🇨🇳 Qwen2.5-72B-Instruct | 97,2 % | 28 |
| 705 | SciQ O Multiple choice science questions. | Science QA | 95,9 % | 95,1 %gesättigt | 0,67 | 🇨🇳 Qwen3 8B Base | 96,1 % | 30 |
| 706 | Ruler 4k O RULER benchmark at 4k context window. | Long-context eval | 96,3 % | 97,3 %gesättigt | 0,77 | 🇨🇳 Qwen2.5 14B Instruct | 97,7 % | 17 |
| 707 | HHEM v2.1 O Hughes Hallucination Evaluation Model (Vectara) — lower is better. | Hallucination detection | 96,4 % | — | 0,45 | G AntGroup Finix_S1_32b | ↓ 0,6 % | 21 |
| 708 | AlpacaEval 1.0 O Original AlpacaEval: win rate against text-davinci-003 judged by GPT-4. | Instruction following | 96,5 % | 96,5 %gesättigt | 1,23 | 🇺🇸 GPT-4 Turbo (11-06) | 97,7 % | 45 |
| 709 | StrongREJECT O StrongREJECT jailbreak benchmark; score = share of forbidden prompts safely handled. | Safety (jailbreak refusal) | 97,3 % | 99,5 %gesättigt | 0,47 | 🇨🇳 Kimi-K2.6 Thinking | 99,8 % | 17 |
| 710 | HarmBench O Harmfulness and safety compliance benchmark across a variety of risky prompts. | Safety | 97,5 % | 99,6 %gesättigt | 0,72 | 🇺🇸 GPT-realtime-1.5 | 100,0 % | 12 |
| 711 | ARC-Easy O Easier subset of AI2 Reasoning Challenge. | Science QA | 97,6 % | 99,5 %gesättigt | 1,59 | G Clef-Omni | 99,8 % | 172 |
| 712 | AA Conversational Dynamics: Turn-taking O Artificial Analysis Conversational Dynamics (Full-Duplex-Bench subset): smooth turn-taking (%). | Full-duplex speech interaction | 97,8 % | 100 %gesättigt | 0,62 | 🇺🇸 GPT Realtime (Aug 2025) | 100,0 % | 37 |
| 713 | VoiceBench AdvBench O VoiceBench adversarial safety evaluation. | VoiceBench | 98,3 % | 99,7 %gesättigt | 0,75 | 🇺🇸 NemotronLabs VoiceChat 11B | 100,0 % | 18 |
| 714 | LMArena Text O Chatbot Arena text evaluation (average win rate). | Crowd eval (text) | — | — | — | 🇺🇸 Gemini 4 Argon (high) | 1524 | 429 |
| 715 | Meta Score Overall O Overall Meta Score across the benchmarks of all Meta Score categories. | Composite capability index | — | 35,3 % | — | 🇨🇳 Qwen3.8-Max | 51,7 % | 303 |
| 716 | Meta Score STEM O | Composite capability index | — | 64,0 % | — | 🇨🇳 Qwen3-VL-235B-A22B Instruct | 100,0 % | 226 |
| 717 | DesignArena O Leaderboard tracking generative design systems across layout, branding, and marketing tasks. | Generative design | — | — | — | 🇺🇸 Claude Sonnet 4.5 (Thinking) | 1410 | 199 |
| 718 | Meta Score Text O | Composite capability index | — | 60,6 % | — | 🇺🇸 Claude Opus 4.5 | 85,7 % | 197 |
| 719 | Meta Score Math O | Composite capability index | — | 65,7 % | — | 🇨🇳 Qwen3-VL-235B-A22B Instruct | 100,0 % | 175 |
| 720 | LMArena Vision O Chatbot Arena vision evaluation leaderboard (ELO ratings). | Crowd eval (vision) | — | — | — | 🇺🇸 Claude Fable 5 (high) | 1308 | 165 |
| 721 | Meta Score Code O | Composite capability index | — | 64,5 % | — | 🇺🇸 Claude Opus 4.5 | 100,0 % | 163 |
| 722 | MT-Bench O Multi-turn chat evaluation via GPT-4 grading. | Chat ability | — | 82,1 % | — | 🇺🇸 Apriel Nemotron 15B Thinker | 85,7 % | 149 |
| 723 | Meta Score Visual O | Composite capability index | — | 79,9 % | — | 🇺🇸 Gemini 3 Pro | 100,0 % | 103 |
| 724 | GDPval-AA v2 Elo O Artificial Analysis GDPval-AA v2 leaderboard Elo (re-based; not comparable to v1 GDPval-AA Elo). | Office tasks | — | — | — | 🇺🇸 Claude Opus 5.5 | 1866 | 100 |
| 725 | AlignBench O Benchmark for instruction-following quality and alignment behavior. | Alignment and instruction following | — | — | — | 🇨🇳 Hunyuan-Large-Instruct | 8,30 | 98 |
| 726 | MME Elo O Elo-style scoring for the MME multimodal evaluation benchmark. | Multimodal perception | — | — | — | 🇨🇳 InternVL2.5-78B | 2495 | 86 |
| 727 | EQ-Bench 3 O A benchmark measuring emotional intelligence in challenging roleplays, judged by Sonnet 3.7. | Emotional intelligence (roleplay) | — | — | — | 🇺🇸 Claude Fable 5 | 2050 | 83 |
| 728 | Meta Score OCR O | Composite capability index | — | 53,9 % | — | 🇺🇸 o3 (low) | 80,0 % | 82 |
| 729 | Codeforces O Competitive programming performance on Codeforces problems (ELO). | Competitive programming | — | — | — | 🇨🇳 DeepSeek V4.1-Flash | 3471 | 78 |
| 730 | GLUE O General Language Understanding Evaluation. | Multi-task NLU | — | 91,2 %gesättigt | — |  🇺🇸 Turing ULR v6 | 91,3 % | 76 |
| 731 | Meta Score Agent O | Composite capability index | — | 73,6 % | — | 🇺🇸 Claude Opus 4.5 | 100,0 % | 76 |
| 732 | AA-Omniscience O Benchmark measuring factual recall and hallucination across economically relevant domains. | Knowledge and hallucination | — | — | — | 🇺🇸 Claude Fable 5 | 40,0 | 73 |
| 733 | WebDev Arena O Arena evaluation for autonomous web development agents. | Web development agents | — | — | — | 🇺🇸 Gemini 3.7 Flash | 1588 | 57 |
| 734 | Meta Score Safety O | Composite safety index | — | 64,6 % | — | G Granite 3.3 8B Instruct | 70,0 % | 53 |
| 735 | LiveCodeBench Pro O LiveCodeBench Pro evaluates competitive programming performance across Codeforces, ICPC, and IOI contests. Elo rating, higher is better. | Competitive coding problems from Codeforces, ICPC, and IOI | — | — | — | 🇺🇸 Gemini 3 Deep Think | 3298 | 46 |
| 736 | GDPVal-AA Elo O GDPVal Artificial Analysis Elo rating for office-style tasks. | Office tasks | — | — | — | 🇺🇸 Claude Fable 5 | 1932 | 44 |
| 737 | AA Speech-to-Speech Time to First Audio O Artificial Analysis speech-to-speech time to first audio in seconds (lower is better). | Voice latency | — | — | — | G Raon-SpeechChat-9B | ↓ 0,0 s | 39 |
| 738 | ContractNLI O Document-level natural language inference over contracts; macro-F1 (Jev Decision Index suite, posed as typed decisions). | Legal NLI | — | — | — | G Clef-Flash | 84,3 | 39 |
| 739 | RAGTruth O Response-level hallucination detection for RAG answers; F1 on the hallucinated class. | Hallucination detection | — | — | — | G pplx-decider-v1-27b | 88,8 | 39 |
| 740 | ACOS (decision) O Aspect-category-opinion-sentiment decisions; per-review F1. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Aspect sentiment | — | — | — | G pplx-decider-v1.2 | 40,9 | 38 |
| 741 | Amazon ESCI O Shopping query/product relevance (Exact, Substitute, Complement, Irrelevant); macro-F1 (Jev Decision Index suite, posed as typed decisions). | Product search relevance | — | — | — | G pplx-decider-v1.2 | 62,4 | 38 |
| 742 | ANLI O Adversarial NLI (entailment / neutral / contradiction); macro-F1 (Jev Decision Index suite, posed as typed decisions). | Natural language inference | — | — | — | G AutoTrust JEV-27B | 77,2 | 38 |
| 743 | BRIGHT (decision) O Rate candidate documents for BRIGHT queries; nDCG@10. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Reasoning-intensive retrieval | — | — | — | G GLiDE (no thinking, 28B) | 50,1 | 38 |
| 744 | FinEntity (decision) O Entity-level financial sentiment; macro-F1. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Financial sentiment | — | — | — | G Clef-Flash | 97,1 | 38 |
| 745 | NLI4CT O NLI over clinical trial reports (SemEval NLI4CT); macro-F1 (Jev Decision Index suite, posed as typed decisions). | Clinical NLI | — | — | — | G vLLM-SR d3 | 86,0 | 38 |
| 746 | RouterBench (decision) O Choose which LLM to route a query to; selected quality (quality objective). Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Model routing | — | — | — | G Kev 9B | 80,0 | 38 |
| 747 | SGD/SGD-X (decision) O Schema-Guided Dialogue (SGD-X) intent/slot decisions; macro-F1. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Dialogue state tracking | — | — | — | G Bespoke-Nimble-9B-v3 | 71,8 | 38 |
| 748 | ToolRet (decision) O Rate usefulness of 32 retrieved tools per request; nDCG@10. Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Tool retrieval | — | — | — | G Clef | 69,2 | 38 |
| 749 | VAST O VAST zero-shot stance detection; macro-F1 (Jev Decision Index suite, posed as typed decisions). | Stance detection | — | — | — | G pplx-decider-v1.2 | 80,2 | 38 |
| 750 | BANKING77 O Bank customer intent among 77 classes; macro-F1 (Jev Decision Index suite, posed as typed decisions). | Intent classification | — | — | — | G Clef-Omni | 94,8 | 36 |
| 751 | CLINC150+OOS O CLINC150 intent detection with out-of-scope class; macro-F1 (Jev Decision Index suite, posed as typed decisions). | Intent classification | — | — | — | G Clef-Omni | 97,7 | 36 |
| 752 | MMHal-Bench (Score) O MMHal-Bench GPT-judged response quality score (0-6) for hallucination in vision-language models. | Multimodal hallucination | — | — | — | 🇨🇳 Qwen3.8-27B | 5,03 | 36 |
| 753 | iSarcasmEval O SemEval-2022 iSarcasmEval track A (English) sarcasm F1 (Jev Decision Index suite, posed as typed decisions). | Sarcasm detection | — | — | — | G pplx-decider-v1.2 | 74,7 | 35 |
| 754 | METR O METR evaluates AI agents on long-horizon coding and agentic tasks, measuring autonomous task completion time. | Long task benchmark | — | — | — | 🇺🇸 Claude Opus 4.5 | 4,8 h | 33 |
| 755 | WMT20 En-Ja O WMT20 English-to-Japanese news translation (Swallow eval, BLEU). | Machine translation | — | — | — | 🇺🇸 GPT-OSS-Swallow-120B-SFT-v0.1 | 27,0 | 31 |
| 756 | WMT20 Ja-En O WMT20 Japanese-to-English news translation (Swallow eval, BLEU). | Machine translation | — | — | — | 🇺🇸 GPT-5 mini | 23,9 | 31 |
| 757 | Real5-OmniDocBench O OmniDocBench v1.5 pages re-captured under five real-world conditions (scanning, warping, screen photography, illumination, skew); Overall score. | Document parsing (real-world distortions) | — | 92,8 %gesättigt | — | G OvisOCR3 | 93,6 % | 30 |
| 758 | AA Speech Agent Arena O Artificial Analysis Speech Agent Arena: blind pairwise voice-agent conversations, overall Elo (GPT-Realtime-1.5 anchored at 1000). | Voice agent preference | — | — | — | 🇺🇸 Gemini 3.1 Flash Live | 1095 | 29 |
| 759 | SuperGLUE O Suite of harder NLU tasks beyond GLUE. | Multi-task NLU | — | 91,1 %gesättigt | — | G Hairuo | 91,4 % | 29 |
| 760 | MM-MT-Bench O Multi-turn multimodal instruction following benchmark evaluating dialogue quality and helpfulness. | Multimodal instruction following | — | — | — | 🇨🇳 Qwen3-VL-235B-A22B Thinking | 8,50 | 26 |
| 761 | OmniDocBench (Edit Distance) O OmniDocBench v1.0 overall normalized edit distance (x100), lower is better. | Document understanding | — | — | — | G Gundam-M | ↓ 12,3 % | 25 |
| 762 | OmniDocBench-CN O Chinese subset of OmniDocBench focusing on OCR-grounded document comprehension and reasoning. | Document understanding (Chinese) | — | — | — | G PP-StructureV3 | ↓ 13,6 % | 25 |
| 763 | MMBench-Video O Long-form multi-shot video understanding with GPT-4-judged free-form answers; overall mean score on a 0-3 scale. | Video QA | — | — | — | 🇨🇳 InternVL2.5-78B | 1,97 | 24 |
| 764 | AA-Briefcase O Artificial Analysis benchmark of long-horizon multi-week knowledge-work projects; Elo from rubric and pairwise judging. | Knowledge work | — | — | — | 🇺🇸 Claude Sonnet 5.5 | 1824 | 23 |
| 765 | Vending-Bench 2 O Long-horizon agentic task benchmark evaluating sustained goal completion. | Long-horizon agentic tasks | — | — | — | 🇺🇸 Claude Opus 4.7 | $10.937 | 23 |
| 766 | Meta Score Writing O | Composite capability index | — | 51,1 % | — | 🇨🇳 Qwen3-235B-A22B-Instruct-2507 | 60,0 % | 22 |
| 767 | MMBench-GUI L2 O MMBench-GUI level-2 element grounding across Windows, macOS, Linux, iOS, Android and Web. | GUI grounding | — | 84,6 % | — | G MAI-UI-8B | 88,8 % | 21 |
| 768 | UI-Vision O Desktop GUI grounding benchmark (basic, functional, spatial) for computer-use agents. | GUI grounding | — | 37,0 % | — | G POINTS-GUI-G-8B | 49,9 % | 20 |
| 769 | Creative Story‑Writing Benchmark V3 O Story writing benchmark evaluating creativity, coherence, and style (v3). | Creative writing | — | — | — | 🇨🇳 Kimi-K2-Instruct-0905 | 8,75 | 18 |
| 770 | Japanese MT-Bench O Japanese adaptation of MT-Bench multi-turn conversational benchmark, LLM-as-judge score on a 1-10 scale. | Multi-turn chat (Japanese) | — | — | — | 🇺🇸 GPT-5.4 | 8,87 | 18 |
| 771 | Online-Mind2web O Online web automation and task execution benchmark. | Web automation | — | 81,5 % | — | 🇺🇸 GPT-5.4 | 92,8 % | 18 |
| 772 | ASR LibriSpeech Other O Automatic speech recognition benchmark on LibriSpeech other split. | ASR | — | — | — | G Ming-flash-omni 2.0 | ↓ 2,2 % | 17 |
| 773 | OCRBench-ELO O OCR benchmark using ELO rating system to rank model performance on text extraction tasks. | OCR (ELO ranking) | — | — | — | 🇺🇸 Gemini 3 Pro | 909 | 17 |
| 774 | OK-VQA O Visual question answering requiring outside knowledge. | Knowledge-based VQA | — | 62,8 % | — | G Emu2-Chat | 64,8 % | 17 |
| 775 | PutnamBench O 672 Putnam competition problems formalized in Lean 4; score = share of problems proved. | Formal theorem proving | — | 80,9 % | — | G Aleph Prover | 99,4 % | 17 |
| 776 | MiniF2F pass@32 O MiniF2F competition benchmark pass@32 accuracy. | Math competition | — | 86,6 % | — | 🇨🇳 LongCat-Flash-Prover | 93,9 % | 16 |
| 777 | TIIF-Bench Long O TIIF-Bench long prompt score for text-to-image generation. | Text-to-image | — | — | — | G Seedream 4.5 | 88,5 | 16 |
| 778 | TIIF-Bench Short O TIIF-Bench short prompt score for text-to-image generation. | Text-to-image | — | — | — | G Nano Banana 2.0 | 91,0 | 16 |
| 779 | ASR LibriSpeech Clean O Automatic speech recognition benchmark on LibriSpeech clean split. | ASR | — | — | — | 🇨🇳 Qwen3-Omni-30B-A3B-Instruct | ↓ 1,2 % | 15 |
| 780 | ParseBench O Document-parsing benchmark covering tables, charts, content faithfulness, semantic formatting and visual grounding; overall over 5 categories. | Document parsing | — | 73,5 % | — | G LightOnOCR-3-4B | 75,1 % | 15 |
| 781 | ASR Common Voice 8 (Japanese) O Japanese automatic speech recognition on the Common Voice 8.0 Japanese test set (character error rate). | ASR | — | — | — | 🇺🇸 LFM2.5-Audio-1.5B-JP | ↓ 4,4 % | 14 |
| 782 | ASR JSUT Basic5000 O Japanese automatic speech recognition on JSUT basic5000 (character error rate). | ASR | — | — | — | G Whisper-large-V3 | ↓ 7,1 % | 14 |
| 783 | ASR ReazonSpeech O Japanese automatic speech recognition on the ReazonSpeech held-out test set (character error rate). | ASR | — | — | — | G ReazonSpeech-NeMo-v2 | ↓ 11,2 % | 14 |
| 784 | BoB-HVR O Hard, Versatile, and Relevant composite score across eight capability buckets. | Composite capability index | — | — | — | 🇺🇸 Gemini 3 Pro | 18,3 | 14 |
| 785 | DPG-Bench O DPG-Bench score for text rendering in image generation. | Text rendering | — | — | — | G Seedream 4.5 | 88,6 | 14 |
| 786 | MathBench-A O MathBench application split covering arithmetic, primary, middle, high school and college levels (overall score). | Math (multi-level) | — | 63,0 % | — | 🇺🇸 GPT-4o-2024-05-13 | 70,9 % | 14 |
| 787 | OneIG-Bench EN O OneIG-Bench English subset score for text-to-image generation. | Text-to-image | — | — | — | G Nano Banana 2.0 | 0,578 | 14 |
| 788 | StreamingBench O Streaming video understanding benchmark. | Streaming video | — | 73,6 % | — | G Seed1.8 | 84,4 % | 14 |
| 789 | VoiceBench WildVoice O VoiceBench evaluation on WildVoice dataset. | VoiceBench | — | 92,2 %gesättigt | — | 🇺🇸 Gemini 2.5 Pro | 93,4 % | 14 |
| 790 | Full-Duplex-Bench V1 Turn-taking Latency O Full-Duplex-Bench v1 smooth turn-taking latency in seconds (lower is better). | Full-duplex speech interaction | — | — | — | G PersonaPlex 7B v1 | ↓ 0,1 s | 13 |
| 791 | ProofNet (Test) O ProofNet test split: undergraduate-level mathematics theorems formalized in Lean 4; score = share proved. | Formal theorem proving | — | 31,9 % | — | 🇨🇳 LongCat-Flash-Prover | 47,3 % | 13 |
| 792 | SimpleQA (with search) O SimpleQA answered with web search / tool use enabled (agentic setting), not closed-book. | QA | — | 94,4 %gesättigt | — | 🇨🇳 DeepSeek V3.2-Exp | 97,1 % | 13 |
| 793 | BBQ O Bias Benchmark for Question Answering evaluating social biases across contexts. | Bias evaluation | — | 21,9 % | — | 🇫🇷 Mixtral 8x 7B | 56,0 % | 12 |
| 794 | FinSearchComp-CN O Chinese financial search and comprehension benchmark measuring retrieval-grounded reasoning over regional financial content. | Financial retrieval (Chinese) | — | 51,6 % | — | G doubao-1-5-vision-pro | 54,2 % | 12 |
| 795 | Hungarian National HS Math Exam O Hungarian national high-school mathematics finals (2023) graded by humans; held-out math reasoning test. | Math (held-out exam) | — | 59,8 % | — | 🇺🇸 GPT-4 | 68,0 % | 12 |
| 796 | ProverBench O ProverBench (DeepSeek-Prover-V2): 325 formalized problems incl. AIME and textbook problems in Lean 4; score = share proved. | Formal theorem proving | — | 54,2 % | — | 🇨🇳 LongCat-Flash-Prover | 66,5 % | 12 |
| 797 | TydiQA O Typologically diverse QA across languages. | Cross-lingual QA | — | 36,6 % | — | G InternLM-20B | 43,2 % | 12 |
| 798 | ARC Average O Average accuracy across ARC-Easy and ARC-Challenge. | Science QA (average) | — | 56,0 % | — |  🇺🇸 SmolLM2 1.7B Pretrained | 60,5 % | 11 |
| 799 | JevBench public (hard) O Accuracy on the hard tier of JevBench's public set. | Typed decisions | — | 73,3 % | — | G H2O-Lightning-4B | 76,6 % | 11 |
| 800 | LBV1-QA O Vision-language QA benchmark v1. | Vision-language | — | 71,4 % | — | 🇺🇸 GPT-5 | 73,7 % | 11 |
| 801 | LBV2 O Vision-language benchmark v2. | Vision-language | — | 60,7 % | — | 🇺🇸 Gemini 2.5 Pro | 65,7 % | 11 |
| 802 | MathBench (ZH) O Chinese portion of MathBench as reported in the InternLM-Math paper. | Math (Chinese) | — | 43,6 % | — | 🇨🇳 Qwen-72B-Chat | 47,8 % | 11 |
| 803 | OneIG-Bench ZH O OneIG-Bench Chinese subset score for text-to-image generation. | Text-to-image | — | — | — | G Nano Banana 2.0 | 0,567 | 11 |
| 804 | AA-Omniscience-Public O AA-Omniscience Index on the publicly released question subset (index from -100 to 100). | Knowledge and hallucination | — | — | — | 🇨🇳 GLM-5.3 | 22,6 | 10 |
| 805 | ActivityNet-QA O Open-ended QA on ActivityNet videos; GPT-judged accuracy. | Video QA (open-ended) | — | 58,3 % | — | 🇺🇸 GPT-4o | 61,9 % | 10 |
| 806 | Business Arena O Simulated business-operation arena; final profit/value in USD achieved by the agent. | Business simulation | — | — | — | 🇺🇸 GPT-5.6 Sol | $168.867 | 10 |
| 807 | CursorBench 3.0 O Cursor's coding-agent benchmark on ambiguous multi-file tasks from real Cursor sessions (v3.0, used in the Composer 2 technical report as CursorBench-3). | Software engineering | — | 59,8 % | — | 🇺🇸 GPT-5.4 | 63,9 % | 10 |
| 808 | JevBench public (easy) O Accuracy on the 48-item easy tier of JevBench's public set. | Typed decisions | — | 100 %gesättigt | — | G Jev 1.13 | 100,0 % | 10 |
| 809 | JevBench public (original/standard) O Accuracy on the 72-item original (standard) tier of JevBench's public set. | Typed decisions | — | 98,3 %gesättigt | — | G Jev 1.13 | 98,6 % | 10 |
| 810 | LiveBrowseComp O Live (continuously refreshed) BrowseComp-style benchmark of hard-to-find web information for deep-search agents. | Agentic web browsing | — | 46,5 % | — | G XYZ-Aquila-pro | 53,7 % | 10 |
| 811 | MathOlympiadBench O Olympiad-level math problems formalized in Lean 4; score = share of theorems proved (Pass@32 in LongCat-Flash-Prover report). | Formal theorem proving | — | 18,8 % | — | 🇨🇳 LongCat-Flash-Prover | 35,8 % | 10 |
| 812 | MMVP O Benchmark for multimodal video understanding and perception. | Multimodal video perception | — | 84,7 % | — | G Seed1.8 | 91,6 % | 10 |
| 813 | PinchBench (Best) O PinchBench OpenClaw-style agent task benchmark, best-of-runs score. | Agent tasks (OpenClaw) | — | 90,8 %gesättigt | — | 🇺🇸 Claude Opus 4.6 | 93,3 % | 10 |
| 814 | PRBench Legal O PRBench legal subset: rubric-graded professional reasoning tasks in law. | Professional reasoning (legal) | — | 54,8 % | — | 🇺🇸 Claude Fable 5 | 57,6 % | 10 |
| 815 | WildSpeech-Bench O WildSpeech-Bench: end-to-end speech-in/speech-out evaluation on real-world spoken queries (%). | Spoken dialogue | — | 76,3 % | — | 🇺🇸 Gemini 3.1 Pro | 77,7 % | 10 |
| 816 | AMO-Bench CH O Chinese subset of AMO-Bench. | Math (competition) | — | 65,1 % | — | 🇺🇸 Gemini 3 Pro | 74,9 % | 9 |
| 817 | ARC-AGI-3 O ARC-AGI-3 interactive reasoning benchmark; Relative Human Action Efficiency (RHAE) score. | General reasoning | — | 29,3 % | — | 🇺🇸 GPT-6 Astra | 99,9 % | 9 |
| 818 | BFCL Live v2 O Financial compliance and literacy questions from the BFCL Live v2 benchmark. | Finance QA | — | 77,2 % | — | 🇺🇸 o1 Mini | 81,0 % | 9 |
| 819 | BFCL v2 o Second release of the BFCL benchmark focusing on functional code correctness and logic. | Code reasoning | — | 73,3 % | — | 🇨🇳 MiniCPM3-4B | 76,0 % | 9 |
| 820 | CC-OCR (KIE) o Key information extraction track of the CC-OCR benchmark. | Key information extraction | — | 91,3 %gesättigt | — | 🇨🇳 Qwen3-VL-235B-A22B | 95,1 % | 9 |
| 821 | CMMMU o Chinese Massive Multi-discipline Multimodal Understanding benchmark. | Chinese multimodal understanding | — | 37,6 % | — | 🇺🇸 GPT-4V(ision) | 42,5 % | 9 |
| 822 | ComplexFuncBench o Complex function reasoning and synthesis benchmark. | Code reasoning | — | 37,6 % | — | 🇺🇸 GPT-4o | 66,5 % | 9 |
| 823 | ComplexFuncBench (Audio) O ComplexFuncBench multi-step, constrained function calling with spoken user requests (%). | Spoken function calling | — | 72,0 % | — | 🇺🇸 Gemini 3.1 Flash Live (high) | 90,8 % | 9 |
| 824 | CursorBench 4.0 o Cursor's coding-agent benchmark on ambiguous multi-file tasks from real Cursor sessions (v4.0). | Software engineering | — | 51,6 % | — | 🇺🇸 Claude Opus 5.5 | 57,8 % | 9 |
| 825 | CVTG-2K CLIPScore o CVTG-2K CLIPScore for text rendering in image generation. | Text rendering | — | — | — | G Seedream 4.5 | 0,807 | 9 |
| 826 | CVTG-2K NED o CVTG-2K normalized edit distance (NED) for text rendering. | Text rendering | — | — | — | 🇨🇳 GLM-Image | 0,956 | 9 |
| 827 | CVTG-2K Word Accuracy o CVTG-2K word accuracy for text rendering in images. | Text rendering | — | — | — | 🇨🇳 GLM-Image | 0,912 | 9 |
| 828 | DeR2 Bench o Dense retrieval and reasoning benchmark for long-context evaluation. | Long-context reasoning | — | 64,1 % | — | 🇺🇸 GPT-5.2 (high) | 69,0 % | 9 |
| 829 | DSBench-Hard (internal) o DeepSeek internal test set of difficult coding-agent problems. | Coding | — | 66,8 % | — | 🇺🇸 Claude Opus 4.8 | 71,7 % | 9 |
| 830 | Ego3D-Bench (RMSE) o Ego-centric multi-view 3D spatial reasoning benchmark on outdoor driving scenes; metric = RMSE of numeric answers (lower is better). | Driving spatial reasoning | — | — | — | 🇨🇳 Qwen-Drive-1.0-4B | ↓ 7,78 | 9 |
| 831 | EgoTempo o Egocentric video temporal reasoning benchmark. | Egocentric temporal reasoning | — | 68,0 % | — | G Seed2.0 Pro | 71,8 % | 9 |
| 832 | Encyclo-K o Encyclopedic knowledge evaluation benchmark. | Encyclopedic knowledge | — | 63,9 % | — | G Seed2.0 Pro | 65,7 % | 9 |
| 833 | Flickr30K Captions o Flickr30K image captioning (Karpathy test split, CIDEr). | Image captioning | — | — | — | 🇨🇳 Qwen-VL | 85,8 | 9 |
| 834 | fr-bench-pdf2md o French PDF-to-Markdown conversion benchmark (baseline, forms, graphics, handwritten, long tables, multi-column, tiny text); overall score. | Document parsing | — | 69,3 % | — | G LightOnOCR-3-4B | 74,1 % | 9 |
| 835 | KGC-Safety o Korean safety benchmark evaluating harmfulness and compliance. | Safety (Korean) | — | 92,6 %gesättigt | — | G K-EXAONE 2.0 | 99,8 % | 9 |
| 836 | Ko-LongBench o Long-context understanding benchmark in Korean. | Korean long-context | — | 89,4 % | — | 🇨🇳 DeepSeek V4-Pro-Thinking (max) | 91,4 % | 9 |
| 837 | KoMT-Bench o Korean multi-turn chat evaluation benchmark. | Korean chat ability | — | — | — | 🇨🇳 Qwen3-30B-A3B-Instruct-2507 | 8,49 | 9 |
| 838 | KORBench o Korean reasoning benchmark evaluating diverse reasoning capabilities. | General reasoning | — | 77,4 % | — | 🇺🇸 GPT-5.2 (high) | 79,2 % | 9 |
| 839 | LongText-Bench EN o LongText-Bench English subset score for text rendering. | Text rendering | — | — | — | G Seedream 4.5 | 0,989 | 9 |
| 840 | LongText-Bench ZH o LongText-Bench Chinese subset score for text rendering. | Text rendering | — | — | — | G Seedream 4.5 | 0,987 | 9 |
| 841 | MBPP-Ko o Korean translation of MBPP code generation benchmark. | Code generation (Korean) | — | 64,4 % | — | 🇨🇳 Qwen3-30B-A3B | 66,8 % | 9 |
| 842 | MDPBench o Document parsing benchmark (reported in OvisOCR3 release). | Document parsing | — | 82,8 % | — | G OvisOCR3 | 88,8 % | 9 |
| 843 | MixEval Hard o Hard subset of MixEval covering diverse reasoning tasks. | Multi-task reasoning (hard) | — | 42,3 % | — | 🇨🇳 Qwen3.5-9B | 50,3 % | 9 |
| 844 | MMAR O MMAR: deep reasoning over speech, sound, music and their mix (%). | Audio reasoning | — | 81,7 % | — | G StepAudio 3 Realtime | 86,5 % | 9 |
| 845 | MMLU Arabic O Arabic-language variant of MMLU evaluating knowledge and reasoning. | Arabic knowledge and reasoning | — | 69,1 % | — | 🇨🇳 Qwen 2.5 72B | 74,1 % | 9 |
| 846 | MMLU Pro MCF o MMLU-Pro common format (MCF) few-shot evaluation. | Multi-domain knowledge (few-shot) | — | 23,5 % | — | 🇨🇳 Qwen3-4B-Base | 41,1 % | 9 |
| 847 | MRCR 1M-8N o OpenAI MRCR v2, 8 needles, 512K-1M context bucket. | Long-context reasoning | — | 82,9 % | — | G Muse Spark 1.3 | 98,1 % | 9 |
| 848 | OCRBench (KIE) o Key information extraction subset of OCRBench (v1). | Key information extraction | — | 92,9 %gesättigt | — | 🇺🇸 Gemini 3.1 Pro | 96,0 % | 9 |
| 849 | OmniSpatial o Spatial understanding and reasoning benchmark (OmniSpatial). | Spatial reasoning | — | 49,7 % | — | 🇨🇳 GLM-4.6V | 52,0 % | 9 |
| 850 | ProcBench o Procedural reasoning benchmark evaluating step-by-step logical reasoning. | Procedural reasoning | — | 93,3 %gesättigt | — | G Seed2.0 Pro | 96,6 % | 9 |
| 851 | ProofBench Basic o Entry-level mathematical proof benchmarking set. | Mathematical proofs | — | 77,2 % | — | 🇨🇳 DeepSeekMath-V2-Heavy | 99,0 % | 9 |
| 852 | PureDocBench o Document parsing benchmark (reported in OvisOCR3 release). | Document parsing | — | 76,2 % | — | G OvisOCR3 | 82,3 % | 9 |
| 853 | QVHighlights o Video moment retrieval / highlight detection benchmark; R1@0.5. | Video temporal grounding | — | 71,6 % | — | G Nova 2 Lite | 77,2 % | 9 |
| 854 | Scale AI Multi Challenge O Scale AI Multi Challenge crowd-evaluated instruction following benchmark. | Chat & instruction following | — | 58,6 % | — | 🇨🇳 Qwen3.5-122B-A10B | 61,5 % | 9 |
| 855 | SVG-Bench o MiniMax internal SVG generation benchmark (build from scratch or edit from text/image inputs), VLM-verified rendering accuracy. | SVG generation | — | 62,4 % | — | 🇺🇸 Claude Sonnet 4.6 | 64,1 % | 9 |
| 856 | SWE-Bench (OpenHands) O SWE-Bench results using the OpenHands autonomous coding agent. | Code repair | — | 60,6 % | — | 🇨🇳 Qwen3.5-35B-A3B | 69,2 % | 9 |
| 857 | TabFact o Table-based fact verification (MMTab subset). | Table fact verification | — | 89,3 % | — | 🇨🇳 Kimi-K2.5 | 90,6 % | 9 |
| 858 | TAU2-Bench o Aggregate tool-augmented agent evaluation across airline, retail, and telecom scenarios (TAU2). | Agent tasks | — | 87,1 % | — | 🇺🇸 Claude Opus 4.5 | 91,6 % | 9 |
| 859 | TOMG-Bench o Text-based open molecule generation benchmark (editing, optimization, customized generation). | Molecule generation | — | 65,3 % | — | 🇺🇸 GPT-5.5 | 69,9 % | 9 |
| 860 | VCRBench o Visual commonsense reasoning benchmark. | Visual commonsense reasoning | — | 51,9 % | — | G Seed1.8 | 59,8 % | 9 |
| 861 | VideoReasonBench o Video reasoning benchmark assessing temporal and causal understanding. | Video reasoning | — | 64,5 % | — | G Seed2.0 Pro | 77,8 % | 9 |
| 862 | ViSpeak o Video-grounded dialogue and description benchmark. | Video dialogue | — | 86,5 % | — | 🇺🇸 Gemini 3 Pro | 89,0 % | 9 |
| 863 | VLAD o Vision-language autonomous-driving QA benchmark (as reported in the Qwen-Drive-1.0 evaluation). | Driving scene QA | — | 60,9 % | — | 🇨🇳 Qwen-Drive-1.0-4B | 66,5 % | 9 |
| 864 | WaymoQA o Driving-scene multiple-choice QA benchmark built on Waymo data; overall accuracy. | Driving scene QA | — | 65,5 % | — | 🇨🇳 Qwen-Drive-1.0-4B | 74,5 % | 9 |
| 865 | WaymoQA (Safety) o Safety-critical subset of WaymoQA; accuracy. | Driving scene QA | — | 62,9 % | — | 🇨🇳 Qwen-Drive-1.0-4B | 70,7 % | 9 |
| 866 | ZeroSCROLLS MuSiQue o ZeroSCROLLS split derived from MuSiQue multi-hop QA. | Long-context reasoning | — | — | — | 🇺🇸 Llama 3.3 70B Instruct | 52,2 | 9 |
| 867 | ZeroSCROLLS SpaceDigest o ZeroSCROLLS SpaceDigest extractive summarization task. | Long-context summarization | — | — | — | 🇺🇸 Llama-3_1-70B-TFree-HAT-SFT | 77,9 | 9 |
| 868 | ZeroSCROLLS SQuALITY o ZeroSCROLLS split based on the SQuALITY long-form summarization benchmark. | Long-context summarization | — | — | — | 🇺🇸 GPT-4 | 22,6 | 9 |
| 869 | ActivityNet-TimeLens o TimeLens-Bench re-annotated ActivityNet Captions split for video temporal grounding (mIoU). | Video temporal grounding | — | 52,3 % | — | 🇨🇳 Keye-VL-2.0-30B-A3B | 58,5 % | 8 |
| 870 | Blueprint-Bench 2 o Andon Labs agentic spatial-reasoning benchmark: reconstruct 2D floor plans from ~20 interior photos per apartment (50 apartments). | Spatial reasoning | — | 31,9 % | — | 🇺🇸 Claude Fable 5 | 38,6 % | 8 |
| 871 | Borda Count (Multilingual) o Borda count aggregate ranking across multilingual benchmarks; lower is better. | Aggregate ranking | — | — | — | 🇨🇳 Qwen3-32B | ↓ 2,90 | 8 |
| 872 | CaseReportBench o Clinical information extraction from medical case reports into structured fields. | Medical information extraction | — | 18,6 % | — | 🇺🇸 LFM2.5 350M | 32,5 % | 8 |
| 873 | Charades-TimeLens o TimeLens-Bench re-annotated Charades-STA split for video temporal grounding (mIoU). | Video temporal grounding | — | 50,3 % | — | 🇨🇳 Keye-VL-2.0-30B-A3B | 58,4 % | 8 |
| 874 | Cybersecurity Capture The Flag Challenges o Capture-the-flag challenge benchmark evaluating cybersecurity problem-solving skills. | Cybersecurity CTF | — | 89,1 % | — | 🇺🇸 GPT-5.6 Sol | 96,7 % | 8 |
| 875 | Docmatix (multimodal RAG) o Docmatix document-VQA based multimodal retrieval-augmented QA evaluation. | Multimodal RAG | — | 53,5 % | — | G Yuan3.0 Ultra | 67,4 % | 8 |
| 876 | DS-1000 o DS-1000: 1000 data-science coding problems across 7 Python libraries. | Data-science code generation | — | 31,3 % | — | 🇨🇳 DeepSeek-Coder-33B-base | 40,2 % | 8 |
| 877 | DSBench-FullStack (internal) o DeepSeek internal full-stack development test set for coding agents. | Coding | — | 72,5 % | — | 🇺🇸 Claude Fable 5 | 77,2 % | 8 |
| 878 | FaithJudge (1-Hallu.) o FaithJudge hallucination rate with 1-hallucination metric (lower is better). | Hallucination detection | — | — | — | G Moonlight-Instruct | ↓ 56,0 % | 8 |
| 879 | GeneBench Pro o Long-horizon genomics and quantitative-biology analysis tasks. | Biology agent tasks | — | 24,1 % | — | 🇺🇸 GPT-6 Astra | 37,1 % | 8 |
| 880 | GraphWalks Parents 256K o GraphWalks Parents long-context graph traversal at 256K context (F1). | Long-context reasoning | — | 98,0 %gesättigt | — | 🇺🇸 Claude Mythos 5 | 100,0 % | 8 |
| 881 | IOI 2025 o International Olympiad in Informatics 2025 problem set; total score out of 600 points. | Competitive programming | — | — | — | 🇨🇳 Kimi-K2.6 Thinking | 585 | 8 |
| 882 | Kimi Code Bench v2 o Moonshot in-house coding-agent benchmark (Kimi Code Bench 2.0): realistic software engineering tasks across 10+ languages and production tech stacks. | Agentic coding | — | 71,1 % | — | 🇺🇸 Claude Fable 5 | 76,9 % | 8 |
| 883 | LogiQA o Reading comprehension with logical reasoning. | Logical reasoning | — | 22,2 % | — | G Pythia 70M | 23,5 % | 8 |
| 884 | MCPMark-Verified O Human-verified edition of MCPMark covering Notion, GitHub, Filesystem, Postgres and Playwright MCP servers. | Agent tool-use (MCP) | — | 90,2 %gesättigt | — | 🇨🇳 Kimi-K3 | 96,1 % | 8 |
| 885 | MLE-Bench o OpenAI MLE-Bench: Kaggle-style machine-learning engineering competitions solved by agents. | ML engineering | — | 54,3 % | — | 🇺🇸 Claude Sonnet 5 | 66,9 % | 8 |
| 886 | MMAU-Pro o MMAU-Pro: challenging audio understanding and reasoning benchmark (speech, sound, music). | Audio reasoning | — | 69,3 % | — | 🇨🇳 Qwen3.5-Omni-Plus | 70,0 % | 8 |
| 887 | MolBench-bind o MolBench binding-affinity subset: scientific agentic tasks on molecular binding prediction. | Molecular science agents | — | 53,0 % | — | 🇺🇸 GPT-5.5 | 62,2 % | 8 |
| 888 | MRCR 512K-8N o OpenAI MRCR v2, 8 needles, 256K-512K context bucket. | Long-context reasoning | — | 92,2 %gesättigt | — | 🇺🇸 GPT-6 Astra | 100,0 % | 8 |
| 889 | MRCR 64K-8N o Multi-Round Coreference Resolution benchmark at 64k context with 8 needles. | Long-context reasoning | — | 61,5 % | — | 🇺🇸 GPT-5.2 | 92,0 % | 8 |
| 890 | Natural2Code o Natural language to code benchmark for instruction-following synthesis. | Code generation | — | 84,7 % | — | 🇺🇸 Gemini 2.0 Flash | 92,9 % | 8 |
| 891 | OCR-VQA o Visual question answering by reading text in book cover images. | Text-based VQA | — | 72,6 % | — | 🇨🇳 Qwen-VL | 75,7 % | 8 |
| 892 | OpenAI Internal Computer-Use Safety o OpenAI internal eval reported in the GPT-6 Astra announcement: rate of unsafe actions during computer use (lower is better). | Safety | — | — | — | 🇺🇸 GPT-6 Astra | ↓ 2,4 % | 8 |
| 893 | OSWorld 2.0 (Offline) O OSWorld 2.0 offline set (v2026.08.08), partial-credit score on long-horizon computer-use workflows without internet access. | GUI agents | — | 70,1 % | — | 🇺🇸 GPT-6 Astra | 73,5 % | 8 |
| 894 | QVHighlights-TimeLens o TimeLens-Bench re-annotated QVHighlights split for video temporal grounding (mIoU). | Video temporal grounding | — | 64,9 % | — | 🇨🇳 Keye-VL-2.0-30B-A3B | 70,1 % | 8 |
| 895 | Random Complex Tasks o Randomly constructed complex task environments for agent generalization. | Agentic tasks (random) | — | 33,2 % | — | 🇨🇳 LongCat-Flash-Thinking-2601 | 35,8 % | 8 |
| 896 | SWE-Bench Pro v2 o SWE-Bench Pro v2 (Scale AI) agentic software-engineering tasks; full set. | Software engineering | — | 86,6 % | — | 🇨🇳 Qwen3.8-27B | 94,9 % | 8 |
| 897 | SWE-Lancer o Software engineering benchmark using real freelance-style issues. | Code repair (freelance tasks) | — | 47,1 % | — | 🇺🇸 GPT-5.1 Codex-Max | 79,9 % | 8 |
| 898 | V* (With Tools) o V* visual search benchmark evaluated with a Python code-interpreter tool. | Multimodal reasoning | — | 96,8 %gesättigt | — | 🇺🇸 GPT-5.4 | 98,4 % | 8 |
| 899 | WebTailBench o Microsoft long-tail web task benchmark (shopping, travel, comparison and other real-world tasks) introduced with Fara-7B. | Web agents | — | 40,8 % | — | 🇺🇸 MolmoWeb-8B | 49,5 % | 8 |
| 900 | Workspace Bench Lite o Lite subset of Workspace Bench: workspace tasks with heterogeneous file dependencies. | Agentic office work | — | 69,7 % | — | 🇨🇳 GLM-5.3 | 71,7 % | 8 |
| 901 | ZeroSCROLLS GovReport o ZeroSCROLLS split based on the GovReport summarization benchmark. | Long-context summarization | — | — | — | G CoLT5 | 41,0 | 8 |
| 902 | ZeroSCROLLS NarrativeQA o ZeroSCROLLS split based on the NarrativeQA reading comprehension benchmark. | Long-context QA | — | — | — | 🇺🇸 Claude v1.3 | 32,6 | 8 |
| 903 | ZeroSCROLLS Qasper o ZeroSCROLLS split based on the Qasper paper QA benchmark. | Long-context QA | — | — | — | G FLAN-UL2 | 56,9 | 8 |
| 904 | ZeroSCROLLS QMSum o ZeroSCROLLS split based on the QMSum meeting summarization benchmark. | Long-context summarization | — | — | — | G CoLT5 | 22,5 | 8 |
| 905 | ZeroSCROLLS QuALITY o ZeroSCROLLS split based on the QuALITY reading comprehension benchmark. | Long-context QA | — | 78,8 % | — | 🇺🇸 GPT-4 | 89,2 % | 8 |
| 906 | ZeroSCROLLS SummScreenFD o ZeroSCROLLS split based on the SummScreenFD summarization benchmark. | Long-context summarization | — | — | — | G CoLT5 | 20,0 | 8 |
| 907 | AG News o AG News four-class news topic classification. | Topic classification | — | 90,5 %gesättigt | — | G Laya | 92,8 % | 7 |
| 908 | AgentBench o Multi-environment agent benchmark (OS, DB, KG, puzzles, household, web shopping, web browsing); overall weighted score. | LLM agents | — | — | — | 🇨🇳 GLM 4 (05-20) | 3,79 | 7 |
| 909 | AgentCompany o Company-level agent reasoning and decision-making benchmark. | Agent reasoning | — | 37,7 % | — | 🇺🇸 Claude Sonnet 4.5 | 41,0 % | 7 |
| 910 | AutoLogi o AutoLogi benchmark evaluating automated logical reasoning accuracy. | Logical reasoning | — | 88,5 % | — | 🇺🇸 Claude Sonnet 4 | 89,8 % | 7 |
| 911 | BioReason Coding o Coding variant-effect prediction from DNA (balanced accuracy). | Genomics | — | 64,4 % | — | G Carbon-X | 74,9 % | 7 |
| 912 | BioReason Non-SNV o Complex (non-SNV) variant-effect prediction from DNA (balanced accuracy). | Genomics | — | 85,4 % | — | 🇺🇸 Claude Opus 5.5 (max) | 94,8 % | 7 |
| 913 | BrokenArXiv o Benchmark of subtly broken/false statements derived from recent arXiv math papers; models must detect and prove or refute them. | Research math | — | 59,2 % | — | 🇺🇸 Claude Opus 5 | 77,7 % | 7 |
| 914 | ClawEval (Pass@3) o ClawEval OpenClaw agent benchmark, pass@3 (any of 3 trials succeeds). | Agent tasks (OpenClaw) | — | 80,0 % | — | 🇺🇸 Gemini 3.1 Pro | 82,9 % | 7 |
| 915 | COCO Detection (F1) o COCO object detection with an MLLM; F1 averaged over IoU 0.5:0.95. | Object detection | — | — | — | G WeVisOne-VL-4B | 57,1 | 7 |
| 916 | CorpFin o Corporate finance reasoning and numerical literacy benchmark. | Corporate finance | — | 70,4 % | — | 🇺🇸 Grok 4 Fast | 73,3 % | 7 |
| 917 | Dense200 Detection (F1) o Dense200 object detection with an MLLM; F1 averaged over IoU 0.5:0.95. | Object detection | — | — | — | G WeVisOne-VL-4B | 71,3 | 7 |
| 918 | DocFigCapBench Localization F1 o DocFigCapBench: localization F1 of document figures. | Document figure captioning | — | 86,0 % | — | G OvisOCR3 | 91,0 % | 7 |
| 919 | ForecastBench (decision) o ForecastBench questions answered as probabilities; Brier score x100 (lower is better). Jev Decision Index adaptation posed as typed decisions; not comparable to the generative benchmark. | Forecasting | — | — | — | G Clef-Flash | ↓ 10,6 | 7 |
| 920 | FrontierVEP o Single-base variant-effect prediction benchmark. | Genomics | — | 66,2 % | — | 🇺🇸 Claude Opus 5.5 (max) | 76,2 % | 7 |
| 921 | GAIA (no file) o GAIA benchmark subset without file inputs. | General AI tasks | — | 77,9 % | — | G Step-3.5 Flash 20260204 | 84,5 % | 7 |
| 922 | GenomeQA o DNA sequence understanding QA (enhancers, splice sites, organism, histone marks, TF binding, motifs). | Genomics | — | 61,6 % | — | 🇺🇸 GPT-6.1 Sol (max) | 67,2 % | 7 |
| 923 | GraphWalks BFS 256K-1M o GraphWalks BFS long-context graph traversal at 256K-1M context (F1). | Long-context reasoning | — | 73,6 % | — | 🇺🇸 Gemini 4 Argon | 84,2 % | 7 |
| 924 | GSM-Symbolic o Symbolic reasoning variant of GSM that tests algebraic manipulation and arithmetic with structured problems. | Math reasoning | — | 83,2 % | — | G Granite-4.0-H-Small | 87,4 % | 7 |
| 925 | HotpotQA o Explainable multi-hop QA with supporting facts. | Multi-hop QA | — | 37,6 % | — | 🇨🇳 Qwen3-0.6B | 64,0 % | 7 |
| 926 | JevBench public set o Accuracy on JevBench's 231 public items (easy, original/standard and hard tiers). | Typed decisions | — | 85,6 % | — | 🇺🇸 strands-decider-26B-A4B-gemma4-v1-2610 | 89,6 % | 7 |
| 927 | Kimi Code Bench 3.0 Long-horizon o Moonshot in-house long-horizon coding-agent benchmark (Kimi Code Bench 3.0, long-horizon split). | Agentic coding | — | 60,2 % | — | 🇺🇸 GPT-6 Astra | 66,5 % | 7 |
| 928 | Kimi Hallu Bench Hard (With Tools) o Moonshot in-house hard hallucination benchmark with tools: percentage of factual claims containing errors (lower is better). | Hallucination | — | — | — | 🇺🇸 GPT-6 Astra | ↓ 5,5 % | 7 |
| 929 | Legal Research Bench o Vals AI Legal Research benchmark of agentic legal research tasks. | Legal research | — | 45,2 % | — | 🇺🇸 Claude Fable 5 | 49,5 % | 7 |
| 930 | LongCodeBench o Long-context code comprehension and repair benchmark (reported at 1M-token context). | Long-context code understanding | — | 82,0 % | — | G Nova 2 Lite | 84,0 % | 7 |
| 931 | LVIS Detection (F1) o LVIS object detection with an MLLM; F1 averaged over IoU 0.5:0.95. | Object detection | — | — | — | G WeVisOne-VL-4B | 55,3 | 7 |
| 932 | Medbullets (4 options) o USMLE Step 2/3-style clinical questions, 4-option version. | Medical QA | — | 61,7 % | — | G MedMO-8B | 66,5 % | 7 |
| 933 | Medbullets (5 options) o USMLE Step 2/3-style clinical questions, 5-option version. | Medical QA | — | 53,4 % | — | G MedMO-8B | 60,2 % | 7 |
| 934 | MindCube-Tiny o Tiny evaluation subset of MindCube spatial mental-model benchmark. | Spatial reasoning | — | 67,2 % | — | G ZDTaichu5.0-9B | 78,3 % | 7 |
| 935 | MM-BrowseComp o Multimodal browsing comprehension benchmark. | Multimodal browsing | — | 49,4 % | — | 🇺🇸 Claude Opus 4.6 | 59,3 % | 7 |
| 936 | MMMU-Med o Medical/health subset of MMMU (validation). | Medical QA | — | 63,8 % | — | G MedMO-8B-Next | 69,3 % | 7 |
| 937 | MSVD-QA o Open-ended QA on MSVD video clips; GPT-judged accuracy. | Video QA (open-ended) | — | 70,2 % | — | G Baichuan-Omni | 72,2 % | 7 |
| 938 | NextBase o Next-base DNA sequence prediction (per-base accuracy). | Genomics | — | 32,3 % | — | G Carbon-X | 39,4 % | 7 |
| 939 | OCW Courses o MIT OpenCourseWare undergraduate STEM problems (from Minerva). | STEM math reasoning | — | 12,9 % | — | G Minerva 540B | 17,6 % | 7 |
| 940 | OIBench EN o English subset of OIBench for code generation. | Code generation | — | 47,6 % | — | 🇺🇸 Gemini 3 Pro | 58,2 % | 7 |
| 941 | OmniMedVQA o Large-scale medical VQA across 12 imaging modalities. | Medical QA | — | 88,0 % | — | G MedMO-8B-Next | 93,3 % | 7 |
| 942 | OpenAI Internal Investment Banking Modeling Tasks o OpenAI internal eval (GPT-5.4/GPT-5.5 announcements): investment-banking spreadsheet modeling tasks. | Finance | — | 85,5 % | — | 🇺🇸 GPT-5.5 Pro | 88,6 % | 7 |
| 943 | OpenAI Internal Management Consulting Tasks o OpenAI internal eval (GPT-5.6 announcement): management-consulting tasks. | Professional work | — | 36,6 % | — | 🇺🇸 GPT-5.6 Sol | 43,2 % | 7 |
| 944 | OSWorld 2.1 (Strict) o OSWorld 2.1 strict pass rate (full task success). | GUI agents | — | 42,4 % | — | 🇺🇸 Claude Opus 5.5 | 48,7 % | 7 |
| 945 | PathVQA o Pathology visual question answering. | Medical QA | — | 57,0 % | — | G MedMO-4B-Next | 73,3 % | 7 |
| 946 | RiemannBench o Research-level mathematics benchmark (reported in Anthropic Fable 5/Mythos 5 system card). | Research math | — | 67,6 % | — | 🇺🇸 Gemini 4 Argon | 76,0 % | 7 |
| 947 | SALAD-Bench o Safety Alignment and Dangerous-behavior benchmark evaluating harmful assistance and refusal consistency. | Safety alignment | — | — | — | G Granite-4.1-3B | ↓ 94,0 % | 7 |
| 948 | SPAR-Bench o Spatial perception and reasoning benchmark derived from the SPAR dataset (single- and multi-view). | Spatial reasoning | — | 50,4 % | — | 🇺🇸 GPT-5.2 | 55,1 % | 7 |
| 949 | SWE-Marathon v1.1 o SWE-Marathon v1.1: very long-horizon software engineering tasks (Abundant AI). | Software engineering | — | 45,4 % | — | 🇺🇸 Claude Opus 4.8 | 48,8 % | 7 |
| 950 | SWE-Perf o Software engineering benchmark focused on performance-oriented fixes. | Code repair | — | 4,2 % | — | 🇺🇸 Gemini 3 Pro | 6,5 % | 7 |
| 951 | SWT-Bench o Software tool-use benchmark for code tasks. | Code repair | — | 75,9 % | — | 🇺🇸 GPT-5.2 Thinking | 80,7 % | 7 |
| 952 | ToolACE o ToolACE function-calling evaluation, posed as decisions. | Tool calling | — | 92,8 %gesättigt | — | G Intern-Decision-2B | 96,5 % | 7 |
| 953 | Typed Decision o Typed-decision evaluation set used in Intern-Decision model card (Laya Typed Decisions). | Typed decisions | — | 75,0 % | — | G Intern-Decision-4B | 80,6 % | 7 |
| 954 | Vibe Average o Aggregate Vibe evaluation score. | Vibe evaluation | — | 75,9 % | — | G MiniMax M2.1 | 88,6 % | 7 |
| 955 | VisDrone Detection (F1) o VisDrone object detection with an MLLM; F1 averaged over IoU 0.5:0.95. | Object detection | — | — | — | G WeVisOne-VL-4B | 44,7 | 7 |
| 956 | VizWiz-VQA o Visual question answering on photos taken by blind people (VQA score). | Accessibility VQA | — | 33,7 % | — | 🇨🇳 Qwen-VL-Chat | 38,9 % | 7 |
| 957 | WebArena o Realistic self-hosted web environments (shopping, forums, GitLab, CMS, maps) for autonomous web agents; task success rate. | Web agents | — | 64,2 % | — | 🇺🇸 Claude Sonnet 4.6 | 65,6 % | 7 |
| 958 | WebQuest-MultiQA o Multi-question web search and interaction tasks. | Web agents | — | 56,1 % | — | 🇨🇳 GLM-4.5V | 60,6 % | 7 |
| 959 | WebQuest-SingleQA o Single-question web search and interaction tasks. | Web agents | — | 72,5 % | — | 🇨🇳 GLM-4.6V | 79,5 % | 7 |
| 960 | WikiTableQuestions o WikiTableQuestions (MMTab subset, image tables). | Table QA | — | 76,5 % | — | 🇺🇸 Gemini 3.1 Pro | 79,6 % | 7 |
| 961 | WorldVQA (With Tools) o WorldVQA evaluated with tool access in an agent harness. | Visual world knowledge | — | 53,8 % | — | G Step 3.7 Flash | 58,1 % | 7 |
| 962 | YC-Bench o YC-Bench: long-horizon startup-management simulation; metric is final assets of the fund in millions of USD. | Long-horizon business simulation | — | — | — | 🇺🇸 Claude Opus 4.7 | $2,2M | 7 |
| 963 | ZeroSCROLLS BookSumSort o ZeroSCROLLS split based on BookSumSort long-form summarization. | Long-context summarization | — | — | — | 🇺🇸 GPT-4 | 60,5 | 7 |
| 964 | AGIEval LSAT-AR o LSAT Analytical Reasoning subset from AGIEval benchmark. | Law exam reasoning | — | 26,0 % | — | 🇨🇳 Qwen2.5 32B Base | 30,4 % | 6 |
| 965 | Claw-Eval Multimodal o Multimodal subset of Claw-Eval OpenClaw-style agent benchmark. | Multimodal agent tasks | — | 23,4 % | — | 🇺🇸 GPT-5.4 | 25,7 % | 6 |
| 966 | CrossTabBench Reconstruction TEDS o CrossTabBench: TEDS of reconstructed cross-page tables. | Cross-page tables | — | 57,0 % | — | 🇨🇳 Qwen3.8-Max | 91,7 % | 6 |
| 967 | CrossTabBench Relation F1 o CrossTabBench: F1 for detecting cross-page table continuation relations. | Cross-page tables | — | 70,0 % | — | G OvisOCR3 | 97,1 % | 6 |
| 968 | CursorBench 3.2 o Cursor's coding-agent benchmark on ambiguous multi-file tasks from real Cursor sessions (v3.2). | Software engineering | — | 70,2 % | — | 🇺🇸 Claude Fable 5.1 | 73,4 % | 6 |
| 969 | Daily-Omni o Daily-Omni: audio-visual question answering on everyday videos with temporal alignment. | Audio-visual QA | — | 81,5 % | — | 🇺🇸 Gemini 3 Pro | 84,2 % | 6 |
| 970 | DetailBench o Evaluates whether LLMs can notice subtle errors and minor inconsistencies in text. | Spot small mistakes | — | 4,3 % | — | 🇺🇸 Llama 4 Maverick | 8,7 % | 6 |
| 971 | EnterpriseOps-Gym o Agentic benchmark of enterprise operations workflows with tool use. | Enterprise agent tasks | — | 26,6 % | — | G Spark-X2.6-4B | 33,6 % | 6 |
| 972 | FeTaQA o Free-form table QA (MMTab subset; BLEU). | Table QA | — | 15,4 % | — | G Yuan3.0 Ultra | 39,2 % | 6 |
| 973 | FORTE o AGI-Eval FORTE general agentic task benchmark. | General agent | — | 75,8 % | — | 🇺🇸 GPT-5.5 | 77,8 % | 6 |
| 974 | FullStackBench (zh) o Chinese language full-stack development tasks and evaluation. | Full-stack development | — | 58,9 % | — | 🇨🇳 Qwen3-235B-A22B | 63,1 % | 6 |
| 975 | FutureOmni o FutureOmni: omni-modal (audio-visual) future event forecasting benchmark from video. | Video forecasting | — | 61,6 % | — | G MiMo V2 Omni | 66,7 % | 6 |
| 976 | Gaokao Math 2026 o Five 2026 Chinese Gaokao math exams (National I, II, Beijing, Shanghai, Tianjin); average score out of 150. | Math exams | — | — | — | 🇨🇳 Qwen3.5-9B | 136 | 6 |
| 977 | GRE Math maj@16 o GRE quantitative section evaluated via majority voting over 16 samples. | Math (standardized tests) | — | 51,7 % | — | 🇨🇳 Qwen2 7B | 58,5 % | 6 |
| 978 | GSM8K-Ko o Korean translation of the GSM8K grade-school math word problems. | Math (grade-school, Korean) | — | 82,5 % | — | 🇨🇳 Qwen3-30B-A3B | 88,1 % | 6 |
| 979 | HierText (F1) o HierText text grounding and recognition, F1 mean. | OCR | — | — | — | G WeVisOne-VL-4B | 39,0 | 6 |
| 980 | HiTab o Hierarchical table QA (MMTab subset). | Table QA | — | 60,3 % | — | 🇺🇸 GPT-5.1 | 77,8 % | 6 |
| 981 | HLE-Diamond o Hard 'Diamond' subset of Humanity's Last Exam, evaluated without tools. | Multidisciplinary reasoning | — | 18,6 % | — | G Muse Spark Open | 26,1 % | 6 |
| 982 | HumanRef o Referring detection of people in images (HumanRef), F1 mean. | Referring expressions | — | — | — | G WeVisOne-VL-4B | 80,7 | 6 |
| 983 | ICDAR2015 (F1) o ICDAR2015 text grounding and recognition, F1 mean. | OCR | — | — | — | G WeVisOne-VL-4B | 47,8 | 6 |
| 984 | IFEval-Code o Instruction following evaluation for code generation tasks. | Instruction following (code) | — | 24,7 % | — | 🇨🇳 Qwen3-32B | 28,0 % | 6 |
| 985 | InfoTabs o Natural language inference over infobox tables (MMTab subset). | Table NLI | — | 75,9 % | — | G Yuan3.0 Ultra | 89,7 % | 6 |
| 986 | Kimi Hallu Bench o Moonshot in-house benchmark: percentage of factual claims containing errors, without tools (lower is better). | Hallucination | — | — | — | 🇺🇸 GPT-6 Astra | ↓ 4,2 % | 6 |
| 987 | KoNET o Korean National Educational Test benchmark (elementary to college entrance exams) with images. | Korean exams (multimodal) | — | 79,7 % | — | 🇺🇸 Gemini 2.5 Pro | 96,2 % | 6 |
| 988 | LegalBench o Suite of legal question answering and reasoning tasks. | Legal QA | — | 81,3 % | — | 🇺🇸 Gemma 4 31B (IT) | 83,1 % | 6 |
| 989 | LifeSciBench o Life-sciences research workflow benchmark. | Life sciences | — | 56,2 % | — | 🇺🇸 GPT-6 Astra | 60,3 % | 6 |
| 990 | MEWC o Multi-page End-to-end Web Comprehension benchmark. | Web comprehension | — | 76,1 % | — | 🇺🇸 Claude Opus 4.6 | 89,8 % | 6 |
| 991 | MLE-bench-30 o 30-competition subset of MLE-bench (Kaggle ML engineering tasks); average position score as used in the Gemini 3.6 Flash model card. | ML engineering | — | 60,4 % | — | G Naive-N0.5-Flash | 73,7 % | 6 |
| 992 | MM-ClawBench o MiniMax MM-ClawBench (MM Claw): end-to-end OpenClaw-style personal-agent benchmark. | Personal agent tasks | — | 67,5 % | — | 🇺🇸 Claude Opus 4.6 | 75,4 % | 6 |
| 993 | MPDocBench-Parse Truncated Table TEDS o MPDocBench-Parse: TEDS on truncated (page-split) tables. | Cross-page tables | — | 90,6 %gesättigt | — | G OvisOCR3 | 92,0 % | 6 |
| 994 | OctoCodingBench o Coding benchmark across multi-language programming tasks. | Code generation | — | 26,8 % | — | 🇺🇸 Claude Opus 4.5 | 36,2 % | 6 |
| 995 | OfficeQA Pro v2 o OfficeQA Pro v2: grounded QA over office/enterprise documents requiring files, tools and context. | Enterprise document QA | — | 55,2 % | — | 🇨🇳 Kimi-K3.1 Pro | 64,4 % | 6 |
| 996 | OpenAI Internal Broken Search Tool Non-Disclosure o OpenAI internal alignment eval: rate of failing to disclose a broken search tool (lower is better). | Safety | — | — | — | 🇺🇸 GPT-6 Astra | ↓ 1,5 % | 6 |
| 997 | OpenAI Internal Factual Error Rate (Difficult Prompts) o OpenAI internal factuality eval on de-identified ChatGPT conversations where users flagged factual errors; error rate (lower is better). | Factuality | — | — | — | 🇺🇸 GPT-6 Astra | ↓ 3,9 % | 6 |
| 998 | OpenAI Internal Reviewer Bypass Attempts o OpenAI internal alignment eval: rate of attempts to bypass a reviewer (lower is better). | Safety | — | — | — | 🇺🇸 GPT-6 Astra | 6 | |
| 999 | OpenAI Internal Warning Circumvention o OpenAI internal alignment eval: rate of circumventing warnings (lower is better). | Safety | — | — | — | 🇺🇸 GPT-6 Astra | ↓ 17,4 % | 6 |
| 1000 | PAgentBench o iFlytek in-house Personal Assistant Benchmark: task management, file operations and system interactions. | Personal assistant agents | — | 58,1 % | — | G Spark-X2.6-4B | 74,1 % | 6 |
| 1001 | PointArena o Language-guided pointing benchmark for multimodal models. | Visual grounding | — | 70,7 % | — | 🇺🇸 GPT-6 Astra | 75,4 % | 6 |
| 1002 | PubTabNet o Image-based table recognition benchmark from PubMed Central articles; TEDS score on validation set. | Table recognition | — | 91,4 %gesättigt | — | G Infinity-Parser2-Pro | 94,8 % | 6 |
| 1003 | Ref-L4 Referring Detection (F1) o Ref-L4 referring object detection, F1 mean over IoU 0.5:0.95. | Referring expressions | — | — | — | G WeVisOne-VL-4B | 78,1 | 6 |
| 1004 | RefCOCO testA Segmentation (cIoU) o RefCOCO testA referring segmentation, cIoU. | Referring segmentation | — | 82,3 % | — | G WeVisOne-VL-4B | 84,5 % | 6 |
| 1005 | RefCOCO+ testA Segmentation (cIoU) o RefCOCO+ testA referring segmentation, cIoU. | Referring segmentation | — | 78,0 % | — | G WeVisOne-VL-4B | 81,1 % | 6 |
| 1006 | RefCOCOg Referring Detection (F1) o RefCOCOg test referring object detection (multi-box), F1 mean over IoU 0.5:0.95. | Referring expressions | — | — | — | G WeVisOne-VL-4B | 81,8 | 6 |
| 1007 | RefCOCOg Referring Detection (Rejection) o Rejection rate on non-existent referents for RefCOCOg referring detection. | Referring expressions | — | 31,2 % | — | G WeVisOne-VL-4B | 67,4 % | 6 |
| 1008 | RefCOCOg test Segmentation (cIoU) o RefCOCOg test referring segmentation, cIoU. | Referring segmentation | — | 75,9 % | — | G WeVisOne-VL-4B | 77,8 % | 6 |
| 1009 | RobotGrounding (in-house) o ACE Robotics in-house robot grounding benchmark. | Embodied grounding | — | 88,9 % | — | G ACE-Brain-0.6-9B | 93,4 % | 6 |
| 1010 | SecCodeBench o Benchmark evaluating secure code generation capabilities. | Secure code generation | — | 65,9 % | — | 🇺🇸 GPT-5.2 | 68,7 % | 6 |
| 1011 | ShareRobot-Affordance o ShareRobot affordance prediction benchmark. | Embodied affordance | — | 42,2 % | — | G ACE-Brain-0.6-9B | 48,8 % | 6 |
| 1012 | SIFO o Single-turn instruction following benchmark. | Instruction following | — | 50,9 % | — | 🇨🇳 Qwen3-VL-30B-A3B Thinking | 66,9 % | 6 |
| 1013 | SIFO Multiturn o Multi-turn SIFO benchmark for sustained instruction adherence. | Instruction following | — | 41,6 % | — | 🇨🇳 Qwen3-VL-30B-A3B Thinking | 60,3 % | 6 |
| 1014 | Spider 2.0-Lite o Spider 2.0-Lite enterprise text-to-SQL over real schemas, scored by executing the query (Interfaze reports the 135-task SQLite subset). | Text-to-SQL | — | 48,5 % | — | G Interfaze | 52,9 % | 6 |
| 1015 | SROIE (F1) o SROIE text grounding and recognition, F1 mean. | OCR | — | — | — | G WeVisOne-VL-4B | 68,1 | 6 |
| 1016 | SWE-Dev o Software engineering development and bug fixing benchmark. | Code repair | — | 64,7 % | — | 🇺🇸 Claude Sonnet 4 | 67,1 % | 6 |
| 1017 | SWE-Lancer Diamond o Diamond subset of SWE-Lancer focusing on the hardest freelance-style issues. | Code repair (freelance) | — | 57,6 % | — | 🇺🇸 GPT-5.3 Codex | 81,4 % | 6 |
| 1018 | SWebBench o iFlytek in-house benchmark for web browsing, information retrieval and research (reported with Spark-X2.6). | Web research agents | — | 43,9 % | — | G Spark-X2.6-4B | 71,7 % | 6 |
| 1019 | TAT-QA o Hybrid tabular and textual financial QA (MMTab subset). | Table QA | — | 58,7 % | — | G Yuan3.0 Ultra | 74,9 % | 6 |
| 1020 | TotalText (F1) o TotalText text grounding and recognition, F1 mean. | OCR | — | — | — | G WeVisOne-VL-4B | 52,0 | 6 |
| 1021 | UniMERNet o UniMER-Test benchmark for universal mathematical expression recognition from images. | Formula recognition | — | 96,6 %gesättigt | — | G Infinity-Parser2-Pro | 97,7 % | 6 |
| 1022 | VisualPuzzle o Visual puzzle solving benchmark evaluating reasoning and pattern recognition capabilities. | Visual reasoning | — | 62,5 % | — | 🇺🇸 Gemini 3 Pro | 71,5 % | 6 |
| 1023 | VitaBench 2.0 o VitaBench 2.0: interactive agent tasks in real-world service scenarios (v2.0, not comparable to v1). | Agentic real-world tasks | — | 20,4 % | — | G Spark-X2.6-4B | 30,8 % | 6 |
| 1024 | VPCT o Visual perception and comprehension test. | Multimodal reasoning | — | 60,0 % | — | 🇺🇸 Gemini 3 Pro | 90,0 % | 6 |
| 1025 | WeRef (F1) o Tencent WeRef referring object detection benchmark, F1 mean. | Referring expressions | — | — | — | G WeVisOne-VL-4B | 78,7 | 6 |
| 1026 | WeRef (Rejection) o Rejection rate on non-existent referents in WeRef. | Referring expressions | — | 75,3 % | — | G WeVisOne-VL-4B | 96,4 % | 6 |
| 1027 | ACEBench (ZH) o Chinese split of ACEBench tool-use evaluation. | Tool use (Chinese) | — | 85,8 % | — | 🇨🇳 ERNIE 5.0 | 89,6 % | 5 |
| 1028 | ACVA o Arabic Cultural Value Alignment benchmark (true/false statements about Arab culture). | Arabic culture | — | 80,4 % | — | G Fanar-2-27B-Instruct | 82,7 % | 5 |
| 1029 | AInstein-SWE-Bench o AInstein agent coding benchmark. | Agentic coding | — | 33,6 % | — | 🇺🇸 Gemini 3 Pro | 42,8 % | 5 |
| 1030 | AndroidBench o Agentic Android app development benchmark; Qwen reports avg@3 on the 95-task public subset. | Software engineering (Android) | — | 72,0 % | — | 🇺🇸 Claude Fable 5 | 84,5 % | 5 |
| 1031 | API-Bank o API-Bank tool-use benchmark. | Tool use | — | 79,6 % | — | 🇺🇸 Llama 3.1 405B | 92,0 % | 5 |
| 1032 | Big Finance Bench o Finance benchmark reported in the GPT-5.6 announcement. | Finance | — | 46,6 % | — | 🇺🇸 GPT-5.6 Sol | 53,0 % | 5 |
| 1033 | BIOBench o Biology knowledge and reasoning benchmark. | Biology reasoning | — | 45,7 % | — | 🇺🇸 Gemini 3 Pro | 51,9 % | 5 |
| 1034 | CFEval o Contest-style coding evaluation with ELO-like scoring. | Coding ELO / contest eval | — | — | — | 🇨🇳 Qwen3-235B-A22B-Thinking-2507 | 2134 | 5 |
| 1035 | Claw-Gym o OpenClaw-style personal-assistant agent environment suite measuring task success on daily tasks. | Agentic assistant tasks | — | 46,3 % | — | G Nanbeige4.2-3B | 65,0 % | 5 |
| 1036 | CLOC Counting o CLOC test object counting, exact-match accuracy. | Visual counting | — | 24,8 % | — | G WeVisOne-VL-4B | 28,3 % | 5 |
| 1037 | COCO Pointing o COCO object pointing accuracy. | Object pointing | — | 82,6 % | — | G WeVisOne-VL-2B | 85,8 % | 5 |
| 1038 | CodeIF-Bench o Code-focused instruction following benchmark. | Code instruction following | — | 46,4 % | — | 🇨🇳 Qwen3-8B Non-Thinking | 50,0 % | 5 |
| 1039 | Collie-Hard o Hard subset of Collie instruction-following tasks. | Instruction following | — | 82,8 % | — | 🇺🇸 GPT-5 (high) | 99,0 % | 5 |
| 1040 | Complex Workflow o Complex workflow benchmark for economically valuable tasks. | Complex workflows | — | 55,1 % | — | 🇺🇸 Gemini 3 Pro | 58,2 % | 5 |
| 1041 | Contract Scrub o Contract-review task: identify and redact/flag clauses in legal contracts (Thomson Reuters eval). | Contract review | — | 42,8 % | — | 🇺🇸 Gemma 4 31B (IT) | 54,8 % | 5 |
| 1042 | Countdown o Countdown-style reasoning and planning benchmark. | Planning and reasoning | — | 21,3 % | — | G K2-V2 | 75,6 % | 5 |
| 1043 | CultCat o Yandex 4-shot benchmark of (Russian) cultural facts. | Cultural knowledge | — | 70,4 % | — | G AliceAI-Foundation-80B-A3B-Base | 86,5 % | 5 |
| 1044 | Customer Support Q&A o Customer support question answering benchmark. | Customer support QA | — | 64,4 % | — | G Seed1.8 | 69,0 % | 5 |
| 1045 | CUTE o CUTE aggregate capability score. | English characters | — | 58,6 % | — | 🇺🇸 Bolmo 7B | 78,6 % | 5 |
| 1046 | D4LA o Diverse and detailed dataset for document layout analysis (12 document types, 27 categories). | Layout analysis | — | 46,7 % | — | G Infinity-Parser2-Pro | 52,4 % | 5 |
| 1047 | DeepConsult o Agentic consulting and writing benchmark. | Agentic writing | — | 41,3 % | — | 🇺🇸 GPT-5 (high) | 57,2 % | 5 |
| 1048 | DeepShop o Online shopping web-agent benchmark with complex, deep product-search queries. | Web agents | — | 29,5 % | — | 🇺🇸 MolmoWeb-8B | 42,3 % | 5 |
| 1049 | Dense200 Pointing o Dense200 object pointing accuracy. | Object pointing | — | 76,5 % | — | G LocateAnything-3B | 87,6 % | 5 |
| 1050 | Do-Anything-Now o Resistance to Do Anything Now (DAN) style jailbreak prompts. | Safety / jailbreak | — | 73,9 % | — | G IQuest-Coder-V1-40B-Thinking | 97,7 % | 5 |
| 1051 | DocFigCapBench Caption Score o DocFigCapBench: caption quality score (0-10 scale). | Document figure captioning | — | — | — | 🇨🇳 Qwen3.8-Max | 7,30 | 5 |
| 1052 | DocLayNet o Human-annotated document layout segmentation dataset; layout analysis mIoU/mAP. | Layout analysis | — | 62,9 % | — | G PaddleOCR-VL-1.5 | 71,1 % | 5 |
| 1053 | DREAM-1K o DREAM-1K dense/detailed video description benchmark of 1,000 clips. | Video captioning | — | 75,0 % | — | 🇺🇸 GPT-6 Astra | 79,1 % | 5 |
| 1054 | EduBench English o Yandex education benchmark built from Alice user queries: English language subset. | Education (English) | — | 73,8 % | — | 🇨🇳 DeepSeek V4-Flash Base | 82,9 % | 5 |
| 1055 | EduBench History o Yandex education benchmark built from Alice user queries: history subset. | Education (history) | — | 71,6 % | — | G AliceAI-Foundation-80B-A3B-Base | 82,0 % | 5 |
| 1056 | EduBench Literature o Yandex education benchmark built from Alice user queries: literature subset. | Education (literature) | — | 60,4 % | — | G AliceAI-Foundation-80B-A3B-Base | 73,8 % | 5 |
| 1057 | EduBench Math o Yandex education benchmark built from Alice user queries: school math subset. | Education (math) | — | 72,4 % | — | 🇨🇳 Qwen3.5-35B-A3B-Base | 80,0 % | 5 |
| 1058 | EduBench Math University o Yandex education benchmark built from Alice user queries: university math subset. | Education (university math) | — | 65,5 % | — | G AliceAI-Foundation-80B-A3B-Base | 70,1 % | 5 |
| 1059 | EduBench Russian o Yandex education benchmark built from Alice user queries: Russian language subset. | Education (Russian language) | — | 53,6 % | — | G AliceAI-Foundation-80B-A3B-Base | 74,2 % | 5 |
| 1060 | Education o Education field evaluation (economically valuable tasks). | Economics/education | — | 55,6 % | — | G Seed1.8 | 60,8 % | 5 |
| 1061 | EGE o Part-A questions of the Russian Unified State Exam (EGE) across subjects. | Russian state exam | — | 85,5 % | — | G AliceAI-Foundation-80B-A3B-Base | 90,5 % | 5 |
| 1062 | EIFBench o Complex instruction-following benchmark. | Instruction following | — | 51,4 % | — | 🇺🇸 GPT-5 (high) | 66,7 % | 5 |
| 1063 | Enterprise RAG o Enterprise retrieval-augmented generation evaluation covering internal knowledge bases. | Retrieval-augmented generation | — | 52,8 % | — | 🇺🇸 Apriel Nemotron 15B Thinker | 69,2 % | 5 |
| 1064 | EXECUTE o Multilingual character-level evaluation benchmark. | Multilingual character tasks | — | 51,7 % | — | 🇺🇸 Bolmo 7B | 71,6 % | 5 |
| 1065 | ExpertFactsQA Law o Yandex expert-written factual QA benchmark, law (hard) subset. | Expert knowledge (law) | — | 33,0 % | — | G AliceAI-Foundation-80B-A3B-Base | 49,6 % | 5 |
| 1066 | ExpertFactsQA Medicine o Yandex expert-written factual QA benchmark, medicine subset. | Expert knowledge (medicine) | — | 55,2 % | — | G AliceAI-Foundation-80B-A3B-Base | 63,6 % | 5 |
| 1067 | FigQA o Figure question answering benchmark evaluating visual reasoning over scientific figures and diagrams. | Figure understanding and QA | — | 62,2 % | — | 🇺🇸 Claude Mythos 5 | 88,9 % | 5 |
| 1068 | FinanceReasoning o Financial reasoning benchmark evaluating quantitative and qualitative finance problem solving. | Financial reasoning | — | 86,3 % | — | G Ling 1T | 87,5 % | 5 |
| 1069 | FinQA 128k o Long-context (128k) variant of FinQA: analytics questions over financial reports. | Long-context financial QA | — | 65,8 % | — | G AliceAI-Foundation-80B-A3B-Base | 74,1 % | 5 |
| 1070 | FinSearchComp (T2&T3) o Finance search competition tasks (tracks T2 and T3). | Finance search | — | 54,0 % | — | 🇺🇸 GPT-5 (high) | 64,5 % | 5 |
| 1071 | FLTEval o Mistral benchmark built from real pull requests to the Fermat's Last Theorem Lean repository: complete all formal proofs and new definitions per PR. | Formal proof engineering | — | 27,4 % | — | 🇺🇸 Claude Opus 4.6 | 39,6 % | 5 |
| 1072 | FreshQA o Question answering benchmark emphasizing up-to-date knowledge and recency. | Recency QA | — | 43,0 % | — | 🇨🇳 Qwen3-4B Thinking 2507 | 66,9 % | 5 |
| 1073 | GeoBench1 o Geospatial visual QA and reasoning (set 1). | Geospatial reasoning | — | 66,8 % | — | 🇨🇳 GLM-4.5V | 79,7 % | 5 |
| 1074 | Gorilla Benchmark API Bench o Gorilla API Bench tool-use evaluation. | Tool use | — | 17,9 % | — | 🇺🇸 Llama 3.1 405B | 35,3 % | 5 |
| 1075 | GraphWalks Parents 1M o GraphWalks Parents long-context graph task at 1M context (F1). | Long-context reasoning | — | 47,4 % | — | 🇺🇸 Claude Opus 4.6 | 72,0 % | 5 |
| 1076 | HardMultiQA o Yandex 5-shot benchmark of hard multi-answer factual questions in Russian. | Russian factual knowledge | — | 56,7 % | — | G AliceAI-Foundation-80B-A3B-Base | 67,9 % | 5 |
| 1077 | Harvey LAB-AA o Artificial Analysis run of Harvey's Legal Agent Benchmark (legal work products from source documents). | Legal agents | — | 8,5 % | — | G Muse Wonder Full | 13,1 % | 5 |
| 1078 | HSS o HSS scene-understanding benchmark over images and video. | Scene understanding | — | 47,6 % | — | 🇺🇸 GPT-6 Astra | 53,6 % | 5 |
| 1079 | HumanEval-V o HumanEval variant with visual programming prompts. | Code generation (vision) | — | 35,7 % | — | G Step3-VL-10B | 66,1 % | 5 |
| 1080 | IFStruct o Instruction-following benchmark for structured outputs (format/schema-constrained generation). | Structured output | — | 68,3 % | — | 🇺🇸 LFM2.5 2.6B | 85,5 % | 5 |
| 1081 | IndicIFEval (ground) o Natively written Indic instruction-following prompts; prompt-level strict. | Indic languages | — | 35,9 % | — | G Sarvam-30B | 40,7 % | 5 |
| 1082 | IndicIFEval (translated) o IFEval-style instruction following, translated into Indic languages; prompt-level strict. | Indic languages | — | 59,3 % | — | 🇺🇸 Gemma 4 31B (IT) | 81,1 % | 5 |
| 1083 | Information Extraction o Information extraction benchmark for economically valuable fields. | Information extraction | — | 45,5 % | — | 🇺🇸 Claude Sonnet 4.5 | 46,9 % | 5 |
| 1084 | Information Processing o Information processing benchmark for economically valuable tasks. | Information processing | — | 54,4 % | — | 🇺🇸 Gemini 3 Pro | 56,5 % | 5 |
| 1085 | Intention Recognition o Intent recognition benchmark for practical applications. | Intent recognition | — | 62,7 % | — | 🇺🇸 Gemini 3 Pro | 65,3 % | 5 |
| 1086 | KernelBench Hard o KernelBench Hard: agents write optimized GPU kernels; score relative to hardware peak throughput. | GPU kernel optimization | — | 21,9 % | — | 🇺🇸 Claude Opus 4.7 | 30,7 % | 5 |
| 1087 | KINA o Knowledge benchmark of culturally grounded factual questions (reported in the Seed2.1 model card). | Knowledge | — | 49,5 % | — | 🇺🇸 Gemini 3.1 Pro | 53,2 % | 5 |
| 1088 | KK-4 People o Keep/kill working-memory benchmark with 4 people entities. | Working memory (4 people) | — | 34,6 % | — | G K2-V2 | 92,9 % | 5 |
| 1089 | KK-8 People o Keep/kill working-memory benchmark with 8 people entities. | Working memory (8 people) | — | 20,8 % | — | G K2-V2 | 82,8 % | 5 |
| 1090 | KSM o Korean STEM and math benchmark | Multilingual math | — | 49,4 % | — | G EXAONE Deep 2.4B | 60,9 % | 5 |
| 1091 | KubriCount (testB) o KubriCount testB object counting, exact-match accuracy. | Visual counting | — | 26,6 % | — | G WeVisOne-VL-4B | 40,1 % | 5 |
| 1092 | Legal Agent Benchmark o Harvey Legal Agent Benchmark, full public set. | Legal agents | — | 11,4 % | — | 🇺🇸 Claude Mythos 5 | 16,9 % | 5 |
| 1093 | LEXam (hard) o Hard subset of the LEXam legal-exam benchmark. | Legal exams | — | 38,2 % | — | G Darwin-180B-RSI | 45,7 % | 5 |
| 1094 | LEXam MCQ (4-choice, EN) o LEXam legal-exam benchmark, English 4-option multiple-choice subset. | Legal exams | — | 76,6 % | — | 🇺🇸 Gemma 4 31B (IT) | 87,4 % | 5 |
| 1095 | LiveMathematicianBench o Live benchmark of research-level mathematics problems. | Research math | — | 35,9 % | — | 🇺🇸 GPT-5.5 | 50,8 % | 5 |
| 1096 | LiveSQLBench o LiveSQLBench (Base-Full v1): contamination-free text-to-SQL / database tasks over PostgreSQL databases, solved agentically. | Text-to-SQL | — | 38,9 % | — | 🇺🇸 Claude Opus 4.7 | 41,0 % | 5 |
| 1097 | LVIS Pointing o LVIS object pointing accuracy. | Object pointing | — | 74,7 % | — | G WeVisOne-VL-4B | 79,5 % | 5 |
| 1098 | MatBench o Materials property prediction benchmark for scientific AI models. | Materials property prediction | — | 60,2 % | — | G Intern-S1-Pro | 72,8 % | 5 |
| 1099 | MATH500 (ES) o Spanish MATH500 benchmark | Math (multilingual) | — | 83,9 % | — | G EXAONE 4.0 1.2B | 88,8 % | 5 |
| 1100 | MBE Bar Exam o Multistate Bar Examination (US bar exam) multiple-choice questions. | Legal exams | — | 82,5 % | — | 🇺🇸 Gemma 4 31B (IT) | 88,8 % | 5 |
| 1101 | MeasureBench o Benchmark of reading measurement instruments (gauges, rulers, meters) from images. | Visual measurement reading | — | 49,2 % | — | G Seed2.1 Pro | 62,9 % | 5 |
| 1102 | MedChemBench (internal) o OpenAI internal eval reported in the GPT-6 Astra announcement: medicinal chemistry tasks. | Medicinal chemistry | — | 39,7 % | — | 🇺🇸 GPT-6 Astra | 49,3 % | 5 |
| 1103 | MemoryBench o Meta-reported benchmark of remembering and retrieving user context across conversations. | Long-term memory | — | 82,6 % | — | G Muse Wonder | 85,6 % | 5 |
| 1104 | MILU o Multi-task Indic Language Understanding benchmark (11 Indian languages + English), accuracy. | Indic languages | — | 77,9 % | — | 🇺🇸 Gemma 4 31B (IT) | 85,3 % | 5 |
| 1105 | MixCount o MixCount test object counting, exact-match accuracy. | Visual counting | — | 21,2 % | — | G WeVisOne-VL-4B | 40,4 % | 5 |
| 1106 | MME-CC o MME-CC multimodal evaluation suite. | Multimodal evaluation | — | 40,8 % | — | 🇺🇸 Gemini 3 Pro | 56,9 % | 5 |
| 1107 | MMLongBench-128K o 128K-context variant of MMLongBench evaluating multimodal long-context understanding. | Long-context multimodal | — | 70,7 % | — | G Seed2.1 Pro | 78,3 % | 5 |
| 1108 | MMMLU (ES) o Spanish MMMLU benchmark | Multilingual knowledge | — | 58,4 % | — | 🇺🇸 SmolLM3-3B | 64,7 % | 5 |
| 1109 | MRCR 128K-2N o Multi-Round Coreference Resolution benchmark at 128k context with 2 needles. | Long-context reasoning | — | 25,8 % | — | 🇫🇷 Ministral-3-R 8B | 50,3 % | 5 |
| 1110 | MRCR 128K-4N o Multi-Round Coreference Resolution benchmark at 128k context with 4 needles. | Long-context reasoning | — | 15,5 % | — | 🇫🇷 Ministral-3-R 8B | 22,7 % | 5 |
| 1111 | MRCR 64K-2N o Multi-Round Coreference Resolution benchmark at 64k context with 2 needles. | Long-context reasoning | — | 27,4 % | — | 🇫🇷 Ministral-3-R 8B | 44,0 % | 5 |
| 1112 | MRCR 64K-4N o Multi-Round Coreference Resolution benchmark at 64k context with 4 needles. | Long-context reasoning | — | 20,1 % | — | 🇫🇷 Ministral-3-R 8B | 35,8 % | 5 |
| 1113 | MSEarth-MCQ o Earth science multiple-choice question benchmark for scientific AI models. | Earth science | — | 61,6 % | — | 🇺🇸 Gemini 3 Pro | 65,8 % | 5 |
| 1114 | NAVSIM (PDMS) o NAVSIM v1 non-reactive simulation benchmark for end-to-end driving planners; score = PDM Score (PDMS, 0-100). | Driving planning | — | — | — | 🇨🇳 Qwen-Drive-1.0-4B RL | 90,7 | 5 |
| 1115 | NoLiMa 128K o NoLiMa (No Literal Match) long-context benchmark at 128k context window. | Long-context eval | — | 9,8 % | — | 🇨🇳 MiniCPM-SALA | 23,9 % | 5 |
| 1116 | NoLiMa 32K o NoLiMa (No Literal Match) long-context benchmark at 32k context window. | Long-context eval | — | 27,3 % | — | 🇨🇳 MiniCPM-SALA | 54,5 % | 5 |
| 1117 | NoLiMa 64K o NoLiMa (No Literal Match) long-context benchmark at 64k context window. | Long-context eval | — | 18,1 % | — | 🇨🇳 MiniCPM-SALA | 43,0 % | 5 |
| 1118 | OBQA o OpenBookQA science question answering benchmark. | Open book QA | — | 43,5 % | — | 🇨🇳 Qwen2.5-Omni-3B | 76,3 % | 5 |
| 1119 | OCRBenchV2 KIE o Key information extraction subset of OCRBench v2 as reported without a language split (OvisOCR3 release). | Key information extraction | — | 74,7 % | — | G OvisOCR3 | 83,0 % | 5 |
| 1120 | OfficeQA o Grounded question answering over office and enterprise documents, as reported by OpenAI for GPT-5.4. | Enterprise document QA | — | 71,2 % | — | 🇺🇸 Claude Opus 4.7 | 86,3 % | 5 |
| 1121 | OmniGAIA o OmniGAIA: GAIA-style omni-modal (image/video/audio) agentic perception and reasoning benchmark. | Multimodal agent | — | 56,4 % | — | 🇺🇸 Gemini 3 Pro | 62,5 % | 5 |
| 1122 | OpenAI Internal Coding Deception o OpenAI internal alignment eval: fraction of coding-agent answers with detected deception (lower is better). | Safety | — | — | — | 🇺🇸 GPT-6 Astra | ↓ 0,5 % | 5 |
| 1123 | OptMATH o OptMATH benchmark targeting challenging math optimization and problem-solving tasks. | Math optimization reasoning | — | 42,3 % | — | G Ling 1T | 57,7 % | 5 |
| 1124 | Order 15 Items o Ordering benchmark requiring models to sequence 15 items correctly. | List ordering | — | 31,5 % | — | G K2-V2 | 87,6 % | 5 |
| 1125 | Order 30 Items o Ordering benchmark requiring models to sequence 30 items correctly. | List ordering (long) | — | 8,4 % | — | G K2-V2 | 40,3 % | 5 |
| 1126 | OvisOCR In-house KIE Bench o OvisOCR team in-house key information extraction benchmark. | Key information extraction | — | 72,9 % | — | G OvisOCR3 | 77,7 % | 5 |
| 1127 | PLawBench o Legal-practice benchmark; Qwen evaluates it with gemini-3.1-pro-preview as judge. | Legal reasoning | — | 68,8 % | — | 🇨🇳 Qwen3.8-Max | 73,2 % | 5 |
| 1128 | PRBench Finance o PRBench finance subset: rubric-graded professional reasoning tasks in finance. | Professional reasoning (finance) | — | 53,7 % | — | 🇨🇳 Qwen3.8-Max | 58,3 % | 5 |
| 1129 | PresentBench o Benchmark for generating presentation decks from source materials, scored with rubrics. | Agentic office work (slides) | — | 57,1 % | — | 🇺🇸 GPT-5.5 | 68,9 % | 5 |
| 1130 | Query Sufficiency o Judging whether a legal research query contains sufficient information to answer (Thomson Reuters eval). | Legal query triage | — | 53,8 % | — | G Thomson-1.0-Small | 59,4 % | 5 |
| 1131 | Realbench o Real-world browsing and QA benchmark. | Web browsing | — | 41,5 % | — | G Seed1.8 | 49,1 % | 5 |
| 1132 | ReasonSeg (val) o ReasonSeg val reasoning segmentation, gIoU. | Reasoning segmentation | — | 61,6 % | — | G WeVisOne-VL-4B | 71,2 % | 5 |
| 1133 | Recruit-Bench o Benchmark of recruiting-domain tasks (resume screening, job matching, candidate communication) used by Nanbeige. | Recruitment / HR tasks | — | 58,1 % | — | 🇺🇸 Gemma 4 12B IT | 69,4 % | 5 |
| 1134 | Ref-ZOM (cIoU) o Ref-ZOM zero/one/many-target referring segmentation, cIoU. | Referring segmentation | — | 67,2 % | — | G WeVisOne-VL-2B | 73,4 % | 5 |
| 1135 | Ref-ZOM (Rejection) o Ref-ZOM rejection rate on non-existent referents. | Referring segmentation | — | 38,4 % | — | G WeVisOne-VL-4B | 86,0 % | 5 |
| 1136 | ReMI o Reasoning over multimodal inputs (ReMI). | Multimodal reasoning | — | 60,2 % | — | G Step3-VL-10B | 67,3 % | 5 |
| 1137 | SafetyBench o Multiple-choice safety evaluation across 7 categories in Chinese and English. | Safety | — | 88,1 % | — | 🇺🇸 GPT 4 0613 | 89,7 % | 5 |
| 1138 | SArena (Icon) o SVG Arena benchmark for icon generation evaluation. | SVG generation | — | 75,0 % | — | G Intern-S1-Pro | 83,5 % | 5 |
| 1139 | SkillsBench v1.1 o Public SkillsBench v1.1 benchmark (87 tasks), average score over three runs per task. | Agent skills | — | 68,2 % | — | 🇺🇸 GPT-5.6 Sol | 73,5 % | 5 |
| 1140 | SmolInstruct o Small molecule instruction-following and understanding benchmark. | Small molecule understanding | — | 54,3 % | — | G Intern-S1-Pro | 74,8 % | 5 |
| 1141 | SWE-fficiency o SWE-fficiency: repository-level performance-optimization tasks; agents must speed up real codebases while keeping tests passing. | Code performance optimization | — | 31,5 % | — | 🇺🇸 GPT-5.5 | 46,6 % | 5 |
| 1142 | SWE-Review o Software engineering review benchmark for assessing code review quality. | Code review | — | 9,1 % | — | 🇺🇸 Claude Opus 4.5 | 16,2 % | 5 |
| 1143 | Terminal-Bench 4.0-AA o Artificial Analysis run of Terminal-Bench 4.0. | Agentic terminal coding | — | 56,0 % | — | 🇺🇸 Claude Opus 5.5 | 59,6 % | 5 |
| 1144 | TriviaQA-Wiki o TriviaQA subset answering using Wikipedia evidence. | Open-domain QA | — | 85,5 % | — | 🇺🇸 Llama 3.1 405B Base | 91,8 % | 5 |
| 1145 | U-Artifacts o Benchmark focusing on generated code artifacts quality. | Agentic coding artifacts | — | 46,9 % | — | 🇺🇸 Gemini 3 Pro | 57,8 % | 5 |
| 1146 | UGC-VideoCap o Short-form user-generated video captioning benchmark. | Video captioning | — | 79,6 % | — | G Glimpse | 82,7 % | 5 |
| 1147 | Vibe Android o Vibe evaluation on Android tasks. | Vibe evaluation (Android) | — | 83,5 % | — | 🇺🇸 Claude Opus 4.5 | 92,2 % | 5 |
| 1148 | Vibe Backend o Vibe evaluation on backend tasks. | Vibe evaluation (backend) | — | 84,3 % | — | 🇺🇸 Claude Opus 4.5 | 98,0 % | 5 |
| 1149 | Vibe iOS o Vibe evaluation on iOS tasks. | Vibe evaluation (iOS) | — | 74,9 % | — | 🇺🇸 Claude Opus 4.5 | 90,0 % | 5 |
| 1150 | Vibe Simulation o Vibe evaluation on simulation tasks. | Vibe evaluation (simulation) | — | 83,3 % | — | 🇺🇸 Gemini 3 Pro | 89,2 % | 5 |
| 1151 | Vibe Web o Vibe evaluation on web tasks. | Vibe evaluation (web) | — | 87,6 % | — | G MiniMax M2.1 | 91,5 % | 5 |
| 1152 | VideoMathQA o Benchmark for mathematical reasoning grounded in video content. | Video math reasoning | — | 52,6 % | — | G JoyAI-VL-Interaction | 56,2 % | 5 |
| 1153 | VisDrone Pointing o VisDrone object pointing accuracy. | Object pointing | — | 58,5 % | — | G WeVisOne-VL-4B | 64,4 % | 5 |
| 1154 | WGO Bench o WGO Bench: end-to-end robot manipulation event understanding (Elorian-reported). | Embodied manipulation | — | 33,5 % | — | G Glimpse | 42,7 % | 5 |
| 1155 | WikiWebFacts o Yandex 5-shot benchmark of factual knowledge in Russian. | Russian factual knowledge | — | 75,0 % | — | G AliceAI-Foundation-80B-A3B-Base | 86,5 % | 5 |
| 1156 | WorldBench o Benchmark of real-world visual knowledge and recognition. | Visual knowledge | — | 61,1 % | — | G Seed2.1 Pro | 67,6 % | 5 |
| 1157 | WorldTravel2 (multi-modal) o WorldTravel2 benchmark multimodal track. | Travel planning (multimodal) | — | 43,5 % | — | 🇺🇸 Gemini 3 Pro | 47,2 % | 5 |
| 1158 | WorldTravel2 (text) o WorldTravel2 benchmark text-only track. | Travel planning (text) | — | 51,9 % | — | 🇺🇸 GPT-5 (high) | 56,4 % | 5 |
| 1159 | xdev-bench-hard o Moonshot internal offensive-security / exploit-development benchmark; problems solved (pass@3). | Cybersecurity | — | 71,4 % | — | 🇺🇸 GPT-5.6 Sol | 100,0 % | 5 |
| 1160 | XorQA-IN o Cross-lingual open-retrieval QA in Indian languages (IndicXTREME), F1. | Indic languages | — | 32,9 % | — | G gnani-evon-v3.3-30B-A3B | 41,8 % | 5 |
| 1161 | XpertBench (Edu) o XpertBench education domain subset. | Economics/education | — | 49,3 % | — | 🇺🇸 GPT-5 (high) | 56,9 % | 5 |
| 1162 | XpertBench (Fin) o XpertBench finance domain subset. | Economics/finance | — | 51,5 % | — | 🇺🇸 GPT-5 (high) | 64,5 % | 5 |
| 1163 | XpertBench (Humanities) o XpertBench humanities domain subset. | Economics/humanities | — | 60,8 % | — | 🇺🇸 GPT-5 (high) | 68,5 % | 5 |
| 1164 | XpertBench (Law) o XpertBench legal domain subset. | Economics/legal | — | 53,6 % | — | 🇺🇸 Claude Sonnet 4.5 | 58,7 % | 5 |
| 1165 | XpertBench (Research) o XpertBench research domain subset. | Economics/research | — | 33,5 % | — | 🇺🇸 GPT-5 (high) | 48,2 % | 5 |
| 1166 | XQuAD-IN o Extractive QA (XQuAD) in Indian languages (IndicXTREME), F1. | Indic languages | — | 51,3 % | — | G gnani-evon-v3.3-30B-A3B | 64,6 % | 5 |
| 1167 | τ³-Banking (Voice) o Sierra τ³ banking knowledge tasks run voice-to-voice; pass@1 (%). | Voice agents | — | 20,2 % | — | 🇺🇸 Gemini 3.8 Live Extended Thinking (high) | 35,1 % | 5 |
| 1168 | AfriMed-QA o Pan-African medical question answering benchmark. | Medical QA | — | 57,0 % | — | 🇺🇸 MedGemma 27B | 72,0 % | 4 |
| 1169 | AgentIF o Instruction-following benchmark built from real agentic scenarios with long, complex system prompts and constraints; score is (CSR+ISR)/2. | Instruction following | — | 43,8 % | — | G openPangu-2.0-Flash Thinking | 44,9 % | 4 |
| 1170 | AIME 2024-Ko o Korean translation of AIME 2024 problems. | Math (competition, Korean) | — | 63,2 % | — | 🇨🇳 Qwen3-30B-A3B-Thinking-2507 | 80,3 % | 4 |
| 1171 | AIME 2026 I (with tools) o AIME 2026 I problems solved with tool (code execution) access. | Math competition (with tools) | — | 93,3 %gesättigt | — | 🇺🇸 Gemini 3.1 Pro | 99,9 % | 4 |
| 1172 | Almieyar o Arabic linguistic competence benchmark across phonology, morphology, syntax, semantics and pragmatics. | Arabic linguistics | — | 68,1 % | — | G Fanar-2-27B-Instruct | 79,5 % | 4 |
| 1173 | Android Daily o Benchmark of long-horizon daily phone-use tasks across multiple Android apps. | GUI agents | — | 57,5 % | — | 🇺🇸 Gemini 3 Flash | 63,2 % | 4 |
| 1174 | ASR AMI o Automatic speech recognition benchmark on AMI meeting speech. | ASR | — | — | — | 🇨🇳 Qwen2.5-Omni-3B | ↓ 15,1 % | 4 |
| 1175 | ASR Earnings22 o Automatic speech recognition benchmark on Earnings22 financial calls. | ASR | — | — | — | G Whisper-large-V3 | ↓ 11,3 % | 4 |
| 1176 | ASR GigaSpeech o Automatic speech recognition benchmark on GigaSpeech. | ASR | — | — | — | G Whisper-large-V3 | ↓ 10,0 % | 4 |
| 1177 | ASR SPGISpeech o Automatic speech recognition benchmark on SPGISpeech. | ASR | — | — | — | 🇺🇸 LFM2.5-Audio-1.5B | ↓ 2,8 % | 4 |
| 1178 | ASR TED-LIUM o Automatic speech recognition benchmark on TED-LIUM. | ASR | — | — | — | 🇺🇸 LFM2.5-Audio-1.5B | ↓ 3,5 % | 4 |
| 1179 | ASR VoxPopuli o Automatic speech recognition benchmark on VoxPopuli. | ASR | — | — | — | 🇨🇳 Qwen2.5-Omni-3B | ↓ 5,6 % | 4 |
| 1180 | BeaverTails o Safety benchmark evaluating harmfulness in model responses. | Safety / harmfulness | — | 70,7 % | — | G IQuest-Coder-V1-40B-Thinking | 76,7 % | 4 |
| 1181 | BharatMath o Math word problems in Indian languages. | Indic languages | — | 72,3 % | — | 🇺🇸 Gemma 4 31B (IT) | 78,6 % | 4 |
| 1182 | BridgeBench o BridgeBench evaluation benchmark. | Reasoning | — | 54,9 % | — | 🇺🇸 Claude Opus 4.6 | 60,1 % | 4 |
| 1183 | CC-Bench-V2 Backend o Z.ai CC-Bench-V2 backend development tasks in a Claude Code-style agent harness. | Agentic coding | — | 23,9 % | — | 🇺🇸 Claude Opus 4.6 | 26,9 % | 4 |
| 1184 | CC-Bench-V2 Frontend o Z.ai CC-Bench-V2 frontend development tasks in a Claude Code-style agent harness. | Agentic coding | — | 69,0 % | — | 🇺🇸 Claude Opus 4.6 | 75,9 % | 4 |
| 1185 | CC-Bench-V2 Repo Exploration o Z.ai CC-Bench-V2 repository exploration tasks in a Claude Code-style agent harness. | Agentic coding | — | 70,6 % | — | 🇺🇸 Claude Opus 4.6 | 74,4 % | 4 |
| 1186 | ChestX-ray14 o NIH ChestX-ray14 classification, macro F1. | Medical imaging | — | 44,0 % | — | 🇺🇸 MedGemma 4B | 50,1 % | 4 |
| 1187 | CheXpert o Chest X-ray condition classification, macro F1 over top 5 conditions. | Medical imaging | — | 44,7 % | — | 🇺🇸 MedGemma 27B | 49,9 % | 4 |
| 1188 | Civil Comments o Toxicity classification of online comments (Jigsaw Civil Comments). | Toxicity classification | — | 93,8 %gesättigt | — | G d1-omni-600M | 95,8 % | 4 |
| 1189 | Countix o Video-based counting benchmark for multiple objects. | Video counting | — | 23,6 % | — | G Seed1.8 | 31,0 % | 4 |
| 1190 | CRAG o Complex Retrieval-Augmented Generation benchmark for grounded question answering. | Retrieval QA | — | 52,8 % | — | G Jamba Mini 1.6 | 76,2 % | 4 |
| 1191 | DeepCodeBench o Qodo's question-answering benchmark over real-world code repositories, requiring retrieval and reasoning across large codebases. | Code QA | — | 73,5 % | — | G openPangu-2.0-Flash Thinking | 76,5 % | 4 |
| 1192 | EHRNoteQA o Clinical-note question answering over MIMIC-IV discharge summaries. | Medical records | — | 82,1 % | — | 🇺🇸 MedGemma 27B | 90,7 % | 4 |
| 1193 | EnConda-Bench o English code editing benchmark for applying conditional modifications. | Code editing | — | 13,0 % | — | G Youtu-LLM-2B | 21,5 % | 4 |
| 1194 | FATE-H o FATE-H: graduate-level abstract algebra problems formalized in Lean 4 (100 problems). | Formal theorem proving (abstract algebra) | — | 72,5 % | — | G Leanstral 1.5 | 87,0 % | 4 |
| 1195 | FATE-X o FATE-X: PhD-level abstract algebra problems formalized in Lean 4 (100 problems). | Formal theorem proving (abstract algebra) | — | 25,3 % | — | G Leanstral 1.5 | 34,0 % | 4 |
| 1196 | FeatBench o Benchmark of feature implementation in real repositories from natural-language requirements (vibe coding). | Code (feature implementation) | — | 46,8 % | — | G openPangu-2.0-Pro | 48,4 % | 4 |
| 1197 | FinanceBench (FullDoc) o FinanceBench full-document question answering benchmark requiring long-context financial understanding. | Finance QA | — | 24,0 % | — | G Jamba Mini 1.6 | 45,4 % | 4 |
| 1198 | FrontierCode o Cognition's agentic coding benchmark of 150 tasks from real PRs, graded on hidden tests plus code-quality rubric (Main set, v1.1). | Software engineering | — | 50,1 % | — | 🇺🇸 Claude Opus 5.5 | 54,4 % | 4 |
| 1199 | Gaokao-MathCloze o Fill-in-the-blank math problems from Chinese Gaokao (AGIEval). | Chinese math (exam) | — | 12,3 % | — | 🇨🇳 DeepSeek Math 7B | 20,3 % | 4 |
| 1200 | Gaokao-MathQA o Multiple-choice math problems from Chinese Gaokao (AGIEval). | Chinese math (exam) | — | 27,1 % | — | 🇨🇳 DeepSeek Math 7B | 35,3 % | 4 |
| 1201 | GeneBench o OpenAI eval of multi-step scientific data analysis in genetics and quantitative biology, with ambiguous or flawed data and hidden confounders. | Scientific data analysis | — | 25,7 % | — | 🇺🇸 GPT-5.5 Pro | 33,2 % | 4 |
| 1202 | GlobalMGSM o Global multilingual grade school math word problems. | Math (multilingual) | — | 54,7 % | — | 🇨🇳 Qwen3-4B | 60,9 % | 4 |
| 1203 | GQA o Compositional question answering over real-world images (scene-graph based); test-dev balanced accuracy. | Visual reasoning VQA | — | 49,7 % | — | 🇨🇳 Qwen-VL | 59,3 % | 4 |
| 1204 | GSM-Infinite Hard (128K) o GSM-Infinite Hard benchmark at 128K context. | Math reasoning | — | 23,1 % | — | G MiMo V2 Flash Base | 29,0 % | 4 |
| 1205 | GSM-Infinite Hard (16K) o GSM-Infinite Hard benchmark at 16K context. | Math reasoning | — | 41,1 % | — | 🇨🇳 DeepSeek V3.2-Exp | 50,4 % | 4 |
| 1206 | GSM-Infinite Hard (32K) o GSM-Infinite Hard benchmark at 32K context. | Math reasoning | — | 36,0 % | — | 🇨🇳 DeepSeek V3.2-Exp | 45,2 % | 4 |
| 1207 | GSM-Infinite Hard (64K) o GSM-Infinite Hard benchmark at 64K context. | Math reasoning | — | 28,7 % | — | 🇨🇳 DeepSeek V3.1 | 34,7 % | 4 |
| 1208 | HELMET LongQA o Long-context subset of the HELMET benchmark focusing on grounded question answering. | Long-context QA | — | 29,7 % | — | G Jamba Mini 1.6 | 46,9 % | 4 |
| 1209 | IMO 2025 o 2025 International Mathematical Olympiad (6 proof problems), graded with MathArena methodology. | Math olympiad (proofs) | — | 50,3 % | — | 🇺🇸 GPT-5.5 | 76,1 % | 4 |
| 1210 | IndicSafe5 o Safety benchmark of harmful/benign prompts across Indic languages; accuracy. | Indic languages | — | 86,4 % | — | 🇺🇸 Gemma 4 31B (IT) | 89,8 % | 4 |
| 1211 | KernelGen 1P o OpenAI internal eval (GPT-5.6 announcement): optimizing GPU kernels. | AI research automation | — | 40,5 % | — | 🇺🇸 GPT-5.6 Sol | 61,1 % | 4 |
| 1212 | Kimi Claw 24/7 Bench o Moonshot in-house benchmark of persistent multi-day coworking tasks (17 scenarios, 610 evaluation points) run through the OpenClaw harness; average pass rate over 3 runs. | Long-horizon agents | — | 48,3 % | — | 🇺🇸 GPT-5.5 | 52,8 % | 4 |
| 1213 | LatentJailbreak o Robustness to latent jailbreak adversarial techniques. | Safety / jailbreak | — | 49,7 % | — | 🇺🇸 GPT-3.5-turbo | 77,4 % | 4 |
| 1214 | LiveClaw o Live benchmark of OpenClaw agent tasks executed in real environments. | Agentic assistant tasks | — | 41,3 % | — | 🇺🇸 LFM2.5 24B A2B | 48,1 % | 4 |
| 1215 | LiveCodeBench-Ko o Korean translation of LiveCodeBench. | Code generation (Korean) | — | 47,1 % | — | 🇨🇳 Qwen3-30B-A3B-Thinking-2507 | 66,3 % | 4 |
| 1216 | M3GIA (CN) o Chinese-language M3GIA benchmark covering grounded multimodal question answering. | Chinese multimodal QA | — | 90,0 % | — | 🇨🇳 Seed1.5-VL-Thinking | 91,2 % | 4 |
| 1217 | mArenaHard o Multilingual variant of Arena-Hard evaluating chat quality across languages. | Chat ability (multilingual) | — | 66,8 % | — | 🇨🇳 Qwen3-4B | 70,1 % | 4 |
| 1218 | MASSIVE intent o Amazon MASSIVE intent classification (English). | Intent classification | — | 85,7 % | — | G Decider 4B (Mapika) | 88,3 % | 4 |
| 1219 | MATH-Ko o Korean translation of the MATH competition benchmark. | Math (Korean) | — | 51,3 % | — | 🇨🇳 Qwen3-30B-A3B | 58,2 % | 4 |
| 1220 | mDolly o Multilingual variant of the Dolly instruction-following benchmark. | Instruction following (multilingual) | — | 73,3 % | — | G Tiny Aya Global | 86,9 % | 4 |
| 1221 | MIMIC-CXR (classification) o Chest X-ray condition classification on MIMIC-CXR, macro F1 over top conditions. | Medical imaging | — | 87,4 % | — | 🇺🇸 MedGemma 27B | 90,0 % | 4 |
| 1222 | MMLB-NIAH (128k) o MMLB-NIAH 128k long-context multimodal benchmark. | Multimodal long-context | — | 68,8 % | — | G Seed1.8 | 72,2 % | 4 |
| 1223 | MMLB-VRAG (128k) o MMLB-VRAG 128k long-context multimodal benchmark. | Multimodal long-context | — | 81,8 % | — | 🇺🇸 Gemini 3 Pro | 88,9 % | 4 |
| 1224 | Mobile Actions o Google Mobile Actions dataset: function calling for on-device Android actions (FunctionGemma eval). | Tool use | — | 86,9 % | — | 🇨🇳 Qwen3.5-9B | 91,8 % | 4 |
| 1225 | MS-CXR-T o Temporal chest X-ray disease progression classification, macro accuracy. | Medical imaging | — | 59,0 % | — | 🇺🇸 MedGemma 1.5 4B | 65,7 % | 4 |
| 1226 | MTalk-Bench o MTalk-Bench: multi-turn speech-to-speech dialogue (semantic, paralinguistic, ambient) evaluation (%). | Spoken multi-turn dialogue | — | 69,2 % | — | G StepAudio 3 Realtime | 72,8 % | 4 |
| 1227 | MTOB (full book) o Long-context book understanding benchmark (full-book setting). | Long-form reasoning | — | 45,3 % | — | 🇺🇸 Llama 4 Maverick | 50,8 % | 4 |
| 1228 | MTOB (half book) o Long-context book understanding benchmark (half-book setting). | Long-form reasoning | — | 48,1 % | — | 🇺🇸 Llama 4 Maverick | 54,0 % | 4 |
| 1229 | MultiLoKo o Multilingual local-knowledge QA benchmark covering 31 languages with locally sourced questions. | Multilingual knowledge | — | 44,5 % | — | 🇨🇳 DeepSeek V4-Pro Base | 51,1 % | 4 |
| 1230 | NanoGPT o OpenAI internal eval (GPT-5.6 announcement): improving a NanoGPT training recipe (speed-up score). | AI research automation | — | 7,1 % | — | 🇺🇸 GPT-5.6 Terra | 14,5 % | 4 |
| 1231 | Nanonets KIE o Nanonets key information extraction benchmark (F1). | Key information extraction | — | 82,4 % | — | G Qianfan-OCR | 86,5 % | 4 |
| 1232 | NIAH-Multi 128K o Needle-in-a-haystack multi-query benchmark at 128K context. | Long-context QA | — | 97,4 %gesättigt | — | 🇨🇳 Kimi-K2 Base | 99,5 % | 4 |
| 1233 | NIAH-Multi 32K o Needle-in-a-haystack multi-query benchmark at 32K context. | Long-context QA | — | 96,1 %gesättigt | — | 🇨🇳 Kimi-K2 Base | 99,8 % | 4 |
| 1234 | NIAH-Multi 64K o Needle-in-a-haystack multi-query benchmark at 64K context. | Long-context QA | — | 96,1 %gesättigt | — | 🇨🇳 Kimi-K2 Base | 100,0 % | 4 |
| 1235 | NoCaps o Novel object captioning benchmark (CIDEr on val). | Image captioning | — | — | — | G InstructBLIP (Vicuna-13B) | 121,9 | 4 |
| 1236 | OCRBenchV2 KIE (EN) o English key information extraction subset of OCRBench v2. | Key information extraction | — | 84,6 % | — | 🇺🇸 Gemini 3.1 Pro | 87,8 % | 4 |
| 1237 | OCRBenchV2 KIE (ZH) o Chinese key information extraction subset of OCRBench v2. | Key information extraction (Chinese) | — | 70,0 % | — | G Qianfan-OCR | 82,3 % | 4 |
| 1238 | ODVBench o Online/streaming video understanding benchmark (accuracy). | Streaming video understanding | — | 56,6 % | — | G JoyAI-VL-Interaction | 68,6 % | 4 |
| 1239 | OlympicArena o Olympiad-style mathematics reasoning benchmark. | Math (competition) | — | 71,2 % | — | 🇨🇳 DeepSeek V3 | 76,2 % | 4 |
| 1240 | Omni3D o Omni3D monocular 3D object detection benchmark across indoor/outdoor datasets (AP3D). | 3D detection | — | 39,6 % | — | 🇨🇳 Qwen3.5-4B | 47,4 % | 4 |
| 1241 | OpenAI Internal Database Migration Tasks o OpenAI internal eval reported in the GPT-6 Astra announcement: database migration coding tasks. | Software engineering | — | 53,7 % | — | 🇺🇸 GPT-6 Astra | 63,9 % | 4 |
| 1242 | OpenAI Internal Research Debugging Evaluation o OpenAI internal eval (GPT-5.6 announcement): debugging real AI research systems. | AI research automation | — | 59,2 % | — | 🇺🇸 GPT-5.6 Sol | 68,3 % | 4 |
| 1243 | OpenAI Internal Unauthorized Agent Interaction o OpenAI internal alignment eval: rate of unauthorized interaction with other agents (lower is better). | Safety | — | — | — | 🇺🇸 GPT-6 Astra | 4 | |
| 1244 | OSWorld 2.1 (Offline) o OSWorld 2.1 offline subset (82 tasks), partial-credit score on long-horizon computer-use workflows without internet access. | GUI agents | — | 55,2 % | — | 🇺🇸 Claude Sonnet 5.5 | 83,9 % | 4 |
| 1245 | OSWorld2 o Second-generation OSWorld GUI agent benchmark. | GUI agents | — | 16,3 % | — | 🇨🇳 GLM-4.5V | 35,8 % | 4 |
| 1246 | PAWS-X o Cross-lingual adversarial paraphrase identification. | Paraphrase identification | — | 71,4 % | — | G d1-omni-600M | 79,5 % | 4 |
| 1247 | PostTrainBench Lite o Lite variant of PostTrainBench (agents post-training base LLMs under fixed compute). | Post-training automation | — | 42,6 % | — | 🇺🇸 GPT-5.6 Terra | 51,5 % | 4 |
| 1248 | RefusalBench o Safety-oriented refusal and policy adherence benchmark. | Safety / refusal | — | 64,4 % | — | 🇺🇸 Hermes 4.3 36B Pyche | 72,3 % | 4 |
| 1249 | SaaS-Bench o Benchmark of agents completing tasks in SaaS application environments (as reported in the Kimi K3 release). | Agentic SaaS tasks | — | 55,4 % | — | 🇺🇸 GPT-5.6 Sol | 61,4 % | 4 |
| 1250 | SAT Math o SAT mathematics questions (DeepSeekMath / Minerva-style evaluation). | Math (exam) | — | 71,9 % | — | 🇨🇳 DeepSeek Math 7B | 84,4 % | 4 |
| 1251 | SimpleVQA-DS o SimpleVQA variant curated by DeepSeek with everyday image question answering tasks. | General VQA | — | 56,6 % | — | 🇨🇳 Seed1.5-VL-Thinking | 61,3 % | 4 |
| 1252 | SOB (Structured Output Benchmark) o Structured Output Benchmark: extract values into a JSON schema from text, images and audio (5,324 records); value accuracy over exact field matches. | Structured output | — | 79,3 % | — | G Interfaze 1.1 Lite | 81,5 % | 4 |
| 1253 | SpreadsheetBench-Verified o Verified subset of SpreadsheetBench real-world spreadsheet manipulation tasks. | Spreadsheet manipulation | — | 75,7 % | — | 🇺🇸 Claude Opus 4.6 | 82,4 % | 4 |
| 1254 | SWE-Bench Pro v2 (Hard) o Hard split of SWE-Bench Pro v2 (Scale AI) for software-engineering agents. | Software engineering | — | 76,7 % | — | 🇨🇳 Kimi-K3 | 88,2 % | 4 |
| 1255 | ToolHop o Query-driven benchmark of 995 multi-hop tool-use queries with 3,912 locally executable tools; answer accuracy. | Multi-hop tool use | — | 46,9 % | — | 🇨🇳 Qwen3.5-9B | 52,0 % | 4 |
| 1256 | TriQA o Triadic question answering benchmark evaluating world knowledge and reasoning. | Knowledge QA | — | 76,7 % | — | 🇫🇷 Mixtral 8x22B | 82,2 % | 4 |
| 1257 | TrustLLM o TrustLLM benchmark for trustworthiness and safety behaviors. | Safety / reliability | — | 77,5 % | — | 🇨🇳 Qwen3-Coder-480B-A35B-Instruct | 88,4 % | 4 |
| 1258 | Vibe Code Bench o Vals AI Vibe Code Bench: building complete web applications from natural-language specs. | App generation | — | 90,5 %gesättigt | — | 🇺🇸 Gemini 4 Argon | 91,9 % | 4 |
| 1259 | VideoHallucer o Benchmark for detecting intrinsic and extrinsic hallucinations in video-language models. | Video hallucination | — | 58,5 % | — | 🇺🇸 Gemini 3 Pro | 66,8 % | 4 |
| 1260 | VisualProbe o High-resolution visual search benchmark requiring iterative zoom/crop tool use (introduced with Mini-o3). | Visual search / fine-grained perception | — | 63,1 % | — | 🇺🇸 Gemini 3 Flash | 69,9 % | 4 |
| 1261 | VoiceBench-ja Elyza o ELYZA-tasks subset of VoiceBench-ja (spoken input), GPT-4o judge score 0-5. | Spoken QA (Japanese) | — | — | — | 🇨🇳 Qwen2.5-Omni-3B | 2,37 | 4 |
| 1262 | VoiceBench-ja M-IFEval o M-IFEval subset of VoiceBench-ja (spoken input), scored 0-1. | Spoken instruction following (Japanese) | — | — | — | G J-Moshi | 0,151 | 4 |
| 1263 | VoiceBench-ja spoken_Elyza o spoken_Elyza subset of VoiceBench-ja, GPT-4o judge score 0-5. | Spoken QA (Japanese) | — | — | — | 🇺🇸 LFM2.5-Audio-1.5B-JP | 2,12 | 4 |
| 1264 | WebTestBench o Benchmark of web-app testing agents that detect defects against a checklist in live web applications. | Web testing agents | — | 51,9 % | — | 🇺🇸 GPT-5.6 Sol | 54,0 % | 4 |
| 1265 | Windows Agent Arena o Windows OS computer-use agent benchmark (WAA), task success rate. | GUI agents | — | 45,2 % | — | G EvoCUA-32B-20260105 | 56,5 % | 4 |
| 1266 | WOD-E2E (RFS) o Waymo Open Dataset end-to-end driving challenge on long-tail scenarios; score = Rater Feedback Score (RFS, 0-10) on the test split. | Driving planning | — | — | — | 🇨🇳 Qwen-Drive-1.0-4B RL | 7,91 | 4 |
| 1267 | WorkBench o Workplace agent benchmark: tasks over simulated email, calendar, CRM, analytics and project-management tools; scored by task completion (outcome-based). | Agentic tool use | — | 43,9 % | — | 🇺🇸 Gemma 4 E4B IT | 46,1 % | 4 |
| 1268 | ZClawBench o Z.ai end-to-end benchmark for OpenClaw agent workflows (environment setup, software dev, retrieval, data analysis, content creation). | Agent tasks (OpenClaw) | — | 57,4 % | — | 🇺🇸 Claude Opus 4.6 | 62,3 % | 4 |
| 1269 | AA-Omniscience-Public Accuracy o Share of questions answered correctly on the publicly released AA-Omniscience subset. | Knowledge and hallucination | — | 43,5 % | — | 🇺🇸 GPT-6 Luna (medium) | 46,9 % | 3 |
| 1270 | AgentIF-OneDay o Agentic instruction-following benchmark of full-day, multi-step everyday and office tasks for general agents. | Agentic instruction following | — | 61,6 % | — | G Nanbeige4.2-3B | 67,5 % | 3 |
| 1271 | APEX-SWE o Mercor/Cognition benchmark of integration (shipping end-to-end systems on cloud/business/infra services) and observability (debugging production-style failures from telemetry) tasks; Pass@1. | Software engineering | — | 56,3 % | — | 🇺🇸 Claude Fable 5 | 58,8 % | 3 |
| 1272 | BEAM 128K o BEAM long-context memory benchmark at 128K tokens. | Long-context memory | — | 62,1 % | — | G Muse Glimmer 30B | 65,1 % | 3 |
| 1273 | BFCL v4 Multi-Turn o Multi-turn category of the Berkeley Function Calling Leaderboard v4. | Function calling | — | 53,9 % | — | G Mercury Voice | 60,5 % | 3 |
| 1274 | BioLP-Bench o Comprehensive biomedical language processing benchmark evaluating LLMs across tasks like NER, relation extraction, and QA. | Biomedical NLP | — | 34,6 % | — | 🇺🇸 Grok 4 | 47,0 % | 3 |
| 1275 | C3-Bench o Comprehensive Chinese reasoning capability benchmark. | Reasoning (Chinese) | — | 76,7 % | — | 🇨🇳 GLM-4.5 Base | 83,1 % | 3 |
| 1276 | CC-OCR (Multilingual) o Multilingual text recognition track of the CC-OCR benchmark. | OCR (multilingual) | — | 61,1 % | — | G Qianfan-OCR | 76,7 % | 3 |
| 1277 | ChartBench o Chart comprehension benchmark covering annotated and unannotated chart types. | Chart understanding | — | 78,0 % | — | G Qianfan-OCR | 85,9 % | 3 |
| 1278 | COCO Captions o COCO image captioning benchmark (Karpathy test split), CIDEr score. | Image captioning | — | — | — | 🇺🇸 T5Gemma 2 4B-4B | 105,4 | 3 |
| 1279 | Cybersecurity CTF o Cybersecurity Capture The Flag challenges benchmark. | Cybersecurity CTF | — | 70,9 % | — | 🇺🇸 GPT-5.3 Codex | 77,6 % | 3 |
| 1280 | E.T. Bench o E.T. Bench: event-level and time-sensitive video understanding benchmark. | Video event understanding | — | 62,3 % | — | G Glimpse | 63,6 % | 3 |
| 1281 | FACTS Parametric o FACTS suite parametric-knowledge split: closed-book factoid questions answered without retrieval. | Factuality | — | 41,2 % | — | 🇨🇳 DeepSeek V4-Pro Base | 62,6 % | 3 |
| 1282 | FaithEval (Inconsistent) o FaithEval inconsistent-context subset: detecting contradictory evidence in retrieved context. | Context faithfulness | — | 97,1 %gesättigt | — | G Thomson-1.0-Small | 97,7 % | 3 |
| 1283 | FrontierCode (Diamond) o Cognition's FrontierCode agentic coding benchmark, Diamond subset; patch correctness graded by held-out tests plus rubric. | Software engineering | — | 16,1 % | — | 🇺🇸 Claude Fable 5 | 29,3 % | 3 |
| 1284 | FrontierMath o Frontier mathematics benchmark. | Math (frontier) | — | 13,7 % | — | 🇺🇸 GPT-5 pro | 32,1 % | 3 |
| 1285 | GSM8K Platinum o Harder subset/setting of GSM8K grade-school math problems. | Math (grade-school, hard) | — | 86,2 % | — | 🇨🇳 Kimi-Linear-Base | 89,6 % | 3 |
| 1286 | HarmfulQA o Harmful question set testing models' ability to avoid unsafe answers. | Safety | — | 84,2 % | — | G K2-THINK | 99,0 % | 3 |
| 1287 | HealthBench-Hard Hallucinations o Measures hallucination and unsafe medical advice under hard clinical scenarios. | Medical hallucination safety | — | — | — | 🇺🇸 GPT-5 | ↓ 1,6 % | 3 |
| 1288 | HelpSteer2 o NVIDIA HelpSteer2 helpfulness/attribute rating, posed as decisions. | Response quality rating | — | 36,9 % | — | G Decider 4B (Mapika) | 42,0 % | 3 |
| 1289 | HLE-VL o Vision-language HLE benchmark. | Holistic language evaluation (vision-language) | — | 28,8 % | — | 🇺🇸 Gemini 3 Pro | 36,0 % | 3 |
| 1290 | HumanEval-FIM o Fill-in-the-middle variant of HumanEval measuring code infilling. | Code infilling | — | 84,9 % | — | 🇫🇷 Codestral 22B | 91,6 % | 3 |
| 1291 | IndicMMLU-Pro o MMLU-Pro translated/adapted to Indic languages. | Indic languages | — | 72,7 % | — | 🇺🇸 Gemma 4 31B (IT) | 79,0 % | 3 |
| 1292 | IPhO 2025 (Theory) o Theory problems of the 2025 International Physics Olympiad. | Physics olympiad | — | 87,9 % | — | 🇺🇸 GPT-5.4 Pro | 93,5 % | 3 |
| 1293 | ISL/OSL 8k/16k o Relative throughput on ISL/OSL 8k/16k context workloads. | Throughput | — | — | — | 🇺🇸 Nemotron-3-Nano-30B-A3B | 3,30 | 3 |
| 1294 | LEXam o LEXam benchmark of law-school exam questions (open and multiple-choice) across jurisdictions. | Legal exams | — | 56,5 % | — | G Darwin-180B-RSI | 68,9 % | 3 |
| 1295 | MiniF2F pass@1 o MiniF2F competition benchmark pass@1 accuracy. | Math competition | — | 22,6 % | — | 🇺🇸 NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 | 50,0 % | 3 |
| 1296 | MMLU (cloze) o Cloze-form MMLU evaluation variant. | Multi-domain knowledge (cloze) | — | 30,8 % | — | 🇺🇸 SmolLM2 135M Base | 31,5 % | 3 |
| 1297 | MMSearch-Plus o Harder MMSearch variant requiring fine-grained visual reasoning plus multi-step search. | Multimodal search | — | 27,1 % | — | 🇨🇳 GLM-5V-Turbo | 30,0 % | 3 |
| 1298 | MRCR 16K-8N o OpenAI MRCR v2 8-needle, 8K-16K context bucket. | Long-context reasoning | — | 91,2 %gesättigt | — | 🇺🇸 GPT-5.5 | 93,0 % | 3 |
| 1299 | MRCR 32K-8N o OpenAI MRCR v2 8-needle, 16K-32K context bucket. | Long-context reasoning | — | 96,3 %gesättigt | — | 🇺🇸 GPT-5.4 | 97,2 % | 3 |
| 1300 | MRCR 8K-8N o OpenAI MRCR v2 8-needle, 4K-8K context bucket. | Long-context reasoning | — | 97,9 %gesättigt | — | 🇺🇸 GPT-5.2 | 98,2 % | 3 |
| 1301 | Needle In A Haystack o Needle In A Haystack test for locating hidden facts in long contexts. | Long-context retrieval | — | 98,4 %gesättigt | — | G MobileLLM P1 Base | 100,0 % | 3 |
| 1302 | Objectron o Objectron benchmark for 3D object detection in video captures. | Object detection | — | 28,3 % | — | 🇨🇳 Qwen3-VL-235B-A22B Thinking | 71,2 % | 3 |
| 1303 | OpenAI Internal Advanced Cybersecurity Completion Rate o OpenAI internal eval (Daybreak announcement): share of advanced, dual-use cybersecurity requests (exploit chains, auth bypass, privilege escalation) the model completes. | Cybersecurity | — | 51,4 % | — | 🇺🇸 GPT-5.6 Cyber | 95,0 % | 3 |
| 1304 | OpenAI Internal Data Science Tasks o OpenAI internal eval reported in the GPT-6 Astra announcement: data-science tasks. | Data science | — | 35,4 % | — | 🇺🇸 GPT-6 Astra | 40,9 % | 3 |
| 1305 | OpenAI Internal Design Tasks o OpenAI internal eval reported in the GPT-6 Astra announcement: professional design tasks. | Design | — | 44,4 % | — | 🇺🇸 GPT-6 Astra | 50,0 % | 3 |
| 1306 | OpenAI Internal Vulnerability Discovery and Report Writing o OpenAI internal eval (Daybreak announcement): open-ended vulnerability finding in a repo with a known vulnerability, PoC and report quality. | Cybersecurity | — | 14,7 % | — | 🇺🇸 GPT-5.6 Sol | 17,9 % | 3 |
| 1307 | OpenAI Internal Zero-Day Discovery Eval o OpenAI internal eval (Daybreak announcement): find zero-day vulnerabilities in current open-source releases, build max-impact PoCs and write-ups; graded on severity, impact and calibration. | Cybersecurity | — | 19,3 % | — | 🇺🇸 GPT-5.6 Cyber | 24,5 % | 3 |
| 1308 | OpenRewrite-Eval o OpenRewrite evaluation; micro-averaged RougeL. | Rewrite quality | — | — | — | 🇨🇳 Qwen2.5 1.5B Instruct | 46,9 | 3 |
| 1309 | PenguinScrolls o Long-context reading comprehension and reasoning benchmark. | Long-context eval | — | 79,2 % | — | 🇨🇳 Hunyuan-Large-Instruct | 85,2 % | 3 |
| 1310 | ScreenSpot-Pro (with tools) o ScreenSpot-Pro evaluated with tool access (e.g. zoom/crop). | Screen UI locators (with tools) | — | 86,2 % | — | 🇺🇸 Claude Opus 4.8 | 87,9 % | 3 |
| 1311 | SpatialSense VQA o SpatialSense spatial-relation recognition posed as VQA (as used in Gemma/PaliGemma evaluations). | Spatial reasoning | — | 50,8 % | — | 🇺🇸 T5Gemma 2 4B-4B | 51,7 % | 3 |
| 1312 | SpeechRole o SpeechRole: speech role-playing agent benchmark (aggregate judge score, unbounded scale). | Spoken role-play | — | — | — | 🇺🇸 Gemini 3.1 Pro | 124,2 | 3 |
| 1313 | SRE-Bench o Software reverse engineering of binaries without source code (single-attempt solve rate). | Cybersecurity | — | 52,1 % | — | 🇺🇸 GPT-6 Astra | 88,0 % | 3 |
| 1314 | SWE-Lancer IC Diamond o Individual Contributor Diamond subset of SWE-Lancer. | Code repair (freelance) | — | 77,3 % | — | 🇺🇸 GPT-5.3 Codex | 81,4 % | 3 |
| 1315 | TallyQA o TallyQA open-ended counting questions over images (simple and complex). | Visual counting | — | 32,7 % | — | 🇺🇸 T5Gemma 2 4B-4B | 39,6 % | 3 |
| 1316 | URO-Bench Pro: Oral Conversation o URO-Bench pro track, oral-conversation dimension, speech-to-speech (%). | Spoken dialogue | — | 99,2 %gesättigt | — | 🇨🇳 Qwen3.5-Omni-Plus | 99,8 % | 3 |
| 1317 | URO-Bench Pro: Reasoning o URO-Bench pro track, reasoning dimension, speech-to-speech (%). | Spoken dialogue | — | 84,7 % | — | 🇨🇳 Qwen3.5-Omni-Plus | 86,3 % | 3 |
| 1318 | URO-Bench Pro: Understanding o URO-Bench pro track, understanding dimension, speech-to-speech (%). | Spoken dialogue | — | 66,5 % | — | 🇺🇸 Gemini 3.1 Pro | 69,1 % | 3 |
| 1319 | Vibe Code Bench v1.1 o Vals AI Vibe Code Bench v1.1: build working web applications from natural-language specs. | Application building | — | 55,3 % | — | G MiniMax M3.1 Flash (max) | 73,9 % | 3 |
| 1320 | WebVoyager2 o Web navigation and interaction tasks for LLM agents (v2). | Web agents | — | 62,7 % | — | 🇨🇳 GLM-4.5V | 84,4 % | 3 |
| 1321 | Winogender o Coreference resolution dataset for measuring gender bias. | Gender bias (coreference) | — | 73,1 % | — | 🇺🇸 Llama 3.3 70B Instruct | 84,3 % | 3 |
| 1322 | WMDP Bio o Weapons of Mass Destruction Proxy benchmark for biosecurity, measuring hazardous biological knowledge without info hazards. | Biosecurity knowledge | — | — | — | 🇺🇸 Zephyr 7B | ↓ 63,7 % | 3 |
| 1323 | WMDP Chem o WMDP benchmark for chemical security, evaluating knowledge relevant to chemical weapons development. | Chemical security knowledge | — | — | — | 🇺🇸 Zephyr 7B | ↓ 45,8 % | 3 |
| 1324 | WMDP Cyber o WMDP benchmark for cybersecurity, assessing knowledge that could aid in cyber weapons development. | Cybersecurity knowledge | — | — | — | 🇺🇸 Zephyr 7B | ↓ 44,0 % | 3 |
| 1325 | XQuAD o Cross-lingual Question Answering Dataset: SQuAD v1.1 subset professionally translated into 10 languages. | Multilingual QA | — | 58,2 % | — | 🇺🇸 T5Gemma 2 4B-4B | 70,6 % | 3 |
| 1326 | AM-2K (Box) Matting (MAD) o AM-2K (Box): box-prompted animal matting; mean absolute difference x10^3 (lower is better). | Image matting | — | — | — | G WeVisOne-VL-4B | ↓ 5,00 | 2 |
| 1327 | ARC-Challenge (DE) o German translation of the ARC Challenge benchmark. | Science QA (German) | — | — | — | 🇺🇸 Llama-3_1-70B-TFree-HAT-SFT | 0,669 | 2 |
| 1328 | ARC-Easy (DE) o German translation of the ARC Easy science QA benchmark. | Science QA (German) | — | — | — | 🇺🇸 Llama-3_1-70B-TFree-HAT-SFT | 0,848 | 2 |
| 1329 | BIG-Bench o BIG-bench overall performance (original). | Multi-task reasoning | — | 45,2 % | — | 🇺🇸 Gemma 2 7B | 55,1 % | 2 |
| 1330 | BixBench o Benchmark of real-world bioinformatics and data-analysis tasks. | Bioinformatics | — | 77,3 % | — | 🇺🇸 GPT-5.5 | 80,5 % | 2 |
| 1331 | BOLD o Bias in Open-ended Language Dataset probing demographic biases in text generation. | Bias evaluation | — | — | — | 🇫🇷 Mixtral 8x 7B | ↓ 0,084 | 2 |
| 1332 | BrowseComp (Agent Swarm) o BrowseComp evaluated with Moonshot's Agent Swarm multi-agent orchestration (parallel sub-agents) instead of a single agent. | Web browsing agents | — | 82,3 % | — | 🇨🇳 Kimi-K2.6 Thinking | 86,3 % | 2 |
| 1333 | ChartQA-Pro (with tools) o ChartQA-Pro evaluated with tool access. | Chart understanding (with tools) | — | 71,1 % | — | 🇺🇸 Claude Opus 4.8 | 72,3 % | 2 |
| 1334 | Circa o Interpreting indirect yes/no answers in dialogue. | Indirect answers | — | 86,9 % | — | G pplx-decider-v1-27b | 89,2 % | 2 |
| 1335 | CloningScenarios o Safety benchmark that red-teams models with cloning-related misuse scenarios to measure compliance and refusal rates. | Biosecurity refusal | — | — | — | 🇺🇸 Grok 4 | ↓ 45,0 % | 2 |
| 1336 | CORE o Comprehensive Ontological Relation Evaluation for Large Language Models. | Ontological reasoning | — | 50,5 % | — | G Nanbeige4.1-3B | 53,5 % | 2 |
| 1337 | CoVoST o CoVoST speech-to-text translation benchmark (BLEU). | Speech translation | — | — | — | 🇺🇸 Gemma 4 E4B IT | 35,5 | 2 |
| 1338 | Dolphin-Page o Dolphin Page benchmark measuring OCR fidelity and structured extraction on multi-layout documents. | Document OCR | — | — | — | 🇺🇸 Dolphin 1.5 | ↓ 7,4 % | 2 |
| 1339 | Economically important tasks o Evaluation suite of real-world, economically impactful tasks across key industries and workflows. | Industry QA (cross-domain) | — | 40,3 % | — | 🇺🇸 GPT-5 | 47,1 % | 2 |
| 1340 | EVG o EVG document OCR benchmark evaluating recognition accuracy and layout extraction. | Document OCR | — | — | — | 🇺🇸 Dolphin 1.5 | ↓ 3,0 % | 2 |
| 1341 | ExploitBench (Jun-Aug 2026) o ExploitBench restricted to vulnerabilities disclosed June-August 2026, after model training cutoffs. | Cybersecurity | — | 22,3 % | — | 🇺🇸 GPT-6 Astra | 39,0 % | 2 |
| 1342 | ExploitGym-Honeypot o ExploitGym honeypot variant measuring how often agents take bait out of scope (lower is better). | Safety | — | — | — | 🇺🇸 GPT-6 Astra | 2 | |
| 1343 | FairPoliticsQA o Sakana AI benchmark of balanced, fair answers to Japanese political questions. | Political fairness (Japanese) | — | 45,2 % | — | G Sakana Namazu | 56,3 % | 2 |
| 1344 | FinancialPhraseBank o Sentence-level financial news sentiment classification. | Financial sentiment | — | 80,6 % | — | G pplx-decider-v1-27b | 84,2 % | 2 |
| 1345 | FLEURS o FLEURS multilingual automatic speech recognition benchmark; word error rate, lower is better. | Speech recognition | — | — | — | 🇺🇸 Gemma 4 E4B IT | ↓ 0,1 % | 2 |
| 1346 | Fox-Page-cn o Fox Page benchmark evaluating OCR accuracy and layout understanding on Chinese document pages. | Document OCR (Chinese) | — | — | — | 🇺🇸 Dolphin 1.5 | ↓ 0,8 % | 2 |
| 1347 | Fox-Page-en o Fox Page benchmark evaluating OCR accuracy and layout understanding on English document pages. | Document OCR (English) | — | — | — | 🇺🇸 Dolphin 1.5 | ↓ 0,7 % | 2 |
| 1348 | Full Text MMLU o Full-context MMLU variant evaluating reasoning over long passages. | Multi-domain knowledge (long-form) | — | — | — | 🇺🇸 Llama 3.3 70B Instruct | 83,0 | 2 |
| 1349 | GSM8K (DE) o German translation of the GSM8K grade-school math word problems. | Math (grade-school, German) | — | — | — | 🇺🇸 Llama-3_1-70B-TFree-HAT-SFT | 0,630 | 2 |
| 1350 | HellaSwag (DE) o German translation of the HellaSwag commonsense benchmark. | Commonsense reasoning (German) | — | — | — | 🇺🇸 Llama-3_1-70B-TFree-HAT-SFT | 0,727 | 2 |
| 1351 | HMMT o Harvard–MIT Mathematics Tournament problems. | Math (competition) | — | 96,7 %gesättigt | — | 🇺🇸 GPT-5 pro | 100,0 % | 2 |
| 1352 | ImageMining o Multimodal tool-use benchmark of mining information from images with tools. | Multimodal tool use | — | 27,5 % | — | 🇨🇳 GLM-5V-Turbo | 30,7 % | 2 |
| 1353 | ImplicitMemBench o Tests implicit memory: whether agents apply previously seen preferences and procedures without being explicitly asked to recall them. | Long-term memory | — | 46,2 % | — | G MemT-1.0-Flash | 49,2 % | 2 |
| 1354 | JudgeBench o Pairwise response-correctness judgement benchmark for LLM judges. | LLM-as-judge | — | 78,4 % | — | G Jev 1.13 | 78,6 % | 2 |
| 1355 | LOCA-Bench o LOCA-Bench long-context agent benchmark (environment description length 256K). | Long-context agents | — | 53,2 % | — | 🇺🇸 Claude Opus 4.7 | 57,0 % | 2 |
| 1356 | LoCoMo o Very long-term conversational memory: QA over multi-session dialogues. | Long-term memory | — | 61,3 % | — | G MemT-1.0-Flash | 65,1 % | 2 |
| 1357 | MakeMeSay o Adversarial benchmark testing model robustness against manipulation attempts. Lower is better. | Adversarial robustness | — | — | — | 🇺🇸 Grok 4.1 (Thinking) | 2 | |
| 1358 | MAVERIX o Benchmark requiring joint video and audio understanding (situational awareness, social sentiment). | Audio-visual video understanding | — | 67,0 % | — | 🇺🇸 Gemini 2.5 Flash | 67,4 % | 2 |
| 1359 | MemBench o Benchmark of factual and reflective memory of LLM-based agents across participation and observation scenarios. | Long-term memory | — | 72,7 % | — | G MemT-1.0-Flash | 73,5 % | 2 |
| 1360 | MemoryAgentBench o Evaluates memory agents on accurate retrieval, test-time learning, long-range understanding and conflict resolution. | Long-term memory | — | 16,4 % | — | G MemT-1.0-Flash | 17,4 % | 2 |
| 1361 | MMDU o Multi-turn, multi-image dialog understanding benchmark with GPT-4o-judged scores. | Multi-turn multi-image dialog | — | 61,5 % | — | 🇺🇸 GPT-4V(ision) | 66,3 % | 2 |
| 1362 | OmniMMI o OmniMMI benchmark for multimodal interaction across video streams. | Multimodal interaction | — | 51,3 % | — | G Seed1.8 | 53,0 % | 2 |
| 1363 | Open Rewrite o Rewrite benchmark assessing open-ended editing and directive-following quality. | Instruction following | — | 46,3 % | — | G MobileLLM P1 | 51,0 % | 2 |
| 1364 | OpenAI Internal Circumvention o OpenAI internal eval reported in the GPT-6 Astra announcement: rate of circumventing safeguards (lower is better). | Safety | — | — | — | 🇺🇸 GPT-6 Astra | 2 | |
| 1365 | OpenAI Internal Computer-Use Safety (AutoReview) o OpenAI internal eval reported in the GPT-6 Astra announcement: unsafe computer-use action rate with AutoReview enabled (lower is better). | Safety | — | — | — | 🇺🇸 GPT-6 Astra | ↓ 1,8 % | 2 |
| 1366 | OpenAI Internal Expert-SWE o OpenAI internal eval (GPT-5.5 announcement): expert-level, long-horizon software-engineering tasks. | Software engineering | — | 70,8 % | — | 🇺🇸 GPT-5.5 | 73,1 % | 2 |
| 1367 | OpenAI Internal Hallucination o OpenAI internal eval reported in the GPT-6 Astra announcement: hallucination rate (lower is better). | Hallucination | — | — | — | 🇺🇸 GPT-6 Astra | ↓ 4,2 % | 2 |
| 1368 | OpenScore String Quartets o Optical music recognition on OpenScore string quartet scores; score is 1 - OMR normalized edit distance. | Optical music recognition | — | — | — | 🇺🇸 GPT-6 Astra | 0,840 | 2 |
| 1369 | P3M-500-NP (Box) Matting (MAD) o P3M-500-NP (Box): box-prompted portrait matting; mean absolute difference x10^3 (lower is better). | Image matting | — | — | — | G WeVisOne-VL-2B | ↓ 6,60 | 2 |
| 1370 | PerMemSafe o Personal-memory safety benchmark: appropriate use and protection of stored user memories. | Memory safety | — | 36,1 % | — | G MemT-1.0-Nano | 38,7 % | 2 |
| 1371 | PersonaMem o Evaluates whether models track evolving user personas and preferences across long interaction histories. | Personalization | — | 57,0 % | — | G MemT-1.0-Flash | 61,6 % | 2 |
| 1372 | Perspective Bench o Re-alignment score over 80 prompts measuring adherence to the Public AI Constitution (balanced perspectives). | Value alignment | — | 54,5 % | — | G Snowdon1.1-Small | 93,0 % | 2 |
| 1373 | PPM-100 (Box) Matting (MAD) o PPM-100 (Box): box-prompted portrait matting; mean absolute difference x10^3 (lower is better). | Image matting | — | — | — | G WeVisOne-VL-2B | ↓ 5,30 | 2 |
| 1374 | PRDBench o Product Requirements Document benchmark for evaluating agentic coding capabilities. | Agentic coding | — | 27,5 % | — | 🇨🇳 LongCat-Flash-Lite | 39,6 % | 2 |
| 1375 | ProtocolQA o Protocol question answering benchmark evaluating understanding of scientific protocols and procedures. | Protocol understanding and QA | — | 77,5 % | — | 🇺🇸 Grok 4.1 (Thinking) | 79,0 % | 2 |
| 1376 | Putnam 2025 o Natural-language proof solutions to the 2025 William Lowell Putnam Mathematical Competition (12 problems, 10 points each); score = points out of 120. | Math (competition, proof writing) | — | — | — | G Nomos 1 | 87 | 2 |
| 1377 | QuALITY o Long-document multiple-choice reading comprehension benchmark. | Long-context reading comprehension | — | — | — | 🇺🇸 Llama-3_1-70B-TFree-HAT-SFT | 48,8 | 2 |
| 1378 | Ref-L4 (test) o Ref-L4 referring expression comprehension on the test split. | Referring expressions | — | 88,3 % | — | 🇨🇳 GLM-4.6V | 88,9 % | 2 |
| 1379 | RULER-100 @1M o RULER-100 evaluation at a 1M context window. | Long-context eval | — | 81,9 % | — | 🇺🇸 NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 | 86,3 % | 2 |
| 1380 | RULER-100 @256k o RULER-100 evaluation at a 256k context window. | Long-context eval | — | 91,2 %gesättigt | — | 🇺🇸 NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 | 92,9 % | 2 |
| 1381 | RULER-100 @512k o RULER-100 evaluation at a 512k context window. | Long-context eval | — | 87,7 % | — | 🇺🇸 NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 | 91,3 % | 2 |
| 1382 | SWE-Interact o Interactive SWE benchmark (75 tasks) where the agent must clarify underspecified issues with a simulated user before fixing them. | Interactive software engineering | — | 20,7 % | — | 🇨🇳 Kimi-K3 | 21,3 % | 2 |
| 1383 | TLDR9+ o Long-form summarization benchmark with nine-domain TLDR prompts plus extended variations. | Summarization | — | 16,8 % | — | G MobileLLM P1 | 16,8 % | 2 |
| 1384 | TruthfulQA (binary) o Binary-choice variant of TruthfulQA. | Truthfulness | — | 88,7 % | — | G Jev 1.13 | 92,0 % | 2 |
| 1385 | TruthfulQA (DE) o German translation of the TruthfulQA benchmark. | Truthfulness / hallucination (German) | — | — | — | 🇺🇸 Llama 3.3 70B Instruct | 0,174 | 2 |
| 1386 | WeRefMat (Category) Matting (MAD) o WeRefMat (Category): category-prompted matting; mean absolute difference x10^3 (lower is better). | Image matting | — | — | — | G WeVisOne-VL-4B | ↓ 16,4 | 2 |
| 1387 | WeRefMat (Referring) Matting (MAD) o WeRefMat (Referring): referring-expression matting; mean absolute difference x10^3 (lower is better). | Image matting | — | — | — | G WeVisOne-VL-4B | ↓ 9,80 | 2 |
| 1388 | WinoGrande (DE) o German translation of the WinoGrande pronoun resolution benchmark. | Coreference reasoning (German) | — | — | — | 🇺🇸 Llama-3_1-70B-TFree-HAT-SFT | 0,793 | 2 |
| 1389 | WMT16 En–De o WMT16 English–German translation benchmark (news). | Machine translation | — | — | — | 🇺🇸 Llama 3.3 70B Instruct | 38,8 | 2 |
| 1390 | WMT16 En–De (Instruct) o Instruction-tuned evaluation on the WMT16 English–German translation set. | Machine translation | — | — | — | 🇺🇸 Llama 3.3 70B Instruct | 37,9 | 2 |
| 1391 | XStoryCloze o Multilingual StoryCloze: choose the correct story ending in 11 languages. | Commonsense reasoning (multilingual) | — | 87,7 % | — | 🇨🇳 Zhipu GLM 4-9B-Chat | 90,7 % | 2 |
| 1392 | XWinograd o Multilingual Winograd schema benchmark (en, fr, jp, pt, ru, zh). | Coreference reasoning (multilingual) | — | 67,4 % | — | 🇨🇳 Zhipu GLM 4-9B-Chat | 73,1 % | 2 |
Keine passenden Benchmarks. Kürzere Suche oder andere Filter versuchen.