Two sets of numbers: independent ones from Artificial Analysis, which runs the same tests on every model it covers, and the ones Mistral published itself. Until the weights are out, nobody can run Mistral Large 4 on their own hardware to check either.
Artificial Analysis tested the preview through Mistral's own API, with thinking on. The first group makes up its Intelligence Index v4.3.2.Source: Artificial Analysis
| Test | Score |
|---|---|
| In the Intelligence Index | |
| AA-Briefcase v1.1Agentic knowledge work | 1,393 Elo |
| GDPval-AA v2.1Agentic real-world work tasks | 1,424 Elo |
| AutomationBench-AAAgentic workflows across business apps | 59.9% |
| Terminal-Bench 4.0Agentic coding and terminal use | 26.8% |
| SciCodeScientific codingMarked “under review” by Artificial Analysis. | 54.2% |
| Humanity's Last ExamReasoning and knowledge | 35% |
| GDP.pdfReasoning over professional documents, all checks passed | 18.6% |
| CritPtPhysics reasoningMarked “under review” by Artificial Analysis. | 10.6% |
| AA-OmniscienceKnowledge, with a penalty for made-up answersOn a scale from −100 to 100, where below zero means more wrong answers than right ones. 25.8% of answers were correct, with a 41.9% hallucination rate. | −5.3 |
| AA-LCR v1.1Long-context reasoning | 81.3% |
| Other tests | |
| MMMU-ProVisual reasoning | 76.4% |
Mistral chose these tests and published the scores in its launch post, some in the text and some in charts that compare it with other models. These are Mistral's own figures, not independent results.Source: Mistral's announcement
| Test | Score |
|---|---|
| Coding | |
| DeepSWE v1.1Software engineering | 61.7% |
| SWE-Atlas-QnAUnderstanding a code repository | 59.4% |
| Terminal-Bench 4Agentic coding and terminal use | 28.3% |
| Coding Agent IndexThe average of the three tests above | 49.8% |
| Blind human review by Surge AICode quality rated by professional annotatorsSecond of five models, behind Claude Opus 5. | 3.74 out of 5 |
| Cybersecurity | |
| AA Cyber IndexFinding and fixing flaws in real software | 50 |
| CyberGym-E2EReproducing, then patching, a real vulnerability | 82% |
| Cybench40 exercises from security competitions | 93% |
| Agents and office work | |
| AutomationBench657 workflows in apps such as Gmail, Sheets, Slack and Salesforce | 59.9% |
| AA-BriefcaseLong-horizon knowledge work | 1,393 Elo |
| Finance Agent v2Financial analysis, run by Vals.ai | 54.7 |
| Harvey's Legal Agent BenchmarkLegal work, run by Vals.ai | 15.8 |
| Vision and science | |
| Dense 200Visual grounding in dense scenes | 42% |
| SciCode-VerifiedScientific coding, first-try pass rate over six runs | 91.8% |
| Safety | |
| B3 AI Security BenchmarkAttacks on AI agents resisted, from Lakera | 93.3% |
| KORA BenchmarkResponsible behaviour with users | 1.691 out of 2 |