2026-09-10MODELDeepSeek-V4.1-Flash released with extreme agent memory efficiency and evaluated across VALS • 2026-09-03MODELGPT-6 Astra released by OpenAI - 71.7% on EMB, 63.3% on Tax Agent Bench • 2026-09-02MODELGoogle Gemini 3.8 Flash takes #1 rank on Finance Agent (v2) with 61.4% and 72.2% on EMB • 2026-09-02MODELMeta releases Muse Spark 1.3 & 1.3 Max - 59.0%-60.0% on Finance Agent (v2) • 2026-09-01MODELClaude Fable 5.1 leads Tax Agent Bench (77.6%) and EMB (76.7%) with 75% cheaper cache reads • 2026-09-11BENCHTax Agent Bench and FAB v2 benchmarks updated with latest frontier model rankings • 2026-09-15UPDATEPlatform updated: 43 models tracked across 41 independent benchmarks • 2026-09-10MODELDeepSeek-V4.1-Flash released with extreme agent memory efficiency and evaluated across VALS • 2026-09-03MODELGPT-6 Astra released by OpenAI - 71.7% on EMB, 63.3% on Tax Agent Bench • 2026-09-02MODELGoogle Gemini 3.8 Flash takes #1 rank on Finance Agent (v2) with 61.4% and 72.2% on EMB • 2026-09-02MODELMeta releases Muse Spark 1.3 & 1.3 Max - 59.0%-60.0% on Finance Agent (v2) • 2026-09-01MODELClaude Fable 5.1 leads Tax Agent Bench (77.6%) and EMB (76.7%) with 75% cheaper cache reads • 2026-09-11BENCHTax Agent Bench and FAB v2 benchmarks updated with latest frontier model rankings • 2026-09-15UPDATEPlatform updated: 43 models tracked across 41 independent benchmarks •
Home/Benchmarks/SWE-bench
Tier 2agentTasks

SWE-bench

Software engineering benchmark evaluating resolution of real-world GitHub issues.

2,294 tasks
22 models evaluated
Updated 2026
arXiv:2310.06770
View Source →Submit your scoreMethodology
97.0%
Top Score
Top model
Claude Opus 5
Model Rankings
financebenchmark.ai · SWE-bench
Anthropic
Anthropic
OpenAI
DeepSeek
xAI
Moonshot AI
Anthropic
DeepSeek
Google
Google
Zhipu AI
OpenAI
OpenAI
xAI
Thinking Machines
Thinking Machines
Moonshot AI
Ant Group
Meta
Meta
NVIDIA
Alibaba
93.4%
🥇AnthropicAnthropicClaude Opus 597.0%
🥈AnthropicAnthropicClaude Sonnet 579.6%
🥉OpenAIOpenAIGPT-5.6 Sol96.2%
4DeepSeekDeepSeekDeepSeek V4 FlashOPEN88.8%
5xAIxAIGrok 4.695.6%
6Moonshot AIMoonshot AIKimi K3OPEN93.4%
7AnthropicAnthropicClaude Fable 595.0%
8DeepSeekDeepSeekDeepSeek V4 Pro (0813)OPEN96.4%
9GoogleGoogleGemini 3.5 Flash Lite75.0%
10GoogleGoogleGemini 3.6 Flash74.8%
11Zhipu AIZhipu AIGLM 5.2OPEN82.8%
12OpenAIOpenAIGPT-5.6 Luna93.0%
13OpenAIOpenAIGPT-5.6 Terra75.2%
14xAIxAIGrok 4.586.6%
15Thinking MachinesThinking MachinesInklingOPEN77.6%
16Thinking MachinesThinking MachinesInkling SmallOPEN82.2%
17Moonshot AIMoonshot AIKimi K2.7 CodeOPEN78.2%
18Ant GroupAnt GroupLing 3.0 FlashOPEN65.2%
19MetaMetaMuse Spark 1.182.0%
20MetaMetaMuse Spark 1.286.6%
21NVIDIANVIDIANemotron 3 UltraOPEN69.0%
22AlibabaAlibabaQwen 3.8 MaxOPEN85.3%
Full Results Table
#ModelOverallSourceDate
🥇
AnthropicClaude Opus 5
Anthropic
97.0arXiv:2310.067702026-07
🥈
DeepSeekDeepSeek V4 Pro (0813)OPEN
DeepSeek
96.4arXiv:2310.067702026-08
🥉
OpenAIGPT-5.6 Sol
OpenAI
96.2arXiv:2310.067702026-07
4
xAIGrok 4.6
xAI
95.6arXiv:2310.067702026-08
5
AnthropicClaude Fable 5
Anthropic
95.0arXiv:2310.067702026-06
6
Moonshot AIKimi K3OPEN
Moonshot AI
93.4arXiv:2310.067702026-07
7
OpenAIGPT-5.6 Luna
OpenAI
93.0arXiv:2310.067702026-07
8
DeepSeekDeepSeek V4 FlashOPEN
DeepSeek
88.8arXiv:2310.067702026-07
9
xAIGrok 4.5
xAI
86.6arXiv:2310.067702026-07
10
MetaMuse Spark 1.2
Meta
86.6arXiv:2310.067702026-08
11
AlibabaQwen 3.8 MaxOPEN
Alibaba
85.3arXiv:2310.067702026-08
12
Zhipu AIGLM 5.2OPEN
Zhipu AI
82.8arXiv:2310.067702026-06
13
Thinking MachinesInkling SmallOPEN
Thinking Machines
82.2arXiv:2310.067702026-07
14
MetaMuse Spark 1.1
Meta
82.0arXiv:2310.067702026-07
15
AnthropicClaude Sonnet 5
Anthropic
79.6arXiv:2310.067702026-06
16
Moonshot AIKimi K2.7 CodeOPEN
Moonshot AI
78.2arXiv:2310.067702026-06
17
Thinking MachinesInklingOPEN
Thinking Machines
77.6arXiv:2310.067702026-07
18
OpenAIGPT-5.6 Terra
OpenAI
75.2arXiv:2310.067702026-07
19
GoogleGemini 3.5 Flash Lite
Google
75.0arXiv:2310.067702026-07
20
GoogleGemini 3.6 Flash
Google
74.8arXiv:2310.067702026-07
21
NVIDIANemotron 3 UltraOPEN
NVIDIA
69.0arXiv:2310.067702026-06
22
Ant GroupLing 3.0 FlashOPEN
Ant Group
65.2arXiv:2310.067702026-07
All evaluated models
Anthropic
Claude Opus 5
97.0%
Anthropic
Claude Sonnet 5
79.6%
OpenAI
GPT-5.6 Sol
96.2%
DeepSeek
DeepSeek V4 Flash
88.8%
xAI
Grok 4.6
95.6%
Moonshot AI
Kimi K3
93.4%
Anthropic
Claude Fable 5
95.0%
DeepSeek
DeepSeek V4 Pro (0813)
96.4%
Google
Gemini 3.5 Flash Lite
75.0%
Google
Gemini 3.6 Flash
74.8%
Zhipu AI
GLM 5.2
82.8%
OpenAI
GPT-5.6 Luna
93.0%
OpenAI
GPT-5.6 Terra
75.2%
xAI
Grok 4.5
86.6%
Thinking Machines
Inkling
77.6%
Thinking Machines
Inkling Small
82.2%
Moonshot AI
Kimi K2.7 Code
78.2%
Ant Group
Ling 3.0 Flash
65.2%
Meta
Muse Spark 1.1
82.0%
Meta
Muse Spark 1.2
86.6%
NVIDIA
Nemotron 3 Ultra
69.0%
Alibaba
Qwen 3.8 Max
85.3%