SWE Bench Leaderboard
Compare models on SWE Bench benchmarks.
Last updated: May 22, 2025
| Rank | Model | Score | Organization | License | Updated At |
|---|---|---|---|---|---|
| 1 | o3-2025-04-16OpenAI | 69.1% | OpenAI | Proprietary | May 22, 2025 |
| 2 | o3OpenAI | 69.1% | OpenAI | Proprietary | May 22, 2025 |
| 3 | o4-miniOpenAI | 68.1% | OpenAI | Proprietary | May 22, 2025 |
| 4 | google-gemini-2.5-pro-previewGoogle | 63.8 | Proprietary | May 18, 2025 | |
| 5 | claude-3-7-sonnet-20250219Anthropic | 62.3% | Anthropic | Proprietary | May 22, 2025 |
| 6 | claude-3-7-sonnet-latestAnthropic | 62.3% | Anthropic | Proprietary | May 22, 2025 |
| 7 | anthropic-claude-3-7-sonnetAnthropic | 62.3% | Anthropic | Proprietary | May 22, 2025 |
| 8 | gpt-4o-miniOpenAI | 61% | OpenAI | Proprietary | May 22, 2025 |
| 9 | o3-miniOpenAI | 61% | OpenAI | Proprietary | May 22, 2025 |
| 10 | gpt-4.1OpenAI | 55% | OpenAI | Proprietary | May 22, 2025 |
| 11 | google-gemini-2.0-flashGoogle | 51.8 | Proprietary | May 18, 2025 | |
| 12 | google-gemini-2.0-flash-liveGoogle | 51.8% | Proprietary | May 18, 2025 | |
| 13 | anthropic-claude-3-5-sonnetAnthropic | 49% | Anthropic | Proprietary | May 22, 2025 |
| 14 | claude-3-5-sonnet-20241022Anthropic | 49 % | Anthropic | Proprietary | May 22, 2025 |
| 15 | claude-3-5-sonnet-20240620Anthropic | 49% | Anthropic | Proprietary | May 22, 2025 |
| 16 | claude-3-5-sonnet-latestAnthropic | 49% | Anthropic | Proprietary | May 22, 2025 |
| 17 | o1OpenAI | 48.9% | OpenAI | Proprietary | May 22, 2025 |
| 18 | claude-3-5-haiku-20241022Anthropic | 40.6% | Anthropic | Proprietary | May 22, 2025 |
| 19 | anthropic-claude-3-5-haikuAnthropic | 40.6% | Anthropic | Proprietary | May 22, 2025 |
| 20 | claude-3-5-haiku-latestAnthropic | 40.6% | Anthropic | Proprietary | May 22, 2025 |
| 21 | gpt-4o-2024-05-13OpenAI | 31% | OpenAI | Proprietary | May 22, 2025 |
| 22 | gpt-4o-2024-11-20OpenAI | 31% | OpenAI | Proprietary | May 22, 2025 |
| 23 | chatgpt-4oOpenAI | 31% | OpenAI | Proprietary | May 22, 2025 |
| 24 | gpt-4oOpenAI | 31% | OpenAI | Proprietary | May 22, 2025 |
| 25 | gpt-4o-2024-08-06OpenAI | 31% | OpenAI | Proprietary | May 22, 2025 |
| 26 | gpt-4.1-miniOpenAI | 23.6% | OpenAI | Proprietary | May 22, 2025 |