Will the highest Humanity's Last Exam (HLE) accuracy score in Epoch AI's benchmark data reach at least 58.5 % for a model released on or before 2026-12-06?
deadline 2026-12-06
confidence medium
tier 1 evidence
39%Asturna estimate
Resolution criterion
Resolves YES if, in a snapshot of Epoch AI's benchmark data (https://epoch.ai/data/benchmark_data.zip, file "hle_external.csv", column "Accuracy") taken on or after 2026-12-20, at least one model with a release date on or before 2026-12-06 has a value >= 58.5 %. Reference at creation: 54.8 % (gpt-6-astra_unknown, 2026-09-03). Voided if Epoch AI revises the methodology so that values are no longer comparable, or retires the file. Data: Epoch AI, CC BY 4.0.
Verified starting state
Computed base rate: 36% of past 60-day windows in Epoch AI's benchmark data saw the running record rise by at least the required amount (threshold 0.585, current record 0.5479999999999999). Data: Epoch AI, CC BY 4.0, epoch.ai/benchmarks. Overlapping windows: small effective sample.
What pushes it up
- The rapid pace of frontier model development, particularly reasoning-focused models like the GPT-6 series, which are designed to excel at difficult academic benchmarks like HLE [tier 1].
- There are nearly 60 days remaining before the deadline, which is a typical window for at least one major AI lab to release a new model or iteration [tier 1].
What pushes it down
- HLE is designed to be extremely difficult, and progress may slow down as models face increasingly complex questions [tier 1].
- No higher score has been recorded in the benchmark data since September 3, 2026, suggesting a potential short-term plateau [tier 1].
What would move this number most
Whether a major AI lab (such as OpenAI, Anthropic, or Google) will release a new frontier model or a significant update before December 6, 2026.
How this number is built
| Deadline | 2026-12-06 |
| Base rate | 36% |
| Best evidence | tier 1 · 1 article(s) used |
| Independent runs | 35 · 42 (spread 7 pts) |
| Confidence | medium |
| Evidence cap | not triggered |
| Last revised | 2026-10-09 13:53 |
| Model | gemini-3.5-flash |
Sources consulted
Probability history
2026-10-09 13:53 39% 1 article(s) on file · gemini-3.5-flash
- What pushes it up. The rapid pace of frontier model development, particularly reasoning-focused models like the GPT-6 series, which are designed to excel at difficult academic benchmarks like HLE [tier 1].
- What pushes it up. There are nearly 60 days remaining before the deadline, which is a typical window for at least one major AI lab to release a new model or iteration [tier 1].
- What pushes it down. HLE is designed to be extremely difficult, and progress may slow down as models face increasingly complex questions [tier 1].
- What pushes it down. No higher score has been recorded in the benchmark data since September 3, 2026, suggesting a potential short-term plateau [tier 1].
- What would move this number most. Whether a major AI lab (such as OpenAI, Anthropic, or Google) will release a new frontier model or a significant update before December 6, 2026.
2026-10-08 19:50 36% 1 article(s) on file · gemini-3.7-flash
- What pushes it up. Rapid frontier model reasoning advances have historically produced record jumps of this magnitude within a 60-day window [tier 1].
- What pushes it down. Humanity's Last Exam is designed specifically to resist rapid saturation, making a +3.7 percentage point increase within two months difficult [tier 1].
- What pushes it down. No verified announcements or test results indicate an imminent model release exceeding 58.5% before December 2026 [tier 4].
- What would move this number most. Whether a major frontier AI lab releases a new flagship reasoning model or update between October 8 and December 6, 2026, that Epoch AI evaluates on HLE.