Brief
AI long-task benchmark post warns some claims are out of date
A research blog post titled 'Measuring AI Ability to Complete Long Software Tasks' carries a banner saying some of its text and figures are out of date.
The page's own notice states that the interactive chart is kept up to date, but static figures and some claims in the text—such as the doubling time—reflect the state of the data at the time of original publication. It directs readers to a dedicated time horizons page and Time Horizon 1.1 for the latest methodology and results.
Our reading
Our reading is that the post's original quantitative claims, including any doubling-time estimate, should be treated as historical rather than current.
What to do or watch
Readers should treat the post's static figures and its doubling-time claim as historical and consult the dedicated time horizons page and Time Horizon 1.1 for the latest methodology and results; the post's interactive chart is described as kept up to date. The unresolved question is what the current doubling-time estimate is under the updated methodology.
Source details and supporting facts
Each line is stated by the page named above it.
Stated by metr.org
- A banner on the page says some of the text and figures in the post are out of date.
- The banner says the interactive chart is kept up to date, but static figures and some claims in the text (e.g. the doubling time) reflect the state of the data at the time of original publication.
- The banner points to a dedicated time horizons page and Time Horizon 1.1 for latest methodology and results.
Sources
- METRText stored 15 September 2026
How this story was checked. Written from the 1 page listed above, stored 15 September 2026; claims checked against that stored text on 15 September 2026.
What that means
- 3 of 4 reported statements were confirmed against the page that carries them; the rest were removed rather than published.
- Figures in the text were required to appear in the stored source text: yes. Identifiers: yes.
- The check reads stored text only: no claim rests on a fresh look that did not happen.
- Where the reporting was silent, the text says so instead of filling the gap.