An AI coding agent can spend an afternoon reading a repository, editing files and waiting for tests. Whether it finishes before the clock expires depends on more than the model’s ability to write code. The machine, the tools and the examiner’s rules help determine what appears in the score.

DeepSeek’s October 17, 2026 update makes that easy to see. The company replaced V4.1 Flash with DeepSeek-V4.1-Flash-1017 behind the existing deepseek-flash API name, while keeping the calling method and prices unchanged. Its Chinese-language announcement also claims shorter outputs and faster task completion.[1]

The strongest numerical story is a comparison within the same test. DeepSeek’s model card reports 31.2 to 42.3 on Terminal-Bench 4.0, a gain of 11.1 percentage points. On DeepSWE v1.1, it reports 74.2 to 77.0, a smaller 2.8-point improvement. These are the company’s results, read here as of October 23, 2026.[2]

Keep the examiner in the picture

The model card names the agent software around each test: Claude Code for Terminal-Bench 4.0 and Mini-SWE-V2 for DeepSWE v1.1. This surrounding software, often called a harness, manages the interaction between the model and its tools. A score describes that combination in its evaluation setting.[2]

There is a tempting wrong comparison in DeepSeek’s own release history. September’s announcement listed 90.6 on Terminal-Bench 2.1, alongside 31.2 on 4.0. Putting October’s 42.3 next to September’s 90.6 would manufacture a collapse by changing the exam halfway through the comparison.[1]

The matched 4.0 row is useful evidence of progress. It remains a reported aggregate: the model-card table does not attach trial counts, confidence intervals or a complete configuration for each run. It supports a promising direction more readily than a precise forecast of how often someone’s own coding task will succeed.[2]

An eight-hour allowance changes the question

Terminal-Bench’s maintainers explain that version 4.0 recalibrates task resources, including time, CPU and memory. Every task receives an eight-hour agent timeout. They report fewer timeouts and errors than in version 3.0, aiming to reduce the noise introduced by insufficient resources.[3]

The task set changed too. Eight tasks were removed for reasons including saturation, refusals, public solutions and unresolved quality or compatibility problems. Another 19 received fixes to instructions, environments or grading. Under the project’s versioning rules, changes to the agent’s environment or task set require fresh trials.[3]

An eight-hour allowance is a ceiling, not a claim that each task takes eight hours. It also asks a different practical question from a developer who needs a repair before a meeting. A generous clock helps examine whether an agent can eventually finish. A short deadline examines whether it can deliver within a particular working rhythm. Both questions matter; their answers should remain separate.

This is the useful lesson of the version number: preserve the conditions before interpreting the change in score.

What survives in a clean checkout

DeepSWE supplies another view of coding work. Its authors assembled 113 original tasks across 91 open-source repositories in TypeScript, Go, Python, JavaScript and Rust. The prompts describe desired behavior, while the agent must investigate the codebase and decide how to implement it. Tests are designed around observable results rather than requiring one particular internal solution.[4]

In v1.1, the agent’s committed patch is transferred into a fresh, separate container for grading. Named test results make missing checks visible. The revision also removes future Git history from the working environment and addresses dependency drift and flaky tests.[5]

That separation gives the result a concrete meaning: the submitted change must work when carried out of the session that produced it. Imagine an agent makes a test pass by changing something in its working environment, then submits a patch that leaves that change behind. Rebuilding the work elsewhere exposes the gap. This example illustrates the reason for isolation; it is not a reported failure by Flash 1017.

Coverage still has limits. DeepSWE’s authors note that their corpus comes from active, relatively popular public repositories. C++ and Java are absent, and proprietary codebases may behave differently. Their own shared-harness evaluation also does not directly reproduce every model’s experience inside its usual developer tools.[4]

Faster needs its own measurement

DeepSeek’s claim of shorter execution time deserves a separate test from its correctness scores. The October announcement asserts the improvement, but does not provide a matching per-task timing table in the text.[1]

DeepSWE v1.1 offers a useful caution: its authors stopped reporting elapsed time because host-machine performance and provider load made comparisons inconsistent.[5] That decision does not make waiting irrelevant. It means elapsed time needs a well-described setting before it can support a claim about the model.

My reading is that Flash 1017 earns a fresh evaluation on work a team actually recognizes. Freeze the task set, repository state, agent software and grading rules; record the precise model version. Then track accepted changes alongside total spending and the distribution of completion times. Keep failed attempts in the accounting, since a cheap unsuccessful run still leaves work to do.

A particularly revealing result would be more accepted patches within the same working-day deadline, with fewer regressions after review. The published gains give a reason to look for that outcome. The benchmark documentation explains what must stay visible while looking: the version of the exam, the resources available and the code that survives the final check.

Sources

1. DeepSeek, “Change Log,” Chinese-language first-party announcements dated September 10 and October 17, 2026 — release dates, API behavior, benchmark versions and efficiency claims.
2. DeepSeek-AI, “DeepSeek-V4.1-Flash-1017,” official model card — reported prior-version comparisons and harness footnote; consulted October 23, 2026.
3. Ryan Marten, “Terminal-Bench 4.0,” official benchmark release explanation — resource calibration, task changes and rerun requirements; consulted October 23, 2026.
4. Wenqi Huang and colleagues, “DeepSWE: Measuring frontier coding agents” — task construction, behavioral verification and limitations; consulted October 23, 2026.
5. Wenqi Huang and Peter Jiang, “DeepSWE v1.1,” June 14, 2026 — isolated patch grading, test reports and the limits of elapsed-time comparisons.
6. Lü Ang, Xinhua Finance, “DeepSeek Growth Season: A Visit to Its Offices,” February 25, 2025 — Chinese-language on-site report and credited photograph of Huijin International Building, Hangzhou.