Short answer: DeepSeek’s release-era benchmark table reports scores of 66.0 on SWE-bench Verified and 31.3 on Terminal-Bench for V3.1. These are reported results from specific evaluations, not a guarantee of performance on your workload. In a separate September 2025 evaluation, NIST said its strongest U.S. reference model outperformed V3.1 across almost every benchmark it tested. (SPLX; NIST CAISI)
Keep model version, benchmark, and test setup attached to each figure. The results below can help identify what to evaluate—not establish a universal ranking.
DeepSeek’s reported V3.1 benchmark scores
SPLX reproduces a comparison table attributed to DeepSeek’s release materials. The column headings identify the earlier models shown alongside V3.1. DeepSeek’s V3.1 release was announced on August 21, 2025; that is the release timeframe, not a stated benchmark evaluation date. (SPLX; DeepSeek release announcement)
| Benchmark | DeepSeek V3.1 | DeepSeek V3-0324 | DeepSeek R1-0528 |
|---|---|---|---|
| SWE-bench Verified | 66.0 | 45.4 | 44.6 |
| SWE-bench Multilingual | 54.5 | 29.3 | 30.5 |
| Terminal-Bench | 31.3 | 13.3 | 5.7 |
These are the figures as reported in the comparison table; the sources cited here do not establish a common scoring unit or provide enough benchmark-harness detail to treat every row as directly comparable with other evaluations. SPLX attributes the figures to DeepSeek’s release page. The release-era table is not an independent measurement, and it does not establish which V3.1 mode was used for each result. (SPLX)
What NIST’s separate evaluation found
In September 2025, the National Institute of Standards and Technology’s Center for AI Standards and Innovation (CAISI) evaluated DeepSeek R1, R1-0528, and V3.1 alongside four U.S. reference models across 19 benchmarks. CAISI reported that the strongest U.S. model outperformed V3.1 across almost every benchmark, and that V3.1 outperformed DeepSeek’s earlier R1 models. (NIST CAISI)
CAISI said the gap was largest on software-engineering and cyber tasks, where the strongest U.S. model solved “20–80% more tasks.” That describes a relative difference in tasks solved, not a 20–80 percentage-point score gap. It is a summary of CAISI’s evaluation, not a prediction for every coding workload. (NIST CAISI)
The report also says one U.S. reference model cost 35% less on average than the best DeepSeek model to perform at a similar level across 13 performance benchmarks. This is an evaluation-level comparison, not a current API price quote. Use the intended provider’s current prices and your expected token usage to estimate your own costs. (NIST CAISI)
Keep V3.1 results separate from older V3 scores
Some widely repeated scores refer to DeepSeek V3, not V3.1. An Artificial Analysis article published December 27, 2024, reported a Quality Index of 80, HumanEval of 92%, and MATH-500 of 85% for V3. Those historical figures should not be attributed to V3.1. (Artificial Analysis article)
Also distinguish the original V3.1 release from V3.1-Terminus. DeepSeek announced Terminus as a later update. Its results should be identified as Terminus results rather than silently substituted for the original V3.1 figures. (DeepSeek V3.1-Terminus model card)
Modes and capabilities to consider for API selection
DeepSeek’s V3.1 release announcement maps deepseek-chat to non-thinking mode and deepseek-reasoner to thinking mode, and states a 128K context window for both. It also describes stronger tool use and agent-task capability and says Strict Function Calling was supported in the Beta API. These are release-announcement statements; confirm current identifiers, limits, and feature status with the provider you plan to use. (DeepSeek API release note)
Use the mode distinction as a test plan, not a performance promise:
- Routine chat or extraction: evaluate non-thinking mode for accuracy, latency, and consistent formatting on representative examples.
- Multi-step reasoning: test thinking mode and measure whether any quality gain justifies its response time and token use.
- Coding or tool-using agents: test the full workflow, including tool-call correctness, task completion, retries, and human correction—not just isolated answers.
A practical production API evaluation
Before adopting V3.1, check the intended provider’s current model identifier and availability, mode behavior, context and output limits, supported tools, and prices. DeepSeek documents a model-list endpoint that returns currently available models and metadata. That documentation does not establish what another provider serves, so confirm the actual model and behavior with your provider. (DeepSeek model-list API documentation)
Then run a small, reproducible pilot:
- Select representative requests, including difficult cases, long inputs, and the tool workflows your application needs.
- Record the provider, exact model identifier, mode, prompts, tool definitions, and generation settings.
- Define success criteria before testing. For code changes, run the same tests; for extraction, compare against labeled answers.
- Track task success, latency, invalid tool calls, retries, and human correction.
- Estimate cost using the provider’s current input and output rates and expected usage, including applicable caching or tool charges. Compare cost per accepted task, not just price per token.
Bottom line: DeepSeek’s reported V3.1 scores provide a reason to evaluate the model on selected coding and terminal tasks, but they are tied to particular reported evaluations and do not establish which mode was used for every result. NIST’s separate evaluation found U.S. reference models ahead across almost all of its tested benchmarks. Use both as inputs to a shortlist, then make the production decision with a provider-specific pilot.
Sources
Deepseek-V3.1 AI Red Teaming: Smarter, Faster…Safer?
Evaluation of DeepSeek AI Models — NIST CAISI
DeepSeek-V3.1 Release — DeepSeek API Docs
A new leader in open source AI — DeepSeek V3 Benchmarking Report