Short answer: GPT-OSS 120B is worth shortlisting for coding if you can accommodate its larger model and want to evaluate it with the reasoning and tools your workflow will use. OpenAI’s model card reports 62.4% on SWE-Bench Verified, 44.4% on Aider Polyglot, and 2,463 Codeforces Elo without tools or 2,622 with a terminal tool, all at high reasoning effort. These are OpenAI-reported benchmark results—not a guarantee for your codebase. [1]
The choice is not simply “120B is always better.” At medium reasoning, GPT-OSS 20B scored slightly higher on SWE-Bench Verified, while its listed checkpoint is much smaller. OpenAI’s documentation describes Responses API compatibility, but OpenAI says gpt-oss is not served through the OpenAI API. A provider-specific example is documented by Modular; the sources reviewed here do not establish that Token Forge Cloud currently offers GPT-OSS 120B. [1, 5]
GPT-OSS 120B coding benchmark results
OpenAI’s model card, dated August 5, 2025, reports the following results. Percentages and Elo are different measures and should not be compared as though they were the same score. The Codeforces results have explicit tool and no-tool conditions; the card describes the terminal tool as similar to Codex CLI. [1]
| Benchmark | GPT-OSS 120B result | Condition and metric |
|---|---|---|
| SWE-Bench Verified | 62.4% | High reasoning; percentage |
| Aider Polyglot | 44.4% | High reasoning; percentage |
| Codeforces | 2,463 Elo | High reasoning; no terminal tool |
| Codeforces | 2,622 Elo | High reasoning; with terminal tool |
OpenAI characterizes GPT-OSS 120B’s coding and tool-use results as exceeding o3-mini and approaching o4-mini. The model-card material cited here does not provide exact o3-mini or o4-mini coding scores for a like-for-like numerical comparison, so treat that as OpenAI’s qualitative comparison—not an independently established ranking. [1]
The scores support evaluating the model, not assuming it will solve a particular issue. Repository structure, task type, prompting, reasoning setting, and whether the model can run tests or other tools can all affect the result. OpenAI’s published Codeforces results themselves differ by tool condition.
GPT-OSS 120B vs. 20B for coding
Here are the model-card results for both sizes. The table keeps each benchmark’s metric and reasoning setting visible. [1]
| Benchmark | Reasoning | GPT-OSS 120B | GPT-OSS 20B |
|---|---|---|---|
| SWE-Bench Verified | High | 62.4% | 60.7% |
| SWE-Bench Verified | Medium | 52.6% | 53.2% |
| Aider Polyglot | High | 44.4% | 34.2% |
| Codeforces, no tools | High | 2,463 Elo | 2,230 Elo |
| Codeforces, with terminal tool | High | 2,622 Elo | 2,516 Elo |
The listed checkpoint sizes are 60.8 GiB for 120B and 12.8 GiB for 20B. These are checkpoint figures, not a complete measure of runtime memory, hosting requirements, latency, or serving cost. [1]
A practical way to choose:
- Evaluate 120B first if coding quality is the priority and your infrastructure or provider can serve it. It leads on the listed high-reasoning results, especially Aider Polyglot.
- Keep 20B in the running if model size matters or your workflow uses medium reasoning: it has the smaller checkpoint, and its reported medium-reasoning SWE-Bench Verified score is slightly higher.
- Decide with your own workload, not checkpoint size or benchmark scores alone. The published figures do not tell you which model will have better latency, cost, or success rates in your environment.
OpenAI also says that higher reasoning effort generally produces longer reasoning and may increase response latency and cost. That makes reasoning level a deployment trade-off to measure alongside answer quality—not just a setting to maximize. [1]
Production API access: what is documented
API compatibility and a model-reference page are not proof that OpenAI hosts the model. OpenAI’s model card describes Responses API compatibility, while OpenAI’s Help Center explicitly says gpt-oss models are not served through the OpenAI API. The OpenAI API model page lists endpoint names and features, but that listing should not be treated as confirmation of an OpenAI-hosted service. [1, 3, 5]
| Source | What it documents | What that does not establish |
|---|---|---|
| OpenAI model card and Help Center | The model card says gpt-oss is compatible with the Responses API; the Help Center says it is not served through the OpenAI API. [1, 5] | OpenAI-hosted access. Use a hosting provider or infrastructure you control, and check that provider’s current documentation. |
| OpenAI API model documentation | A model page for gpt-oss-120b lists Chat Completions (/v1/chat/completions) and Responses (/v1/responses), along with streaming and function calling as supported features. [3] |
An accessible OpenAI-hosted GPT-OSS service. The listing does not override OpenAI’s explicit Help Center availability statement. |
| Modular product page | Shared, Dedicated, and Self-hosted deployment options; a Python streaming chat-completions example uses https://model.api.modular.com and model ID openai/gpt-oss-120b. [2] |
Token Forge Cloud availability or configuration. The example is Modular-specific and uses a general explanation prompt, not a coding test. |
Token Forge Cloud availability is not confirmed by the sources reviewed for this guide. Its public homepage describes production APIs for Qwen, DeepSeek, GLM, and Seedance; that listing does not establish GPT-OSS 120B access. Before integrating, confirm directly with the provider whether the model is available and request its exact model ID, endpoint, authentication method, supported tools and reasoning controls, limits, and deployment options. [4]
Do not copy a model ID or base URL from one provider into another provider’s client configuration without confirming it. Likewise, model-card compatibility with an API format does not prove that a particular hosted service exposes terminal execution: that capability may require a separate tool harness or implementation.
No Token Forge price or billing unit for GPT-OSS 120B is established here. Before comparing hosted API costs, get the provider’s current price, currency, billing unit, and relevant plan differences. Compare hosted usage with self-hosting only after accounting for the infrastructure and operating costs involved; the checkpoint-size figures alone do not determine the cheaper option.
What to test before choosing it for a coding workflow
Run a small, representative evaluation through the provider and setup you actually expect to deploy:
- Select real tasks from your workflow—for example, bug fixes, feature changes, or test-writing tasks from representative repositories.
- Use the intended reasoning level and tool setup. If production will let the model run tests or execute commands, include those tools; if not, test without them too.
- Define success before running the comparison. For repository work, that might include passing relevant tests, meeting review criteria, and avoiding unwanted changes.
- Compare 120B with 20B and any alternatives under the same task and tool conditions.
- Record task success, latency, and actual usage cost. Repeat enough tasks to avoid making a decision from one unusually easy or difficult example.
This is a recommended evaluation plan, not a test result reported by OpenAI, Modular, or Token Forge Cloud. It addresses what the published benchmarks cannot answer: how the model performs with your repositories, provider, tools, and operating constraints.
Bottom line
OpenAI’s reported results make GPT-OSS 120B a credible coding candidate to test—particularly at high reasoning effort and when terminal tools are part of the workflow. GPT-OSS 20B remains worth testing when model size matters, and it slightly leads on the reported medium-reasoning SWE-Bench Verified score. Choose based on matched tests in your environment, and verify API availability and configuration with your intended hosting provider before building around it. OpenAI itself says gpt-oss is not served through its API. [5]
Sources
[1] OpenAI, gpt-oss-120b & gpt-oss-20b Model Card (August 5, 2025).
[2] Modular, GPT-OSS 120B: OpenAI’s Open-Weight Reasoner on Modular.
[3] OpenAI, gpt-oss-120b Model API documentation.
[4] Token Forge Cloud homepage.
[5] OpenAI Help Center, OpenAI open-weight models (gpt-oss).
Sources
gpt-oss-120b & gpt-oss-20b Model Card
GPT-OSS 120B: OpenAI's Open-Weight Reasoner on Modular
gpt-oss-120b Model | OpenAI API