An API becomes unusable when it fails to deliver enough valid responses within the application's time and traffic requirements. That failure may appear as explicit rate limits, long queues, timeouts, broken streams, or unstable performance. None is visible in the price per million tokens.
Take DeepSeek as an example: the same model can be offered at an attractive price by several providers, while the usable capacity of those services differs substantially.
1. Recognize the Signals Hidden by a Low Price
The most obvious warning is a low RPM or concurrency limit. The endpoint may work perfectly in a demo but cannot support the expected number of users. Capacity available only through a manual increase should not be assumed until it is confirmed and tested.
Queueing is less visible. Requests continue returning 200 responses, but TTFT p95 rises sharply as traffic increases. A provider may preserve a high success rate by making users wait. For a background job this may be acceptable; for interactive chat it can make the service functionally unavailable.
Aggressive rejection creates the opposite profile. Accepted requests stay fast, but 429 responses increase under load. A dashboard that reports latency only for successful completions can make this endpoint look excellent unless rejection rate is shown beside it.
Other warning signs include:
- Large gaps between TTFT p50 and p95
- Performance that changes dramatically across repeated runs
- Missing request IDs or unclear error categories
- Token-speed claims without a measurement definition
- Capacity claims without prompt lengths, output lengths, or test duration
- Pricing that excludes the cost of retries and fallback calls
No single signal proves that a provider is unreliable. Together, they show where a controlled test is needed.
2. Increase Load Until the Service Reveals Its Boundary
Run a step test with representative prompts. Begin below expected production traffic, then increase RPM while keeping request parameters and pacing fixed. Continue sending at the target rate even when responses slow down.
At every level, collect offered and completed RPM, input and output TPM, active concurrency, TTFT p50 and p95, end-to-end latency, 429s, 5xx responses, timeouts, and incomplete streams. Validate the returned content separately.
Define “usable” before looking at the result. For an interactive endpoint, the rule might require at least 99% completed requests and TTFT p95 below a product-specific threshold. For a batch pipeline, the rule might focus on completed jobs per hour and a maximum deadline.
The critical point is the highest level where all requirements remain true. Do not describe the first unstable level as usable capacity merely because some requests succeeded. Also avoid converting a short burst into a sustained-capacity claim.
Repeat the test at different times and add a longer soak run. Cheap capacity that exists only during one quiet window cannot support a predictable production system.
3. Convert the Test Into a Capacity-Adjusted Cost
Token Forge Cloud used fixed pacing and a combined stability rule in its DeepSeek test. For approximately 100 input tokens and 140 output tokens per request, the observed stable baselines were:
| Model | Highest stable tested load | Success rate | TTFT p95 | Token throughput |
|---|---|---|---|---|
deepseek-v4.1-flash |
5,000 RPM | 100% | 1.97 s | >1.21M TPM |
deepseek-v4-pro |
2,500 RPM | 100% | 2.82 s | >617K TPM |
kimi-k3 |
5,000 RPM | 100% | 1.26 s | ~5M TPM |
glm-5.3 |
4,200 RPM | 99.95% | 5.62 s | ~5.4M TPM |
glm-5.3-flash |
2,400 RPM | 100% | 5.29 s | ~3.0M TPM |
The Kimi K3 and GLM rows come from separate tests, the Kimi K3 load test report and the GLM load test report: about 1,000 input and up to 300 output tokens per request, Kimi K3 with reasoning off, and GLM at low reasoning (it cannot be turned off). GLM is held to a TTFT p95 of 8 seconds rather than 5.
The value of the table is not the peak number alone. Each RPM result is attached to a success rate, a tail-latency range, a token workload, and defined test conditions.
Use the same structure for cost. Calculate total spend across original calls, retries, and fallback requests, then divide by valid results delivered within the required time. Add the operational cost of maintaining multiple routes if one provider cannot cover the workload.
A very cheap endpoint may still be useful for delay-tolerant overflow or offline work. The problem is not low price. The problem is treating unverified capacity as interchangeable with production capacity.
The practical test is simple: can the provider repeatedly complete the required DeepSeek workload, at the required rate, within the required latency, for an acceptable effective cost? If any part of that sentence is missing, the price has not yet been proven useful.