How Should Failed Requests Be Classified When a Provider Reports Nonzero Usage Despite Returning an Error?

When an API returns an error and delivers no contractually valid result, classify the client outcome as failed. Record nonzero consumption separately as execution or usage reported, and leave billing unconfirmed until provider billing records establish whether a charge applies. A practical state is outcome=failed, execution=usage_reported, and billing=unconfirmed; field names may vary.

Classify the Outcome as Failed, the Execution as Usage-Reported, and Billing as Unconfirmed

An API response can contain several facts that appear contradictory but describe different parts of the request lifecycle. The client may have received an error, the provider may have performed some computation, and billing may not yet be known.

The safest normalization model preserves those facts independently:

Dimension	Recommended state	Meaning
Client outcome	failed	No valid result was delivered under the API contract.
Execution or consumption	usage_reported	The provider returned one or more nonzero usage values.
Billing	unconfirmed	Usage telemetry exists, but a charge has not been established by billing records.
Response validity	invalid or not_delivered	The application cannot treat the response as a completed result.
Retryability	evaluate	Another attempt requires an error-specific policy decision.

Nonzero usage means only that the provider reported some consumption. It does not independently prove that the request completed successfully, produced usable output, or became a billable event. It also does not establish that the reported amount will precisely match an invoice.

The reverse assumption is equally risky: an error does not prove that no provider-side computation occurred. A provider may process input, begin generation, or complete work before a gateway, transport, policy, or response-delivery failure becomes visible to the client.

Why a Single Success-or-Failure Status Produces Misleading Records

A binary status forces several operational questions into one field:

• Did the application receive a valid result?
• Did the provider report input, output, cached, or other usage?
• Is the attempt expected to be charged?
• Is the response safe to use?
• Should the system retry, fall back, or stop?

For example, marking the request simply as failed accurately describes the client outcome but can hide reported consumption. Marking it as successful because token usage is nonzero incorrectly implies that a usable result was delivered. A compound label such as failed_with_usage is better, but it still does not show whether billing is confirmed or another attempt is appropriate.

This distinction matters for different teams. Reliability teams need to understand delivery and retry state. Product teams need to know whether the user received an acceptable result. FinOps teams need to track potential consumption without prematurely recognizing it as invoiced cost. Platform teams need attempt-level records when a logical request crosses multiple providers or models.

A multidimensional record lets each team answer its question without changing the historical facts. Derived rollups can still simplify dashboards—for example, “failed requests with reported usage”—while the underlying event retains the separate states.

Implement Separate Fields for Outcome, Execution, Billing, and Retryability

A normalized event should represent one provider attempt, not only the overall logical request. This becomes important when a timeout triggers a retry or fallback: one user action may create two or more provider calls, each with its own outcome and usage report.

The following is an illustrative implementation example, not a provider standard or Token Forge Cloud event format:

{
  "logical_request_id": "req-internal-7842",
  "attempt_id": "attempt-2",
  "provider": "provider-a",
  "model": "model-version-x",
  "provider_request_id": "provider-request-123",
  "outcome": "failed",
  "execution": "usage_reported",
  "billing": "unconfirmed",
  "response_validity": "not_delivered",
  "retryability": "manual_or_policy_evaluation",
  "http_status": 500,
  "provider_error_code": "upstream_error",
  "usage_reported": {
    "input_units": 1200,
    "output_units": 85
  },
  "idempotency_key": "logical-operation-456",
  "started_at": "2026-09-25T10:15:20Z",
  "ended_at": "2026-09-25T10:15:24Z"
}

Useful design principles include:

1. Keep observed and derived fields distinct. HTTP status, provider error code, request ID, and raw usage values are observations. outcome, retryability, and normalized billing state are internal interpretations.
2. Preserve raw provider values. Normalize them for reporting, but retain the original error and usage payload where operational and data-handling policies permit.
3. Store one record per attempt. Do not overwrite the failed first attempt when a retry succeeds.
4. Link attempts to one logical request. This supports user-outcome reporting while preserving the cost and reliability implications of every provider call.
5. Allow states to change without rewriting history. Billing may move from unconfirmed to charged, not_charged, or adjusted after later records arrive. Store the transition and its source rather than replacing the original observation without traceability.

The vocabulary can be adapted to an organization’s data model. The important point is that delivery, reported consumption, billing, response validity, and retryability should not be inferred from one another.

Capture the Evidence Needed to Interpret the Error and Reported Usage

Preserve the request evidence bundle as soon as the attempt completes. Provider dashboards, usage exports, and billing records may update on different schedules, while short-lived application logs may be rotated before a discrepancy is investigated.

At a minimum, capture the fields that are available from the provider and your own gateway or application:

• Provider and model identifier, including the model version when available
• Internal logical request ID and attempt ID
• Provider request or trace ID
• HTTP or transport status
• Provider error type, code, message, and relevant headers
• Raw usage fields and their units
• Request start, first-byte, failure, and completion timestamps where available
• Whether a complete, valid, partial, malformed, or no response was delivered
• Streaming state, including whether any output reached the application or end user
• Retry, fallback, and circuit-breaker decisions
• Idempotency key or equivalent operation identifier
• Route, region, endpoint, and account context when relevant

Not every provider exposes every field, and similarly named fields may have different meanings. Preserve the provider’s original representation alongside your normalized values so future investigations can account for those differences.

Response validity should be evaluated independently from the transport status. A body accompanying an error might contain diagnostic text rather than model output. A partially streamed response might be technically parseable but unusable for the application. Conversely, an API contract may define a narrow case in which a response remains valid despite a nonstandard status. The application’s contract—not nonzero usage alone—should determine whether the result was delivered successfully.

Base Retry and Fallback Decisions on Error Semantics, Validity, and Idempotency

Do not use reported usage as the sole retry signal. A retry may create additional usage, repeat a side effect, or duplicate work that the provider completed even though the client did not receive the result.

Use a policy that considers the provider’s documented error semantics, response validity, application behavior, idempotency safeguards, and the uncertainty surrounding provider-side completion.

Situation	Operational interpretation	Retry or fallback considerations
Authentication or authorization error	The request was rejected or access was not accepted.	Correct credentials or policy first; repeated immediate attempts are generally unhelpful unless provider guidance says otherwise.
Validation or unsupported-parameter error	The submitted request did not meet the endpoint contract.	Fix the request rather than repeating it unchanged. Preserve any reported usage without assuming it will be billed.
Rate limit or transient service error	Capacity or service availability may have interrupted processing.	Follow provider guidance, backoff instructions, and application deadlines. Another attempt may report additional usage.
Timeout or connection interruption	The client does not know whether processing stopped, continued, or completed.	Treat completion as uncertain. Check request IDs, idempotency behavior, and side effects before retrying.
Partial or malformed response	Some output may have been generated, but the result failed application validation.	Decide whether partial output is usable under the application contract; otherwise classify the outcome as failed and evaluate retry cost and duplication risk.
Error with a complete-looking body	Transport status and payload appear inconsistent.	Validate the payload against the API contract. Do not promote the attempt to successful solely because content or usage is present.

No error category is universally safe to retry across providers. Even familiar HTTP codes can have provider-specific meanings, and streaming or asynchronous APIs introduce additional completion states.

For operations that can produce external side effects—such as tool calls, transactions, notifications, or workflow state changes—idempotency deserves particular attention. A new model request may duplicate the model computation, while replaying downstream actions can duplicate business work. Use a stable logical operation identifier and application-level safeguards where appropriate, but do not assume that an idempotency key controls provider billing unless the provider explicitly documents that behavior.

Reconcile Request Logs, Provider Usage Telemetry, and Billing Exports

Initial classification should remain provisional where billing is concerned. Request logs describe what the application observed. Provider usage telemetry describes what the provider reported. Billing exports or invoices describe what the provider ultimately recognized for charging. These records can represent different stages and may appear at different times.

A practical reconciliation workflow is:

1. Create the attempt record immediately. Record outcome=failed, execution=usage_reported, and billing=unconfirmed when those facts apply.
2. Match provider telemetry. Use the provider request ID, account, model, route, and timestamp window. Avoid matching only by token quantity because different attempts may report identical usage.
3. Compare later billing records. Determine whether the attempt appears as a charge, is excluded, is aggregated in a way that requires allocation, or remains unresolved.
4. Update the billing dimension. Move the record to an internally defined state such as charged, not_charged, or adjusted, retaining the source and time of the update.
5. Preserve unresolved exceptions. Do not force unmatched records into success, zero cost, or confirmed billing merely to close a reporting period.

Reconcile at both attempt and logical-request levels. Attempt-level accounting shows the potential cost of retries and fallbacks. Logical-request reporting shows whether the user ultimately received a valid result. Together, these views help teams distinguish user success rate from provider-call volume and invoiced consumption.

Token Forge Cloud Managed Model APIs offer an API-first path to model access and usage data. Provider-specific error fields should still be interpreted using the applicable provider documentation, and operational telemetry should be compared with billing records rather than treated as final invoice evidence.

Use Centralized Routing and Telemetry to Apply the Classification Consistently

As organizations add models and providers, normalization becomes a serving-layer concern rather than an isolated application fix. Different APIs can report errors, usage, streaming termination, and request identifiers in different formats. A centralized routing and telemetry layer gives teams a common operational point at which to apply their own classification vocabulary and preserve attempt-level context.

Token Forge Cloud supports enterprise LLM operations through model routing, policy-aware access, usage data, and telemetry under enterprise control. Token Forge Cloud Private LLM Inference focuses on private deployment and serving-layer optimization, including caching, routing, batching, quantization, and GPU scheduling. Token Forge Cloud Managed Model APIs offer an API-first entry point for teams validating model demand before considering private deployment.

For the classification pattern in this guide, the practical objective is consistency across model access paths: preserve the raw provider response, normalize operational dimensions separately, and maintain enough context to evaluate retries and later cost records. Classification rules should remain aligned with each provider’s documented semantics and the application’s definition of a valid result.

Centralization does not make provider behavior uniform, and it should not be treated as proof of billing status or retry safety. It can, however, help platform, reliability, product, and FinOps teams work from a shared event model while retaining provider-specific detail.