A second AI provider can help when the first cannot answer. It cannot tell you whether the first agent already published a page, sent a message or changed a record before the connection failed. That distinction is the centre of a reliable failover design: repeat a safe inference step when the route qualifies, but reconcile uncertain external effects before restarting work.
OpenAI's October guidance on provider failover makes the same boundary explicit. A self-contained read-only request can sometimes move to another approved route; an ongoing agent session does not automatically transfer its conversation, saved work or prior actions. A matching model name is therefore only one part of a recovery plan.
This article develops an illustrative campaign-publishing workflow and a proposed test matrix. It does not report a Digital Applied outage or a measured reliability improvement. The goal is to give application owners an acceptance contract they can test before an outage makes the decision for them.
- 01Separate inference from external actions.Generating another draft can be safe when the first result was unused. Repeating a publication requires knowledge of the destination state.
- 02A status code is not the whole decision.Temporary throttling, exhausted spend limits and access failures need different responses.
- 03Only one result should win.Late primary responses must not trigger another action after a secondary result has already been accepted.
- 04Recovery belongs to the workflow.The router can enforce a policy, but the application must define safe replay and reconcile its own side effects.
01 — Lost acknowledgementThe page may already be live
Imagine an agent preparing a campaign page. It gathers approved material, drafts the copy, requests review and publishes the accepted version. During drafting, an inference request times out before any answer is used. If the alternate route has the required capabilities and controls, regenerating the draft may be a reasonable recovery action. The workflow has not yet changed an external system.
Now move the timeout to the publication step. The content platform accepts the page, but the response never reaches the agent. Replaying the whole workflow could create a second page, repeat a notification or overwrite a later edit. The absence of an acknowledgement is not proof that nothing happened. Read back the destination and the application's operation record before deciding what remains unfinished.
This is an illustrative design problem, not a claim about a particular platform. The same pattern occurs when an agent submits a form, sends an email or updates a database. A retry policy needs to know where the workflow is in that sequence. Treating every timeout as a fresh start can turn a recoverable communication problem into a duplicate business action.
Draft generation
A result that was never consumed may be regenerated through an approved route, within the task deadline.
Publication request
A missing response leaves the result uncertain. Check the destination before creating or changing anything again.
02 — Two changesModel fallback and provider failover differ
Model fallback changes which model serves the request. Provider failover changes where the request is served. Both can happen together, but they introduce different questions. Another model may differ in tool use or answer quality; another provider may differ in API support, region, capacity, logging or available hosted tools. A familiar model name does not remove those differences.
OpenAI's provider-failover guidance recommends validating the actual route against the workload. A workflow that depends on web search cannot silently lose that capability during recovery. A route that can return ordinary text is not necessarily able to continue a hosted agent session or access the same files. These are contract checks, not details to discover after the primary fails.
Write the fallback requirement in terms of the operation. “Generate a draft from these supplied sources without external tools” is much narrower than “continue the agent.” The first can have a clear input, output and acceptance test. The second may depend on provider-managed state and actions already taken. Start with the narrow operation and expand only where recovery semantics are understood. A smaller reliable failover surface is more useful than a broad route that works only in the easiest case.
| Change | Validation needed |
|---|---|
| Different model | Quality, schema, tools and task-specific behaviour |
| Different provider | Endpoint, region, controls, capacity and supported features |
| Different session | Saved work, prior actions and recovery context |
| Different network path | Approved connectivity, identity and destination restrictions |
Operational comparison based on first-party failover guidance; the table is a design checklist, not a measured provider census.
03 — ClassificationDo not turn every error into permission to retry
Temporary service failures can justify another attempt, but only after replay safety and route eligibility are established. An authentication failure usually calls for fixing credentials or access. A safety refusal calls for handling the refusal. An exhausted allowance calls for a budget decision. Automatically changing routes in those cases can defeat the control that produced the error.
The OpenAI error documentation distinguishes temporary throttling from credit or spend-limit problems, even though both can appear within rate-related HTTP responses. Inspect the provider's error details rather than routing solely on the status code. Keep unknown errors conservative: an unclassified failure is a reason to diagnose or defer, not evidence that a second provider is allowed to continue.
For the campaign example, a malformed tool argument should be corrected before another publication attempt. A temporary model-service outage during drafting may qualify for recovery. A missing permission to publish should not cause the agent to try a different account. The policy needs to preserve the application's authority and budget boundaries as carefully as its availability target. Reliability that bypasses those boundaries is not successful recovery.
Classify the provider error, then check the operation’s replay safety. Neither a timeout nor a 429 response alone establishes that a secondary call is appropriate.
04 — Retry controlCount the whole operation, not one attempt
Retries can multiply when the client, SDK, gateway and workflow each believe they own recovery. An application configured for a few attempts may make many more network calls once nested retry layers are included. The useful limit is the total logical operation: how many attempts can occur, how much time remains and how much spend is permitted across every route.
OpenAI's rate-limit guidance notes that unsuccessful requests can still consume per-minute capacity. Immediate repeated calls can therefore worsen the condition they are intended to solve. Respect valid retry timing, use bounded backoff and make the remaining deadline visible to the component that controls retries. A per-attempt timeout is not a whole-workflow deadline.
In a publishing workflow, waiting may be preferable to switching providers if the page is not urgently required. In a time-sensitive application, there may be no useful result after the deadline. Define that outcome in advance: queue, return a reduced result, hand off or stop. Do not let an agent keep working indefinitely simply because each individual request remains within its own timeout. The completion criterion belongs to the user’s task.
Choose one layer to control total attempts and deadlines. Other layers may implement transport details, but their retries must fit inside the same budget.
05 — Recovery contextA failed turn can leave completed work
The Agents API recovery documentation warns that a failed turn may already have changed files or invoked external tools. A disconnected stream also does not establish that the turn itself failed. Recovery starts by checking the saved session, turn and work rather than interpreting the client connection as the complete source of truth.
For the campaign page, keep a logical operation identifier and record the intended destination and approved content version. After an uncertain publication, inspect whether that version exists and whether a later change has occurred. Continue only the unfinished work. The exact mechanism depends on the destination API; a universal cross-provider idempotency key is not something the cited guidance promises.
The HTTP standard's discussion of idempotency explains why automatic retries of non-idempotent operations need additional knowledge. In application terms, the team must know that repeating the operation has the intended effect, or that the original was not applied. Do not infer that from a POST request alone. Design and test the destination-specific behaviour, including the case where an acknowledgement is lost after success.
| Known state | Recovery direction |
|---|---|
| No result consumed; no external effect | Consider an eligible inference retry |
| Action confirmed complete | Record completion; do not repeat it |
| Action confirmed failed without effect | Retry only under the operation’s policy |
| Action outcome unknown | Inspect destination and operation record before replay |
Proposed state model for application recovery. It does not imply that every API exposes all four states directly.
06 — Race conditionsAccept one result and reject the late duplicate
A primary request can finish after the application has timed out and accepted a secondary response. If both answers remain connected to the next tool step, the workflow may publish twice even when each model call behaved correctly. The problem is result ownership: which response is allowed to advance the logical operation?
Make acceptance explicit. Once an eligible result is selected, a late response should be recorded for diagnosis but should not trigger another external action. Test this by deliberately delaying the primary until the secondary has completed. Inspect the destination and the action log, not only the number of successful model responses. The acceptance condition is one permitted business effect, not one or two fluent answers.
Cancellation helps only to the extent the provider and operation support it. A request to cancel is not proof that processing stopped or that a tool did not already run. The application still needs a rule for late results and uncertain actions. This is especially important when a model call and a tool execution are part of one hosted workflow, because the client may not control every intermediate step.
Delay the primary, accept the secondary, then release the primary response. Assert that the late result cannot cause a second publication, message or record change.
07 — Fault injectionTest the recovery policy before enabling it
A useful test suite checks when the secondary must remain silent as well as when it should be called. Build local stubs that return known errors, delay responses and simulate a lost acknowledgement. Keep external networking disabled for tests that might otherwise publish or send something. Synthetic values belong in fixtures; they are not recommended production limits.
The matrix below is a proposed acceptance checklist. It is not a dataset of incidents, a claim that twelve tests prove universal reliability, or a measured comparison between providers. Its value is that each row connects an injected condition to an observable outcome. Extend it for the tools, state and dependencies in your application.
Also test the shared services. Two model providers can still depend on the same retrieval index, identity system or gateway. Switching the model route will not repair a failure in a dependency both routes need. Finally, exercise a constrained secondary. A route that works under light test traffic may fail when it receives its normal workload plus the primary's recovery traffic. Capacity validation should use the approved environment and a bounded test plan.
| Injected condition | Required observation |
|---|---|
| Read-only inference times out | Secondary called only if eligible and within deadline |
| Primary replies late | Only one result advances the workflow |
| Publication acknowledgement lost | No blind duplicate write |
| Tool fails but assistant claims success | Workflow reports actual tool outcome |
| Retry delay exceeds deadline | Defer or stop under the defined policy |
| Spend limit exhausted | No automatic route bypass |
| 401/403 or refusal | No automatic alternate route |
| Required tool missing on secondary | No silent capability loss |
| Shared retrieval service fails | Failure is not misdiagnosed as model-only |
| SDK and gateway both retry | Total attempts stay within one budget |
| Failed turn has saved work | Completed work is reconciled before continuation |
| Unknown error class | No permissive fallback by default |
Digital Applied’s proposed fault-injection checklist, informed by the linked provider guidance. No live failure experiment was run for this article.
08 — OwnershipMake recovery an owned product behaviour
The platform team may own the router, credentials, route health and traffic controls. The application owner still needs to define which operations are safe to repeat, what counts as a valid result and how to reconcile business effects. In a small team, one person may hold both responsibilities, but the decisions remain distinct.
Release failover in a bounded scope after the secondary has passed the workflow's quality, capability and control checks. Keep a manual disable path and a plan for restoring traffic gradually. A provider dashboard returning to green is useful evidence, but it does not prove that the exact model, region and tool combination used by the application is healthy. Use representative requests and the agreed acceptance criteria.
Report recovery in terms the business can understand: completed tasks before deadline, duplicate actions, review required and total cost. A high request-success rate can conceal repeated publications or unusable answers. The strongest failover system is not the one that always finds another route. It is the one that knows when another route can safely complete the task and when stopping protects the work already done.
Enable a secondary for a named operation with tested recovery semantics. Keep broader session recovery disabled until its state and side effects have an explicit, verified design.
Continue with cloud model availability, rollback and checkpoints, agent session design. For implementation support, explore our AI transformation services.
Recover the unfinished operation, not the entire story.
A second provider is useful when it can safely supply the missing result. Keep error classification, deadlines, result ownership and destination state in the same recovery design. The point is to finish the task once, with the intended permissions and a record that proves what happened.