Keep a Voice Agent Useful While Its Tools Are Running

A voice agent stays useful during a slow tool call by saying what it knows, accepting new information and keeping its spoken updates aligned with the task that is still active. It should not fill every pause with narration, and it should not announce success merely because a backend request has started.

GPT-Live’s separate voice and backend layers make this interaction possible, but the application still decides how corrections, cancellations and late results affect the work. This guide proposes an application pattern for that boundary. The examples are design examples, not results from a deployed voice system.

Useful progress tells the caller something that changes their understanding: the request was accepted, the system needs a detail, the lookup remains pending, or an action has completed. Repeating “still working” on a timer can make a slow interaction more intrusive without making it more informative.

A hypothetical travel assistant can explain a baggage rule while a separate lookup checks availability. It should say that availability is pending and avoid presenting the general rule as a confirmed offer. The two subjects can coexist in conversation if their evidence remains separate.

The GPT-Live guide describes delegation as work sent to a backend while conversation continues. That ability changes the user experience, but it does not decide which updates deserve speech. Make the speaking policy part of the product requirements.

Keep the application’s task state explicit. Pending means work has been requested. Running means execution has begun. Completed means the chosen acceptance condition is met. Failed and uncertain need different handling: a confirmed rejection can be explained, while a missing acknowledgment may require reconciliation.

When multiple tools contribute to a result, report the task status rather than narrating each internal call. If one tool found availability and another has not created the booking, the task remains incomplete. The caller needs that distinction more than the name of the function being executed.

The delegation documentation distinguishes instructions, quiet context and spoken commentary. Instructions steer the live model. Quiet context can inform later speech without being spoken on append. Commentary supplies material the model should speak, with paraphrasing.

Use quiet context for background facts the caller may ask about; use commentary for a useful result or necessary status update. Quiet context is not a secret storage channel: it can influence what the model says later. Keep sensitive execution details and full tool payloads in the backend.

For client delegation, preserve the original delegation identifier on task updates and use the appropriate session-wide form for general context. Keep a separate application operation ID and task revision. The provider’s delegation ID associates a conversation request; it is not the authority for deciding whether an external action may run.

Suppose the caller changes a request from an afternoon departure to a morning departure while a lookup is running. Update the application’s active task revision immediately. When the afternoon result arrives, compare its revision with the current one before offering it to the voice layer.

For a read-only lookup, discarding the stale result may be enough. For a write, such as creating a reservation, the application must find out whether the old action was already executed. Cancelling a future step and reversing an existing action are different operations, with different consequences.

The provider guidance explicitly says that spoken interruption does not automatically cancel backend work. Do not rely on a prompt that says “stop” to enforce an action block. Apply the block in application state and verify any requested cancellation. Our safe agent stopping guide covers the broader shutdown problem.

The session documentation says append acknowledgments are not evidence of completed speech or playback. A result can be accepted into context and still be interrupted before the caller hears it. Track delivery separately when the confirmation matters.

If the caller asks again, query or retrieve the known result rather than repeating the action that produced it. If they ask whether the booking exists, answer from the authoritative record. If they ask to change it, create a new authorized task tied to the existing record.

Use the voice latency worksheet to distinguish backend completion, result submission and client playback. This makes it possible to diagnose whether the delay is in execution, result selection or delivery.

Test a correction before tool dispatch, during a lookup, after an external action and during its spoken confirmation. Include a connection loss and a duplicated result event. Check the final record and the audio, not only the model’s transcript.

Closing the voice session needs its own policy. The documented close flow finalizes voice events and usage, while application-managed client work requires a separate finish-or-cancel decision. Do not leave a backend action ownerless because the caller has hung up.

The GPT-Live API decision guide helps choose the delegation path. For a production pilot, our AI transformation service can help make the task states, speaking policy and acceptance checks part of one implementation brief.