The Hidden Cost of AI Agents Waiting for Their Next Turn

An AI agent can spend an hour attached to a customer session while doing only a few minutes of work. If its execution environment reserves memory and compute for that entire hour, a cheap model call can sit inside an expensive service. The question is how much capacity the session occupies while waiting, and whether releasing that capacity is worth the cost of restoring it.

Google's November 5 announcement of Agent Substrate 1.0 brings that question into focus. It describes separating an agent's persistent state from the worker executing it, allowing idle sessions to suspend and later resume on available workers. This is a useful architecture to evaluate, but a vendor's density claim cannot establish the saving for your workload. Google's announcement

For a founder operating an agent service, the first useful deliverable is a capacity worksheet. It should distinguish session duration, active work, idle reservations, restore overhead and the bill that actually changes.

An agent service contains several resources with different lifetimes. A conversation record can remain in a database for months. A browser process might survive between turns. A container can retain a working directory, loaded packages and memory. The model itself may run in a completely separate provider's infrastructure.

Draw those resources before analysing cost. Otherwise, the word “agent” hides the difference between a stored session, an active program and an inference request. Suspending the program does not necessarily reduce token consumption. Keeping conversation history does not necessarily require keeping a browser or container running.

Consider a hypothetical research assistant that opens pages, collects notes and waits for a customer to approve the next step. The wait could be a useful product feature. Removing the approval step to increase utilisation would change the service's controls. Releasing execution capacity during that wait addresses a different problem and can preserve the intended experience.

For each resource, record its creation event, release event and owner. Identify whether an idle reservation prevents another task from using the resource. Include resources held indirectly: a browser connection, a database session or an allocated volume may survive after application code stops running.

In the design described in Google's announcement, an agent retains its identity and state while execution moves between reusable workers. The stated aim is to avoid dedicating a running environment to every waiting session. Google's reported performance belongs to its implementation and test conditions; it is not a measured result for your application. Agent Substrate announcement

Documentation checked on November 10 describes snapshots of memory and local files, restored onto a shared pool of warm workers. It also specifies deployment on GKE Standard clusters. Treat that as current implementation documentation, separately from the November 5 announcement. GKE architecture documentation

This separation introduces two questions. Can the application resume correctly from the saved state? Can the infrastructure reuse the released capacity economically? A successful answer to the first does not prove the second.

Start with a single session and keep the units visible. The following is an illustrative calculation, not a benchmark, provider quotation or Digital Applied operating result.

Assume a session lasts 60 minutes, requires six minutes of active execution and resumes 20 times. Assume each resume occupies a worker for an additional 0.5 seconds. The overhead value is a hypothetical input, not an adoption of Google's performance claim.

Occupied worker time is 360 + (20 × 0.5) = 370 seconds, or approximately 6.17 minutes. Relative to a continuously occupied 60-minute worker, the calculated occupancy fraction is 370 ÷ 3,600 = 10.28%.

That is an occupancy estimate. It is not evidence of an 89.72% reduction in total costs. The calculation excludes snapshot creation, storage, transfer, warm capacity, billing granularity, control services and model calls. It also assumes the worker can actually be released throughout every remaining interval.

Replace each input with observations from your service before using it for planning. In particular, distinguish an application reporting “idle” from the infrastructure having released its reservation. Those timestamps can differ materially.

Only after the capacity calculation should money enter the worksheet. Use the billing model of the actual service or infrastructure agreement. A per-second execution price, a minimum session charge and a reserved machine price can produce different outcomes from the same activity trace.

Separate at least four cost groups rather than reducing everything to a single token price.

Model inference remains a separate line unless the change demonstrably alters it. A resume strategy that causes the assistant to reconstruct context could increase calls even while reducing sandbox occupancy. Conversely, preserving state might avoid some repeated setup. Both are hypotheses to measure.

Use a comparison over a defined period: existing total cost minus proposed total cost, with one-time implementation work displayed separately. If future volume is uncertain, show low, expected and high usage cases rather than presenting a single confident payback date.

Also identify who receives the benefit. A platform may improve underlying resource utilisation while its customer still pays a flat monthly subscription. For an agency purchasing that platform, improved infrastructure efficiency becomes a cost saving only if pricing or required capacity changes.

Average active time can make an attractive spreadsheet while concealing an uncomfortable morning rush. Customers may respond after the same meeting, scheduled jobs may wake at the same minute, and a service recovery may release a backlog of triggers together.

For capacity planning, retain the event sequence rather than only the daily total. Calculate concurrent active sessions over short intervals, the number of restores requested together and how long requests wait for a worker. Include the time after a worker becomes available but before the application is ready to continue useful work.

A simple sensitivity table makes the first worksheet more useful. Keep the hypothetical 60-minute session and 0.5-second resume overhead, but vary the workload.

These values describe individual-session occupancy, not fleet size. A fleet must accommodate overlapping activity, required headroom and failures. Do not multiply the first row by a customer count and assume the resulting average is a sufficient capacity target.

Set a product-facing latency budget too. A half-second infrastructure operation may be acceptable within one workflow and frustrating when repeated across many visible interactions. Measure the customer's wait from request to useful continuation, not just the restore subsystem's timing.

Preserving local memory and files does not freeze the outside world. A network connection can close, a token can expire, a remote document can change and a customer's permission can be withdrawn while the agent is suspended.

Treat resumption as a point where external assumptions may need checking. The required checks depend on the action about to run. Reading an unchanged local draft has different consequences from retrying an external write whose earlier outcome is uncertain.

Create a small recovery matrix. For each external dependency, record whether it can be recreated, revalidated or safely retried. A connection that can be reopened is simpler than a partially completed transaction with no reliable status lookup. Identify those cases before using suspension in the middle of consequential work.

For customer actions, preserve an operation identifier and outcome record wherever the integration supports them. A resumed process should not infer “nothing happened” merely because it did not receive the earlier response. This is an application correctness issue, not a property guaranteed by saving memory.

The runtime dependency reference is useful for identifying these external assumptions. Include secrets, identity and access lifetime in the same inventory, and define how expired sessions are cleaned up without losing required records.

Choose one workflow with representative idle periods and explicit acceptance conditions. Establish the existing behaviour first, including useful completions, occupied capacity, customer-visible latency and recovery failures. Keep model choice and task selection stable enough to interpret the comparison.

Then test a suspension policy with a clear trigger. It might suspend only after an approval request, or after a measured period with no active tool work. Avoid an aggressive timer that repeatedly suspends an environment between closely spaced operations unless that is deliberately part of the experiment.

Track both successful and unsuccessful sessions. Removing failed resumes from the denominator would make cost per completion look better than the customer's experience. Record retries and reconstruction work against the session that caused them.

Our agent observability guide explains the broader role of traces and evaluations. For this trial, a compact event record is enough: session start, active work, suspend request, capacity release, resume request, ready state and completion outcome.

Define the decision rule before reading the result. For example, require no unexplained duplicate actions, an acceptable tail of user waiting time and a positive cost difference under realistic demand. The thresholds should come from your product requirements and operating budget, not an arbitrary universal target.

Suspension is one option among several. A short-lived task may be able to persist its progress explicitly and start a fresh process on the next event. A managed sandbox provider may already offer suitable lifecycle controls. A long-lived interactive environment may need richer state preservation. The workload determines which option deserves investigation.

For a small service, the largest avoidable cost may be an environment left running overnight rather than fine-grained inefficiency between turns. A session expiry rule and reliable cleanup could address that problem with much less implementation work.

For a larger service with many waiting sessions and scarce memory, reusable workers may be more valuable. The case becomes stronger when state reconstruction is expensive, idle intervals are substantial and the team already operates the required platform. None of those conditions means every agency should run Kubernetes.

Assign an owner to the cost comparison after rollout. Usage can shift: customers may run longer tasks, concurrency can rise and snapshots may grow as working directories accumulate files. The original business case should be revisited when those inputs change, rather than treated as a permanent property of the architecture.