When Your AI Agent Should Ask You Instead of Guessing

An AI agent should ask a question when missing information could materially change the action or its consequences. It should usually proceed when a reversible choice has a sensible default and the user’s intent is already clear. The hard part is separating uncertainty about facts, preferences and authority; asking the same vague question for all three creates friction without improving the result, and never asking turns every gap into a guess the user discovers later.

A confidence percentage does not solve that design problem by itself. Build an explicit decision rubric around what is missing, what can be checked independently and what happens if the assumption is wrong. The result should be fewer unnecessary interruptions and clearer questions when a person’s answer matters. This guide sets out the rubric, explains why each row is where it is, shows the shape of a question that actually resolves something, and describes how to measure whether the policy is working once it ships.

The following is a proposed product policy. It is not a measured optimum. Apply it to the concrete action the agent is about to take, rather than to the conversation as a whole: an agent can be confident about the task and still be about to send the wrong customer an invoice. Each row pairs a situation with the response and with what the user should see, because the visible output is how a person tells a good question from a reflexive one.

Read the rows as an order of preference. Proceeding is cheapest and right whenever the choice is reversible and has a default. Checking is next: it costs a tool call and can often resolve a factual gap without a human. Asking is reserved for the two things no tool can settle, a preference and a permission. Stopping is for the case where the agent has discovered the task cannot be done the way it was asked.

For a factual gap, use available authorized evidence. For a consequential preference, ask what outcome the person wants. For missing authority, prepare the exact action for review. A confidence score cannot settle all of these questions, because they are not the same kind of uncertainty. The agent can become more certain about a fact by looking it up. It cannot become more certain about what the user prefers by thinking harder, and it cannot grant itself permission by being confident.

This is why a single “are you sure?” threshold produces both too many questions and too few. It fires on factual gaps the agent could have closed with a lookup, and it stays silent on an authority gap where the agent is very confident and still not allowed. Classify the gap first; the response follows from the class.

Just Ask for Calibration studied verbalized confidence in question-answering settings. Treat confidence as a signal to evaluate on your workflow, not permission to act. The paper’s setting is not your setting: a model that is well calibrated on question-answering benchmarks such as TriviaQA, SciQ and TruthfulQA has told you little about how calibrated it is when deciding which of two customer records to update.

An August 2026 paper on certified deferral in small language models studies deferral under particular statistical assumptions. Test any proposed threshold against your own consequential errors and review workload. The paper’s risk certificate rests on an i.i.d. deployment assumption, which is the kind of assumption a production workflow can break: a shifting input distribution, a new product line, a prompt someone edited on Tuesday.

For a local test, retain the model’s proposed answer, confidence and outcome label on representative cases. Compare cases it would execute with cases it would defer. If the confident failures are the expensive ones, an attractive average accuracy does not rescue the policy. Report the types of failure and the resulting review workload. An illustrative result: a threshold that defers one case in five and catches every wrong-recipient error is a good policy even if it also defers some cases that would have been fine; one that defers one case in fifty and lets a wrong recipient through is not, whatever its accuracy.

“There are two customer records with this name; which one should receive the update?” resolves a decision. “Should I continue?” usually leaves the underlying ambiguity untouched: the person says yes, the agent still has to pick a record, and now the guess has a signature on it. A good question names the specific missing decision, offers the options the agent actually found, and says what changes depending on the answer.

The test for a question is whether the person can answer it without opening anything. If answering requires them to go and look at the two records, the agent should have shown the distinguishing fields. If answering requires them to understand the agent’s plan, the agent should have stated the plan in one line. Questions that pass this test are quick to answer; questions that fail it are easy to set aside, which is one way pending work turns into abandoned work.

Keep one question per message where possible. A list of five open points reads as a status report and gets a status-report reply. Where several decisions genuinely block the same step, group them under the step they unblock so the person sees why they are being asked together.

Partition the task by dependency. An unanswered audience question may block the final introduction while leaving source collection and fact-checking available. An ambiguous destination may block sending while leaving the draft reviewable. Continue those independent steps so the user’s answer unlocks a prepared result rather than restarting a stalled one.

Do not treat silence as approval. A timeout can select a default only when the workflow explicitly permits that default for a reversible choice. It cannot grant missing authority for an external action. Store the pending question and its dependent action together so a later response is applied to the right decision; an answer that arrives a day later should resume the exact step it blocked, not be interpreted against whatever the agent is doing now.

Keep a record of recurring questions. If the agent repeatedly asks about the same documented preference, the problem may be retrieval or instruction design. If different users regularly choose different answers, the product may need a persistent setting rather than a longer prompt. This builds on approval-gate design and where agents read their instructions. The question policy should use those sources of authority, not compete with them, and the handoff of a half-finished task to a waiting person is covered in who owns unfinished work.

The policy fails in opposite directions, and it is easy to notice only one. The visible failure is over-asking: an agent that confirms every step trains its users to click yes without reading, which quietly converts every question into a guess with a human signature. The invisible failure is under-asking: an agent that proceeds on an assumption nobody would have agreed to, and produces work that looks complete until a customer replies.

An illustrative under-ask: a drafting agent is told to “send the renewal reminder to the account owner”. Two contacts on the account carry owner-like titles. The agent picks the one with the more recent login, sends, and reports success. Nothing in its output looked uncertain. The rubric’s third row exists for exactly this case, and the fix is not a higher confidence threshold; it is a rule that two plausible recipients is always an ask.

An illustrative over-ask: a reporting agent asks whether to use the team’s usual date format, every time, on every report, although the format is documented in its instructions. Each question is harmless; the accumulation is a tool nobody wants to use. The fix is the recurring-questions log from the previous section, which points at the retrieval problem rather than at the user.

Review a representative set of completed tasks and classify each interruption: necessary, avoidable or unresolved. Look at the answer the user supplied and whether it changed the outcome. Separately record cases where the agent proceeded under an assumption that should have been clarified. Those two lists are the policy’s two error rates, and a dashboard that shows only one is hiding the other.

A low interruption count can hide reckless assumptions. A high completion count can hide work that nobody wanted. Choose acceptance examples that expose both errors: a harmless formatting ambiguity the agent should handle, and a consequential object ambiguity it must resolve. If the agent asks about the first or proceeds on the second, the policy has failed in a way a pass rate would not show.

When changing models or prompts, rerun those cases. Our model-switch testing guide explains the broader discipline of using representative work. For question behavior, the labels should include whether the agent asked the right question at the right point, not just whether its final prose was correct. Teams that want this instrumented rather than reviewed by hand can work with our AI transformation practice to build the log and the acceptance set into the workflow itself.