Your AI Agent Needs an Unknown State
Cover diagram by the author: a completed external effect with a lost response leaves the executor un 2026-9-25 11:25:54 Author: hackernoon.com(查看原文) 阅读量:0 收藏

Cover diagram by the author: a completed external effect with a lost response leaves the executor uncertain.

Your agent submits a command. The external service performs it. The response disappears somewhere between the service and your worker. A timeout handler marks the task as failed, and the agent tries again.

That retry can be the second successful execution of an operation your system believes has never happened. The model may understand the user's request perfectly. The approval may be valid. The failure sits in the executor's account of reality: it has confused a missing response with an absent effect.

For consequential tool calls, I would give the operation an explicit UNKNOWN outcome. It means execution may have happened, and another write requires evidence or a provider contract that makes repetition safe.

The state must change what the system can do next. Adding a yellow badge to the dashboard is insufficient.

Approval leaves an unanswered question

In Pharos Production's published article-authoring case, the agent prepared the article and populated the composer while publication remained a separate authority. That workflow separated the ability to prepare content from permission to create a public event. It is useful context for the next boundary: an authorized executor can still lose track of what happened after dispatch.

Suppose a human approves one version of a customer announcement. The executor checks the destination account, content and approval, then sends the publication request. The connection closes before a response arrives. Asking the human to approve the same announcement again does not resolve the first request. A second approval cannot tell you whether the audience already received it.

The missing fact lives outside the approval system. The remote platform may have committed the publication, queued it, rejected it or never received it. Each possibility calls for different behavior. A recovery process that starts by generating a new command has skipped the investigation that distinguishes them.

AWS states the underlying problem directly in Timeouts, retries, and backoff with jitter:

A timeout or failure doesn't necessarily mean that side effects haven't happened.

An agent does not remove that ambiguity.

Its ability to explain an error convincingly can make the ambiguity harder to notice, especially when the interface displays its narrative in the same place where operators expect a verified execution result.

Separate permission, dispatch and outcome

I would store three dimensions instead of one overloaded task status. Permission describes whether this exact operation may be attempted. Dispatch records whether an attempt crossed the local execution boundary. Outcome describes what the available evidence establishes about the external effect.

A useful permission vocabulary includes pending, authorized, expired and revoked. Dispatch can be not started, started or completed. Outcome needs its own values: unobserved, accepted, succeeded, rejected and unknown. These are suggested names, not a requirement to build one enormous enumeration containing every combination.

ACCEPTED matters when a service queues work. It proves the service accepted a request under the meaning of its API response; it does not automatically prove that a public article exists or a downstream job finished. SUCCEEDED should name the relevant business effect and the evidence that establishes it. Otherwise the same green badge can mean both queued and completed.

REJECTED should require evidence that the operation did not take effect under the provider's contract. A local exception is not enough. Even an error response needs interpretation: a service might fail while processing a later step after creating the primary resource. The HTTP status alone cannot replace documented operation semantics.

UNKNOWN is therefore an outcome of observation, not a claim about what the provider did. Permission can remain authorized while outcome is unknown. That combination must not automatically produce another dispatch. In this design, retry eligibility is a separate decision with its own reason, evaluated outside the model.

Follow one operation through the failure

Consider a hypothetical platform that accepts an announcement through a non-idempotent endpoint. It offers no client request key and no immediate authoritative lookup. The approved intent is to publish one specific announcement from one specific account. A second identical publication would still be a second public event.

Before the call, the executor creates operation op-217, saves the approved payload fingerprint and reserves attempt attempt-1. It durably records dispatch started before calling the platform. At the remote end, the platform creates a post, but the worker receives a timeout. The executor records the timeout as evidence and sets the outcome to unknown. It does not generate attempt-2.

The conversational agent receives a constrained result. The operation was attempted, publication is unverified and automatic resubmission is blocked. It can explain that result and show the operation reference. It cannot convert the timeout into a new publication request merely because the user originally wanted the announcement online.

An operator later finds the post in an authoritative account view and verifies its identity and content. The operation can then become succeeded with that evidence attached. If the operator finds nothing in a search index, the operation remains unresolved: absence from an index may reflect delayed indexing rather than absence of publication.

There is a deliberately uncomfortable interval here. The announcement might be live while the system reports uncertainty. I would preserve that uncertainty until evidence resolves it. Reporting a false failure invites duplication. An unsupported success gives the user confidence the executor has not earned.

Make the dispatch record survive the worker

The durable operation record is the recovery entry point. Before dispatch, I would store the operation ID, account and destination, action type, canonical payload hash, approval reference, approval expiry, attempt ID and the applicable retry policy. Where a provider supports idempotency, the record also contains the provider key and the scope in which it is valid.

Use an atomic local transition to claim the authorized operation and record dispatch started. Only the worker that wins that transition may initiate the call. The record must survive a process crash.

A message in the model's conversation and an in-memory boolean do not provide that property.

This local transaction cannot make an unrelated external service participate in the same commit. A crash after the local record is written but before the network request leaves produces uncertainty even if nothing happened remotely. Recording dispatch after the call creates the opposite problem: a crash can erase local evidence of a completed external action. The design chooses conservative uncertainty and plans for it.

Queue redelivery should recover the existing operation.

It should not create a fresh intent from the original natural-language request. Otherwise a reliable queue can repeatedly deliver a command into an executor that repeatedly mistakes recovery for permission to act again.

Concurrent recovery workers need attention too. A local lease can reduce overlap, but an expired lease does not stop a paused worker from waking and sending its request. Fencing helps only where the execution path actually enforces the token. For an external service that cannot enforce it, local coordination alone is not a proof against duplicate effects; provider idempotency or a stricter one-attempt policy remains necessary.

Bind permission to the operation that will run

Approval is still necessary where the action requires it. I would bind it to the resolved account, destination, action, material payload, expiry and permitted number or class of attempts. Approving a summary while allowing the executor to change the actual recipient or content leaves a gap before the network problem even begins.

Compute the fingerprint from a documented canonical representation. Exclude only fields whose variation cannot alter the authorized effect, and explicitly define those exclusions. A timestamp used only for local logging differs from a scheduled publication time. Treating both as irrelevant metadata would allow the executor to change when an approved message becomes public.

Credentials and provider access belong to the executor boundary. The model can propose an operation and interpret a constrained receipt, but it should not be the sole component deciding whether its proposal satisfies authorization. The OWASP AI Agent Security Cheat Sheet supports least privilege and explicit controls around sensitive agent actions; the binding fields here are my proposed implementation choices.

After an unknown outcome, a new approval is a policy decision about additional exposure. It is not evidence that the first attempt failed. If an operator decides that another attempt is necessary despite unresolved duplication risk, record it as a separately authorized decision linked to the original operation. A deployment with an absolute one-shot policy must refuse even that path until its policy is explicitly changed.

Let the provider contract decide whether retry is safe

Some APIs are designed to make repeating the same intent safe. Others are not. The correct recovery path depends on the specific operation, its parameters, the provider's idempotency scope and retention, and what the previous response actually means. It cannot be inferred from the presence of an HTTP client retry option.

The HTTP semantics specification, RFC 9110, defines idempotency in terms of the intended server effect of multiple identical requests. That is an operation-level property. It does not promise that every surrounding notification, application workflow or incorrectly implemented endpoint will behave as your business process expects.

For a provider with a suitable idempotency contract, retain the same operation identity and the same key when recovering the same intent. Changing the key because the first request timed out may turn recovery into another operation.

Changing the payload while reusing the key also needs the provider's documented handling; do not assume it updates the earlier request.

AWS's Making retries safe with idempotent APIs describes client request identifiers and the distinction between repeated requests and different intent. That is the relevant foundation. The agent's confidence that two requests mean the same thing is not a substitute for a stable identifier and an enforced service contract.

I would encode the retry decision in an adapter policy: which actions qualify, how long the guarantee lasts, which fields must remain identical, which responses can be recovered and how many attempts are allowed. When any required condition cannot be established, the adapter returns unresolved rather than silently downgrading to an ordinary write.

Browser automation deserves its own recovery policy

A browser click often exposes less of the underlying operation contract than an API. A spinner, button disappearance or navigation might show progress without proving the requested effect. Clicking again may create another resource. Refreshing can also be consequential if the application repeats a submission or resumes an unfinished workflow.

For a one-shot browser publishing workflow, I would treat a submit click with an unavailable result as an unresolved stop. The automated agent must not click submit again, reconstruct the composer or switch to another route to perform the same action. Those are alternative submission attempts, regardless of how the orchestration layer labels them.

If the specific workflow authorizes read-only reconciliation, define the allowed views beforehand. Verification might involve opening the destination account's published items and checking the exact item. It should not involve changing the content, deleting a possible duplicate or using an edit operation as a probe.

Other workflows may forbid automatic reconciliation after an uncertain submit because even further browser activity introduces unacceptable uncertainty. In that case the executor should stop and leave a precise handoff. An API recovery recipe should never be copied wholesale into a browser policy that intentionally permits only one submit invocation.

This costs convenience. It can leave a human with a stalled task that an optimistic agent would mark finished. The benefit is narrower and concrete: the recovery code cannot create a second external event through a path that the original authorization did not cover.

Reconciliation needs evidence rules

When reconciliation is permitted, identify the strongest available evidence before writing the code. A provider operation ID with a documented terminal status is different from a title match in search results. An account page containing the exact resource is different from a cached preview of the draft. These observations should not collapse into one generic found flag.

Match the evidence to the intended effect. For the hypothetical announcement, verify the publishing account, canonical item identifier, published state and material content. A matching title alone is weak because titles can be reused. A URL alone may identify a draft, moderation record or scheduled item that has not reached its audience.

Negative evidence deserves its own standard. A bounded list that omits the item may be paginated, filtered or eventually consistent. Record what was queried, the scope and time of the observation, and the provider guarantees that make absence meaningful. If no guarantee supports a conclusion, keep the outcome unknown instead of increasing confidence through repeated identical searches.

Reconciliation also has a stopping rule. Set a budget for authorized reads and an escalation condition. The budget limits resource use; it does not manufacture certainty when exhausted. After the last permitted check, the state should say unresolved with the observations attached, not failed because the timer ran out.

Preserve contradictory evidence. If a success response refers to one resource and the account view appears to show another, create a reviewable discrepancy. Do not let the model choose the more convenient story and overwrite the receipt that made the contradiction visible.

Give the operator a receipt they can act on

A useful handoff answers what was authorized, what was attempted, what was observed, what remains unknown and which actions are still permitted. It should link to the operation record and any safe destination evidence. Include timestamps and account identity with enough precision to distinguish this event from similar work.

For the example, the receipt might contain:

  • Operation op-217 with the announcement fingerprint unchanged.
  • One dispatch recorded, followed by a connection timeout.
  • External publication unverified; automatic write attempts disabled.
  • Authorized next step: manual account inspection.

These are illustrative fields, not a claim that a particular product already exports this receipt.

Keep the structured record authoritative. A language model may translate it into readable prose, but the UI should display the unresolved status and blocked action directly from execution state. If the prose says published while the record says unknown, the mismatch is a rendering defect to fix, not a reason to promote the operation to succeeded.

Operators also need ownership and an escalation destination. An unknown queue without a responsible person can become a cemetery of stalled work. Assign a handler, expose the age of unresolved operations and define when the business owner must decide whether to wait, investigate through another authorized channel or abandon the intended action.

Measure this queue separately from ordinary application errors. Useful operational measures include the number of unresolved operations, time since dispatch, time until authoritative resolution and how often a proposed retry was refused. None of these measurements proves that the underlying model is intelligent or safe. They show whether the organization can handle the uncertainty created by its execution boundary.

Avoid rewarding the agent for clearing unknown states quickly. A completion metric that treats unresolved work as failure creates pressure to relabel uncertainty or attempt the action again. Review the evidence behind resolution instead. Sample a resolved operation and ask whether another engineer can reconstruct the transition from the retained receipt without trusting the conversation summary.

Keep sensitive payloads out of ordinary status logs. A fingerprint, operation reference and access-controlled evidence location can often support triage without copying the announcement, credentials or customer data into every monitoring system. The receipt should disclose enough to support the authorized operator's decision while keeping the underlying material under its existing access policy.

Abandoning the task does not establish that the external effect never occurred. Record the business decision separately from the outcome. That distinction prevents a later worker from interpreting closed as safe to recreate when an old request, delayed by the provider, finally becomes visible.

Test the places where certainty disappears

Before trusting this design, I would use deterministic failure injection around the executor boundary. A better prompt does not test whether the operation record survives a crash. Start with the points where an action could happen while its evidence is lost, and assert the resulting dispatch permissions.

The following matrix is a proposed acceptance checklist. It describes expected behavior for the architecture, not results of tests performed for this article.

Injected condition

Required observation

Allowed next write

Crash after durable dispatch record, before network call

Existing operation recovered as potentially attempted

Only under a valid recovery contract

Provider commits, response is lost

Outcome remains unknown until evidence resolves it

No ordinary retry

Provider accepts asynchronous work

Accepted remains distinct from completed effect

No fresh submission of the same intent

Queue delivers the same job twice

Both deliveries resolve to the same operation

One winning dispatch path

Payload changes after approval

Fingerprint mismatch blocks execution

Requires matching authorization

Idempotency guarantee has expired

Recovery policy refuses to assume protection

Requires a new explicit decision

Search results omit the created item

Absence is evaluated against source guarantees

No write based only on weak absence

Add a concurrent-worker case where a paused worker resumes after losing its lease. The assertion must cover the external effect or enforced provider key, not merely the number of rows in your local database. A beautifully deduplicated operation table can coexist with duplicate remote resources.

Test the operator interface too. It must preserve unknown after an application restart, show the original evidence and refuse an unauthorized retry button. Check that cancellation, task closure and a model-generated apology do not reset execution eligibility. These transitions are easy to overlook because they sit outside the happy path of the tool call itself.

Decide how much uncertainty the product can carry

This architecture is most useful when duplicate effects matter and the external boundary cannot participate in your local transaction. Publication, customer communication and resource provisioning are obvious examples. The policy should be proportionate to the action; a harmless read does not need the same escalation path as a public write.

It also has costs. Durable records require retention and access controls. Reconciliation needs provider-specific semantics. Strict stopping can delay legitimate work. Operators must understand what their decisions authorize. These are implementation costs to compare with the cost of repeating an action whose first outcome remains unresolved.

An unknown state does not create exactly-once execution across arbitrary systems. It makes the limits of observation explicit and prevents one common software shortcut from pretending to solve them. Where the provider offers stronger guarantees, use them. Where it offers weaker evidence, keep the executor's claims correspondingly narrow.

For the next consequential tool you give an agent, write down the response to one failure before enabling it: the request may have succeeded, but the worker cannot tell. Name the state, preserve the operation identity and specify who can authorize the next step. If the implementation only has success, failure and retry, that unanswered case already exists. It is merely waiting for a timeout to become visible.


文章来源: https://hackernoon.com/your-ai-agent-needs-an-unknown-state?source=rss
如有侵权请联系:admin#unsafe.sh