The next challenge for autonomous AI systems is not making them respond faster. It is teaching them how to wait without losing state, holding resources, duplicating work, or breaking the workflow.

Most AI agent architectures are designed around a request. A user asks something, the agent thinks, it calls a tool, the tool responds, the agent thinks again, and eventually something comes back to the user. That model works surprisingly well when everything involved in the workflow is fast. A database query takes milliseconds. An API responds in a few hundred milliseconds. The model generates an answer in a few seconds. The whole operation fits comfortably inside the lifetime of an HTTP request.

Then an agent enters the real world.

Suddenly, one of its tools takes thirty seconds. Another service takes two minutes. A payment provider accepts a request but does not immediately confirm the transaction. A document has to be reviewed by a person. A deployment is waiting for an approval gate. An external system sends the result through a webhook sometime later. A background job is processing millions of records. A downstream service goes temporarily offline.

The agent now has a problem that is easy to describe but surprisingly difficult to engineer. It has to wait. And waiting is where many agent architectures start to fall apart. We have traditionally treated waiting as a failure condition. A request takes too long, so we introduce a timeout. The timeout fires, the request is cancelled, and the caller gets an error. That model makes sense for a synchronous API call. It becomes much less useful when the thing doing the work is an autonomous agent executing a workflow that might legitimately take five minutes, five hours, or even several days.

An agent waiting for a human approval is not necessarily stuck. An agent waiting for a payment confirmation has not necessarily failed. An agent waiting for a deployment to complete is not necessarily doing nothing. It may simply be waiting for the next event that allows the workflow to continue. That sounds like a subtle distinction, but it changes the architecture completely.

The Problem With Treating Every Agent Task Like an HTTP Request

The simplest agent implementation often looks something like this:

User Request
     |
     v
   Agent
     |
     v
   Tool
     |
     v
 Tool Result
     |
     v
   Agent
     |
     v
 Final Response

There is nothing inherently wrong with this architecture. For short-lived operations, it is exactly what you want. The problem starts when developers take this pattern and stretch it across workflows that were never designed to fit inside a request lifecycle.

Imagine an agent responsible for onboarding a new enterprise customer. It needs to create an account, verify documents, perform compliance checks, request approval from an administrator, provision access, notify the customer, and wait for confirmation that the customer has completed the final setup.

That workflow might take an hour. It might take a day. It might take three days because the administrator is on vacation. Keeping the original HTTP request open for that entire period would be absurd. Even if the infrastructure technically allowed it, you would be tying together too many independent failure domains. A load balancer could close the connection. A worker could restart. A container could be redeployed. A network connection could disappear. A browser could close. A process could run out of memory.

The workflow itself may still be perfectly valid. The request is what ended. Those are not the same thing. This is one of the fundamental architectural changes that agent systems require. The lifetime of the user's request should not necessarily determine the lifetime of the work.

Once that boundary is understood, the system starts looking less like a traditional request-response application and more like a durable workflow engine.

A Timeout Does Not Tell You What Happened

There is an even more dangerous problem with timeouts. A timeout tells you that the caller did not receive a response within a particular period. It does not necessarily tell you what happened to the operation on the other side. This distinction has existed in distributed systems for years, but autonomous agents make it much more important because agents are increasingly capable of performing side effects.

Suppose an agent calls a payment service. The request reaches the payment provider. The provider processes the payment successfully. Before the response reaches the agent, the network connection breaks. The agent receives a timeout.

What happened?

From the agent's perspective, the operation failed to return. From the payment provider's perspective, the payment succeeded. If the agent simply interprets the timeout as failure and retries the payment, it may charge the customer twice. This is why a timeout is not a state. It is an observation about communication. That difference matters enormously. A distributed system can have states such as SUCCESS, FAILED, PENDING, or UNKNOWN. A timeout may be the reason you move into UNKNOWN, but it should not automatically move you into FAILED.

The next action may need to be a status lookup rather than a retry. The system might query the payment provider using an idempotency key or transaction identifier and discover that the original operation completed successfully. This is one of the reasons idempotency becomes so important in long-running agent workflows. A retry should not automatically mean another side effect. It should mean another attempt to safely determine or achieve the desired state. That is a very different design philosophy.

The Agent Needs to Understand Waiting as a First-Class State

Once waiting becomes part of the workflow, it should be represented explicitly. Instead of having a process sitting in memory doing something like:

while not approved:
    time.sleep(10)

The workflow should persist something closer to:

workflow_id: 84721
state: WAITING_FOR_APPROVAL
waiting_for: approval.completed
created_at: 14:32
deadline: 18:00
resume_policy: continue_workflow

The worker can stop. The compute can disappear. The container can be replaced. The HTTP request can finish. None of those events should destroy the workflow. When the approval event arrives, the workflow runtime can load the persisted state, verify the event, reconstruct the necessary context, and resume execution. This is the key idea behind durable execution. The workflow does not remain alive merely because a process is alive. The workflow remains alive because its state has been persisted. That distinction is extremely powerful. It means compute becomes disposable while work remains durable.

Waiting Is Not the Same as Sleeping

There is a subtle but important difference between sleeping and waiting. Sleeping usually means a process remains responsible for remembering that it needs to wake up. Waiting in a durable workflow means the system persists the condition that will allow the workflow to continue and releases the resources associated with active execution. Consider an agent who needs to wait 24 hours before following up with a customer.

The naive implementation might keep a worker alive or schedule a background task that has to survive the next deployment. A durable implementation can record:

WAITING
resume_at = 2026-09-12T10:00:00

and release the worker. At the appropriate time, the workflow scheduler creates a new execution attempt. The same principle applies to events. If the workflow is waiting for a webhook, it should not continuously poll unless polling is actually necessary. It can persist the expected event and resume when the event arrives.

This changes the economics of the architecture. Instead of paying for compute while nothing is happening, you pay for a durable state and resume work when something meaningful occurs. For large agent workloads, that distinction can become significant.

The Real Unit of Work Is No Longer the Request

Traditional web applications are often organised around requests. An incoming request creates a unit of work. The server processes it. A response is returned. Agent workflows increasingly need a different abstraction. The unit of work becomes the workflow execution.

A workflow may contain dozens of activities and multiple waiting periods. For example:

Create Application
       |
       v
Validate Documents
       |
       v
Run Compliance Check
       |
       v
WAIT FOR HUMAN APPROVAL
       |
       v
Provision Account
       |
       v
WAIT FOR EXTERNAL CONFIRMATION
       |
       v
Send Notification
       |
       v
Verify Completion

There is no reason for the same process to execute every step. In fact, expecting it to do so creates unnecessary coupling. A durable workflow can execute the first three steps, persist its position, suspend, and disappear. When approval arrives, another worker can resume the workflow. When the external confirmation arrives, another execution can continue. The workflow identity remains constant even though the compute handling it changes. That is a much better fit for autonomous systems.

Queues Are Not Just Infrastructure. They Are Part of the Control Plane

Queues become particularly important when agents stop operating synchronously. Suppose an agent receives 10,000 tasks and decides to process all of them immediately. The agent may generate thousands of tool calls. The downstream API may only support 100 requests per second. Without some form of admission control, the agent can overwhelm the dependency it is trying to use. This is where a queue provides something more important than asynchronous execution. It provides a boundary between producing work and consuming work.

The agent can create work items. Workers can consume them according to available capacity. That introduces backpressure. If the downstream system slows down, the queue grows rather than causing every upstream process to block indefinitely. The system can monitor queue depth, processing latency, retry rates, dead-letter volume, and age of the oldest message.

Those metrics tell you something much more meaningful than average agent response time. For example, an agent platform might report a median task latency of two seconds while quietly accumulating 50,000 tasks in a queue. The model is fast, but the system is not. That distinction becomes increasingly important as agent workloads become asynchronous.

Backpressure Matters Because Agents Create More Work

There is another reason queues matter for agents: agents can create more work while they are already working. A single task might need five API calls, while another might need twenty. One agent may even trigger another agent, which creates more work of its own. The workload can grow quickly, so the system needs a way to control that flow.

That is where backpressure becomes important. The runtime needs to decide how many workflows and tool calls can run at once, how much work can sit in the queue, which tasks should get priority, and what to do when a downstream service is already overloaded. It may need to slow down new work, delay low-priority tasks, or temporarily reject requests rather than allowing the whole system to collapse. These are runtime decisions, not things the language model should be trusted to manage. The agent decides what work it wants to do, and the platform decides how much work the system can safely handle.

Polling Works Until It Becomes the Architecture

Polling is one of those mechanisms that looks harmless when the system is small. An agent checks:

Is the job complete?
No.

Wait.

Is the job complete?
No.

Wait.

Is the job complete?
Yes.

For one workflow, this is perfectly reasonable. For 100,000 workflows, the architecture becomes very different. Now the system may be generating millions of unnecessary status requests. The external service is receiving traffic that does not represent meaningful work. The agent infrastructure is spending resources asking the same question repeatedly. And worse, the polling interval becomes a compromise. Poll too frequently and you waste resources. Poll too slow, and the workflow responds late. Events provide a better model when the underlying system supports them. Instead of repeatedly asking whether something changed, the system can receive:

job.completed

or:

payment.confirmed

or:

approval.granted

The workflow can then resume immediately. This is not simply a performance optimisation. It changes the semantics of the system. The workflow is no longer guessing when something might have happened. It is reacting to evidence that something happened.

But Events Create Their Own Problems

Event-driven architecture is not magic. An event can be duplicated. An event can arrive late. Events can arrive out of order. A consumer can fail after processing an event but before acknowledging it. A producer can publish an event and then crash before updating another piece of state. An event schema can change. A consumer can be unavailable for hours.

That means a durable agent architecture needs to treat events as part of the distributed-systems problem rather than assuming the event bus has solved it. Consider an event:

payment.completed

The consumer receives it and updates the workflow. Then the consumer crashes before acknowledging the message. The queue delivers the event again. If the workflow blindly processes it again, the system may perform duplicate work. This is why consumers often need idempotent processing. A useful pattern is to associate every event with a stable identifier:

event_id = evt_8f192
workflow_id = wf_84721
operation_id = op_29381

The consumer can record which event or operation identifiers it has already processed. If the same event arrives again, the system can recognise it rather than treating it as a new instruction. Again, none of this is particularly new. That is actually the point. Agent systems are rediscovering distributed-systems problems that the industry has been solving for decades.

Human Approval Is the Ultimate Waiting Problem

Human-in-the-loop workflows make the weakness of synchronous agent architecture particularly obvious. Imagine an agent preparing a production deployment. It can analyse the change, run tests, inspect the risk, generate the deployment plan, and request approval. The human reviewer might approve it in two minutes.

Or thirty minutes. Or tomorrow morning. The agent should not remain in an active reasoning loop throughout that period. It should persist the workflow state and wait. When the approval arrives, the system should verify that the approval corresponds to the correct workflow, correct deployment version, correct environment, and correct authorisation context.

Then it can resume. This also makes auditability much easier. The system can record:

deployment.requested
risk.analysis.completed
approval.requested
approval.granted
deployment.started
deployment.completed

Now the workflow has a history. If someone asks six months later why a deployment occurred, the answer does not depend on reconstructing a model conversation from logs. The system has a durable execution trail. That is much closer to how critical enterprise workflows should operate.

Deadlines Are Different From Timeouts

One of the most useful distinctions in long-running workflows is between a timeout and a deadline. A timeout often describes how long a particular operation is willing to wait for a response. A deadline describes when the business or workflow requirement expires. Those are not the same thing. An API call might have a five-second timeout because there is no reason to keep the network connection open longer. The workflow itself might have a deadline of 5 PM.

If the API call times out after five seconds, the workflow does not necessarily fail. It can retry, use another provider, schedule another attempt, or wait for an asynchronous confirmation. The deadline remains the larger constraint. This gives the runtime more flexibility. Instead of thinking:

"The operation timed out, the workflow failed."

The system can reason: "This attempt did not complete within its execution timeout. The workflow still has 42 minutes before its business deadline." That is a much more useful model. It also enables smarter scheduling. A task with a deadline in five minutes may need priority over a task that has six hours remaining. An agent platform that understands deadlines can make better resource-allocation decisions without asking the model to manage infrastructure directly.

Durable State Is More Important Than a Durable Process

A common mistake when building long-running agents is to focus on keeping the process alive. Teams add worker pools, heartbeats, process supervisors, container restarts, and health checks. Those things are useful. But they are not enough.

The process can always disappear. A machine can fail. A deployment can kill the worker. A network partition can isolate it. A cloud region can become unavailable. A truly durable workflow assumes that execution can disappear at any time. The important question is not:

How do we keep this worker alive?

It is:

How do we make sure the workflow can continue when this worker disappears?

That leads directly to checkpoints. After completing a meaningful step, the workflow records enough information to reconstruct where it is. For example:

workflow_id
current_state
completed_steps
pending_steps
tool_results
operation_ids
event_position
state_version
deadline
retry_count

The exact structure depends on the system, but the principle is the same. The workflow should not depend on the memory of a particular process.

But Checkpointing Alone Is Not Enough

There is an important trap here. Suppose an agent executes:

1. Call payment API
2. Save checkpoint saying payment succeeded

What happens if the payment succeeds but the worker crashes before step two?

The payment happened. The checkpoint does not know. When the workflow resumes, it may call the payment API again. This is the effect gap between external side effects and internal workflow state. The workflow engine can make its own state durable, but it cannot automatically make an external API call and its own checkpoint one atomic transaction. This is why idempotency matters so much. The payment request should have a stable operation identity.

If the workflow retries, the payment provider should be able to recognise that the request corresponds to an existing operation. The retry becomes safe. This pattern becomes even more important as agents become responsible for financial transactions, infrastructure changes, customer communications, provisioning, and other operations where duplicate side effects are expensive. Durability without idempotency can simply make failures recoverable enough to repeat the same mistake.

Agent Reasoning Makes Recovery More Interesting

There is an AI-specific complication here. Traditional deterministic software can often replay a computation and expect the same result. LLM calls are different. The same prompt can produce different output depending on the model version, sampling configuration, tool state, retrieved context, provider behaviour, or other runtime conditions. If an agent crashes after a model call, blindly invoking the model again during recovery can produce a different decision. That may be acceptable for some applications. It can be dangerous for others. A durable agent runtime, therefore, needs to decide which model outputs are part of the execution history and which steps are safe to recompute.

For important workflows, a completed model result can be checkpointed so that recovery does not automatically create a new reasoning branch. Conceptually:

LLM Call
   |
   v
Model Result
   |
   v
Persist Result
   |
   v
Continue Workflow

If the worker crashes afterwards, the runtime can recover the stored result rather than calling the model again. This does not mean every token generated by an LLM needs to be persisted forever. It means the architecture needs to identify which outputs are execution decisions and which are disposable intermediate computations. That is a fundamentally different question from simply storing chat history.

Waiting Should Be Observable

A system that supports long-running agents also needs to make waiting visible. A conventional monitoring dashboard might show:

Request latency: 1.8 seconds
Error rate: 0.4%
CPU: 42%

Those metrics are useful, but they don't tell you much if a workflow has been waiting for three hours. In that situation, you need to know why it is waiting and what needs to happen next. Is it waiting for an external event, a tool response, or a human approval? How long has it been waiting, and is it still within its deadline? Are other workflows stuck on the same dependency? Are events arriving but not being processed, or is the queue simply getting longer? And if a human approval is holding everything up, has it been waiting long enough to require escalation? These are the kinds of signals that make a long-running agent workflow understandable and operationally manageable.

A useful state might look like:

workflow = wf_84721
status = WAITING
reason = HUMAN_APPROVAL
waiting_since = 10:42
deadline = 18:00
owner = compliance-team

That gives operators something actionable. "Agent latency increased" is not particularly useful. "2,400 compliance workflows have been waiting for approval for more than four hours". The observability model has to evolve from request-centric metrics to workflow-centric metrics.

The Agent Should Not Be Responsible for Its Own Waiting Strategy

Another mistake is allowing the LLM to decide how to wait. You do not want the model generating something like:

"I will wait ten minutes and then check again."

That is application logic masquerading as reasoning. The model should express an intent such as:

WAIT_FOR = compliance.approved

or:

WAIT_UNTIL = 2026-09-12T10:00:00

The runtime should decide how that wait is implemented. It may use a queue. It may use a scheduler. It may subscribe to an event. It may create a durable timer. It may request human intervention.

This separation is important because infrastructure policies should not depend on probabilistic output. The model can decide what it needs. The runtime decides how to provide it safely.

Long-Running Agents Change the Meaning of "Done"

There is another problem that appears when agents can wait. The word "done" becomes much more complicated. An agent might successfully submit a request.

Is the workflow done? Probably not. The request might be pending. The downstream system might need to process it. A human might need to approve it. The final state might not be known yet. This means a mature agent workflow needs explicit terminal states. For example:

CREATED
RUNNING
WAITING
RETRYING
FAILED
CANCELLED
COMPLETED

And even COMPLETED should have a meaningful definition. It should not mean "the model said the task looked complete." It should mean the workflow reached a verified terminal state. That distinction becomes especially important when agents operate across multiple systems. A deployment is not complete because the deployment API accepted a request. A payment is not complete because the payment request was submitted. An account is not provisioned because the provisioning job has started. Completion should correspond to an observable system state.

That might require another API call, an event, a database state transition, or a verification step. The agent should not be allowed to confuse intention with outcome.

The Architecture Starts Looking Like a Workflow Runtime

When you put all of these pieces together, the architecture starts to look much less exotic. You have a queue handling work, a durable state keeping track of where each workflow is, workers executing tasks, timers handling scheduled actions, events triggering the next step, and retries dealing with temporary failures. Around that, you have deadlines, idempotency keys, checkpoints, state transitions, failure-handling policies, observability, and audit records. The LLM sits inside this system and decides what should happen next, but most of the infrastructure around that decision is familiar to anyone who has worked with distributed systems.

That is what makes agent engineering interesting. Agents are not replacing the foundations of distributed systems. They are making those foundations more important. A queue gives an agent a safe way to create and consume work. State machines make their transitions explicit. Idempotency makes retries safe when an action might already have happened. Timeouts still matter, but they now need to be understood separately from the overall deadline of a workflow. And waiting is no longer something the system treats as an exception. For long-running agents, waiting is simply another part of execution.

The Goal Is Not to Make Agents Wait Faster

There is a tendency in AI infrastructure to optimise almost everything around latency. We want lower model latency, faster retrieval, faster tool calls, faster inference and faster responses. Those things matter, especially when someone is sitting in front of an application waiting for an answer. But autonomous workflows introduce a different dimension that is easy to overlook: sometimes the right thing for the system to do is simply nothing.

An agent may need to wait for an approval, an external event, a scheduled time, a downstream service to recover, or a person to provide missing information. None of those situations necessarily means the workflow has failed. The workflow is simply not ready for its next step yet. The real problem starts when the system waits without knowing or recording what it is waiting for.

If an agent is sitting inside a running process, that process becomes a point of failure. If it keeps polling an external system, it may waste resources. If it keeps an HTTP connection open, the lifetime of the request becomes tied to the lifetime of the workflow. And if the workflow does not persist where it stopped, a restart can leave the system unsure about what had already happened.

A better approach is to treat waiting as part of the workflow itself. The system records its current state, releases the resources it does not need, waits for the relevant event or deadline, and then resumes from a durable checkpoint when it is time to continue. Once you design it this way, waiting stops looking like a failure condition and becomes just another state in the execution lifecycle. The architectural shift is not about making agents wait less. It is about making them wait reliably.

The Agent Runtime Needs to Know How to Stop

There is one final idea that is easy to miss. A good agent runtime is not just responsible for starting work. It needs to know when to stop active execution. If a workflow has reached a waiting state, the worker should be able to stop. If a downstream service is unavailable, the workflow should be able to pause. If the workflow has exceeded its deadline, it should stop rather than continuing indefinitely.

If a human approval is required, the agent should not keep reasoning in the background, trying to guess what the human might decide. Stopping is not failure. Sometimes, stopping is the correct state transition. This is particularly important because LLM-based systems make it very easy to keep generating another step. One call leads to another. Another reasoning cycle begins. Another retry happens. Without explicit workflow boundaries, the agent can remain busy indefinitely. A durable runtime gives the system a different option. It can say:

Nothing needs to happen right now. Persist the state. Release the worker, wait. Resume when there is a reason to continue. That is not a limitation of autonomy. It is what makes autonomy manageable.

The Next Generation of Agents Will Be Built Around Time

The first generation of AI applications was mostly request-driven. A user asked a question, the model generated a response, and the interaction was over. Agentic systems change that model because the work does not necessarily end when the response is sent back to the user. An agent can start a task, call several external systems, wait for an approval, respond to an event, continue the workflow later, and sometimes keep working long after the original conversation has disappeared.

Once that happens, time becomes part of the architecture. Not just latency, but time as an actual part of the workflow. The system needs to know when the workflow started, when its last step was completed, how long it has been waiting, what it is waiting for, when its deadline expires, how many times an operation has been retried, when something needs to be escalated, how long a particular state is allowed to remain valid, and when the workflow should eventually be cancelled. Those are runtime concerns. The model can help decide what should happen next, but it cannot reliably manage the lifecycle of a workflow by itself.

That is why I think the next step in agent infrastructure is not simply making agents reason better. It is making them wait correctly. A reliable agent should be able to start a workflow, complete a few steps, reach something that may take hours, persist exactly where it stopped, release the resources it no longer needs, and disappear without losing the workflow. When the expected event eventually arrives, the system should be able to bring that workflow back, reconstruct its state, verify that the world has not changed in a way that invalidates the next step, and continue from there.

This also means the workflow has to be designed to survive the things that inevitably happen in production. A worker can crash. A deployment can restart the service. A network connection can disappear. An external API can accept a request without returning a response. A retry can accidentally repeat an operation that already succeeded. A human approval can sit untouched in an inbox for hours. The agent needs to be able to recover from these situations without assuming that the world is the same as it was when it stopped.

Perhaps the most important distinction is knowing the difference between nothing happening yet and something going wrong. A payment that is still pending is not necessarily failed. An approval that has not arrived is not necessarily an error. A deployment that is still running is not necessarily stuck. The system needs enough state and context to understand which situation it is actually dealing with.

That distinction will become increasingly important as agents move beyond chat windows and into real business processes. Real systems do not always respond immediately. People take time, APIs fail, queues fill up, payments remain pending, approvals sit in inboxes, deployments take minutes, external services disappear and return, events arrive late, and networks break. Sometimes, the most correct thing an autonomous system can do is simply wait for the information it needs before taking another action.

That is not a timeout problem. It is a waiting problem. And the agents that can wait without losing state, duplicating work, consuming unnecessary resources, or losing track of what is actually happening will be the ones that are capable of operating reliably in production.