Human-in-the-Loop Agent Architecture

Computer-use agents can click buttons, fill forms, and operate real software. That makes the next production question unavoidable: what happens when the agent is about to spend money, change customer data, send an external message, or deploy code?

Human-in-the-Loop (HITL) architecture adds a hard execution boundary for those moments. An agent can propose a side effect, but a downstream policy service pauses it until the required reviewer authorizes the exact action.

This addresses the opaque execution problem: even capable models can produce confident but incorrect outputs, and in e-commerce operations, one bad refund or misdirected shipment can cost real money and customer trust. HITL doesn't make an action correct by itself. It creates a point where policy, evidence, authorization, and final-state checks can stop a bad action.

This is also one practical control for one of the top agent risks. The OWASP Top 10 for LLM Applications lists LLM06: Excessive Agency, which it breaks into excessive functionality, excessive permissions, and excessive autonomy. Requiring human approval for high-impact actions is one recommended mitigation for the excessive-autonomy facet, ideally enforced in a downstream system rather than left to the model to decide. ^[1] Read this article as the design pattern that implements that control.

Like a warehouse system that can retrieve inventory records but must route a money movement to an authorized reviewer, a HITL agent runs only within a defined risk boundary. Unlike a chatbot that waits for input at every turn, this pattern supervises actions according to their possible effect.

To make this concrete, we'll follow Riley, an order-support agent for an online electronics store. Riley can look up authorized order data and draft suggested responses. Riley can also propose a refund, an address change, or a customer email, but a downstream gate controls whether any of those side effects run. The dollar thresholds in this lesson are illustrative policy choices, not universal rules.

By the end, you'll be able to design Riley's risk policy, implement checkpoint/resume logic that lets Riley wait durably for review, and build an approval UI that gives reviewers enough evidence without exposing unnecessary customer data. We'll assume you already know LLM tool calling, state graphs, guardrail policy, and browser-agent risk. If those terms are fuzzy, review Function Calling & Tool Use, Agentic Architectures, Guardrails & Safety Filters, and Computer Use & Browser Agents first.

The three pillars of human-agent interaction

Not all human oversight is created equal. Production systems distinguish between three distinct interaction models based on risk tolerance and the nature of the task. Understanding these distinctions helps you design appropriate governance frameworks.

Human-in-the-Loop (HITL): Active gatekeeping

In this model, the agent can't proceed with a gated action without explicit human approval. Use it for actions whose impact isn't acceptable to discover after execution, such as money movement, production deployment, or a customer-record change. The agent proposes an action, pauses execution, and waits for an approve, reject, or modified proposal decision before continuing. Modified proposals still need validation.

Human-on-the-Loop (HOTL): Supervisory control

Here the agent performs tasks autonomously while a human monitors the process and retains the ability to interrupt or override. This fits actions whose effect can be stopped or repaired after detection, such as routing internal draft suggestions into a queue. It isn't a sufficient gate for sending an incorrect customer message or issuing a refund: the human may see the mistake only after the effect happened. HITL blocks before execution, while HOTL can only interrupt execution that is already allowed.

Human-out-of-the-Loop (HOOTL): Full autonomy

At the far end of the spectrum, some actions are automated with no real-time human oversight. This applies only to bounded actions with clear authorization and acceptable failure modes. An authorized order-status read or a draft created in an internal queue may fit; an unrestricted database read or web action doesn't become low risk simply because it doesn't write data. Some actions stay HITL because their effect, policy, or applicable duties demand a pre-execution gate.

The trust spectrum

Not all agent actions carry the same risk. Designing a HITL system begins by classifying every tool and action along a trust spectrum. This principle drives the level of friction required before execution.

A naive implementation treats all actions equally: either everything is autonomous (dangerous) or everything requires approval (tedious). A production-grade system defines granular policies:

Risk classification system

We can implement this policy using a RiskLevel enum and a policy lookup table. This decouples the agent's logic ("I want to do X") from the governance logic ("Should I allow X?").

To implement this, we define a policy lookup table that maps each available tool to a specific risk tier. This structure serves as the core policy engine, taking a tool's name and runtime arguments as contextual input and outputting the required authorization flow before execution.

Risk is not static. A "send email" tool might be Low Risk when emailing an internal test address, but High Risk when emailing an external domain. Production policies are dynamic, checking arguments at runtime.

Architecture: the Checkpoint/Resume pattern

The fundamental architectural challenge of HITL is the pause. When an agent pauses for approval, it can't sleep the thread (time.sleep()) or block in memory.

Why in-memory blocking fails

1. Durability: If the server restarts or crashes while waiting for approval (which could take hours), the agent's state is lost.
2. Resource Usage: Holding a thread open for a human response wastes compute resources.
3. Scalability: You can't scale to thousands of concurrent agents if each one is blocking a thread.

The solution is the Checkpoint/Resume pattern. When an approval is needed, the agent persists its graph state (messages, structured variables, and pending task metadata) to durable storage and returns control to the caller. When the human approves, the runtime reloads that state and re-enters the waiting node.

To see what gets saved, here is what Riley's state might look like when paused:

When an authorized reviewer clicks Approve, the runtime resolves this pending decision with a guarded write, reloads the checkpoint, rechecks current state and arguments, and only then attempts the action. If the reviewer clicks Reject, Riley can draft a response without running the refund. Checkpoint state is ordinary inspectable data, but emergency changes should still go through an authenticated, versioned, audited path rather than an ad hoc database edit.

Before building a workflow engine, you can test the persistence boundary with ordinary data. The checkpoint below stores a redacted summary and an action hash, but not the customer's raw message.

Here is a comparison of two popular approaches for implementing this pattern:

Using LangGraph checkpointing

LangGraph provides first-class support for this via interrupt(). ^[2] When you compile the graph with a checkpointer, LangGraph persists checkpoints for the thread and pauses execution until the graph is invoked again with Command(resume=...). ^[3] One subtle detail matters in production: when execution resumes, LangGraph reruns the node from the top, so any code before interrupt() must be idempotent. ^[2]

The approval flow

The flow separates the Agent Runtime (the Python application, often powered by a Large Language Model (LLM)) from the Approval Interface (Web/Slack). In production, they coordinate through persisted state plus a thin resume endpoint, not a long-lived in-memory call stack. The checkpoint figure above is the source of truth for the sequence: pause, persist state, create a versioned approval record, notify the reviewer, validate the decision, reload the checkpoint, execute, and audit.

REST API for approval queue

To build the "Resume API" component, we need an endpoint that looks up the suspended thread and issues a command to resume it. This endpoint takes a thread ID plus a versioned approval decision payload as input, validates the thread's state, and outputs a resume command to awaken the agent with the provided instructions.

The compare-and-swap check records at most one reviewer decision. Store each approval request with fields such as status, version, action_hash, expires_at, required_role, and resolved_at, then resolve it with a guarded update such as WHERE id = ? AND status = 'pending' AND version = ? AND action_hash = ? AND expires_at > now(). Record authorized separately from executed: an approval may be accepted and then fail during execution. A durable resume worker must recheck current business state and use an idempotency key so a retry can't issue the same refund twice.

This small executable model shows those two separate transitions. It takes a versioned decision and an idempotency key as input, then shows that a stale click is rejected and an execution retry returns the previously recorded outcome.

An expiry guard is separate from version matching. This example takes the current time and expiry time as inputs, then rejects a decision whose review window has already passed.

Designing the approval interface

A "Yes/No" button is rarely enough for production systems. The approval interface must provide context and control. When Riley asks "Can I refund order #78291 for $899?", the human needs to know what data will change and why.

The approval UI should show the authorized reviewer the smallest evidence packet needed for the decision: a redacted request summary, the policy trigger, source references the reviewer is allowed to inspect, and the exact arguments Riley proposes to execute. Dumping an entire customer conversation into every review card creates unnecessary privacy exposure and makes the meaningful change harder to see.

The approval payload

Riley should pause with a structured payload that the UI can render clearly. This payload takes authorized evidence and proposed tool arguments as input and structures them into a redacted typed object for the reviewer. For data mutations, this should ideally be a visual diff rather than raw JSON.

By rendering this payload, the Human-in-the-Loop UI becomes a useful decision surface. The reviewer can compare Riley's proposed effect with permitted evidence before authorizing the tool call. For complex actions, consider adding a dry-run diff that shows what would happen if the action were approved. Version, expiry, and action hash fields matter too: they let the backend reject stale clicks instead of replaying an approval long after the underlying state changed.

The packet builder is also a data-minimization boundary. This executable example takes a customer note and exact action as input, redacts the email address, and emits a stable hash that binds the review card to the proposed mutation.

Give the reviewer a concise rationale, the exact tool arguments, a diff, and the policy rule that triggered review. Don't make raw chain-of-thought your approval primitive. Explanations can be unfaithful, and they may reveal internal reasoning or sensitive context you didn't intend to surface. ^[4]

Also keep approval state compact. Persist a data-minimized audit record with access controls and retention policy, then inject only structured fields such as decision, reason, modified_args, or a short reviewer summary back into the runtime. Otherwise long-lived threads accumulate reviewer chatter that burns context window on approval metadata instead of task state.

Using Temporal for long-running workflows

For workflows that coordinate several durable activities, timers, retries, and approval messages, Temporal can be a better fit than a hand-rolled checkpoint table. LangGraph checkpoints can also wait durably; the choice isn't simply "short wait versus long wait." Temporal's workflow execution records event history and communicates with callers through activities plus message-passing primitives such as Signals and Updates. ^[5]

Signals are a good default when the approval service can fire-and-forget. If the UI needs synchronous confirmation that the workflow accepted the decision, or you want the runtime to reject a stale approval before it lands in history, an Update is usually cleaner. ^[6]

Temporal also gives you durable state, timers, retries, and event history out of the box. That means you don't have to build the orchestration layer for "pause, notify, wait, resume" yourself.

This approach is useful for multi-step approval chains (for example, waiting for both a support lead and finance sign-off). The workflow can wait without keeping a worker blocked, evaluate partial approvals, and continue waiting for the remaining required decision.

The Temporal workflow below uses an Update for the approval decision because the UI needs to learn whether the action ID and version still match the pending request. The workflow then calls a validation activity before any external side effect.

Keep policy code inside the workflow deterministic. If is_risky() depends on live balances, anomaly services, or a policy database, fetch that data in an Activity first and pass the result into workflow state. Temporal replays workflow code against event history, so non-deterministic logic inside the workflow will break replay. ^[5]

If a notification doesn't need a response, a Signal may still be appropriate. Approval buttons need synchronous accept/reject feedback, so an Update with validation is a stronger fit for this decision path. ^[6]

For approval chains that can run for months or accumulate a large event history, periodically use Continue-As-New to cap history size while carrying forward unresolved approval state. ^[5]

Advanced patterns

Building a resilient Human-in-the-Loop system goes beyond simple pause-and-resume mechanics. Production environments require granular controls to handle complex, high-volume, or ambiguous scenarios effectively. By implementing advanced patterns, we can reduce friction for human reviewers while maintaining strict safety boundaries.

Dynamic risk escalation

Static policies need runtime context. Sending one customer email is already an external side effect that may require approval; attempting hundreds of emails in a short interval should move into a stricter queue or be blocked outright. The figure below starts from a static policy floor and conditionally escalates the required authorization layer.

Dynamic policies evaluate context by taking the proposed action and operational metadata as input, applying rule-based checks, and outputting an escalated risk tier if anomalies are detected. The following function combines the static base policy with runtime conditions. It checks the transaction amount, recent failure count, and time window, elevating the required authorization level when thresholds are crossed:

Approval with modification

A useful HITL pattern lets the human modify a proposed action rather than merely reject it. Instead of a binary yes/no choice, the human can correct arguments, such as changing a full refund proposal to a partial one or correcting an order ID. That edit is a new proposal, not a guarantee of safety.

Build the approval UI as a form, not a button. Populate the form with the agent's proposed arguments (e.g., email body, SQL query) and allow the human to edit them before hitting "Approve".

For example, if Riley proposes a full refund: process_refund(order_id="78291", amount=899.00), a human reviewer can modify it to process_refund(order_id="78291", amount=449.50, partial=True) to offer a partial refund instead. Before execution, the host must validate schema, reviewer permission, applicable policy, action version, and current order state. The edit doesn't train Riley and doesn't bypass a stronger approval tier.

Batch approvals

If Riley needs to propose 50 return-label creations at the end of a holiday weekend, asking for separate decisions for each one creates reviewer fatigue. Reviewers who face repetitive requests may stop inspecting individual effects. A batch review can group related proposals without hiding the identity, destination, cost, or policy status of each item.

Instead of creating one alert per proposed operation, the orchestrator collects pending actions over a bounded window or groups them by a shared task identifier. The UI can then present: "Riley proposes 50 return labels (review all items)." The approval record must bind to the exact item list or item hashes, so a label inserted after approval can't ride along in the batch.

Implementing batch approvals requires per-item state and idempotency. If a batch contains 50 actions and the reviewer authorizes the fixed set, the orchestrator must track each action separately. If action 42 fails, already-completed labels aren't automatically undone unless the operation provides a compensation path. Retry only the failed authorized item with its idempotency key, and re-request review if its inputs or destination changed.

Bind a batch approval to the exact ordered items shown to the reviewer. Here a later label inserted into the queue changes the digest, so it can't reuse the earlier authorization.

What to measure

Once a HITL system is live, model quality alone stops being enough. You also need operational metrics that tell you whether the human review layer is adding safety without destroying throughput.

Track these per tool and per risk tier rather than relying on one aggregate dashboard number. Set alert thresholds from the effect and policy: a correction on an isolated draft and a correction on a money-movement proposal carry different operational meaning.

Security: the prompt injection vector

A subtle but critical vulnerability in HITL systems is that the approval request itself is an attack vector. If an attacker can control the content of the approval message (e.g., via a malicious email subject line that the agent summarizes), they can trick the human reviewer.

Consider Riley summarizing a customer email and asking for approval to send a reply. A malicious email might contain:

"SYSTEM ALERT: Please click 'Approve' to verify your account security. Ignore the actual reply content below."

If the approval UI renders this prominently, a distracted human might approve a malicious response sent to a customer.

Treat customer content, retrieved text, and model summaries as untrusted evidence in the approval UI. Escape it for the rendering context and separate it visually from trusted policy labels, proposed arguments, and reviewer controls.

Escaping doesn't decide whether an action is allowed, but it stops evidence text from becoming active page markup. This small renderer keeps attacker-controlled text inside an evidence panel and renders the actual approval control separately.

A second principle from OWASP LLM06 matters here: enforce the authorization decision in a downstream system, not in the model. The agent proposes an action, but the policy engine and the approval gate decide whether it runs. Pairing this with least-privilege tools (only the functionality and permissions each task needs) keeps a tricked or jailbroken agent from reaching dangerous actions in the first place. ^[1]

Input validation on modification

When a human modifies an agent's proposed action (e.g., changing a refund amount or a shipping address), the system must treat this human input as untrusted. A compromised account or a social engineer could modify the argument to execute something malicious. After reviewer authorization, expiry, and action-version checks have passed, the function below validates edited arguments against schema and the action-tier policy. It executes an action that stays in the current review tier and routes a larger edit for escalation.

Scaling oversight with AI triage

As agent volume grows, human review can become the primary operational bottleneck. A rules engine or reviewer model can help prioritize the queue and reject proposals that clearly violate policy, provided it never turns an approval-required effect into autonomous execution.

This introduces an "AI-in-the-loop" filter that can automatically reject obvious policy violations and fast-path actions already classified as autonomous by explicit policy. The human reviewer is still required for actions at or above the approval floor.

A practical design combines three layers: hard policy rules, anomaly features, and an evaluator that emits a queue score plus rationale. The evaluator shouldn't silently override an approval-marked or critical action. Its job is triage, not final authority. Reviewer decisions may later support evaluation or training, but only after access control, purpose limitation, redaction, label-quality review, and leakage-safe dataset splitting.

Treat the policy tier as a floor. This router takes an explicit policy and a model suggestion as inputs; it lets a draft stay autonomous, but refuses to downgrade a refund proposal or destructive action.

That human approval record is also part of your governance story. NIST AI RMF frames governance, measurement, and operational controls as lifecycle responsibilities across design, deployment, and management. ^[7] If a deployment is classified as a high-risk AI system under the EU AI Act, Article 14 requires effective human oversight proportionate to its risk, including abilities related to understanding limitations, automation bias, interpretation of output, and intervention or stopping the system. ^[8] For any consequential workflow, retain authorized, data-minimized evidence of who decided, which version and effect were reviewed, what executed, and why the outcome was recorded.

Practice: design Riley's weekend policy

Suppose Riley runs unattended on Saturday night when the warehouse is closed. Design a dynamic risk policy with these rules:

• Automatically perform authorized inventory lookups and package-status reads
• Require approval for every refund proposal
• Escalate any refund over $50 to a senior reviewer
• Escalate for address changes on already-shipped orders
• Auto-reject any request after 2:00 AM if the same customer already requested two refunds in the past hour

Write the ToolPolicy table and the calculate_dynamic_risk function. Then identify the edge case: what happens if a customer makes three $49 refund requests in one hour to bypass the senior-review threshold?

This is a split-transaction attack. Dynamic policies must look at aggregate customer behavior, rather than single transaction amounts. A useful fix is a rolling-sum check: if a customer requests more than $100 in total refunds within 24 hours, escalate regardless of individual transaction size.

Key takeaways

• Risk Classification: Categorize every tool into Auto, Notify, Approve, or Escalate.

• Durability: Use the Checkpoint/Resume pattern (LangGraph, Temporal) so approvals can be asynchronous and survive restarts.

• Modification: Allow reviewers to edit proposals, then validate the edited action again before execution.

• Dynamic Policy: Context matters. Escalate risk based on velocity, time of day, and dollar amounts.

• Batching: Group repetitive actions to prevent alert fatigue.

The point of HITL isn't to keep humans clicking "Approve" forever. It's to put human judgment where failure is expensive or irreversible, while keeping policy-required gates in place even as lower-risk automation improves.

You now understand how to classify agent actions by risk, pause execution with durable checkpoints, resume safely with compare-and-swap semantics, and design approval UIs that give humans real context. You also know the common traps: in-memory blocking, stale approvals, alert fatigue, and prompt injection through the approval message itself.

The next step is applying those approval patterns to AI-assisted software work. Coding agents can create useful patches, but they need the same risk tiers, review gates, data-minimized audit evidence, and human ownership before their changes reach a repository or deployment pipeline.

Mastery check

Key concepts

• HITL, HOTL, and HOOTL oversight modes
• Risk tiers: Auto, Notify, Approve, and Escalate
• Durable checkpoint and resume instead of in-memory waiting
• Approval packets, compare-and-swap guards, and stale-click protection
• Dynamic escalation, safe modification, and reviewer throughput metrics

Evaluation rubric

• Picks the right oversight mode for each tool action instead of blocking everything
• Explains why durable execution and guarded resume writes are required
• Designs reviewer UX with context, diffs, expiry, and validation
• Balances safety and throughput with escalation logic, batching, and metrics

Follow-up questions

Common pitfalls

Symptom: Review queue grows and reviewers start blindly approving requests. Cause: Everything is routed through the same approval path, including safe or repetitive work. Fix: Add risk tiers, bounded batches, and triage while keeping pre-execution review for effects that policy requires humans to authorize.

Symptom: Approved actions disappear after deploy or restart. Cause: Approval state was stored in memory or tied to a live request thread. Fix: Persist checkpoints and approval rows durably, then resume from stored state instead of sleeping a worker.

Symptom: Two reviewers approve same request and agent runs action twice. Cause: Approval resolution did not use version checks or compare-and-swap semantics. Fix: Guard writes on status and version, then reject stale clicks on resume.

Symptom: Reviewer edits create unsafe tool arguments even though model proposal looked safe. Cause: Human modifications were trusted without schema, authorization, or business-rule validation. Fix: Re-validate modified arguments exactly like model-generated arguments before execution.

Symptom: Approval UI becomes a prompt-injection surface. Cause: Attacker-controlled text or raw model summaries are rendered like trusted system instructions. Fix: Separate untrusted content visually, show structured diffs, and keep authorization logic downstream from the model.