Human-in-the-Loop Agent Architecture

Vega, a release assistant for an internal AI platform, has prepared one change: move recommendation-reranker-v17 from a shadow deployment to a canary, a small slice of live traffic. A release manager might approve that change now, but Vega's worker may not run it for hours. During that pause, deployment state or reviewer session can change. A Boolean such as approved: true in chat history can't prove that the exact action is still safe.

Computer-Use & GUI Browser Agents can reject a proposed click before the host executes it. Vega needs a slower gate, one that can survive a long pause and recheck the world before it writes. Follow one proposal through that loop:

Vega may read authorized evaluation results and prepare this proposal, but it can't promote the model. A trusted host must pause before the write, save enough state for review, authenticate a qualified reviewer, and bind the decision to these exact arguments. The worker then reads current deployment state and executes with a separate narrow credential. That fresh-state check prevents a time-of-check-to-time-of-use (TOCTOU) failure, where an approved target changes before its queued action runs.

That sequence is Human-in-the-Loop (HITL) architecture. HITL doesn't prove Vega's proposal is correct. It gives policy, evidence, human judgment, and fresh-state validation a place to stop an unsafe effect.

OWASP LLM06:2025 Excessive Agency names three roots: too much functionality, too much permission, and too much autonomy. Mitigations include downstream authorization, least privilege, user-context execution, and human approval before high-impact actions.^{[1]Reference 1OWASP Top 10 for Large Language Model Applicationshttps://genai.owasp.org/llm-top-10/}

Function Calling & Tool Use introduced typed tool calls. Agentic Architectures introduced stateful control loops. Here, the model's call is an untrusted proposal inside a durable approval workflow.

The traffic thresholds are teaching fixtures, not universal release policy.

Choose oversight from possible effect

Before choosing a queue, ask when a person must be able to stop the effect. Human involvement usually sits before, during, or outside real-time execution. Choose among those positions from what can happen if the system is wrong, not from model confidence.

An unrestricted read isn't automatically safe: it may expose another tenant's data or a secret. HITL isn't automatically safe either. A reviewer who lacks authority, sees stale evidence, or approves an unbound action adds ceremony without control.

What is the difference between HITL, HOTL, and HOOTL?

Answer

HITL blocks before a high-risk action executes, HOTL lets the agent run while a human can monitor and interrupt, and HOOTL runs without real-time oversight for bounded, authorized work with acceptable failure modes.

Oversight mode says when a person can intervene. A policy tier must still decide which Vega actions need that intervention.

Put policy before approval

Routing every tool call to a reviewer creates fatigue. Routing none of them creates excessive autonomy. Start with a deterministic policy floor based on possible effect, then allow runtime context to move an action only to a stricter tier.

The model can choose a tool, but it can't choose its own authority. This policy table maps admitted tools to a minimum tier outside the model. Guardrails & Safety Filters covers the wider rule and enforcement layer. Here, policy has already returned APPROVE or ESCALATE, and we follow the resulting workflow.

The small Python model below turns that policy floor into a lookup. Its escalate_risk() helper is monotonic: a stricter runtime signal can raise a tier, never lower one.

Why should a HITL policy classify tool calls by risk tier instead of asking for approval on everything?

Answer

Approval on everything creates reviewer fatigue and slows safe work. Risk tiers let read-only or reversible actions run while costly, external, or irreversible actions pause for review.

Risk isn't static. A draft for an internal queue may stay autonomous, while publishing the same text externally needs approval. Arguments, tenant, cumulative traffic change, recent failures, and time window can raise the tier.

Runtime rules must never silently lower the static floor. The tool name alone still can't settle the decision.

Why is the tool name alone not enough to decide whether approval is required?

Answer

The same tool can have different risk based on arguments and context: audience, traffic percentage, endpoint tier, time of day, recent failures, or deployment history.

Once policy returns APPROVE, Vega needs a durable state machine transition. An open request handler or a Boolean attached to chat history can't carry one action safely across a long pause.

One approval moves through explicit states

An Approve button captures only the decision. The system also has to preserve one action's identity across a long pause and keep authorization separate from execution.

Use these states as a contract rather than a loose approved: true flag:

Keep these states distinct in the audit trail. authorized doesn't mean executed, and failed doesn't mean safe to retry without checking whether a downstream effect happened.

Pause without holding a process open

When Vega enters pending, ask what the worker should do while a reviewer is away. It has nothing useful to do, so it can't wait with time.sleep() or keep a request handler in memory.

1. Durability: Review can take hours. If the process restarts while Casey is still looking at the packet, in-memory state is gone.
2. Resources: A blocked thread isn't doing useful work. It's just occupying a worker.
3. Scale: Thousands of paused promotions can't each own a live process.

Use the checkpoint/resume pattern. Persist graph state and the approval record, return the worker, then reload by stable workflow ID after a decision. A guarded compare-and-swap (CAS) update ensures that only one current decision resolves the pending row. If the reviewer never arrives, the row should fail closed: expire and refuse the write instead of auto-promoting.

The pause is a conversation among four roles, not a function that blocks:

Why can't a production HITL agent wait with time.sleep() or an open Python thread?

Answer

Human review can take minutes or hours. In-memory waiting loses state on deploys or crashes, wastes resources, and prevents the system from scaling to many paused agents.

Vega's paused state might look like this:

When an eligible reviewer approves, the runtime resolves this version with a guarded write. A worker reloads the checkpoint, rechecks current deployment state, and attempts the action with an idempotency key.

Rejection closes the request without promotion. An edit validates new arguments, computes a new hash and version, and returns to pending; it doesn't reuse the old authorization.

Before building a workflow engine, test the persistence boundary with ordinary data. The checkpoint below stores a redacted summary and an action hash, but not the raw request message.

What should be present in a useful approval checkpoint?

Answer

A stable thread ID, requester and tenant scope, necessary redacted evidence, exact tool arguments, action hash, status, version, expiry, reviewer requirement, and enough workflow state to resume safely.

Two orchestration approaches expose the same lifecycle differently:

Framework primitives persist control flow, but they don't authenticate reviewers, choose risk policy, or grant downstream permissions. Those decisions remain application responsibilities.

Map the lifecycle to LangGraph

LangGraph's interrupt() emits JSON-serializable review data and pauses a graph compiled with a checkpointer. Resumption uses the same thread_id plus Command(resume=...).^{[2]Reference 2LangGraph Interruptshttps://docs.langchain.com/oss/python/langgraph/interrupts[3]Reference 3LangGraph Persistencehttps://docs.langchain.com/oss/python/langgraph/persistence}

On resume, the node restarts from its beginning. Code before interrupt() must therefore be idempotent or live in an earlier node. The primitives carry a decision back into the graph, but they don't authenticate the reviewer or grant production access.

LangChain documents middleware on top of the same interrupt primitive: approve, edit, reject, or respond. That wrapper can run edited arguments immediately.

Don't use respond to deny a write. respond looks like a successful tool result to the model. Treat an edit as a new proposal instead, because a 10% canary isn't the action the reviewer originally locked.

Use a custom decision payload when an edit should become a new proposal. The excerpt leaves graph construction and application helpers out so the pause/resume boundary stays visible.

Why must code before interrupt() be idempotent in LangGraph?

Answer

On resume, LangGraph restarts the node from its beginning before interrupt() returns the resume value. Any earlier side effect can run again unless it's idempotent or moved after the interrupt.

A LangGraph thread ID recovers graph state, but it doesn't say which human may decide. It also isn't the only way to wait. When a pause spans timers, retries, and other services, Temporal records the same lifecycle as event history.

Pause with Temporal when the wait spans services

LangGraph checkpoints can wait for hours. Temporal fits when the same workflow also needs durable timers, retries, and cross-service activities, not because "long wait" is a different problem.^{[4]Reference 4Temporal Workflow Execution Overviewhttps://docs.temporal.io/workflow-execution}

Signals are fire-and-forget. If the review UI needs to know whether the workflow accepted the click, or you want to reject a stale version before it lands in history, use an Update with a validator.^{[5]Reference 5Temporal Python SDK: Workflow message passinghttps://docs.temporal.io/develop/python/workflows/message-passing}

Multi-step chains, such as a release manager followed by a second approver, can wait for each required decision without holding a worker.

The workflow below uses an Update because the UI needs accept/reject feedback. Validation of edited arguments and current deployment state still happens in an Activity, not inside workflow code.

The validator rejects a stale version and a second click while the first accepted Update is waiting to be consumed.

Keep policy lookups deterministic inside the workflow. If is_risky() depends on live metrics or a policy database, fetch that data in an Activity and pass the snapshot in.

Temporal replays workflow code against event history, so a non-deterministic call inside the workflow breaks replay.^{[4]Reference 4Temporal Workflow Execution Overviewhttps://docs.temporal.io/workflow-execution} For months-long chains, Continue-As-New caps history size while carrying forward unresolved approval state.

When should a Temporal approval use an Update instead of a Signal?

Answer

Use an Update when the approval UI needs synchronous confirmation that the workflow accepted or rejected the decision. Signals are better for fire-and-forget messages.

A thread ID or workflow ID recovers paused work. Neither one names the human who may click Approve, or the credential that may write to production.

Keep four identities separate

Vega's thread ID tells us which workflow resumes, not who may authorize it or which credential may execute it. Approval intent, human identity, and tool capability are different objects.

Combining them creates a confused-deputy path where a valid click can authorize the wrong tenant or give Vega a reusable production credential.

OWASP's excessive-agency guidance makes both user context and least privilege explicit: downstream systems should enforce authorization for the current user scope, and extensions should receive only permissions needed for the task.^{[1]Reference 1OWASP Top 10 for Large Language Model Applicationshttps://genai.owasp.org/llm-top-10/} For Vega, bind requester_subject, tenant_id, resource scope, reviewer_subject, and executor identity to separate audit fields.

High-impact actions also need separation of duties (SoD). An eligible release manager still can't sole-approve a promotion they requested. Critical changes may require two distinct qualified reviewers.

A break-glass route needs its own narrow role, expiry, reason, alert, and later review. A free-text note isn't break-glass authorization.

Record decision before execution

The decision endpoint authenticates the reviewer from the server session, loads the pending record, checks tenant and role, and performs one guarded transition. It doesn't accept reviewer identity from the request body or call the production tool inline.

The compare-and-swap update should include id, status = 'pending', version, action_hash, and non-expired time, then require exactly one affected row. That conditional write decides which reviewer wins a race.

Displaying a shortened digest is fine, but storage and comparisons need the full digest. authorized stays separate from executed, and the approval update plus outbox insert commit together. A reconciler can detect damaged invariants, but it isn't a substitute for atomic persistence.

Reviewer eligibility also deserves an executable check. It takes server-derived subjects, tenant, and roles as inputs, then rejects self-approval even when the requester has the required role.

Why does HITL fail if the proposer can also sole-approve?

Answer

A compromised requester-plus-agent path that can approve its own request defeats the gate. Separation of duties keeps a high-risk effect dependent on a distinct qualified reviewer, or on dual control when policy requires it.

Now make the two transitions visible: authorization resolves one version, and execution reuses the recorded result on retry. The example takes a versioned decision and an idempotency key, then rejects a stale click.

Version matching isn't an expiry check. Compare current time with the review deadline, and reject a decision after that window closes.

Why does the approval endpoint need compare-and-swap semantics?

Answer

Approval records are shared state. A guarded update on status and version prevents stale clicks or two reviewers from resolving the same action twice.

Backend guards protect the transition. The reviewer still needs a card that makes the exact effect and the evidence boundary obvious.

Design review card around exact effect

A Yes/No card isn't enough. When Vega asks to route 25% of traffic to reranker-v17, the reviewer needs enough trusted context to predict the effect: policy facts, current state, exact before/after arguments, permitted evidence, and the consequence of each decision.

Show the smallest packet that supports the decision: a redacted request summary, policy trigger, links the reviewer may open, proposed mutation, requester scope, required role, expiry, and a dry-run or diff. Don't dump the entire incident thread or model history into the card. Extra context can expose sensitive data and hide the change that matters.

The packet is also a data-minimization boundary. The example below takes a deployment note and exact action, redacts the email, and emits a stable hash that binds the card to the proposed mutation. For a data write, render a diff, not a blob of JSON.

What should an approval UI show beyond Approve and Reject buttons?

Answer

It should show a redacted request summary, proposed tool and arguments, policy rule, risk tier, diff or dry-run effect, action hash, version, expiry, and enough authorized evidence for a reviewer to decide.

Give the reviewer a concise rationale, the exact tool arguments, a diff, and the policy rule that triggered review. Don't make raw chain-of-thought your approval primitive. Explanations can be unfaithful, and they may reveal internal reasoning or sensitive context you didn't intend to surface.^{[6]Reference 6Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptinghttps://arxiv.org/abs/2305.04388}

Why shouldn't raw chain-of-thought be the main approval artifact?

Answer

It may be unfaithful, verbose, sensitive, or misleading. Reviewers need structured action context, evidence, diffs, and policy triggers rather than private reasoning text.

Also keep approval state compact. Persist a data-minimized audit record with access controls and a retention policy. Inject only structured fields such as decision, reason, modified_args, or a short reviewer summary back into the runtime.

Long-lived threads otherwise accumulate reviewer chatter that burns context window on approval metadata instead of task state.

A compact card still isn't enough if the queue is noisy. Live context and batching shape what reviewers see; reviewer metrics tell us whether they keep catching real mistakes.

Escalate, batch, and measure the queue

Pause and resume is the core mechanism. A busy queue adds three design questions: when should risk escalate, which proposals can share a review, and how will we know whether reviewers are helping? Dynamic escalation, bounded batches, and reviewer-load metrics answer them without weakening the policy floor.

Dynamic risk escalation

Static policies need runtime context. Publishing one external incident update may already require approval. Attempting hundreds of updates in a short interval should move into a stricter queue or be blocked outright. The examples start from a static policy floor and raise the required authorization layer when context adds risk.

A 50% traffic shift moves from APPROVE to ESCALATE. An otherwise autonomous eval read moves to APPROVE after an anomaly spike. Disabling a safety filter stays at ESCALATE even when the night looks quiet. escalate_risk() implements that monotonic rule.

A dynamic policy starts from the static floor, then raises the tier for blast radius, recent failures, and off-hours promotions. Each output combines that floor with a runtime signal:

Why should dynamic risk logic promote requests to stricter tiers rather than silently downgrade them?

Answer

Runtime context is best used as an extra safety signal. Downgrading a policy-marked high-risk action can bypass the static control that was created for known-dangerous operations.

Batch approvals

If Vega needs to propose 50 low-risk release-note publications after a migration freeze, asking for separate decisions for each one creates reviewer fatigue. Reviewers who face repetitive requests may stop inspecting individual effects. A batch review can group related proposals without hiding the service, audience, destination, or policy status of each item.

Instead of creating one alert per proposed operation, the orchestrator collects pending actions over a bounded window or groups them by a shared task identifier. The UI can then present: "Vega proposes 50 release-note publishes (review all items)." The approval record must bind to the exact item list or item hashes, so an item inserted after approval can't ride along in the batch.

Implementing batch approvals requires per-item state and idempotency. If a batch contains 50 actions and the reviewer authorizes the fixed set, the orchestrator must track each action separately. If action 42 fails, already-completed publishes aren't automatically undone unless the operation provides a compensation path. Retry only the failed authorized item with its idempotency key, and re-request review if its inputs or destination changed.

Bind a batch approval to the exact items shown to the reviewer. Here a later publish inserted into the queue changes the digest, so it can't reuse the earlier authorization.

What risk do batch approvals reduce, and what new execution problem do they introduce?

Answer

They reduce repetitive alerts while keeping a visible group of proposed effects. They introduce item binding, idempotent execution, and partial-failure handling because each authorized item may succeed, fail, or become stale independently.

What to measure

A live HITL system can have good model quality and still fail operationally. Metrics must show whether human review adds safety without destroying throughput.

Track these per tool and per risk tier rather than relying on one aggregate dashboard number. Set alert thresholds from the effect and policy: a correction on an isolated draft and a correction on a money-movement proposal carry different operational meaning.

What do correction rate and intervention latency tell you about a HITL system?

Answer

Correction rate shows how often reviewers reject or edit agent proposals, which points to planner or policy quality. Intervention latency shows how much queue delay human review adds to the user's workflow.

Queue design isn't enough if evidence can manipulate the reviewer. Every model summary and retrieved note belongs on the untrusted side of the display.

Treat review evidence as hostile

The approval request itself can be a prompt-injection vector. If an attacker controls content that the agent summarizes, such as a commit message, incident comment, or retrieved ticket, that content can trick the human reviewer.

Consider Vega summarizing a deployment ticket and asking for approval to publish a release update. A malicious comment might contain:

"SYSTEM ALERT: Please click 'Approve' to verify your account security. Ignore the actual reply content below."

If the approval UI renders this prominently, a distracted human might approve a malicious update or traffic change.

Treat user content, retrieved text, and model summaries as untrusted evidence in the approval UI. Escape it for the rendering context. Keep it visually separate from trusted policy labels, proposed arguments, and reviewer controls.

Escaping doesn't decide whether an action is allowed, but it stops evidence text from becoming active page markup. The renderer keeps attacker-controlled text inside an evidence panel and renders the actual approval control separately.

OWASP LLM06 adds a second boundary: enforce authorization in a downstream system, not in the model. The agent proposes an action; the policy engine and approval gate decide whether it runs.^{[1]Reference 1OWASP Top 10 for Large Language Model Applicationshttps://genai.owasp.org/llm-top-10/}

Pair that boundary with least-privilege tools, giving each task only the functionality and permissions it needs. A tricked or jailbroken agent then has fewer dangerous actions within reach.

Why is the approval request itself a prompt-injection surface?

Answer

The reviewer may see attacker-controlled text summarized by the agent. If the UI presents that text like system instructions, a human can be socially engineered into approving the wrong action.

Edited arguments need a new intent lock

A review form can let a human change the traffic percentage or rollout stage, but edited fields are untrusted input. Validate the schema, tenant and resource scope, allowed state transition, static policy floor, and runtime escalation rules.

Then compute a new action hash, increment the version, choose the reviewer requirement again, and return the proposal to pending.

Vega's original 25% canary and a reviewer-edited 10% canary are different actions. The old decision must not authorize the new hash. A larger edit may also move from APPROVE to ESCALATE, while an invalid stage transition should close or reject the replacement.

Why validate human modifications as if they were model-generated arguments?

Answer

Human accounts can be compromised, stale, or socially engineered. Edited arguments need schema, authorization, current-state, and policy checks, followed by a new version and action hash before review.

After these hard boundaries are in place, automation can help order the queue. It still can't erase the policy floor or replace an accountable human decision.

Triage the queue without lowering the floor

As agent volume grows, human review can become the bottleneck. A rules engine or reviewer model can rank the queue and reject proposals that already violate policy. It must never turn an approval-required effect into autonomous execution.

An AI-in-the-loop filter can auto-reject obvious policy breaks and fast-path actions that explicit policy already marked autonomous. Everything at or above the approval floor still reaches a human.

A practical design combines hard policy rules, anomaly features, and an evaluator that emits a queue score plus rationale. The evaluator shouldn't silently override an approval-marked or critical action. Its job is triage, not final authority.

Reviewer decisions may later support evaluation or training, but only after access control, purpose limitation, redaction, label-quality review, and leakage-safe dataset splitting.

Treat the policy tier as a floor. This router takes an explicit policy and a model suggestion as inputs; it lets a draft stay autonomous, but refuses to downgrade a model-promotion proposal or destructive action.

What should an AI reviewer model be allowed to do in a HITL queue?

Answer

It can triage obvious cases, rank queue priority, or fast-path low-risk items covered by explicit policy. It shouldn't override critical policy gates or become final authority for high-risk actions.

The human approval record also belongs in governance evidence. NIST AI RMF 1.0 asks organizations to document human-AI roles (GOVERN 3.2), define and assess oversight processes (MAP 3.5), and measure whether those controls still work.^{[7]Reference 7Artificial Intelligence Risk Management Framework (AI RMF 1.0)https://www.nist.gov/itl/ai-risk-management-framework}

The EU AI Act doesn't classify every agent workflow as high risk. When a separate legal analysis places a system in that category, Article 14 requires effective oversight proportionate to risk and context, including the ability to understand limitations, resist automation bias, interpret output, override it, and stop the system safely.^{[8]Reference 8EU AI Act: Regulation laying down harmonised rules on artificial intelligencehttps://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689} Consequential workflows should retain authorized, data-minimized evidence of who decided, which version and effect they reviewed, what executed, and why the outcome was recorded.

Diagnose broken approval workflows

Once intent, identity, durability, and execution are separate, production failures become easier to localize.

The final pre-write check asks three independent questions: does the approval record still match, does the current deployment state still permit the transition, and does the executor identity have the minimum scope for this tenant and resource? One yes can't substitute for the other two.

Practice: design Vega's weekend policy

Suppose Vega runs unattended on Saturday night during a migration freeze. The weekday floor still stands: authorized reads can run, every promotion needs approval, and a required safety-filter change escalates. Add weekend rules:

• Escalate any promotion over 10% traffic
• Auto-reject write-tier requests after 2:00 AM if that service already had two failed rollouts in the past hour
• Catch threshold splitting: three 9% shifts in one hour shouldn't sneak past senior review

Inspect a timestamped one-hour window alongside the current traffic_percent, rather than trusting one aggregate field. Each event records incremental traffic added by a promotion. Use (now - 1 hour, now]: an event exactly at the left boundary is excluded, an event at now is included, and future events are ignored. The first 9% proposal stays at approve. A second 9% proposal sees 9% already accumulated inside the window, crosses 10%, and escalates. At 3:00 AM, a promotion after two failed rollouts is rejected instead of waiting in a dead queue. Authorized reads still run. Because new_traffic_24h_percent had no timestamps, it couldn't express any of those boundaries.

The host should load traffic_events from a trusted, service-scoped deployment ledger. Neither the model nor the incoming action should be able to rewrite the history used to classify its own risk.

How does this policy define its one-hour traffic window, and why does the boundary matter?

Answer

It sums incremental promotion events with timestamps in (now - 1 hour, now]. An event exactly at the left boundary is excluded, while an event at now is included, so adjacent windows don't count one event twice or miss a current event. The current request is added to that sum before comparing it with 10%.