AI Coding Workflow with Agents

Suppose a payment provider sends the same billing event twice. The application should create one invoice, not two. You hand billing_event_78291 to a coding agent, and it returns a confident pull request (PR) with a debounce on PayButton.tsx, a changed lockfile, and a green suite whose new test creates an invoice once. The duplicate still appears when the provider retries the webhook, because the server never enforces the rule that one event ID maps to one invoice.

The patch is valid TypeScript, and the suite is green. The workflow still failed because no change fixed the server behavior, no boundary kept unrelated files out, and no test replayed the same event. A reliable coding-agent workflow turns each requirement into evidence a reviewer can inspect.

View larger

Earlier lessons established this split. Code Generation & Sandboxing showed how to isolate generated code while it runs. Human-in-the-Loop Agent Architecture showed the same boundary for a risky write: a release assistant named Vega could propose promote_model, but a trusted host owned authorization.

A merge into main is that gated write again. In this workflow, the agent may prepare a local patch; publishing a branch or PR and merging it are separately authorized actions. Keep one question in view: can a reviewer point to a bounded change and replay its evidence before allowing it into main? The running example uses a fictional billing service whose internal handler returns an invoice ID. Replaying billing_event_78291 must return the original ID instead of creating another invoice. A real webhook may return only an acknowledgment; inspect persistent invoice state too.

Treat the agent as a worker inside a system

An inline coding assistant suggests text near your cursor. A coding agent combines a large language model (LLM) with tools that can inspect files, choose actions, edit the working tree, and run commands. Product labels overlap, so classify the enabled capabilities rather than the name. More reach means the workflow must define what the agent may touch, what it must prove, and who can release the result.

As checked on October 10, 2026, products expose that action surface in different ways. GitHub Copilot's cloud agent researches a repository, plans, edits a branch, and runs tests in an ephemeral cloud development environment.^{[1]Reference 1GitHub Copilot cloud agenthttps://docs.github.com/en/copilot/concepts/agents/cloud-agent/about-cloud-agent} Claude Code reads a codebase, edits files, runs commands, and works across terminal, editor, desktop, and web surfaces.^{[2]Reference 2Claude Code overviewhttps://code.claude.com/docs/en/overview} Their surfaces will change, but four responsibilities stay stable:

Research often describes a tool-using loop with the ReAct pattern: interleave reasoning about the next step with actions and observations.^{[3]Reference 3ReAct: Synergizing Reasoning and Acting in Language Modelshttps://arxiv.org/abs/2210.03629} For repository work, hidden reasoning isn't the review artifact. A reviewer can inspect the files read, diff produced, commands run, failures observed, and final evidence packet.

SWE-agent calls the tool layer an agent-computer interface (ACI) and shows that repository navigation, editing, and execution tools change software-engineering results.^{[4]Reference 4SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringhttps://arxiv.org/abs/2405.15793} Clear tool names, structured arguments, bounded output, and fast checks give the agent a better working environment. They improve execution; they don't grant authority.

For this bug, keep the path from diagnosis to review visible. A feature, documentation change, or open-ended investigation may need different evidence:

1. Contract scoping: Pin down the behavior invariant, owned files, non-goals, and reproduction verifier before any code changes.
2. Repository exploration: Trace call paths and locate definitions using indexed search tools rather than flooding the prompt context window.
3. Reproduction test creation: Write a failing test that reproduces the bug against the unpatched base code, establishing the red baseline.
4. Patch iteration: Make a focused change with an editing interface suited to the file; inspect the resulting diff.
5. Verification and handoff: Run the unchanged reproduction and regression checks, then return results tied to the tested source and environment.

Tool design affects exploration and editing, but doesn't determine success by itself. Start from the webhook route, follow the handler to invoice creation, and inspect the storage constraint. Targeted text search, Language Server Protocol (LSP) definition/reference queries, and Abstract Syntax Tree (AST) searches can help locate those links. Availability and results depend on language support, indexing, and the tool implementation. Neither a line count nor a search method establishes a universal token budget or millisecond latency.

Choose an editing tool that checks its assumptions. A patch can require matching context; a replacement tool can reject absent or ambiguous matches; a syntax-aware refactor can update related references. Each tool defines its own interface, so inspect the contract before supplying a target. Whole-file rewrites can omit unrelated content, while a four-line replacement can still be semantically wrong. Inspect the full diff and run the relevant check either way.

Write a task contract before code changes

Once the agent's role is clear, give it a contract before it edits. “Fix duplicate invoices” names a symptom, not an acceptance condition. Without more detail, the agent may propose changes to the frontend, payment configuration, dependencies, or deployment code. An underspecified request isn't permission for all those actions.

Write the contract around behavior, ownership, exclusions, checks, and execution policy:

Investigate duplicate invoice creation when billing_event_78291 is replayed.

Owned files

• web/src/billing/webhook.ts
• web/src/billing/webhook.test.ts

Non-goals

• Don't edit payment-provider configuration, auth, checkout UI, dependencies, or deployment files.
• Don't call production services or use production credentials.

Acceptance evidence

• Add a reproduction that replays one event ID twice.
• Show it failing before the patch and passing after the patch.
• Preserve normal invoice creation for a new event ID.
• Run pnpm test --run billing and pnpm lint.

Allowed tools and checks

• Repository read/search tools; patch tool restricted to the two owned files.
• pnpm test --run billing and pnpm lint through the approved runner.
• Host-controlled status and diff inspection, with external diff helpers disabled.
• No dependency installation, shell setup changes, commits, pushes, PR creation, or merge.

Delivery

• The host provides branch agent/billing-event-78291 and records its base revision in the restricted project runner.
• Report changed files, exact commands, outputs, and remaining risk.

Stop condition

• After three failed repair attempts after the baseline reproduction, escalate with the last failure, suspected missing context, and a proposed rescope. Stop sooner for denied capability, scope expansion, or suspected secret exposure.

The pnpm commands assume this example project's manifest defines a billing test selector and a non-fixing lint script. Inspect the actual scripts and test collection before adopting them elsewhere. The executable examples below are standalone Python checks; they don't run a TypeScript service or start a coding-agent product.

Read each field as a separate guardrail. The written contract guides the agent and reviewer; host enforcement is what blocks actions. Each field answers a different question:

File ownership is a review boundary, not a prediction that the bug must live in those files. If diagnosis shows that the invariant belongs in another module, the agent pauses and requests a rescope before editing it. That pause keeps a valid diagnosis from silently becoming a larger diff.

A narrower diff does not bind the command that produces the patch. Check whether that command can reach past the repository.

Separate execution from integration

A task contract narrows the patch. It doesn't stop a command from reaching beyond the repository. Three tools often get confused here: a Git branch, a Git worktree, and an execution sandbox.

A Git branch is a named reference to a commit, not a process boundary. pnpm test runs with the authority the operating system and runner grant it, regardless of branch name. On an unrestricted host that may include home-directory credentials and outbound network access. A branch can't restrict those capabilities or stop a production API call.

Git worktrees give the agent a separate checkout with its own index and HEAD, avoiding many working-file/index collisions. git worktree add <path> <branch> links that checkout to the same repository. Objects, most refs, and repository configuration are shared; default hooks can be shared too.^{[5]Reference 5git-worktree: Manage multiple working treeshttps://git-scm.com/docs/git-worktree} A process with filesystem access can still edit another checkout or shared Git state. Restrict that access separately, and coordinate operations on shared refs/configuration.

An execution sandbox constrains processes under its configured isolation model. Containers, application-kernel sandboxes, and microVMs provide different boundaries; isolation alone doesn't deny network requests, supplied credentials, or writable host mounts. Docker containers can make outgoing connections by default.^{[6]Reference 6Docker networking overviewhttps://docs.docker.com/engine/network/} For this billing task, configure and verify filesystem, credential, egress, resource, and lifetime limits independently. A network namespace by itself isn't an outbound deny policy.

View larger

For the billing task, the runner needs source files, fixtures, and named checks. It doesn't need production tokens, package publication, Terraform, email, or arbitrary outbound network access.

An allowed command string isn't an execution sandbox. pnpm test runs a manifest-defined script, which can load project code and dependencies; a lint script may even use --fix.^{[7]Reference 7pnpm runhttps://pnpm.io/cli/run} Review the scripts, pin dependencies in a trusted setup phase, and keep credentials, filesystem restrictions, and network policy in force for the check and its child processes. A modified test file can execute code too. Prefer host-selected executables and argument arrays over a model-built shell string, but don't confuse shell-injection prevention with safe execution of the program itself.

Enforce integration separately. On GitHub, configure required reviews and status checks on the protected target branch, and inspect bypass permissions. Dismiss stale approvals or require approval of the latest push according to team policy.^{[8]Reference 8About protected brancheshttps://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches} The runner identity shouldn't be able to change those settings or approve its own integration. A denied push confirms that the boundary is working. Route policy changes through the approved process.

The boundary must cover every tool. Some agents use the Model Context Protocol (MCP) to reach repository services, documentation, or other tools.^{[9]Reference 9Introducing the Model Context Protocolhttps://www.anthropic.com/news/model-context-protocol} An MCP server can expose credentials or side effects. Admit only task-appropriate capabilities, protect token audiences/scopes, and restrict local tool processes. Official MCP guidance discusses confused-deputy risks and process sandboxing for command-spawning proxies.^{[10]Reference 10Security Best Practiceshttps://modelcontextprotocol.io/docs/2026-07-28/tutorials/security/security_best_practices} A protocol connection alone does not provide a security boundary.

Resource text may inform a patch; it can't authorize a new command, widen file ownership, or grant production access. Apply the same rule to README.md, issue comments, fixtures, and generated logs. Explicitly admitted project instructions can guide style and checks within the task, but repository content can't grant itself higher privilege. OWASP identifies prompt injection and excessive agency as separate risks in LLM applications; a coding workflow needs both instruction isolation and limited capabilities.^{[11]Reference 11OWASP Top 10 for LLM Applications 2025https://genai.owasp.org/resource/owasp-top-10-for-llm-applications-2025/}

Close the loop with executable evidence

The agent now has a bounded task and a safe place to work. A plausible patch can still be wrong, so give it feedback tied to the behavior you asked it to change.

A verifier is a check that speaks directly to the behavior you're changing. For billing_event_78291, a useful, inexpensive verifier replays the same event twice, checks the returned identity, and counts stored invoices. Lint is useful, but it can't establish idempotency; a broad green suite may never exercise the reported retry.

View larger

Feedback only helps when the retry loop has an end. Without a budget, an agent can churn through speculative edits, inflate the diff, and spend compute while learning little. Here the expected baseline failure doesn't consume a repair attempt. Each patch-and-check cycle does. Classify timeouts and missing dependencies as environment failures, preserve that result, and request help instead of rewriting code until any command turns green.

Prove the invariant before and after

For this regression, first run the reproduction on the base implementation. A green result alone doesn't demonstrate that the check distinguishes the defective behavior. Inspect four traps:

1. Tautological assertions: The agent writes expect(response).toBeDefined() or checks for HTTP status 200. Both webhook deliveries return 200 OK, but the server minted two distinct invoice IDs in the database. The assertion never verifies identity or state.
2. Mock masking: The agent mocks the database with an in-memory dictionary or stubs the payment client so aggressively that the deduplication query is completely bypassed.
3. Selector zero-matches: The selected pattern collects no relevant tests. Exit behavior depends on the runner/configuration: Vitest fails an empty collection by default, while passWithNoTests: true permits success.^{[12]Reference 12Vitest configuration: passWithNoTestshttps://vitest.dev/config/passwithnotests} A wrapper or skipped suite can also produce a misleading success. Inspect what actually ran rather than inferring it from exit zero.
4. Setup errors masquerading as reproductions: The test fails on the base revision with exit code 1, but inspecting the logs reveals a syntax error or missing import rather than an assertion failure on duplicate invoices. When the patch fixes the import, the test passes, but the bug was never reproduced.

A reliable reproduction contract enforces a strict red-to-green transition:

• Red on base: The unchanged reproduction runs against the base implementation and fails specifically on the business invariant (retry != first), with the test ID and assertion in retained stdout/stderr or structured results.
• Green on head: The exact same test file runs against the candidate commit and passes without any modifications to test assertions.
• Regression checks: Relevant billing suites remain green on the candidate. Inspect adjacent paths separately because the suite only covers its selected behaviors.

Hold the input event constant so the behavior change is visible. The executable example runs one verifier against two in-memory implementations, then prints invoice IDs instead of a generic “tests pass.” A second event checks the neighboring contract: a genuinely new event still creates a new invoice. Checking stored count also catches a handler that returns the old ID while secretly creating another invoice. This is a sequential demonstration, not a checkout of two real service revisions.

from typing import Protocol

EVENT_ID = "billing_event_78291"
NEW_EVENT_ID = "billing_event_99002"

class InvoiceStore(Protocol):
    def create_for(self, event_id: str) -> str: ...
    def invoice_count(self) -> int: ...

class BuggyInvoiceStore:
    def __init__(self) -> None:
        self.invoices: list[str] = []

    def create_for(self, event_id: str) -> str:
        invoice_id = f"inv_{len(self.invoices) + 1}"  # ignores event_id
        self.invoices.append(invoice_id)
        return invoice_id

    def invoice_count(self) -> int:
        return len(self.invoices)

class FixedInvoiceStore:
    def __init__(self) -> None:
        self.by_event: dict[str, str] = {}

    def create_for(self, event_id: str) -> str:
        if event_id not in self.by_event:
            self.by_event[event_id] = f"inv_{len(self.by_event) + 1}"
        return self.by_event[event_id]

    def invoice_count(self) -> int:
        return len(self.by_event)

def replay(store: InvoiceStore) -> tuple[str, str, str]:
    first = store.create_for(EVENT_ID)
    retry = store.create_for(EVENT_ID)
    other = store.create_for(NEW_EVENT_ID)
    return first, retry, other

def run_receipt(label: str, store: InvoiceStore) -> bool:
    first, retry, other = replay(store)
    if retry != first:
        print(f"{label}: FAIL (retry {retry} != {first})")
        return False
    if other == first:
        print(f"{label}: FAIL (new event reused {first})")
        return False
    if store.invoice_count() != 2:
        print(f"{label}: FAIL (expected 2 stored invoices, got {store.invoice_count()})")
        return False
    print(f"{label}: PASS (retry kept {first}; new event {other}; stored 2)")
    return True

assert not run_receipt("before patch", BuggyInvoiceStore())
assert run_receipt("after patch", FixedInvoiceStore())

first, retry, other = replay(FixedInvoiceStore())
assert first == retry == "inv_1"
assert other == "inv_2"

before patch: FAIL (retry inv_2 != inv_1)
after patch: PASS (retry kept inv_1; new event inv_2; stored 2)

The dictionary isn't a production fix. It disappears on restart, isn't shared by multiple workers, and its check-then-insert isn't a storage transaction. Two workers can both observe an absent event and each create an invoice. A real implementation usually needs a durable uniqueness rule and an atomic operation over the deduplication key and invoice creation. If another service performs the side effect, a local database transaction alone can't make that remote call exactly once.

Choose the key from the actual provider contract, including account or tenant scope when needed. Event-ID deduplication covers delivery retries of the same event; distinct event IDs referring to the same business operation may require an additional business key. For the real patch, test concurrency, restart, and the crash between invoice creation and recording completion where those paths exist. None of those properties follows from the sequential example.

Before-and-after evidence must also identify what ran. Keep the new reproduction unchanged while running it against the base implementation and the candidate. Confirm the baseline fails on the duplicate-invoice assertion, not import, setup, or unrelated failures. Record collected test IDs, result, command arguments, environment, source revision or tree digest, and the verifier version. A command that exits zero after collecting no relevant tests proves nothing about retries.

View larger

The replay check is the strongest receipt for this bug, but other changes need different evidence:

Use coding benchmarks as context for local tests

The replay receipt answers a local question: did this patch fix this behavior? Coding benchmarks answer a broader question, and they shouldn't replace the local check.

SWE-bench introduced repository-level tasks drawn from real GitHub issues and pull requests.^{[13]Reference 13SWE-bench: Can Language Models Resolve Real-World GitHub Issues?https://arxiv.org/abs/2310.06770} SWE-bench Pro later added longer-horizon tasks across public, held-out, and proprietary repositories.^{[14]Reference 14SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?https://arxiv.org/abs/2509.16941} These benchmarks can compare a model plus its agent scaffold under a stated protocol. They don't prove that one patch fixed your webhook.

Benchmark interpretation also changes as suites and agent setups change. In February 2026, OpenAI reported test and task-description defects in an audit of 138 hard SWE-bench Verified problems, plus evidence that frontier models could reproduce gold patches or problem specifics, a sign of training-set contamination.^{[15]Reference 15Why we no longer evaluate SWE-bench Verifiedhttps://openai.com/index/why-we-no-longer-evaluate-swe-bench-verified/} The hard-problem sample wasn't a random audit of the full 500-task suite. OpenAI stopped reporting Verified scores and recommended SWE-bench Pro in that report. Apply contamination checks independently to each benchmark. Record the split, benchmark version, scaffold, tools, and attempt budget, then demand repository-specific evidence for each merge.

Review scope before implementation detail

When the agent returns a branch and receipts, start your review with the contract and artifacts. The agent's summary comes later:

1. Compare the complete candidate diff with owned paths, including deletions, both sides of renames, file-mode changes, and untracked files in a local handoff.
2. Check that before-and-after evidence exercises acceptance criteria.
3. Inspect sensitive surfaces such as auth, billing, dependencies, shell execution, workflows, and deployment.
4. Read implementation for correctness, failure handling, observability, and maintainability.
5. Confirm CI and any rendered or integration behavior that local checks couldn't prove, against the exact candidate being reviewed.

Scope comes first because a correct four-line fix can hide inside an unrelated eighteen-file patch. The helper below turns ownership and receipt metadata into early findings. Its inputs come from the host: the complete changed-path set and results from the test runner. Try the fixture's older-candidate receipt: a passing check on that revision can't validate the current head. This checker screens metadata; it doesn't authenticate logs, inspect code, or approve a merge.

from dataclasses import dataclass, replace
from pathlib import PurePosixPath

OWNED = {
    "web/src/billing/webhook.ts",
    "web/src/billing/webhook.test.ts",
}
REQUIRED_TESTS = frozenset({"same_event_replay", "new_event_creates_invoice"})
# Fixture identities, not hashes collected from a real repository.
BASE = "a" * 40
HEAD = "b" * 40
VERIFIER = "d" * 64

@dataclass(frozen=True)
class Receipt:
    revision: str
    verifier_digest: str
    collected: frozenset[str]
    executed: frozenset[str]
    failed: frozenset[str]
    exit_code: int | None  # None means interrupted or no terminal result.

def canonical_path(path: str) -> bool:
    parsed = PurePosixPath(path)
    return (bool(parsed.parts) and not parsed.is_absolute() and ".." not in parsed.parts
            and "\\" not in path and "\x00" not in path and str(parsed) == path)

def needs_focused_review(path: str) -> bool:
    parts = PurePosixPath(path).parts
    return (bool({"auth", "billing", "deploy"}.intersection(parts))
            or parts[:2] == (".github", "workflows")
            or PurePosixPath(path).name in {"pnpm-lock.yaml", "package.json"})

def screen_pull_request(
    changed_files: set[str],
    *,
    before: Receipt,
    after: Receipt,
    base: str,
    head: str,
    verifier_digest: str,
) -> list[str]:
    findings: list[str] = []

    if not changed_files:
        findings.append("no candidate changes supplied")
    if any(not canonical_path(path) for path in changed_files):
        findings.append("invalid repository-relative path")
    outside_scope = sorted(changed_files - OWNED)
    if outside_scope:
        findings.append(f"scope drift: {', '.join(outside_scope)}")

    sensitive = sorted(path for path in changed_files if needs_focused_review(path))
    if sensitive:
        findings.append(f"focused review: {', '.join(sensitive)}")

    if before.revision != base or after.revision != head:
        findings.append("receipt revision differs from reviewed base or head")
    if any(receipt.verifier_digest != verifier_digest for receipt in (before, after)):
        findings.append("receipt uses a different verifier")
    if any(receipt.collected != REQUIRED_TESTS for receipt in (before, after)):
        findings.append("expected test collection differs")
    if any(receipt.executed != REQUIRED_TESTS for receipt in (before, after)):
        findings.append("required behavior checks did not all execute")
    if any(receipt.exit_code is not None and type(receipt.exit_code) is not int
           for receipt in (before, after)):
        findings.append("invalid terminal exit-code metadata")
    if before.exit_code != 1 or before.failed != frozenset({"same_event_replay"}):
        findings.append("baseline did not show the expected behavior failure")
    if after.exit_code != 0 or after.failed:
        findings.append("candidate did not finish with both checks passing")

    return findings

before = Receipt(BASE, VERIFIER, REQUIRED_TESTS, REQUIRED_TESTS,
                 frozenset({"same_event_replay"}), 1)
after = Receipt(HEAD, VERIFIER, REQUIRED_TESTS, REQUIRED_TESTS, frozenset(), 0)
changed = {
    "web/src/billing/webhook.ts",
    "web/src/checkout/PayButton.tsx",
    "pnpm-lock.yaml",
}
stale_after = replace(after, revision="c" * 40)
for finding in screen_pull_request(changed, before=before, after=stale_after,
                                  base=BASE, head=HEAD, verifier_digest=VERIFIER):
    print(f"REVIEW: {finding}")

clean_findings = screen_pull_request(OWNED, before=before, after=after,
                                   base=BASE, head=HEAD, verifier_digest=VERIFIER)
print("scoped current packet:", clean_findings)
print("merge authorized:", False)
skipped_after = replace(after, executed=frozenset({"new_event_creates_invoice"}))
skipped_findings = screen_pull_request(
    OWNED, before=before, after=skipped_after,
    base=BASE, head=HEAD, verifier_digest=VERIFIER,
)
print("skipped replay rejected:", "required behavior checks did not all execute" in skipped_findings)
assert not canonical_path(".")
assert "invalid terminal exit-code metadata" in screen_pull_request(
    OWNED, before=before, after=replace(after, exit_code=False),
    base=BASE, head=HEAD, verifier_digest=VERIFIER,
)

REVIEW: scope drift: pnpm-lock.yaml, web/src/checkout/PayButton.tsx
REVIEW: focused review: pnpm-lock.yaml, web/src/billing/webhook.ts
REVIEW: receipt revision differs from reviewed base or head
scoped current packet: ['focused review: web/src/billing/webhook.test.ts, web/src/billing/webhook.ts']
merge authorized: False
skipped replay rejected: True

Here, scope and evidence freshness need different responses. A button debounce is out of scope; test server idempotency with concurrent duplicate requests. The lockfile changes the dependency surface without justification. Rerun the old-head receipt on the reviewed candidate. Even the scoped, current packet still flags billing for focused review. Integration authorization requires the designated review and checks; an automated screen reports only its own findings.

The path helper checks spelling, not filesystem containment or symlink targets. The runner must enforce actual read/write paths. The receipt fixture rejects boolean exit codes explicitly because Python otherwise treats False as zero; it still relies on host-supplied test identities and results.

The skipped fixture collected both named tests and exited zero with no failed tests, yet never executed the replay check. Collection establishes discovery, not execution. The host must obtain executed IDs and terminal outcomes from runner events or structured reports; it can't construct them by copying the collected IDs. This compact screen assumes failed describes the executed tests' terminal outcomes. A real adapter must also reject pending, skipped, interrupted, or expected-failure outcomes for required acceptance checks.

GitHub's required-status-check gate can accept a check with a skipped or neutral conclusion.^{[8]Reference 8About protected brancheshttps://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches} Configure a required workflow that reports failure when the acceptance checks don't execute; branch protection alone doesn't interpret your test collection or prove the replay ran.

A review packet should distinguish the identities behind each result:

1. Base revision: Identify the base implementation, plus the separately supplied unchanged reproduction. A new regression test usually isn't present in the base commit itself.
2. Candidate revision: Identify the proposed commit or tested merge result. Results on B remain evidence about B; they don't automatically validate a later C. Repository policy may require new checks on C even when its source tree matches B.
3. Source snapshot: git rev-parse HEAD^{tree} identifies the committed tree, not dirty, untracked, or ignored files. git write-tree identifies the index.^{[16]Reference 16git-write-tree: Create a tree object from the current indexhttps://git-scm.com/docs/git-write-tree} Test from a controlled checkout or inventory the actual snapshot, test overlay, dependencies, generated inputs, and relevant configuration. A Git object ID alone doesn't identify every input a process reads.
4. Verifier identity: Hash the unchanged reproduction so the runner can detect changed test bytes. Review assertion meaning, authenticate runner logs, and protect the test environment against hiding defects.

Try the snapshot claim directly. This temporary-repository exercise stages a file, changes its working copy, and creates an untracked input. The index tree stays unchanged. It creates no commit, remote, branch publication, or production action.

View larger

from pathlib import Path
from tempfile import TemporaryDirectory
import subprocess

with TemporaryDirectory() as directory:
    root = Path(directory)
    empty_hooks = root / "empty-hooks"
    empty_hooks.mkdir()

    def git(*args: str) -> str:
        return subprocess.run(
            ["git", "-c", "core.fsmonitor=false", "-c",
             f"core.hooksPath={empty_hooks}", *args],
            cwd=root, check=True, capture_output=True, text=True,
        ).stdout.strip()

    git("init", "--quiet")
    source = root / "handler.py"
    source.write_text("VALUE = 1\n")
    git("add", "handler.py")
    staged_tree = git("write-tree")
    source.write_text("VALUE = 2\n")
    (root / "local-input.json").write_text('{"mode":"fixture"}\n')

    assert git("write-tree") == staged_tree
    assert git("show", ":handler.py") == "VALUE = 1"
    assert source.read_text() == "VALUE = 2\n"
    assert "local-input.json" in git("ls-files", "--others", "--exclude-standard").splitlines()
    print("index tree unchanged:", True)
    print("working source differs from index:", True)
    print("untracked input absent from tree:", True)

index tree unchanged: True
working source differs from index: True
untracked input absent from tree: True

A trusted runner must bind actual source/test inputs, environment, collected test IDs, terminal results, and retained artifacts. Expected baseline failure needs the relevant assertion, not merely exit_code=1; runners may use other exit conventions. A model can report computed identifiers, but authoritative evidence must be obtained or verified outside its claims. Verify check execution from runner evidence. Branch rules gate integration only when required reviews/checks and bypass permissions are configured correctly.

Before integration, re-read the current head, required checks, reviews, and target-branch state. Test the merge result or use the repository's merge queue when a moving base matters. Bind the final authorization and integration write to the reviewed revision so a new push can't slip between them. This workflow reserves merge and deployment decisions for authorized humans; other teams may configure policy-controlled automation. The model's confidence doesn't grant either authority.

Diagnose common workflow failures

Use roles to define handoffs and runtime policy to control authority

Once a patch passes the first scope check, roles make the next handoff explicit. One agent can work in several passes, or several agents can own separate artifacts. Either way, each role should name the output it owes the next role:

A worker should run checks while coding. A later verifier pass still adds value because it begins from acceptance criteria and the final diff, not from the worker's edit path. Fresh context, different instructions, CI, or a human reviewer can make that pass more independent; calling the same model twice doesn't create independence by itself.

Self-review can catch omissions before handoff, but it isn't external verification. Reflexion is a more specific research pattern: evaluated trials produce verbal feedback that enters episodic memory for later attempts.^{[17]Reference 17Reflexion: Language Agents with Verbal Reinforcement Learninghttps://arxiv.org/abs/2303.11366} A single critique of the draft in front of the same agent is better described as self-review.

Parallel work needs coordinated ownership. Assign one writer to shared files or sequence the changes; another workflow might use explicit patch integration. Worktrees separate ordinary source edits, while shared databases, ports, caches, and fixture directories still need access controls or serialization. Verify the combined patch with integrated tests and review.

Start coding agents where feedback is local and judgment is bounded: narrow bug fixes, focused tests, one-module migrations, documentation synced from source, and contained refactors with strong coverage. Keep humans close to auth, billing, dependency policy, migrations, security boundaries, and architectural choices.

Assemble the merge packet

Return to billing_event_78291. “I'm confident” isn't a reviewable completion. The reviewer needs a short chain of artifacts:

The packet doesn't need a transcript of every agent thought. It needs enough evidence to reproduce behavior, plus enough provenance to identify the code, command, environment, verifier, and final revision behind each result. Redact secrets and cap noisy output before attaching logs. “Not run,” “timed out,” “failed,” and “passed on an older revision” are different states; retain them rather than collapsing all four into “checked.”

If the team later auto-merges a narrow class of low-risk generated patches, keep policy outside the model: explicit repository class, owned paths, required checks, rollback plan, audit trail, and a fast way to disable automation. “Agent confidence” isn't a merge policy.

Use the packet to adjudicate this final case:

Agent says: “Fixed duplicate invoices. All tests pass.” Diff changes the billing handler, PayButton.tsx, auth middleware, and the lockfile. A new test calls invoice creation once.