Causal Inference and Counterfactuals

An AI assistant resolves 54% of its tickets, while the old workflow resolves 76%. A release report calls the assistant harmful and recommends pulling the plug. Then the tickets are split by pre-routing risk: the assistant's observed resolution rate is ten percentage points higher inside both easy and difficult groups. Which comparison should drive the rollout decision?

Experiment Design and A/B Testing Foundations introduced fair comparisons. Causal inference asks whether an action caused an observed outcome, then makes the assignment assumptions behind that conclusion mathematically explicit.^{[1]Reference 1Causal Inference in Statistics: A Primerhttps://bayes.cs.ucla.edu/PRIMER/} You can't rely on the unadjusted aggregate label on the report.

Watch the aggregate reverse the group results

Work with a synthetic log of 200 tickets. Treatment means routing a ticket to a fixed assistant workflow; control means routing it to the existing human workflow. Success means resolution within 24 hours, measured the exact same way in both arms. Risk is recorded before routing, not inferred from the assistant's response.

Within each risk group, the assistant's observed success rate is ten percentage points higher. Before looking at the totals, predict what the aggregate would say if both arms had the same risk mix.

The totals compare different populations. The assistant receives 90 difficult tickets; control receives 90 easy tickets. That unequal mix flips the aggregate direction.

This reversal is Simpson's paradox: group composition changes the direction of the aggregated association. Observational counts alone still don't establish a causal improvement.

The next snippet computes both within-group differences and the reversed aggregate. It reports percentage points, not relative percentage changes: 90% minus 80% is 10 points, whereas the relative increase is 10/80 = 12.5%.

The reversal disappears once both workflows are compared against the same risk distribution. That repairs the visible mix, but it still leaves a deeper question: for one assistant-handled ticket, what would control have done?

Pearl's ladder: association, intervention, and counterfactuals

Standard machine learning models optimize conditional expectations like $P(Y \mid X)$. When evaluating systems, this observational conditioning mixes the effect of the model with the behavior of upstream routers, triage heuristics, and customer selection. Judea Pearl organized causal queries into three distinct levels known as the Ladder of Causation.^{[1]Reference 1Causal Inference in Statistics: A Primerhttps://bayes.cs.ucla.edu/PRIMER/}

Rung 1: association (seeing)

Observational queries ask: If we see event $X$, what's our belief about $Y$? In notation, this is $P(Y \mid X)$. This is the native language of statistical learning, correlation, and passive monitoring. In the support log, $P(Y=1 \mid A=1) = 0.54$. This number answers what happened to tickets observed under the assistant, but it reflects the difficult queue the assistant was assigned to rather than the tool's standalone capability.

Rung 2: intervention (doing)

Interventional queries ask: What happens to $Y$ if we actively set variable $X$ to value $x$? Pearl denotes this operation with the $\operatorname{do}(\cdot)$ operator: $P(Y \mid \operatorname{do}(A=1))$. Intervening isn't passive conditioning. An intervention cuts the natural causal arrows flowing into $A$, replacing the upstream routing rule with an external mandate. If we force every incoming ticket to the assistant regardless of risk, the resolution rate is $P(Y=1 \mid \operatorname{do}(A=1))$, which doesn't equal $P(Y=1 \mid A=1)$ whenever confounding exists.

Rung 3: counterfactuals (imagining)

Counterfactual queries ask retrospective questions about specific units: Given that ticket #102 received the legacy control workflow and failed to resolve within 24 hours, what would have happened if we had routed that exact ticket to the assistant? In notation, this queries $P(Y_{A=1} \mid A=0, Y=0)$. Rung 3 lives in hypothetical alternative worlds. It enables individual credit assignment, root-cause debugging, and algorithmic fairness audits that observational data alone can't answer.

A dashboard shows that users who trigger an LLM code explanation feature retain 15% better than users who don't. Does this justify rolling out the feature to all users?

Answer

No. That comparison sits on Rung 1: P(Retain | Feature = 1) vs P(Retain | Feature = 0). Highly motivated users might use the feature more frequently. Deciding a rollout requires an interventional query on Rung 2: P(Retain | do(Feature = 1)), which separates user motivation from feature impact.

Separate observed outcomes from missing alternatives

Take one difficult ticket that received the assistant and resolved within 24 hours. Its observed outcome is 1. Define $A=1$ for assistant assignment and $A=0$ for control. The potential outcomes $Y(1)$ and $Y(0)$ are that ticket's binary resolution outcomes under the two workflows. Their difference $Y(1)-Y(0)$ is its individual treatment effect.

Only one potential outcome is ever observed for a given unit. Here $Y(1)=1$, but the counterfactual $Y(0)$ could be 0 or 1. The assistant may have caused this success, or the ticket may have resolved either way. Another control ticket can help estimate an average, but can't reveal this ticket's missing outcome. This limitation is known as the Fundamental Problem of Causal Inference.^{[2]Reference 2Causal Inference: What Ifhttps://miguelhernan.org/whatifbook}

Causal estimands: ATE and ATT

The average treatment effect averages those individual differences across the entire population:

$$\operatorname{ATE} = \mathbb{E}[Y(1) - Y(0)]$$

For binary resolution, an ATE of 0.10 means ten additional resolutions per 100 tickets on average under assistant assignment versus control.

Engineers also inspect the Average Treatment Effect on the Treated (ATT):

$$\operatorname{ATT} = \mathbb{E}[Y(1) - Y(0) \mid A = 1]$$

When treatment assignment is completely randomized, ATE and ATT coincide. When routing policies select specific subgroups, they can diverge sharply. If the assistant provides a larger boost on difficult tickets than on easy tickets, and the triage system routes mostly difficult tickets to the assistant, ATT will exceed ATE.

Latent response types and the counterfactual matrix

Because individual counterfactuals are hidden, every unit in a binary-outcome setting belongs to one of four latent response types:

1. Always Resolves (Immune): $Y(1)=1, Y(0)=1$. The ticket succeeds under either workflow.
2. Helped (Beneficiary): $Y(1)=1, Y(0)=0$. The assistant succeeds where legacy control fails.
3. Harmed (Adverse): $Y(1)=0, Y(0)=1$. Legacy control succeeds, but the assistant fails.
4. Never Resolves (Doomed): $Y(1)=0, Y(0)=0$. The ticket fails under both workflows.

Consider two hypothetical populations of 100 tickets, with both potential outcomes filled in for illustration:

Both populations produce 78 assistant successes and 68 control successes. In Population A, ten tickets benefit and zero are harmed. In Population B, twenty benefit and ten are harmed. The net gain is ten points in both cases. Even an ideal randomized experiment estimating marginal averages can't reveal which pairing of potential outcomes is true without structural assumptions like monotonicity ($Y_i(1) \ge Y_i(0)$ for all $i$).

The next snippet verifies both populations from their underlying counts:

Why can't the control group's 76% aggregate success rate serve as the missing outcome for every assistant ticket?

Answer

The control group contains a much larger share of easy tickets, so its aggregate doesn't provide a comparable average for the assistant-assigned cases. Even a perfectly comparable control group identifies an average, not the missing binary outcome of each individual ticket.

Causal graphs, d-separation, and the three canonical junctions

The routing table shows an imbalanced mix, but not why routing produced it. Suppose the triage system sends difficult tickets to the assistant more often. Risk exists before routing and influences both assignment and resolution, making it a confounder.

Directed acyclic graphs (DAGs) encode causal assumptions as non-parametric structural equations. Arrows represent direct causal influence, not correlations. To understand how statistical dependencies flow through graphs, Pearl developed the rules of d-separation (directional separation), built from three elementary three-node junctions.^{[1]Reference 1Causal Inference in Statistics: A Primerhttps://bayes.cs.ucla.edu/PRIMER/}

1. The chain (mediator): $A \to M \to Y$

Treatment $A$ causes an intermediate variable $M$, which in turn causes outcome $Y$. For example, the assistant ($A$) produces an automated diagnostic summary ($M$), which helps staff resolve the ticket ($Y$).

• Unconditioned state: The path is active. Association flows from $A$ to $Y$, transmitting the causal effect.
• Conditioned state: Conditioning on mediator $M$ blocks the path. Holding the summary fixed isolates any direct effect $A \to Y$, but blocks the indirect mechanism through which the assistant works. If you adjust for a mediator when estimating the total effect, you wipe out the very benefit you're trying to measure.

2. The fork (confounder): $A \leftarrow C \to Y$

A pre-treatment common cause $C$ influences both treatment assignment $A$ and outcome $Y$. In our triage system, baseline ticket risk ($C$) determines routing ($A$) and changes resolution difficulty ($Y$).

• Unconditioned state: The path is active. Association flows through the backdoor $A \leftarrow C \to Y$, creating a non-causal association that produces Simpson's reversal.
• Conditioned state: Conditioning on confounder $C$ blocks the backdoor path. Stratifying by risk allows within-stratum comparisons to isolate the true causal effect.

3. The collider (inverted fork): $A \to K \leftarrow Y$

Two independent variables point to a shared effect $K$. For instance, suppose an automated quality audit flag ($K$) is triggered if the assistant was used ($A$) or if the ticket failed to resolve ($Y$).

• Unconditioned state: The path is naturally blocked. No association flows between $A$ and $Y$ through $K$.
• Conditioned state: Conditioning on collider $K$ (such as filtering an evaluation dashboard to only tickets that triggered quality audit flags) opens an artificial path between $A$ and $Y$. This phenomenon is Berkson's bias or selection bias. If an audited ticket didn't fail, it probably used the assistant; this induces an artificial negative correlation between assistant usage and ticket resolution!

The backdoor criterion

Pearl formalized when a set of observed covariates $Z$ is sufficient to identify the causal effect of $A$ on $Y$:^{[1]Reference 1Causal Inference in Statistics: A Primerhttps://bayes.cs.ucla.edu/PRIMER/}

A set of variables $Z$ satisfies the backdoor criterion relative to an ordered pair of variables $(A, Y)$ in a DAG if:

1. No node in $Z$ is a descendant of $A$.
2. $Z$ blocks every path between $A$ and $Y$ that contains an arrow pointing into $A$ (backdoor paths).

If $Z$ satisfies this criterion, the causal effect is identified by the backdoor adjustment formula:

$$P(Y = y \mid \operatorname{do}(A = a)) = \sum_{z} P(Y = y \mid A = a, Z = z) P(Z = z)$$

Ticket risk is recorded before routing, while an escalation note is created by the assistant after routing. Which variable can support backdoor adjustment, and why can't the other replace it?

Answer

Pre-treatment risk can block the common-cause fork when it captures the relevant assignment and outcome differences. The assistant-generated escalation note occurs after treatment, so conditioning on it can block the mediator path or act as a collider, introducing new selection bias.

Standardize both arms to one population

Suppose the planned deployment has 70% low-risk and 30% high-risk tickets. What would each workflow's resolution rate be at that same mix? The assistant rate becomes $0.7(0.90)+0.3(0.50)=0.78$; the control rate becomes $0.7(0.80)+0.3(0.40)=0.68$. Their standardized difference is:

$$\widehat{\Delta}_{\text{standardized}} = 0.7(0.90-0.80) + 0.3(0.50-0.40) = 0.10$$

The hat marks an estimate from observed data. Both within-group differences are ten points, so this weighted difference is ten points for any shared risk mix. The individual arm rates depend on the mix, and unequal subgroup differences would make the estimated effect depend on it too. Calling this difference an identified causal effect requires explicit identification conditions.^{[2]Reference 2Causal Inference: What Ifhttps://miguelhernan.org/whatifbook}

The next snippet continues the earlier session and applies one shared population distribution to both arms:

The arithmetic answers a target-population comparison. It estimates an identified causal effect only under explicit assumptions:

The smallest strata contain only ten tickets. If one low-risk assistant success became a failure, its rate would fall from 90% to 80%, and the standardized difference would fall from ten points to three. That's a sensitivity check, not a formal confidence interval, but it shows why the estimate needs uncertainty analysis before an engineering rollout. More data reduce sampling variance; they don't remove unmeasured confounding.

Both observed risk groups show a ten-point advantage, but a hidden customer-tier field affects both assistant routing and resolution. Does standardizing only by recorded risk identify the causal effect?

Answer

Not necessarily. The arithmetic still returns a risk-adjusted ten-point association, but an unmeasured common cause violates conditional exchangeability. Without measuring and adjusting for customer tier, or switching to a randomized design, that association isn't guaranteed to be causal.

Weight the assignment, then check its support

Standardization averages group-specific outcomes at a chosen mix. Inverse Probability of Treatment Weighting (IPTW) instead weights each observation by the inverse of the probability of receiving the treatment it actually received. The propensity score is the conditional assignment probability $e(r) = P(A=1 \mid R=r)$, not the probability of successful resolution.^{[2]Reference 2Causal Inference: What Ifhttps://miguelhernan.org/whatifbook}

Estimate it from the log: low-risk tickets went to the assistant 10 times out of 100, so $\hat{e}(\text{low}) = 0.10$. High-risk tickets went 90 times out of 100, so $\hat{e}(\text{high}) = 0.90$. An assistant ticket gets weight $1/\hat{e}(r)$; a control ticket gets $1/(1-\hat{e}(r))$.

The ten low-risk assistant tickets each get weight 10, for total weight 100. The ninety high-risk assistant tickets each get weight $1/0.9$, also totaling 100. The control arm gets the same 100/100 weighted mix. These weights recover the logged population's 50%/50% risk mix, not the planned deployment's 70%/30% mix. Weighting and standardization agree when they target the same population.

This snippet works directly with the grouped counts:

Positivity violations and extreme weights

If high-risk tickets always receive the assistant, then $e(\text{high}) = 1$: no high-risk control outcomes can appear in the logs. This violates positivity (overlap). Without examples from both actions, no reweighting can identify the counterfactual without extra parametric assumptions.

Even non-zero probabilities can destabilize estimators. A rare assignment probability of 0.005 yields an inverse weight of 200, allowing a single noisy ticket to dominate the estimate. Production systems inspect weight distributions and employ stabilized weights (the Hajek estimator, normalizing weights to sum to one within each arm) or apply weight clipping, accepting a small amount of bias to rein in variance.

The weighted assistant rate is 70%, while the 70/30 standardized assistant rate was 78%. Did one calculation fail?

Answer

No. The inverse-probability weights here target the log's 50/50 risk mix: 0.5(0.90) + 0.5(0.50) = 0.70. Deployment standardization targets 70/30: 0.7(0.90) + 0.3(0.50) = 0.78. The differences between arms happen to agree because both risk groups share the same ten-point difference.

Instrumental variables when unmeasured confounders lurk

What happens when an unmeasured confounder $U$ (like customer technical frustration or urgency) affects both treatment routing $A$ and resolution $Y$? Conditional exchangeability fails because $U$ isn't recorded in the logs. Backdoor adjustment and IPTW can't eliminate the bias.

An instrumental variable (IV) $Z$ offers an alternative identification route. A valid instrument satisfies three conditions:^{[1]Reference 1Causal Inference in Statistics: A Primerhttps://bayes.cs.ucla.edu/PRIMER/}

1. Relevance: $Z$ causally affects treatment assignment $A$ ($\operatorname{Cov}(A, Z) \neq 0$).
2. Exclusion Restriction: $Z$ affects outcome $Y$ only through treatment $A$ (no direct arrow $Z \to Y$).
3. Exogeneity / Independence: $Z$ shares no common causes with $Y$ ($Z \perp U$).

In an AI support platform, $Z$ could be a randomized client gateway latency experiment or canary flag that stochastically pushes incoming requests toward the assistant without affecting customer frustration.

Wald estimator and local average treatment effects

When treatment is binary and compliance is imperfect, the Wald estimator computes the causal effect by taking the ratio of the intention-to-treat outcome difference to the assignment compliance difference:

$$\operatorname{ATE}_{\text{IV}} = \frac{\mathbb{E}[Y \mid Z=1] - \mathbb{E}[Y \mid Z=0]}{\mathbb{E}[A \mid Z=1] - \mathbb{E}[A \mid Z=0]}$$

Suppose setting canary flag $Z=1$ raises assistant usage from 30% to 70% ($\Delta A = 0.40$), while overall ticket resolution rises from 68% to 72% ($\Delta Y = 0.04$). The Wald estimate is:

$$\operatorname{ATE}_{\text{IV}} = \frac{0.72 - 0.68}{0.70 - 0.30} = \frac{0.04}{0.40} = 0.10 \quad (+10\text{ percentage points})$$

Under one-sided non-compliance and monotonicity (no defiers who choose the opposite of their assignment), this identifies the Local Average Treatment Effect (LATE): the causal effect on compliers whose routing was changed by the instrument.

Why can't an engineer use the ticket's word count as an instrument for assistant routing?

Answer

Ticket word count correlates with issue complexity, which directly affects resolution time. This violates the exclusion restriction: word count has a direct causal path to resolution that doesn't pass through assistant routing.

Causal inference in production AI systems

Causal principles are fundamental to machine learning and AI agent engineering, particularly when models interact with environments, user behavior, and multi-step tool calls.

Offline policy evaluation (OPE)

When testing a new agent policy $\pi(a \mid s)$ (such as a redesigned triage prompt), deploying it live to production traffic can degrade user experience. Instead, teams evaluate $\pi$ on historical interaction logs collected under a legacy logging policy $\pi_0(a \mid s)$.

This problem matches observational causal inference. The action $a$ is the treatment, state $s$ is the covariate context, and reward $r$ is the outcome. The standard importance-sampling estimator (IPS) weights logged rewards by the policy probability ratio:

$$\hat{V}_{\text{IPS}}(\pi) = \frac{1}{N} \sum_{i=1}^N \frac{\pi(a_i \mid s_i)}{\pi_0(a_i \mid s_i)} r_i$$

For multi-step agents, importance ratios compound exponentially over the trajectory horizon $T$: $\prod_{t=1}^T \frac{\pi(a_t \mid s_t)}{\pi_0(a_t \mid s_t)}$, causing massive variance.

To tame this variance, teams turn to the Doubly Robust (DR) estimator. DR fits a baseline regression model $\hat{Q}(s, a) \approx \mathbb{E}[R \mid S=s, A=a]$ and uses importance weights only to correct the residual:

$$\hat{V}_{\text{DR}}(\pi) = \frac{1}{N} \sum_{i=1}^N \left[ \hat{Q}(s_i, \pi) + \frac{\pi(a_i \mid s_i)}{\pi_0(a_i \mid s_i)} \left( r_i - \hat{Q}(s_i, a_i) \right) \right]$$

The Doubly Robust estimator is unbiased if either the propensity model $\pi_0$ is accurate OR the reward model $\hat{Q}$ is accurate.

The next snippet verifies how the Doubly Robust estimator corrects a biased direct model using importance-weighted residuals from our 200-ticket log:

Recommendation and search debiasing

In recommendation engines and search rankers, user interaction logs suffer from severe position bias and popularity bias. Users click the top-ranked item because it's displayed first, not necessarily because it's the most relevant. Training ranking models directly on raw clicks creates a self-reinforcing feedback loop. By treating presentation position as an observed confounder and weighting training losses by inverse examination probabilities, models learn true user preferences rather than interface artifacts.

Counterfactual reasoning in LLM agents

When an autonomous agent executes a sequence of tool calls (e.g. database query, code generation, test execution) and fails at step 5, diagnosing which step caused the failure is non-trivial. Did tool call 2 return subtly corrupted data, or did tool call 4 fail on valid input?

By constructing a causal DAG of the execution trace, developers can perform counterfactual interventions: holding the agent's intermediate state fixed up to step 2, intervening on tool call 2's return value, and re-running downstream generation. This isolates the causal contribution of individual agent steps and provides principled credit assignment for agent reinforcement learning.

Decide what the comparison supports

A properly implemented randomized design makes assignment independent of potential outcomes in expectation. It balances baseline characteristics across arms, but it doesn't automatically repair post-assignment attrition, interference, or measurement errors. For support tickets sharing an agent pool, assigning one ticket to an AI assistant can free staff time for another, violating the no-interference assumption.

For this release report, the raw -22-point difference compared unequal case mixes. The +10-point standardized difference answers a better-specified comparison, but remains an estimate conditioned on routing, measurement, and interference assumptions. It doesn't prove that every ticket benefited. A randomized evaluation with sufficient tickets in both risk strata resolves routing bias; consistent measurement horizons and queue designs ensure the resulting numbers can be trusted.