Monitoring Predictive Models

Friday's cluster ran 160 batch jobs against a weekly-lag forecast of 132. That leftover became a reviewable alert. The forecasts, and the job-risk scores that decide which jobs get SLA review, can still fail after the endpoint looks healthy: inputs break now, labels arrive after the promised window, and a newer model can look better offline while costing more after promotion.

Monitoring splits those questions onto separate clocks. Can this request be scored safely? Has traffic changed? Did eventual outcomes stay good enough? Should a candidate replace the current release?

Start with a served release and feature traces

Stay with that serving cluster. Now score each live job for SLA-miss risk. High scores enter review. Each request carries two model inputs:

Store a feature trace with each prediction. A feature trace records the values, timestamps, and feature-contract versions that produced one score. It lets an engineer reproduce the decision after traffic, code, or source data changes.

Build six incoming requests against deployed release job-risk-v3. Two rows contain failures that should be caught before delayed SLA labels exist.

At request time, you don't know which jobs will miss their SLA. You do know whether required inputs exist and whether their timestamps are fresh enough to trust. Each trace binds those values to the model and feature-contract versions that consumed them.

Block invalid inputs before waiting for labels

Define two request-time invariants:

1. Required features can't be missing.
2. Feature snapshots can't be older than 60 minutes. That bound is a feature freshness contract: age is prediction time minus the snapshot's feature_as_of, and a typed non-null value can still be unusable if that gap is too large.

An invariant is a condition that has to stay true for the system to operate safely. Check each trace and block unsafe rows instead of sending a plausible-looking score downstream.

Job s-103 lacks backlog data even though its snapshot is only 15 minutes old. Job s-104 carries a three-hour-old snapshot. A label-based accuracy report would discover either unsafe path too late.

Sculley et al. treat unstable data dependencies, input-data tests, and live monitoring as first-class production concerns, including prediction-bias checks and alerts from upstream feature producers.^{[1]Reference 1Hidden Technical Debt in Machine Learning Systems.https://research.google/pubs/hidden-technical-debt-in-machine-learning-systems/} The practical point is that models depend on surrounding data systems, not model code alone.

Why block s-104 before its SLA label arrives?

Answer

The three-hour-old feature snapshot violates a request-time freshness contract. A future label may measure outcome quality, but it can't make an unsafe current input trustworthy.

Measure drift without calling it failure

Healthy rows can still look different from training traffic. Data drift means an input distribution changed over time. Klaise et al. treat those distribution checks as a production signal when labels lag, and they distinguish covariate shift ($P(x)$ moves) from label shift ($P(y)$ moves).^{[2]Reference 2Monitoring Machine Learning Models in Production.https://arxiv.org/abs/2007.06299} A shift is a reason to inspect. It isn't proof that accuracy fell.

Compare the historical and current distributions for hours_since_last_heartbeat:

Older heartbeats are more common now. A scheduler outage could cause that shift, but so could a holiday or runner-pool mix change.

Compress the four bucket pairs into one diagnostic with Population Stability Index (PSI):

$$\text{PSI} = \sum_i (a_i - e_i)\log\left(\frac{a_i}{e_i}\right)$$

Here $e_i$ is the historical share for bucket $i$, and $a_i$ is its current share. This exercise uses non-zero shares in every bucket so the logarithm is defined. Real monitoring code must choose an explicit policy for empty buckets, such as a small floor, before taking the log.

For continuous numerical inputs without predefined bins, two-sample tests like the Kolmogorov-Smirnov (KS) test (measuring the maximum divergence between empirical cumulative distribution functions, $D = \sup_x |F_1(x) - F_2(x)|$) or Wasserstein distance detect shifts directly. When features are binned or categorical, PSI and Chi-square ($\chi^2$) tests provide standard discrete summaries.

Work the largest term by hand. The 24h+ bucket moved from $e=0.05$ to $a=0.20$:

$$(0.20 - 0.05)\ln\left(\frac{0.20}{0.05}\right) = 0.15 \ln 4 \approx 0.15 \times 1.386 = 0.208$$

The other three terms add about $0.102 + 0.009 + 0.051$, so PSI is about $0.370$. Credit-scoring practice often treats PSI below $0.10$ as small and $0.20$ as large. Treat those bands as a local starting point, not a retraining law. Confirm the sum in code and apply a local investigation threshold of 0.20.

PSI is a compact diagnostic, not an accuracy verdict. The 0.20 cutoff is a local policy calibrated on stable windows. This result calls for inspection; it doesn't prove release v3 is inaccurate or that retraining is the first fix.

Why shouldn't the 0.37 PSI result automatically start retraining?

Answer

PSI only reports a distribution change. Inspect source-data health, business context, and delayed outcomes first. Retraining on corrupted inputs can produce a newer broken model.

Data drift isn't concept drift

Data drift (covariate shift) is a change in the input distribution $P(x)$. Feature PSI measures that class of change. Concept drift is a change in the relationship $P(y \mid x)$: the same feature values start mapping to different outcomes. Label / prior shift is a change in $P(y)$ that may or may not accompany either of the others.

Delayed quality metrics and slice calibration are concept-drift evidence: they compare scores to outcomes after the label window closes. Feature PSI alone can't prove concept drift. Don't retrain on corrupt or skewed inputs either; fix data path and parity first, then use mature delayed quality to decide whether the learned relationship still holds.

Score distributions are a useful early signal between feature PSI and delayed labels. If risk scores shift sharply while feature mix and parity look stable, inspect serving config, threshold policy, and recent transform deploys before outcomes mature. A score histogram isn't an accuracy verdict; it only shortens the time to notice that something changed.

Reproduce training-serving skew

Training-serving skew means production computes an input differently from training. Suppose training divided backlog count by staffed runner-pool capacity, while a serving change divides by a fixed 100.

Probe the same raw row through both transformations.

The same runner pool becomes 0.4 in training and 0.2 online. A model retrained on the existing training function won't repair that serving bug. Fix the shared transformation contract and keep a parity test in the release gate.

Google Cloud's MLOps guidance includes unit tests for feature-engineering logic, prediction-service tests that call the API with expected inputs, data validation, and a check that the candidate meets predictive-performance targets before deployment.^{[3]Reference 3MLOps: Continuous Delivery and Automation Pipelines in Machine Learning.https://docs.cloud.google.com/architecture/mlops-continuous-delivery-and-automation-pipelines-in-machine-learning}

Join delayed labels back to stored predictions

Request-time checks move fast because they don't need outcomes. Accuracy moves slower. An SLA-miss label may arrive only after the promised job window closes.

Store model ID, score, job ID, decision context, and the time when each outcome becomes mature. Later, join outcomes by immutable prediction ID. Compute quality on cohorts whose outcome windows have closed, and report label coverage inside those cohorts. Labels that happen to arrive early can be a biased subset; a missing label in a mature cohort is a coverage gap, not an on-schedule outcome.

Prediction p-006 is excluded because its SLA outcome window hasn't closed. That's the right-hand clock in the opening figure: five bars sit before noon, and +1h stays out of the cohort. Label arrival doesn't define cohort maturity. The five-row mature cohort has 100% label coverage. If a mature row were still unlabeled, the report would expose the coverage gap and withhold the quality decision rather than assume the observed subset is representative.

Why exclude p-006 from this quality cohort instead of counting it as an on-schedule job?

Answer

Its outcome window hasn't closed, so the prediction isn't mature. Replacing that unknown outcome with a negative label biases the report. Separately, missing labels for predictions that are already mature must reduce reported coverage and can block quality decisions.

Evaluate decisions and calibration

A binary decision uses deployed threshold 0.50: scores at or above the threshold enter job-SLA review. Require complete label coverage for this mature lab cohort, then compute precision and recall, priority-job misses, and a small review-cost receipt.

The local cost policy assigns 2 units to an unnecessary review and 10 to a missed SLA. It's an example decision policy, not a universal business value. If a denominator is zero, report None: missing evidence isn't the same as measured failure.

Scores also need a calibration check. Guo et al. showed that modern neural classifiers are often miscalibrated: a score near 0.80 doesn't have to mean an 80% chance unless outcomes from adequately covered mature cohorts support that reading.^{[4]Reference 4On Calibration of Modern Neural Networkshttps://arxiv.org/abs/1706.04599} The same habit applies to this risk model. Compare average score with observed job-SLA rate inside two tiny buckets.

Five labels with full mature-cohort coverage are enough to explain the receipt, not enough to approve probability quality. An empty bucket should report None instead of pretending evidence exists. A production report needs larger mature windows, explicit overall and slice coverage, and slices such as runner pool, service tier, geography, and forecast age.

Triage repair, inspection, and retraining separately

Drift, skew, and delayed quality degradation can appear together. They don't imply the same action:

Encode that order in a function so a dirty window can't skip repair and jump straight to training. The first scenario has broken data and parity. The second has clean inputs plus delayed quality regression.

Continuous training doesn't mean every alert immediately overwrites production. Google Cloud documents on-demand, scheduled, new-data, performance-degradation, and feature-distribution triggers, plus metadata that includes a pointer to the previous trained model.^{[3]Reference 3MLOps: Continuous Delivery and Automation Pipelines in Machine Learning.https://docs.cloud.google.com/architecture/mlops-continuous-delivery-and-automation-pipelines-in-machine-learning} Their trigger list labels a feature-distribution change as concept drift. In this chapter that's still data drift until delayed outcomes show that $P(y \mid x)$ moved.

Freeze an immutable candidate bundle

When clean evidence justifies retraining, freeze the snapshot, training pipeline, feature contract, decision policy, evaluation policy, and previous model pointer. The resulting candidate bundle is an immutable release proposal. Promotion moves an alias; it doesn't rewrite the bundle.

Create a deterministic bundle ID from the candidate configuration.

Google Cloud's MLOps guidance recommends recording pipeline versions, arguments, produced artifacts, evaluation metrics, and a pointer to the previous trained model for rollback or comparison.^{[3]Reference 3MLOps: Continuous Delivery and Automation Pipelines in Machine Learning.https://docs.cloud.google.com/architecture/mlops-continuous-delivery-and-automation-pipelines-in-machine-learning}

Gate offline metrics and canary traffic

Compare candidate v4 with current v3 before changing live traffic. These thresholds are explicit local policies:

Offline evidence is necessary but incomplete. Google Cloud's same MLOps page puts a newly trained model through online validation in a canary or A/B setup before it serves full traffic.^{[3]Reference 3MLOps: Continuous Delivery and Automation Pipelines in Machine Learning.https://docs.cloud.google.com/architecture/mlops-continuous-delivery-and-automation-pipelines-in-machine-learning} Two live patterns sit between a frozen bundle and a full alias move:

• Shadow evaluation scores the candidate on production-shaped requests and logs the result. Users still see v3. You learn about errors, latency, and disagreement without giving v4 authority.
• A canary rollout sends a limited slice of real traffic to the candidate. Those scores are user-visible, so blast radius is the canary fraction.

Gate the canary on fast signals: service errors, latency, request count, and immediate review-queue pressure. Those checks can stop a bad rollout quickly. They don't replace delayed-label quality.

Argo Rollouts documents canary steps, metric analysis, unsuccessful-analysis aborts, and post-promotion analysis that can switch traffic back to a previous stable release.^{[5]Reference 5Argo Rollouts - Kubernetes Progressive Delivery Controllerhttps://argoproj.github.io/argo-rollouts/} The exact deployment tool may differ, but promotion needs measured conditions and a rollback path.

Promote an alias, then rehearse rollback

Keep immutable bundle IDs separate from a movable production alias. If both gate sets pass, move the alias from v3 to the candidate. Then simulate a slower delayed-label guardrail arriving after promotion. The cost threshold is evaluated only after the declared mature cohort reaches full label coverage; an incomplete cohort would report insufficient evidence rather than treat early labels as representative.

The rehearsal promotes v4, waits for a fully labeled 500-row mature cohort, observes delayed review cost 8 above local limit 5, and restores alias v3. Its receipt binds candidate lineage, offline evidence, canary evidence, delayed-label coverage, and the derived rollback reason. A rollback drill is stronger than a diagram: it proves the pointer and policy work together.

When monitoring breaks