Capstone: Demand Forecasting

Warehouse teams need to forecast daily parcel volume before staffing and packing decisions are locked. A point forecast alone isn't enough: planners also need to know when actual demand falls outside an expected range and which evidence supports an alert.

This capstone ships a forecast and alert artifact. It doesn't automatically hire labor, move inventory, or page an operator on every miss. It creates a versioned expectation, detects unusually large forecast errors, and records evidence for a planner's human-in-the-loop decision.

Earlier, you built a same-weekday seasonal baseline and separated forecast errors from anomaly review. This capstone packages those mechanics into a release candidate with shadow evidence and a rollback pointer.

Choose the Series and Decision

Predict daily shipped parcels for each warehouse seven days ahead. Use an explicit planning contract:

Demand can change around promotions, holidays, seller campaigns, inventory shortages, and data outages. Those known drivers should appear as features only if they are scheduled and available before the forecast cutoff.

Hyndman and Athanasopoulos explain why forecast evaluation must use later observations and rolling forecasting origins rather than random splits.^{[1]Reference 1Forecasting: Principles and Practice, Third Edition.https://otexts.com/fpp3/} For this project, each backtest run records its training cutoff, horizon, model version, and the actual values that arrived afterward.

Build a Reviewable Artifact

Your repository surface should look like:

The candidate can be a tree model over lag features, rolling means, service tier, weekday, and known promotions. It must beat the seasonal baseline on later windows, especially where underforecasting is expensive. A candidate that marginally improves MAE but misses peak-volume days should remain blocked.

Freeze issued forecasts before outcomes arrive

The runnable receipt below starts after training. Seasonal baseline and candidate point forecasts are pre-filled fixtures so this lesson grades freeze, join, alert, and receipt contracts. Fitting lag features under a rolling origin is a separate training lab; don't claim the harness trained a tree.

It freezes two seven-day forecast windows issued one week apart. Each issued row stores the cutoff, target date, horizon, baseline, candidate, expected range, and known event context before the target day arrives.

Actual parcel counts belong in a separate append-only observation stream. The backtest joins each later observation to the immutable forecast it evaluates. A real pipeline should materialize the same boundary after every rolling-origin fold.

The fixture is intentionally compact: one fulfillment center and one service tier make every row inspectable. Its ID includes center, tier, issue date, and target date so additional series can't collide during replay. A production report should repeat the same contract by center, tier, horizon, and event slice.

Evaluate Forecasts and Alerts Separately

Mean absolute error (MAE) answers how far point forecasts miss on average. Capacity planning needs a second view because a large underforecast on a peak day can cost more than a small overforecast on a routine day. The next cell assigns a local cost of 3 units to each underforecast parcel on a day with at least 150 observed parcels.

An expected range answers a different question. It should contain a stated share of later observations when measured across enough held-out windows. Hyndman and Athanasopoulos describe prediction intervals as forecast ranges with a specified coverage probability and explain why distributional forecasts need their own accuracy measures.^{[1]Reference 1Forecasting: Principles and Practice, Third Edition.https://otexts.com/fpp3/} The local candidate ± 6 policy only demonstrates release plumbing; it isn't a calibrated production interval.

The candidate improves average error and peak-day cost, but one seller-campaign day still escapes its expected range. That row belongs in a planner queue. The queue preserves which immutable forecast was issued, what happened later, who owns review, and which interval and alert policies created the alert.

Publish a Shadow-Review Receipt

Forecast quality and alert usefulness require separate evidence. A narrow range can create an exhausting queue. A broad range can hide events a planner needed to see. Measure alert precision and recall on historical alerts after reviewers label whether each event required action. Keep the forecast and alert-policy versions on those review rows; otherwise a candidate could pass using evidence produced by a different model or threshold.

The final cell publishes one candidate receipt. It advances to planner shadow review, not production replacement. The previous forecast alias remains explicit so a later promotion process can roll back cleanly.

candidate_for_planner_shadow_review is intentionally narrower than launch approval. Frozen forecasts, later observation joins, and reviewed alerts say this bundle deserves planner observation beside current production forecasts. They don't prove that every center, tier, event slice, or future week will behave well. A reviewed-alert window with no triggered or actionable rows reports None, not an invented precision or recall score.

Plan Refresh and Monitoring

New outcomes arrive daily, but model replacement should happen on a scheduled or triggered review cycle. Store:

Practice: break the forecast contract

Use the runnable examples as a release harness. Change one condition at a time, predict the failure, then rerun the examples.

1. Change every fold-2 issue date from 2026-02-08 to 2026-02-15. Which temporal gates fail?
2. Change fold-1 Friday candidate from 140 to 132. Which cost gets worse even though only one row changed?
3. Set INTERVAL_HALF_WIDTH = 0. Why can MAE stay unchanged while range and queue gates fail?
4. Mark A-104 as not actionable. Which alert metric improves, and which stays unchanged?
5. Set PREVIOUS_FORECAST = "". Which executable gate fails?
6. Give first observation forecast ID missing:standard:2026-02-01:2026-02-02. Which replay gate fails?
7. Set every reviewed alert's triggered and actionable values to False. Why do alert-evidence gates fail instead of crashing or passing?
8. Change every reviewed alert's policy_version to outside-range-review-v0. Which provenance gate fails?

Practice answer sketches

Which gates fail when fold-2 is issued on 2026-02-15?

Answer

targets_after_cutoff fails because targets from February 9 through February 15 no longer occur after their issue cutoff. horizons_within_seven_day_contract fails too because those derived horizons are no longer between 1 and 7. That is hindsight, not a backtest.

What changes when the first Friday candidate falls from 140 to 132?

Answer

Candidate MAE rises by 8 / 14, but peak underforecast cost rises by 3 * 8 = 24. The slice metric makes capacity risk visible.

Why do zero-width expected ranges fail local range and queue gates without changing MAE?

Answer

MAE reads point forecasts only. Local coverage and queue volume read whether observations stay inside expected ranges. Different artifacts answer different questions.

What changes when A-104 becomes not actionable?

Answer

Recall rises from 2 / 3 to 2 / 2 = 1.0 because the former missed alert is no longer actionable. Precision stays 2 / 3 because triggered rows don't change.

Which gate fails when PREVIOUS_FORECAST becomes an empty string?

Answer

rollback_pointer_recorded fails. Shadow evidence may justify a later promotion, but promotion still needs a known rollback target.

Which gate fails when an observation points at forecast ID missing:standard:2026-02-01:2026-02-02?

Answer

observations_join_issued_forecasts fails. A later count can't become evaluation evidence unless it joins one immutable forecast issued before the target day.

What happens when reviewed history contains no triggered or actionable alerts?

Answer

Precision and recall become None, then both alert-evidence gates fail. No denominator means no evidence. It isn't a crash, measured success, or measured failure.

Which gate fails when reviewed alerts come from outside-range-review-v0?

Answer

alert_review_matches_candidate_policy fails. Precision and recall only support this release when every reviewed row was produced by the candidate forecast and outside-range-review-v1 policy under review.