Capstone: Image Damage Classifier

A damage classifier receives photos that may be crushed, torn, blurred, dark, or unrelated to the reported case. Unlike a tabular prediction, its input can fail before classification begins because the evidence itself isn't usable.

Earlier, you traced a convolutional neural network (CNN) over a damaged-package image patch. This capstone turns that spatial reasoning into an image-triage endpoint that flags likely visible damage, rejects unusable photos, preserves evidence for human review, and never turns an uncertain image score directly into a refund.

Define the photo decision first

The service receives return photos from customer uploads and intake cameras. The useful product question isn't "does the model recognize every defect?" It's: which photo should a specialist inspect first, and when is the photo too weak to support any decision?

Use three operational outcomes:

The classifier isn't a refund policy. Product eligibility still depends on order ownership, item type, return window, and specialist judgment. This separation prevents a shadow or reflection in a photo from issuing a costly action.

A model card should state the intended use, input constraints, decision threshold, evaluated slices, and known failure cases. Model cards were proposed as structured reports for exactly this type of deployed-model context: users need more than a metric without its operating conditions.^{[1]Reference 1Model Cards for Model Reportinghttps://arxiv.org/abs/1810.03993}

Build a dataset that can't leak

For tabular models, leakage may be a future delivery timestamp. For photos, leakage often hides in nearly identical pixels. A customer may upload three bursts of the same crushed box. A warehouse may photograph one parcel from four angles. If related images land in both train and test sets, the model can memorize one package rather than generalize to new damage.

Your manifest should contain:

Evaluate at least daylight versus dark uploads, customer versus warehouse source, packaging type, and visible-defect size. A global score can hide the exact failure that matters: small tears disappearing in dark phone images.

Use the CNN learned earlier as a baseline, then fine-tune a pretrained image encoder only if you record its preprocessing and measure it under the same split. A later deep-dive explains Vision Transformer image encoders; this capstone doesn't require that architecture.^{[2]Reference 2An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale.https://arxiv.org/abs/2010.11929}

Freeze a grouped photo manifest

Start with the dataset boundary. Multiple photos from one physical package belong in one split even when filenames differ. Keep time direction intact too: later return cases should test a model trained on earlier cases.

The local manifest below is small enough to inspect line by line. This historical evaluation fixture contains frozen specialist labels, reviewer IDs, and guideline versions for dataset audit. Damage labels appear only when a specialist judged the image usable. Unusable photos keep confirmed_damage=None because poor evidence shouldn't become supervision. Candidate scores arrive in a separate object, so route code can't quietly read labels as model output. Manifest confirmed_damage fields are offline audit only; the serving path must not read them. MODEL_OUTPUTS below are frozen fixtures for the routing receipt. Fitting train_cnn_baseline.py and applying the preprocessing version to pixels is a separate training lab.

The fixture uses explicit splits so the invariant stays visible. A larger pipeline can use a group-aware splitter, then freeze and audit the resulting manifest. The important claim isn't that one splitter solves every dataset: no physical case may cross evaluation boundaries, and later cases remain later.

Encode quality-first review traces

The model endpoint should receive a preprocessing result and a damage score, then choose a review route. Quality checks run before the damage threshold. Otherwise a confidently scored blur, dark frame, or unrelated object can create an unsupported escalation.

The score fixture below is separate from the manifest. Labels stay available for offline evaluation, but the router never reads them. A production system may run its cheap quality checks before an expensive damage model; this compact fixture keeps both outputs so you can test that an unsupported high damage score still abstains.

Cases R-406-a and R-409-a are the important failure tests. Both have high damage scores. Neither score counts as usable evidence because quality checks fail first. The endpoint asks for another photo instead of escalating an unsupported claim.

Package the Vision Service

Submit an inspectable repository, not a notebook screenshot:

The serving bundle must pin image resize and crop behavior, color normalization, model weights, label version, route policy, damage threshold, quality-gate thresholds, and previous-bundle pointer. A change from center crop to full-frame resize may change whether a torn corner remains visible; it's a model behavior change even when weights remain constant.

The routing cells emit that trace shape directly. They let a reviewer reconstruct the route:

Join delayed outcomes before promotion

Photo models drift when the image source changes. A new warehouse camera, winter lighting, a mobile upload compressor, or new packaging graphics can alter pixels before a confirmed-damage label exists.

Separate immediate checks from delayed quality:

Google Cloud's MLOps guidance treats serving, monitoring, validation, metadata, and continuous training as connected stages rather than a one-time deploy step.^{[3]Reference 3MLOps: Continuous Delivery and Automation Pipelines in Machine Learning.https://docs.cloud.google.com/architecture/mlops-continuous-delivery-and-automation-pipelines-in-machine-learning} Apply that same discipline here: a change in image quality creates an investigation or candidate run, never an automatic production replacement.

The final local receipt appends specialist outcomes only after routing, then joins each outcome to a physical case. Each delayed outcome also carries its reviewer and label guideline, so precision and recall can't silently mix definitions. It measures review precision and recall once per usable test case, records immediate request rates by source, proves unusable photos abstain, and keeps the previous bundle beside the candidate. When one package has multiple usable photos, the case escalates if any usable photo crosses the priority threshold. These tiny counts teach the contract; production promotion still needs larger slices and shadow traffic.

candidate_for_specialist_shadow_review is narrower than launch approval. The receipt says this frozen bundle deserves comparison beside current production routing. It doesn't claim that four usable local cases prove every camera, lighting condition, package type, or damage shape.

Practice: break the vision contract

Use the runnable examples as a release harness. Change one condition at a time, predict the failure, then rerun the examples.

1. Change R-401-b split from train to validation. Which dataset gate fails?
2. Change R-406-a brightness in MODEL_OUTPUTS from 0.12 to 0.30. Why does this expose a quality-detector failure rather than prove the escalation is safe?
3. Remove damage_threshold from the response trace. Which receipt gate fails?
4. Change R-408-a damage probability in MODEL_OUTPUTS from 0.63 to 0.75. Which delayed metrics improve?
5. Set BUNDLE["previous_bundle"] = "". Why is shadow evidence no longer promotion-ready?
6. Replace SPECIALIST_OUTCOMES with an empty list. Why do evidence gates fail instead of crashing or passing?
7. Add SpecialistOutcome("R-999", 12, True, "S-12", "visible-damage-v1"). Which join gate fails?
8. Change R-408 specialist outcome to guideline visible-damage-v2. Which provenance gate fails?

Practice answer sketches

Which gate fails when R-401-b moves to validation?

Answer

case_groups_do_not_cross_splits fails because two photos of physical return R-401 now cross training and validation. Evaluation can reward memorization of one package.

Why isn't raising R-406-a brightness a safe escalation?

Answer

The router now sees an apparently usable high-score image, but specialist label still says the evidence is unusable. Both unusable_photos_abstain and unsafe_priority_routes_absent fail. That mismatch belongs in quality-model evaluation before promotion.

Which gate fails when route traces omit damage_threshold?

Answer

route_traces_replayable fails. A reviewer can't reconstruct whether a policy threshold change altered the route.

What changes when R-408-a score rises from 0.63 to 0.75?

Answer

It becomes a true-positive priority route. Case-level recall rises from 1 / 2 to 2 / 2 = 1.0, and precision rises from 1 / 2 to 2 / 3 = 0.667.

Why keep previous_bundle beside candidate?

Answer

Shadow evidence may justify a later promotion, but promotion still needs a known rollback target. A candidate without previous alias doesn't prove rollback can restore earlier behavior.

What happens when no specialist outcomes have arrived?

Answer

Precision and recall become None, then outcome-completeness and metric-evidence gates fail. No denominator means no evidence. It isn't a crash, measured success, or measured failure.

Which gate fails when outcome R-999 appears?

Answer

specialist_outcomes_join_usable_test_cases fails. A delayed specialist label can't become evaluation evidence unless it joins a usable test case routed by this bundle.

Which gate fails when R-408 uses visible-damage-v2?

Answer

specialist_outcome_label_contract_matches_bundle fails. Delayed metrics only support this bundle when every outcome records a reviewer and uses the bundle's visible-damage-v1 label contract.