Capstone: Image Damage Classifier

You have shipped models over shipment rows, ranked items, and warehouse time series. A customer return adds a new input type: a photo of a package that may be crushed, torn, blurred, dark, or unrelated to the order.

Earlier, you traced a convolutional neural network (CNN) over a damaged-package image patch. This capstone turns that spatial reasoning into a product: an image triage endpoint that flags likely visible damage, rejects unusable photos, preserves evidence for human review, and never turns an uncertain image score directly into a refund.

Define the Photo Decision First

ShopFlow receives return photos from customers and warehouse intake stations. The useful product question is not "does the model recognize every defect?" It is: which photo should a specialist inspect first, and when is the photo too weak to support any decision?

Use three operational outcomes:

The classifier isn't a refund policy. Product eligibility still depends on order ownership, item type, return window, and specialist judgment. This separation prevents a shadow or reflection in a photo from issuing a costly action.

A model card should state the intended use, input constraints, decision threshold, evaluated slices, and known failure cases. Model cards were proposed as structured reports for exactly this type of deployed-model context: users need more than a metric without its operating conditions.^[1]

Build a Dataset That Cannot Leak

For tabular models, leakage may be a future delivery timestamp. For photos, leakage often hides in nearly identical pixels. A customer may upload three bursts of the same crushed box. A warehouse may photograph one parcel from four angles. If related images land in both train and test sets, the model can memorize one package rather than generalize to new damage.

Your manifest should contain:

Evaluate at least daylight versus dark uploads, customer versus warehouse source, packaging type, and visible-defect size. A global score can hide the exact failure that matters: small tears disappearing in dark phone images.

Use the CNN learned earlier as a baseline, then fine-tune a pretrained image encoder only if you record its preprocessing and measure it under the same split. A later deep-dive explains Vision Transformer image encoders; this capstone doesn't require that architecture.^[2]

Encode a Safe Routing Policy

The model endpoint should receive a preprocessing result and a damage score, then choose a review route. The gate below refuses to use high damage confidence when the image evidence is unusable.

Case R-402 is the important failure test: an apparent high damage probability isn't usable evidence because blur fails first. The endpoint asks for another photo rather than escalating an unsupported claim.

Package the Vision Service

Submit an inspectable repository, not a notebook screenshot:

The serving bundle must pin image resize and crop behavior, color normalization, model weights, label version, damage threshold, and quality-gate thresholds. A change from center crop to full-frame resize may change whether a torn corner remains visible; it is a model behavior change even when weights remain constant.

Return a trace that lets a reviewer reconstruct the route:

Monitor Photos and Human Outcomes

Photo models drift when the image source changes. A new warehouse camera, winter lighting, a mobile upload compressor, or new packaging graphics can alter pixels before a confirmed-damage label exists.

Separate immediate checks from delayed quality:

Google Cloud's MLOps guidance treats serving, monitoring, validation, metadata, and continuous training as connected stages rather than a one-time deploy step.^[3] Apply that same discipline here: a change in image quality creates an investigation or candidate run, never an automatic production replacement.

Mastery Check

Evaluation rubric

Common Failures