LeetLLM
Tracks
Intermediate25 lessons

Production ML Systems

Build predictive ML systems from validated data and baselines through feature pipelines, deployment, monitoring, and rollback.

ML engineers, data scientists, and backend engineers shipping predictive systems beyond notebooks.

You can build a reproducible training pipeline, preserve batch and online feature parity, and operate a monitored model release.

  1. 1Python for AI EngineeringLearn Python as the first AI engineering loop: read JSONL rows, validate fields, compute exact-match accuracy, and harden the scorer with pytest, prompt snapshots, leakage checks, seeded runs, and a CI gate.Computing FoundationsEasy16 min
  2. 2NumPy and Tensor ShapesLearn NumPy shape reasoning from first principles: name axes, predict indexing and broadcasting, reduce safely, distinguish reshape from transpose, and add shape guards.Computing FoundationsEasy17 min
  3. 3SQL and Data ModelingTurn an in-memory support retriever into durable SQL tables. Create rows and keys, query with parameters and joins, enforce permissions, roll back failed work, deduplicate retries, inspect indexes, and see where pgvector fits.Computing FoundationsEasy14 min
  4. 4Probability for Machine LearningA beginner-first probability article that teaches events, priors, conditional probability, independence, Bayes rule, and base-rate mistakes through one API abuse-risk detector story.Math & StatisticsEasy19 min
  5. 5Statistics and UncertaintyEstimate abuse risk in a flagged review queue from finite labels, using bootstrap intuition, score intervals, sampling bias checks, and calibrated reporting.Math & StatisticsEasy12 min
  6. 6Linear Regression from ScratchFit key-rotation assistant latency by hand, implement least squares and gradient descent in NumPy, then test failure cases and held-out behavior.ML Algorithms & EvaluationMedium21 min
  7. 7Logistic Regression and MetricsRoute access-change requests with logistic regression from scratch: derive sigmoid and log loss, fit NumPy weights, select a cost-aware threshold on validation data, audit ranking and calibration, then compare with scikit-learn.ML Algorithms & EvaluationMedium25 min
  8. 8Decision Trees, Forests, and BoostingModel access-change review with decision trees from scratch: compute impurity, test a non-perfect stump on held-out cases, compare forests and boosting, and audit feature explanations.ML Algorithms & EvaluationMedium18 min
  9. 9Validation and LeakageMake model and policy claims honestly: define the decision moment, split access-review episodes by time and user, expose feature and preprocessing leakage, and audit LLM evaluation contamination.ML Algorithms & EvaluationMedium16 min
  10. 10Dataset Pipelines and Data QualityBuild versioned AI datasets with schema gates, grouped splits, contamination checks, and auditable receipts.ML Algorithms & EvaluationMedium17 min
  11. 11Feature Engineering for Production MLTurn training-job events into stable prediction inputs while preventing leakage and training-serving mismatch.Production ML SystemsMedium11 min
  12. 12Batch and Streaming Feature PipelinesBuild point-in-time training-run features from events and preserve the same meaning in online serving.Production ML SystemsMedium12 min
  13. 13Gradient Boosted Trees in ProductionTrain a boosted SLA-risk baseline from tabular features, evaluate slices, and package deployment evidence.Production ML SystemsMedium14 min
  14. 14Ranking and Recommendation SystemsRank documents for a developer using candidate retrieval, relevance metrics, and feedback-loop safeguards.Production ML SystemsMedium14 min
  15. 15Forecasting and Anomaly DetectionForecast batch-job demand with time-aware evaluation and turn large forecast errors into reviewable operational alerts.Production ML SystemsMedium16 min
  16. 16CNNs from ScratchTrace a CNN over a cracked equipment-panel photo patch: shared kernels, feature-map shapes, pooling, padding failures, and a NumPy-to-PyTorch forward pass.Preparation & PrerequisitesEasy10 min
  17. 17Monitoring Predictive ModelsMonitor predictive models from feature freshness through delayed labels, then gate retraining, promotion, and rollback.Production ML SystemsMedium16 min
  18. 18Experiment Design and A/B TestingDesign a trustworthy online experiment for an AI support change: randomize customers, measure useful outcomes, quantify uncertainty, and reject false wins.ML Algorithms & EvaluationMedium20 min
  19. 19Experiment Tracking with MLflow and W&BTurn a live LLM regression into a reproducible candidate decision by logging inputs, metrics, artifacts, and promotion evidence.Applied LLM EngineeringMedium16 min
  20. 20Model Versioning & DeploymentTurn an evaluated LLM change into an immutable release bundle, promote it through measured traffic, and roll back without losing lineage.Applied LLM EngineeringMedium19 min
  21. 21Capstone: Delivery ETA PredictionShip a delivery-delay warning service with as-of features, versioned policy gates, baseline evidence, and monitored fallback.Portfolio CapstonesHard14 min
  22. 22Capstone: Product RankingShip a marketplace ranking candidate with eligible retrieval, separate recall and NDCG gates, replayable exposure rows, and an A/B-ready rollback receipt.Portfolio CapstonesHard15 min
  23. 23Capstone: Demand ForecastingShip a demand forecast and capacity-alert artifact with rolling backtests, alert review, and retraining policy.Portfolio CapstonesHard16 min
  24. 24Capstone: Image Damage ClassifierShip a damaged-package photo triage service with quality checks, slice evaluation, serving bundles, and review monitoring.Portfolio CapstonesHard17 min
  25. 25Capstone: Production ML PipelineAssemble predictive ML artifacts into validated training, registry promotion, canary monitoring, and rollback.Portfolio CapstonesHard17 min