Skip to content
LeetLLM
Tracks
Advanced26 lessons

Training and Alignment

Choose a model behavior you want to improve. Learn how data, feedback, and training experiments can help you teach it.

Readers moving from API usage into model adaptation, post-training, and training infrastructure.

You can explain the lifecycle of a model update and choose the right adaptation method for a product constraint.

  1. 1The LLM LifecyclePolicy P-7 raises the minimum key age from 30 days to 60. Trace which artifact should change, from training data through the served answer.Preparation & PrerequisitesEasy24 min
  2. 2PyTorch Training LoopsA falling loss can hide a broken ticket-router loop, so follow one batch through logits, gradients, validation, and a checkpoint that still matches after reload.ML Algorithms & EvaluationMedium36 min
  3. 3Scaling Laws & Compute-Optimal TrainingKaplan, Chinchilla, and later inference-aware fits split a training budget between parameters and tokens, including when a smaller model wins on lifetime cost.Advanced Training & AdaptationHard43 min
  4. 4Dataset Pipelines and Data QualityTwo incident rows that differ only by spaces can train and test on the same conversation. Gates, grouped splits, and a manifest make that leak visible.ML Algorithms & EvaluationMedium30 min
  5. 5Pre-training Data at ScaleA 1.4T token target estimates training scale. Measure filtering, deduplication, and packing before data reaches a training step.Advanced Training & AdaptationHard45 min
  6. 6Build GPT from Scratch LabA tiny Shakespeare model can move from `And the I the` to `I'll do` and still be wrong. Train the pipeline, then separate learning from a shifted target or a leaked future token.Advanced Training & AdaptationHard38 min
  7. 7Instruction Tuning & Chat TemplatesA correct policy answer can still fail if the server's chat template differs from training. Serialize the turns, choose the loss targets, and catch that drift before you change the weights.Core LLM FoundationsMedium34 min
  8. 8Continued Pretraining for Domain ShiftThe 3 AM assistant blames Wi-Fi for `AUTH_TIMEOUT`. Continued pretraining tests whether more domain text fixes that, without hiding forgetting or a task that never improved.Advanced Training & AdaptationHard40 min
  9. 9LLM-as-a-Judge EvaluationTwo rollback replies both follow DEP-27, but a string match can't say which one an on-call engineer can use.Applied LLM EngineeringMedium26 min
  10. 10Synthetic Data PipelinesA judge score of 9/10 and a passing generated test can still admit a duplicate or a wrong answer. Gate synthetic rows before they become training data.Advanced Training & AdaptationHard37 min
  11. 11Supervised Fine-Tuning PipelineThe assistant sounds reasonable and still misses the policy. Supervised fine-tuning can raise the likelihood of the right reply, but the loss, split, and checkpoint have to measure that behavior.Advanced Training & AdaptationHard41 min
  12. 12Experiment Tracking with MLflow and W&BA routing repair can look right on the incident that inspired it and still be unproven until the run records what it saw and what it must not decide.Applied LLM EngineeringMedium32 min
  13. 13CUDA for ML TrainingFour access tickets can have the right shape and still fail: wrong device, a batch that doesn't fit, or a timer that stops before the GPU finishes.Computing FoundationsEasy25 min
  14. 14Mixed Precision TrainingA 16-bit store can erase a tiny update near one or overflow a large activation, which is why the optimizer step still keeps an FP32 path.Advanced Training & AdaptationHard47 min
  15. 15Distributed Training: FSDP & ZeROOne replicated 70B AdamW state exceeds an 80 GB GPU. Shard the optimizer, gradients, and parameters, then see when the layer or the sequence still has to be split.Advanced Training & AdaptationHard68 min
  16. 16LoRA & Parameter-Efficient TuningFull fine-tuning stores gradients and Adam state for every weight. LoRA trains a low-rank update, and held-out policy checks still decide whether that update is enough.Advanced Training & AdaptationHard54 min
  17. 17Training Run OperationsYesterday's weights can reload while the data cursor and warmup start over. Resume the optimizer, batch, and scheduler with the checkpoint that actually reached storage.Advanced Training & AdaptationHard34 min
  18. 18Data Labeling and Human FeedbackA preference can look obvious and still be the wrong training record if the trace leaks private data or belongs to the evaluation set.Applied LLM EngineeringMedium32 min
  19. 19Reward Modeling from Preference DataThe helpful answer can still be the wrong one. Fit a scalar reward on chosen and rejected pairs, then test whether the score learned the policy or a shortcut such as length.Advanced Training & AdaptationHard45 min
  20. 20RLHF & DPO AlignmentAn internal judge score can rise while blinded comparisons stay flat. Turn preference labels into a PPO update or a DPO loss, then test the score on held-out behavior.Advanced Training & AdaptationHard59 min
  21. 21Constitutional AI & Red TeamingTrace Constitutional AI from critique and revision to AI preference labels, then build honest red-team evaluations that separate model failures, false refusals, and judge errors.Advanced Training & AdaptationHard36 min
  22. 22RLVR & Verifiable RewardsA checker can mark 168 correct and 148 wrong. RLVR trains on that executable result, then tests what the checker still misses.Advanced Training & AdaptationHard43 min
  23. 23Knowledge Distillation for LLMsA smaller model can copy a teacher's probabilities and still miss the expired-admin case, so the release gate must check policy behavior and speed.Advanced Training & AdaptationHard49 min
  24. 24LLM Benchmarks & LimitationsA fluent CodeAssist answer invents on-call deploy permission the freeze policy never granted. A public score will not catch that.Core LLM FoundationsMedium35 min
  25. 25Model Versioning & DeploymentAn engineer reverts the prompt in Git, but the image and the evidence have already moved. The release ID has to name the combination that is actually answering.Applied LLM EngineeringMedium33 min
  26. 26Capstone: Fine-Tuned ClassifierTrain an intake classifier that sends a freeze exception to a person, then hold the release when a required escalation is missed.Portfolio CapstonesHard47 min