Skip to content
LeetLLM
Tracks
Advanced26 lessons

Training and Alignment

Follow the training stack from scaling laws and data pipelines through SFT, LoRA, training-run operations, RLHF, DPO, rewards, and distillation.

Readers moving from API usage into model adaptation, post-training, and training infrastructure.

You can explain the lifecycle of a model update and choose the right adaptation method for a product constraint.

  1. 1The LLM LifecycleFollow a decoder language model from training to serving, check the artifacts after compression, and turn production symptoms into evidence-led investigations.Preparation & PrerequisitesEasy22 min
  2. 2PyTorch Training LoopsTrace logits, gradients, updates, and validation in runnable PyTorch loops. Work through unequal micro-batches, CUDA AMP ordering, independent snapshots, and checkpoint resumption.ML Algorithms & EvaluationMedium34 min
  3. 3Scaling Laws & Compute-Optimal TrainingLearn how Kaplan, Chinchilla, and inference-aware fits split a training budget across parameters and tokens, and when a smaller over-trained model wins on lifetime cost.Advanced Training & AdaptationHard40 min
  4. 4Dataset Pipelines and Data QualityBuild versioned AI datasets with schema gates, grouped splits, contamination checks, and auditable receipts.ML Algorithms & EvaluationMedium29 min
  5. 5Pre-training Data at ScaleUnderstand how web-scale pre-training data is extracted, filtered, deduplicated, mixed, tokenized, and packed into training-ready shards, including decontamination, late-stage annealing, and synthetic-data tradeoffs.Advanced Training & AdaptationHard41 min
  6. 6Build GPT from Scratch LabBuild and train a tiny GPT end to end on Shakespeare: tokenize with GPT-style subwords, remap active token IDs, run causal self-attention, track validation loss, save a checkpoint, and sample text.Advanced Training & AdaptationHard34 min
  7. 7Instruction Tuning & Chat TemplatesTeach a base language model to answer as an assistant: curate grounded SFT rows, serialize chat turns exactly, choose loss targets, pack safely, and detect serving-time template drift.Core LLM FoundationsMedium30 min
  8. 8Continued Pretraining for Domain ShiftLearn when to keep the causal language-modeling objective and continue pretraining on domain text instead of jumping straight to SFT, and how to evaluate the trade-off against forgetting, cost, and downstream gain.Advanced Training & AdaptationHard37 min
  9. 9LLM-as-a-Judge EvaluationBuild rubric-based judges, detect unstable preferences, and measure agreement without mistaking a plausible score for ground truth.Applied LLM EngineeringMedium25 min
  10. 10Synthetic Data PipelinesBuild post-training synthetic data as a gated pipeline: Self-Instruct, Evol-Instruct, grounded execution, calibrated judges, preference pairs, diversity, decontamination, and versioned shards.Advanced Training & AdaptationHard33 min
  11. 11Supervised Fine-Tuning PipelineRun supervised fine-tuning as a real training system: choose the learning objective before the update surface, verify response-token loss and packing, track the real batch budget, save resumable checkpoints, and export on held-out behavior.Advanced Training & AdaptationHard37 min
  12. 12Experiment Tracking with MLflow and W&BRecord the inputs, results, artifacts, and limits behind a candidate decision, then verify them with local MLflow and W&B examples.Applied LLM EngineeringMedium31 min
  13. 13CUDA for ML TrainingFollow one access-ticket batch from CPU memory into CUDA kernels. Learn thread and memory hierarchy, coalesced access, roofline decisions, safe device placement, honest timing, and first-line diagnosis for setup, OOM, and throughput failures.Computing FoundationsEasy24 min
  14. 14Mixed Precision TrainingExplore disappearing gradients, asymmetric rounding, and loss scaling. Test why ordinary AMP keeps FP32 updates before comparing training speed and memory.Advanced Training & AdaptationHard43 min
  15. 15Distributed Training: FSDP & ZeROMaster ZeRO stages, FSDP2 fully_shard architecture, 4D parallelism topology (DP, TP, PP, CP), and NCCL collective communication trade-offs.Advanced Training & AdaptationHard62 min
  16. 16LoRA & Parameter-Efficient TuningUnderstand the mathematics of Low-Rank Adaptation (LoRA), modern adapter targeting strategies, and the real memory tradeoffs compared to full fine-tuning and QLoRA.Advanced Training & AdaptationHard51 min
  17. 17Training Run OperationsRecover training state after interruption, budget checkpoint time, preserve effective batch and scheduler progress, and choose an adaptation recipe that fits the data and memory.Advanced Training & AdaptationHard30 min
  18. 18Data Labeling and Human FeedbackTurn traces into reviewed feedback data: preserve privacy, measure agreement, select useful cases, and protect independent evaluation.Applied LLM EngineeringMedium30 min
  19. 19Reward Modeling from Preference DataTrain reward models as a first-class post-training stage: validate chosen/rejected pairs and splits, fit a scalar reward head with Bradley-Terry loss, audit generalization, and decide when explicit rewards are worth the extra complexity.Advanced Training & AdaptationHard40 min
  20. 20RLHF & DPO AlignmentFollow preference labels into PPO-style RLHF or DPO. Calculate KL-shaped rewards, clipped policy updates, and response-only DPO gradients, then check whether better training scores improve held-out behavior.Advanced Training & AdaptationHard52 min
  21. 21Constitutional AI & Red TeamingTrace Constitutional AI from critique and revision to AI preference labels, then build honest red-team evaluations that separate model failures, false refusals, and judge errors.Advanced Training & AdaptationHard32 min
  22. 22RLVR & Verifiable RewardsUnderstand RLVR, a post-training approach that uses programmatic verification instead of learned human-preference rewards to improve checked outcomes in math, code, and other contract-driven tasks.Advanced Training & AdaptationHard41 min
  23. 23Knowledge Distillation for LLMsUnderstand the main forms of knowledge distillation for LLMs, from logit matching and response-based supervision to on-policy KD. Learn when distillation helps, where student capacity becomes the bottleneck, and how to implement a correct teacher-student training loop.Advanced Training & AdaptationHard45 min
  24. 24LLM Benchmarks & LimitationsBuild an evaluation suite for a policy-answering LLM: score evidence use, understand public benchmark contracts, control judge bias, and make release decisions from private tests.Core LLM FoundationsMedium33 min
  25. 25Model Versioning & DeploymentTurn an evaluated LLM change into an immutable release bundle, promote it through measured traffic, and roll back without losing lineage.Applied LLM EngineeringMedium31 min
  26. 26Capstone: Fine-Tuned ClassifierFit a ticket classifier, compare held-out baselines, and check labels, thresholds, scoring evidence, and agent intake.Portfolio CapstonesHard44 min