LeetLLM
Tracks
Advanced20 lessons

AI Research Scientist

Build the statistical, experimental, and model-level judgment needed to investigate new AI methods and defend empirical claims.

Aspiring research scientists and research engineers working on language models, training methods, evaluation, or interpretability.

You can frame a research question, design controlled experiments, reproduce model training, evaluate evidence, and diagnose what changed inside a model.

  1. 1Linear Algebra for MLFind hidden directions in a support-incident matrix with SVD, then use rank, PCA, truncation, and condition numbers without losing sight of what the numbers mean.Math & StatisticsEasy13 min
  2. 2Adam, Momentum, SchedulersTrace SGD, momentum, Adam, AdamW, schedules, and gradient clipping on one uneven loss surface. Learn what each optimizer buffer measures and how to validate a training choice.Math & StatisticsEasy15 min
  3. 3Probability for Machine LearningA beginner-first probability article that teaches events, priors, conditional probability, independence, Bayes rule, and base-rate mistakes through one API abuse-risk detector story.Math & StatisticsEasy19 min
  4. 4Statistics and UncertaintyEstimate abuse risk in a flagged review queue from finite labels, using bootstrap intuition, score intervals, sampling bias checks, and calibrated reporting.Math & StatisticsEasy12 min
  5. 5Distributions and SamplingModel an incident assistant with binary outcomes, request routes, tool-call counts, and tail latency, then challenge each simulation before trusting it.Math & StatisticsEasy12 min
  6. 6Hypothesis Tests, Intervals, and pass@kCompare a code-generation model with paired evidence, uncertainty for lift, and pass@k under a fixed sampling budget.Math & StatisticsEasy12 min
  7. 7Validation and LeakageMake model and policy claims honestly: define the decision moment, split access-review episodes by time and user, expose feature and preprocessing leakage, and audit LLM evaluation contamination.ML Algorithms & EvaluationMedium16 min
  8. 8Experiment Design and A/B TestingDesign a trustworthy online experiment for an AI support change: randomize customers, measure useful outcomes, quantify uncertainty, and reject false wins.ML Algorithms & EvaluationMedium20 min
  9. 9PyTorch Training LoopsBuild a PyTorch classifier from raw logits through autograd, validation, and reloadable checkpoints.ML Algorithms & EvaluationMedium18 min
  10. 10The Transformer Architecture End-to-EndTrace a support reply through masked attention, a decoder block, and next-token logits with readable NumPy and PyTorch code.Preparation & PrerequisitesEasy13 min
  11. 11Scaled Dot-Product AttentionLearn scaled dot-product attention from first principles, including Q/K/V routing, variance scaling, masks, multi-head shapes, KV-cache costs, and FlashAttention.Transformer Deep DivesHard41 min
  12. 12Scaling Laws & Compute-Optimal TrainingLearn the empirical power laws governing LLM performance, from Kaplan's parameter-heavy frontier through Chinchilla-optimal ratios to modern inference-aware training strategies.Advanced Training & AdaptationHard36 min
  13. 13Pre-training Data at ScaleUnderstand how web-scale pre-training data is extracted, filtered, deduplicated, mixed, tokenized, and packed into training-ready shards, including decontamination, late-stage annealing, and synthetic-data tradeoffs.Advanced Training & AdaptationHard37 min
  14. 14Build GPT from Scratch LabBuild and train a tiny GPT end to end on Shakespeare: tokenize with GPT-style subwords, remap active token IDs, run causal self-attention, track validation loss, save a checkpoint, and sample text.Advanced Training & AdaptationHard24 min
  15. 15Supervised Fine-Tuning PipelineRun supervised fine-tuning as a real training system: choose the learning objective before the update surface, verify response-token loss and packing, track the real batch budget, save resumable checkpoints, and export on held-out behavior.Advanced Training & AdaptationHard24 min
  16. 16Reward Modeling from Preference DataTrain reward models as a first-class post-training stage: validate chosen/rejected pairs and splits, fit a scalar reward head with Bradley-Terry loss, audit generalization, and decide when explicit rewards are worth the extra complexity.Advanced Training & AdaptationHard20 min
  17. 17LLM Benchmarks & LimitationsBuild an evaluation suite for a policy-answering LLM: score evidence use, understand public benchmark contracts, control judge bias, and make release decisions from private tests.Core LLM FoundationsMedium20 min
  18. 18Mechanistic InterpretabilityLearn how sparse autoencoders decompose transformer activations into candidate interpretable features, support circuit tracing, and enable controlled activation-steering experiments.Transformer Deep DivesHard30 min
  19. 19Reasoning & Test-Time ComputeUnderstand how reasoning models trade extra inference compute for better answers, and what that means for search, verifiers, KV cache pressure, and routing.Inference & Production ScaleHard43 min
  20. 20Recursive Language Models (RLM)Learn Recursive Language Models (RLMs): keep long context in a programmable environment, delegate targeted sub-calls, and release the design only after measured quality, cost, and safety checks.Advanced Training & AdaptationHard42 min