# LeetLLM > LeetLLM is an open AI engineering curriculum with 177 lessons, 8 learning tracks, 47 practice problems, and 17 public blog posts covering ML foundations, LLM systems, RAG, agents, training, inference, evaluation, AI lab interview prep, and open-source project deep dives. Use this file to route agents and retrieval systems to the most relevant LeetLLM pages. Lesson pages are public and include explanations, code, diagrams, illustrations, references, practice prompts, pitfalls, and follow-up questions. Blog pages cover timely AI engineering topics and include published or updated dates in their metadata. Canonical URLs below point to public web pages. This file is for inference-time navigation and context selection; it does not replace robots.txt, sitemap.xml, or the site license. ## Primary Pages - [Home](https://leetllm.com): Site overview and current entry point. - [Build a Plan](https://leetllm.com/start): Three-question planner that turns a goal, experience level, and weekly time into a focused four-to-twenty-week curriculum path. - [Learn](https://leetllm.com/learn): 177 ordered lessons across foundations, applied LLM engineering, training, agents, retrieval, inference, system design, interviewing, and open-source project deep dives. - [Tracks](https://leetllm.com/tracks): 8 curated subsets of lessons for different learner goals. - [Practice](https://leetllm.com/practice): 47 coding practice problems plus global theory quizzes in the practice UI. - [Blog](https://leetllm.com/blog): 17 public posts on AI engineering, models, tools, infrastructure, interviews, and learning paths. - [Blog RSS](https://leetllm.com/blog/rss.xml): RSS feed for public blog posts. - [Sitemap](https://leetllm.com/sitemap.xml): Full XML sitemap for crawlers. ## Learning Tracks - [AI Engineer Foundations](https://leetllm.com/tracks/ai-engineer-foundations): Beginner; 23 lessons; Software engineers, students, and career switchers new to ML systems. Outcome: You can read model diagrams, run Python experiments, call LLM APIs, and explain the... - [Production ML Systems](https://leetllm.com/tracks/production-ml-systems): Intermediate; 25 lessons; ML engineers, data scientists, and backend engineers shipping predictive systems beyond notebooks. Outcome: You can build a reproducible training pipeline,... - [RAG and Search Systems](https://leetllm.com/tracks/rag-and-search-systems): Intermediate; 22 lessons; Builders working on search, support bots, internal knowledge assistants, and document QA. Outcome: You can design a retrieval pipeline, debug faithfulness... - [Agents and Tool Use](https://leetllm.com/tracks/agents-and-tools): Intermediate; 25 lessons; Engineers building coding agents, workflow agents, browser agents, or production tool-use systems. Outcome: You can design agent loops with tools, state, recovery... - [Inference and Serving](https://leetllm.com/tracks/inference-and-serving): Advanced; 21 lessons; Engineers responsible for latency, cost, local deployment, model gateways, and GPU serving reliability. Outcome: You can reason about model fit, slow responses, and... - [Training and Alignment](https://leetllm.com/tracks/training-and-alignment): Advanced; 25 lessons; Readers moving from API usage into model adaptation, post-training, and training infrastructure. Outcome: You can explain the lifecycle of a model update and choose... - [AI Research Scientist](https://leetllm.com/tracks/ai-research-scientist): Advanced; 44 lessons; Aspiring research scientists and research engineers who need rigorous experimental judgment, implementation depth, and public evidence. Outcome: You can frame a... - [AI Lab Interview Prep](https://leetllm.com/tracks/ai-lab-interview-prep): Advanced; 20 lessons; Engineers preparing for AI lab, applied research engineer, or LLM systems interviews. Outcome: You can practice the concepts interviewers probe while still learning... ## Curriculum: Computing Foundations - [Git, Shell, Linux for AI](https://leetllm.com/learn/git-shell-linux-reproducible-ai): Easy; 11 min; MLOps & Deployment; Build a reproducible AI repo with safe Git defaults, a clean-clone eval gate, shell diagnostics, and Linux process habits. - [Docker for Reproducible AI](https://leetllm.com/learn/docker-containerization-reproducible-ai): Easy; 11 min; MLOps & Deployment; Package the access-request scorer in a portable Docker image with a small build context, runtime data and secrets, a non-root user, and a reproducible gate. - [Python for AI Engineering](https://leetllm.com/learn/python-for-ai-engineering): Easy; 14 min; MLOps & Deployment; Learn Python as the first AI engineering loop: read JSONL rows, validate fields, compute exact-match accuracy, and harden the scorer with pytest, prompt... - [NumPy and Tensor Shapes](https://leetllm.com/learn/numpy-and-tensor-shapes): Easy; 16 min; NLP Fundamentals; Learn NumPy shape reasoning from first principles: name axes, predict indexing and broadcasting, reduce safely, distinguish reshape from transpose, and add... - [CUDA for ML Training](https://leetllm.com/learn/cuda-gpu-computing-for-ml): Easy; 14 min; Fine-Tuning & Training; Build beginner-first CUDA intuition for model training: CPU vs GPU roles, host-device copies, asynchronous execution, PyTorch device placement, and... - [MPS & Metal for ML on Mac](https://leetllm.com/learn/mps-metal-mac-ml-training): Easy; 11 min; Fine-Tuning & Training; Build beginner-first intuition for training on Apple silicon: what Metal and MPS are, why unified memory changes the CUDA mental model, how PyTorch... - [Data Structures for AI](https://leetllm.com/learn/data-structures-for-ai-systems): Easy; 12 min; MLOps & Deployment; Choose lists, inverted indexes, heaps, queues, and caches by the operations an AI system must serve. - [SQL and Data Modeling](https://leetllm.com/learn/sql-databases-data-modeling-ai): Easy; 13 min; MLOps & Deployment; Turn an in-memory support retriever into durable SQL tables. Create rows and keys, query with parameters and joins, enforce permissions, roll back failed... - [Algorithms for ML Engineers](https://leetllm.com/learn/algorithms-complexity-ml-engineers): Easy; 11 min; MLOps & Deployment; Learn to count retrieval work, express growth with Big-O, avoid wasteful selection and pairwise loops, and enforce a latency budget with runnable Python. ## Curriculum: Math & Statistics - [Gradients and Backprop](https://leetllm.com/learn/calculus-gradients-backpropagation-optimization): Easy; 15 min; Fine-Tuning & Training; Learn why training works by nudging one response-latency weight, tracing and summing chain-rule paths, checking gradients, and confirming them with... - [Vectors, Matrices & Tensors](https://leetllm.com/learn/vectors-matrices-tensors): Easy; 11 min; NLP Fundamentals; Turn one gradient vector into batches of model inputs while learning dot products, matrix transforms, tensor axes, and shape debugging. - [Linear Algebra for ML](https://leetllm.com/learn/linear-algebra-machine-learning-deeper): Easy; 13 min; Fine-Tuning & Training; Find hidden directions in a support-incident matrix with SVD, then use rank, PCA, truncation, and condition numbers without losing sight of what the... - [Adam, Momentum, Schedulers](https://leetllm.com/learn/optimization-algorithms-adam-schedulers): Easy; 14 min; Fine-Tuning & Training; Trace SGD, momentum, Adam, AdamW, schedules, and gradient clipping on one uneven loss surface. Learn what each optimizer buffer measures and how to... - [Probability for Machine Learning](https://leetllm.com/learn/probability-for-machine-learning): Easy; 15 min; Evaluation & Benchmarks; Use one API abuse-risk detector to learn events, priors, conditional probability, independence, Bayes rule, and base-rate mistakes. - [Statistics and Uncertainty](https://leetllm.com/learn/statistics-estimation-uncertainty): Easy; 11 min; Evaluation & Benchmarks; Estimate abuse risk in a flagged review queue from finite labels, using bootstrap intuition, score intervals, sampling bias checks, and calibrated... - [Distributions and Sampling](https://leetllm.com/learn/distributions-sampling-simulation): Easy; 12 min; Evaluation & Benchmarks; Model an incident assistant with binary outcomes, request routes, tool-call counts, and tail latency, then challenge each simulation before trusting... - [Hypothesis Tests, Intervals, and pass@k](https://leetllm.com/learn/hypothesis-tests-confidence-intervals-passk): Easy; 13 min; Evaluation & Benchmarks; Compare a code-generation model with paired evidence, uncertainty for lift, and pass@k under a fixed sampling budget. ## Curriculum: Preparation & Prerequisites - [Neural Networks from Scratch](https://leetllm.com/learn/neural-networks-from-scratch): Easy; 12 min; NLP Fundamentals; Trace an incident-risk network from one neuron to a batched NumPy forward pass, then diagnose activation, shape, scale, and numerical-stability failures. - [CNNs from Scratch](https://leetllm.com/learn/convolutional-neural-networks-from-scratch): Easy; 10 min; Model Architecture; Trace a CNN over a cracked equipment-panel photo patch: shared kernels, feature-map shapes, pooling, padding failures, and a NumPy-to-PyTorch forward pass. - [Training & Backpropagation](https://leetllm.com/learn/training-backpropagation-gradient-descent): Easy; 14 min; NLP Fundamentals; Follow a decode-latency model through prediction, loss, gradients, parameter updates, scalar autograd, mini-batches, validation checks, and PyTorch. - [Softmax, Cross-Entropy & Optimization](https://leetllm.com/learn/softmax-cross-entropy-optimization): Easy; 13 min; NLP Fundamentals; Turn raw class scores into stable probabilities and a useful learning signal, then apply the same loss to next-token predictions. - [RNNs, LSTMs, GRUs, and Sequence Modeling](https://leetllm.com/learn/rnns-lstms-grus-sequence-modeling): Easy; 12 min; Model Architecture; Trace an RNN over ordered events, see why gradients fade or grow, and use LSTM and GRU gates to control memory. - [Autoencoders and VAEs](https://leetllm.com/learn/autoencoders-vaes-generative-modeling-basics): Easy; 14 min; Model Architecture; Compress an inspection-image patch, turn its latent code into a sampleable distribution, and implement VAE loss and training. - [The Transformer Architecture End-to-End](https://leetllm.com/learn/transformer-architecture-end-to-end): Easy; 13 min; Transformer Architecture; Trace an incident note through masked attention, a decoder block, and next-token logits with readable NumPy and PyTorch code. - [Language Modeling & Next Tokens](https://leetllm.com/learn/language-modeling-next-token-prediction): Easy; 21 min; NLP Fundamentals; Learn how next-token prediction becomes a trainable language model, from bigram counts and neural n-grams to causal Transformer generation and KV-cache... - [From GPT to Modern LLMs](https://leetllm.com/learn/gpt-to-modern-llms): Easy; 22 min; NLP Fundamentals; Trace how decoder-only models grew into modern LLMs, then inspect scaling, instruction tuning, open weights, MoE, and serving tradeoffs with runnable... - [Prompt Engineering Fundamentals](https://leetllm.com/learn/prompt-engineering-fundamentals): Easy; 18 min; NLP Fundamentals; Build and test grounded prompts with clear roles, few-shot examples, structured outputs, evidence checks, and failure-focused evaluation. - [Calling LLM APIs in Production](https://leetllm.com/learn/calling-llm-apis-production): Easy; 17 min; MLOps & Deployment; Turn a grounded prompt into a reliable API boundary with server-side secrets, typed results, bounded retries, safe actions, and useful telemetry. - [First AI App End-to-End](https://leetllm.com/learn/first-ai-app-end-to-end): Easy; 15 min; MLOps & Deployment; Ship one traceable rotation-decision workflow: validated input, model boundary, stored status, clear UI states, failure tests, and deploy checks. - [The LLM Lifecycle](https://leetllm.com/learn/llm-lifecycle-training-to-deployment): Easy; 15 min; NLP Fundamentals; Follow one key-rotation assistant from base-model training to post-training, retrieval, serving, evaluation, and the fix chosen after a real failure. ## Curriculum: ML Algorithms & Evaluation - [Linear Regression from Scratch](https://leetllm.com/learn/linear-regression-from-scratch): Medium; 19 min; Evaluation & Benchmarks; Fit key-rotation assistant latency by hand, implement least squares and gradient descent in NumPy, then test failure cases and held-out behavior. - [Logistic Regression and Metrics](https://leetllm.com/learn/logistic-regression-classification-metrics-from-scratch): Medium; 22 min; Evaluation & Benchmarks; Route access-change requests with logistic regression from scratch: derive sigmoid and log loss, fit NumPy weights, select a cost-aware threshold on... - [Decision Trees, Forests, and Boosting](https://leetllm.com/learn/decision-trees-random-forests-gradient-boosting): Medium; 21 min; Evaluation & Benchmarks; Model access-change review with decision trees from scratch: compute impurity, test a non-perfect stump on held-out cases, compare forests and... - [Reinforcement Learning Basics](https://leetllm.com/learn/introduction-to-reinforcement-learning-fundamentals): Medium; 14 min; Evaluation & Benchmarks; Learn reinforcement learning through the access-review workflow from earlier lessons. Define an MDP, compute discounted returns and Bellman backups,... - [Validation and Leakage](https://leetllm.com/learn/cross-validation-leakage-generalization): Medium; 15 min; Evaluation & Benchmarks; Make model and policy claims honestly: define the decision moment, split access-review episodes by time and user, expose feature and preprocessing... - [Clustering and PCA](https://leetllm.com/learn/clustering-pca-representation-learning): Medium; 18 min; Embeddings & Vector Search; Inspect unlabeled developer-message embeddings with k-means and PCA, then stress-test whether apparent neighborhoods survive scale, metric, and... - [Core Retrieval Algorithms](https://leetllm.com/learn/core-retrieval-algorithms): Medium; 16 min; RAG & Retrieval; Build and evaluate the evidence-selection stage of a technical-docs assistant with BM25, dense similarity, rank fusion, reranking, and approximate search... - [Decoding Algorithms](https://leetllm.com/learn/decoding-algorithms): Medium; 16 min; Inference Optimization; Turn retrieved evidence into controlled text by implementing stable softmax, sampling filters, constrained decoding, beam search, and reproducible... - [Experiment Design and A/B Testing](https://leetllm.com/learn/experiment-design-ab-testing): Medium; 20 min; Evaluation & Benchmarks; Design a trustworthy online experiment for an AI support change: randomize customers, measure useful outcomes, quantify uncertainty, and reject... - [PyTorch Training Loops](https://leetllm.com/learn/pytorch-training-loops): Medium; 17 min; Fine-Tuning & Training; Build a PyTorch classifier from raw logits through autograd, validation, and reloadable checkpoints. - [Dataset Pipelines and Data Quality](https://leetllm.com/learn/dataset-pipelines-data-quality): Medium; 16 min; MLOps & Deployment; Build versioned AI datasets with schema gates, grouped splits, contamination checks, and auditable receipts. ## Curriculum: Production ML Systems - [Feature Engineering for Production ML](https://leetllm.com/learn/feature-engineering-production-ml): Medium; 9 min; MLOps & Deployment; Turn training-job events into stable prediction inputs while preventing leakage and training-serving mismatch. - [Batch and Streaming Feature Pipelines](https://leetllm.com/learn/batch-streaming-feature-pipelines): Medium; 12 min; MLOps & Deployment; Build point-in-time training-run features from events and preserve the same meaning in online serving. - [Gradient Boosted Trees in Production](https://leetllm.com/learn/gradient-boosted-trees-production): Medium; 12 min; MLOps & Deployment; Train a boosted SLA-risk baseline from tabular features, evaluate slices, and package deployment evidence. - [Ranking and Recommendation Systems](https://leetllm.com/learn/ranking-recommendation-systems): Medium; 16 min; Evaluation & Benchmarks; Rank documents for a developer using candidate retrieval, relevance metrics, and feedback-loop safeguards. - [Forecasting and Anomaly Detection](https://leetllm.com/learn/forecasting-anomaly-detection): Medium; 14 min; Evaluation & Benchmarks; Forecast batch-job demand with time-aware evaluation and turn large forecast errors into reviewable operational alerts. - [Monitoring Predictive Models](https://leetllm.com/learn/monitoring-predictive-models): Medium; 16 min; MLOps & Deployment; Monitor predictive models from feature freshness through delayed labels, then gate retraining, promotion, and rollback. ## Curriculum: Core LLM Foundations - [The Bitter Lesson & Compute](https://leetllm.com/learn/bitter-lesson-compute-over-heuristics): Medium; 15 min; Reasoning & Scaling; Use Sutton's Bitter Lesson to compare rules, learning, and search through a measured AI-incident routing lab. - [BPE, WordPiece, and SentencePiece](https://leetllm.com/learn/tokenization-bpe-wordpiece-sentencepiece): Medium; 19 min; NLP Fundamentals; Build a small subword tokenizer, compare BPE, WordPiece, and SentencePiece, then audit token cost and Unicode behavior. - [Static to Contextual Embeddings](https://leetllm.com/learn/word-embeddings-contextual-representations): Medium; 15 min; NLP Fundamentals; Turn token IDs into vectors, learn what nearby usage captures, and see why a word such as charge needs sentence-dependent representations. - [Perplexity & Model Evaluation](https://leetllm.com/learn/perplexity-language-model-evaluation): Medium; 16 min; Evaluation & Benchmarks; Compute perplexity from held-out token probabilities, compare models under a fixed protocol, normalize across tokenizers, and decide what PPL can't... - [File Ingestion for AI](https://leetllm.com/learn/file-ingestion-pdfs-html-ocr-markdown): Medium; 14 min; RAG & Retrieval; Turn PDFs, scans, HTML, and Markdown into faithful evidence records with provenance and quality checks before retrieval. - [Chunking Strategies](https://leetllm.com/learn/chunking-strategies-document-processing): Medium; 14 min; RAG & Retrieval; Turn clean documents into retrieval units that preserve answers, citations, and measurable search quality. - [LLM Benchmarks & Limitations](https://leetllm.com/learn/llm-evaluation-benchmarks-limitations): Medium; 21 min; Evaluation & Benchmarks; Build an evaluation suite for a policy-answering LLM: score evidence use, understand public benchmark contracts, control judge bias, and make... - [Instruction Tuning & Chat Templates](https://leetllm.com/learn/instruction-tuning-chat-templates): Medium; 18 min; Fine-Tuning & Training; Teach a base language model to answer as an assistant: curate grounded SFT rows, serialize chat turns exactly, choose loss targets, pack safely, and... ## Curriculum: Applied LLM Engineering - [Dimensionality Reduction for Embeddings](https://leetllm.com/learn/dimensionality-reduction-embeddings): Medium; 20 min; Embeddings & Vector Search; Shrink and inspect embedding indexes without guessing: measure recall while testing PCA, projections, native shortening, and quantization. - [CoT, ToT & Self-Consistency Prompting](https://leetllm.com/learn/chain-of-thought-advanced-prompting): Medium; 16 min; Prompt Engineering; Build and evaluate reasoning controllers: single traces, answer voting, and bounded tree search for multi-step LLM decisions. - [Function Calling & Tool Use](https://leetllm.com/learn/function-calling-tool-use-llms): Medium; 16 min; LLM Agents & Tool Use; Build a safe tool-calling runtime that validates model requests, executes controlled actions, feeds observations back, and evaluates complete... - [MCP & Tool Protocol Standards](https://leetllm.com/learn/mcp-tool-protocol-standards): Medium; 19 min; LLM Agents & Tool Use; Trace a pinned MCP compatibility session, build a working stdio integration, and apply current deprecation and trust-boundary guidance. - [Context Engineering](https://leetllm.com/learn/long-context-engineering-beyond-management): Medium; 24 min; LLM Agents & Tool Use; Move past fitting tokens into the window and learn context engineering: curate a high-signal working set, package reusable Agent Skills, and build... - [Prompt Injection Defense](https://leetllm.com/learn/prompt-injection-defense-strategies): Medium; 16 min; Alignment & Safety; Build a prompt-injection-resistant agent boundary: quarantine untrusted tool content, validate typed action proposals, require approval, and measure... - [Responsible AI Governance](https://leetllm.com/learn/responsible-ai-governance-ethics-compliance): Medium; 17 min; Alignment & Safety; Turn a tool-bearing LLM workflow into auditable evidence: classify its use, own risks, version controls, preserve traces, and gate releases. - [Data Labeling and Human Feedback](https://leetllm.com/learn/data-labeling-human-feedback-active-learning): Medium; 16 min; MLOps & Deployment; Build a trustworthy human-feedback data flywheel: redact traces, write rubrics, measure agreement, select useful examples, prevent leakage, and promote... - [Evaluating AI Agents](https://leetllm.com/learn/agent-evaluation-benchmarks): Medium; 18 min; LLM Agents & Tool Use; Evaluate model-promotion agent runs by final state, observable trace, safety gates, cost, and repeatability, then map private tests to public... - [Production RAG Pipelines](https://leetllm.com/learn/design-production-rag-pipeline): Medium; 17 min; RAG & Retrieval; Design a secure, traceable RAG service around versioned policy evidence, grounded answers, abstention, release gates, and latency budgets. - [Hybrid Search: Dense + Sparse](https://leetllm.com/learn/hybrid-search-dense-sparse-retrieval): Medium; 18 min; RAG & Retrieval; Upgrade a permission-safe RAG retriever with BM25, semantic scores, rank fusion, and recall gates for exact codes and paraphrased policy questions. - [Reranking and Cross-Encoders for RAG](https://leetllm.com/learn/reranking-cross-encoders-rag): Medium; 15 min; RAG & Retrieval; Turn a permission-safe hybrid candidate list into precise context using cross-encoder reasoning, ordering metrics, latency gates, and traceable evidence... - [RAG Evaluation for Reliable Answers](https://leetllm.com/learn/rag-evaluation-faithfulness-relevance): Medium; 14 min; Evaluation & Benchmarks; Evaluate a permission-safe RAG answer trace with context, claim, citation, failure-attribution, and release gates before automating softer judgments. - [LLM-as-a-Judge Evaluation](https://leetllm.com/learn/llm-as-judge-automated-evaluation): Medium; 18 min; Evaluation & Benchmarks; Add calibrated soft judgments to a RAG evaluation trace without letting an LLM override deterministic evidence gates. - [Bias & Fairness in LLMs](https://leetllm.com/learn/bias-detection-fairness-llms): Medium; 15 min; Alignment & Safety; Build a matched-pair fairness audit for an LLM judge, measure routing gaps, and block release when evidence is too weak. - [Hallucination Detection & Mitigation](https://leetllm.com/learn/hallucination-detection-mitigation): Medium; 15 min; Alignment & Safety; Build a claim-level grounding gate for incident updates that verifies evidence, catches confident fabrication, abstains safely, and records release... - [LLM Observability & Monitoring](https://leetllm.com/learn/observability-monitoring-llm-systems): Medium; 18 min; MLOps & Deployment; Turn claim-level answer traces into production metrics, actionable alerts, privacy-safe debugging records, and reproducible incident evidence. - [Experiment Tracking with MLflow and W&B](https://leetllm.com/learn/experiment-tracking-mlflow-wandb): Medium; 17 min; MLOps & Deployment; Turn a live LLM regression into a reproducible candidate decision by logging inputs, metrics, artifacts, and promotion evidence. - [Prompt Optimization with DSPy](https://leetllm.com/learn/programmatic-prompt-optimization-dspy): Medium; 23 min; Prompt Engineering; Move beyond manual prompt editing. Use DSPy to search prompt and few-shot candidates from data, then release only after held-out evaluation. - [Model Versioning & Deployment](https://leetllm.com/learn/model-versioning-continuous-deployment): Medium; 19 min; MLOps & Deployment; Turn an evaluated LLM change into an immutable release bundle, promote it through measured traffic, and roll back without losing lineage. - [Semantic Caching & Cost Optimization](https://leetllm.com/learn/semantic-caching-cost-latency-optimization): Medium; 17 min; Inference Optimization; Reuse stable policy answers across paraphrased questions without crossing release, access, or freshness boundaries; then prove the cache is both safe... - [LLM Cost Engineering & Token Economics](https://leetllm.com/learn/llm-cost-engineering-token-economics): Medium; 16 min; MLOps & Deployment; Build an auditable LLM cost ledger from usage traces, cache decisions, output contracts, offline batch work, and release budget gates. - [Model Gateways, Routing, and Fallbacks](https://leetllm.com/learn/model-gateways-routing-fallbacks): Medium; 16 min; MLOps & Deployment; Turn an audited cost contract into a model gateway that preserves data, schema, review, and budget requirements across routing and fallback. - [Design an Automated Support Agent](https://leetllm.com/learn/design-conversational-ai-support-bot): Medium; 21 min; System Design; Assemble a stateful support agent that grounds replies, gates credit actions, preserves gateway policy, and hands difficult cases to humans. ## Curriculum: Portfolio Capstones - [Capstone: Delivery ETA Prediction](https://leetllm.com/learn/capstone-delivery-eta-prediction): Hard; 13 min; MLOps & Deployment; Ship a delivery-delay warning service with as-of features, versioned policy gates, baseline evidence, and monitored fallback. - [Capstone: Product Ranking](https://leetllm.com/learn/capstone-product-ranking-recommendation): Hard; 14 min; Evaluation & Benchmarks; Ship a marketplace ranking candidate with eligible retrieval, separate recall and NDCG gates, replayable exposure rows, and an A/B-ready rollback... - [Capstone: Demand Forecasting](https://leetllm.com/learn/capstone-demand-forecasting-alerts): Hard; 14 min; MLOps & Deployment; Ship a demand forecast and capacity-alert artifact with rolling backtests, alert review, and retraining policy. - [Capstone: Image Damage Classifier](https://leetllm.com/learn/capstone-damaged-package-vision-classifier): Hard; 16 min; Multimodal Models; Ship a damaged-package photo triage service with quality checks, slice evaluation, serving bundles, and review monitoring. - [Capstone: Production ML Pipeline](https://leetllm.com/learn/capstone-production-ml-pipeline): Hard; 21 min; MLOps & Deployment; Assemble predictive ML artifacts into validated training, registry promotion, canary monitoring, and rollback. - [Capstone: Document QA](https://leetllm.com/learn/capstone-document-qa-product): Hard; 36 min; System Design; Ship a policy-evidence service with controlled admission, cited answers, abstention, replayable eval rows, and source-bound semantic adjudication. - [Capstone: Eval Dashboard](https://leetllm.com/learn/capstone-evaluation-dashboard): Hard; 22 min; Evaluation & Benchmarks; Build a release dashboard for document QA that turns replayable evidence rows into exact-coverage gates, uncertainty checks, and inspectable decisions. - [Capstone: Fine-Tuned Classifier](https://leetllm.com/learn/capstone-fine-tuned-classifier): Hard; 24 min; Fine-Tuning & Training; Train and gate an access-ticket encoder that exports exact-receipt evidence and safe intake decisions to a production agent. - [Capstone: Reproducible ML Study](https://leetllm.com/learn/capstone-reproducible-research-study): Hard; 31 min; Evaluation & Benchmarks; Turn one research paper into a falsifiable, public-safe ML study with paired experiments, uncertainty, reproducible artifacts, and a defensible report. ## Curriculum: Transformer Deep Dives - [Sentence Embeddings & Contrastive Loss](https://leetllm.com/learn/sentence-embeddings-contrastive-learning): Hard; 40 min; Embeddings & Vector Search; Learn how contrastive losses train sentence embeddings, why hard negatives matter, and how retrieval systems combine bi-encoders, rerankers, and... - [Embedding Similarity & Quantization](https://leetllm.com/learn/embeddings-cosine-dot-product-quantization): Hard; 40 min; Embeddings & Vector Search; Learn vector scoring contracts, evaluate Matryoshka widths, and measure scalar, product, and binary quantization before deploying compressed... - [Scaled Dot-Product Attention](https://leetllm.com/learn/scaled-dot-product-attention): Hard; 41 min; Transformer Architecture; Learn scaled dot-product attention from first principles, including Q/K/V routing, variance scaling, masks, multi-head shapes, KV-cache costs, and... - [Vision Transformers and Image Encoders](https://leetllm.com/learn/vision-transformers-image-encoders): Hard; 25 min; Multimodal Models; Understand how Vision Transformers split images into patches, build visual tokens, train encoders, and connect to CLIP and multimodal LLMs. - [Positional Encoding: RoPE & ALiBi](https://leetllm.com/learn/positional-encoding-rope-alibi): Hard; 32 min; Transformer Architecture; Understand why transformers need position information, how sinusoidal encodings work, how RoPE and ALiBi encode relative position, and why... - [Layer Normalization: Pre-LN vs Post-LN](https://leetllm.com/learn/layer-normalization-pre-ln-post-ln): Hard; 29 min; Transformer Architecture; Understand LayerNorm mechanics, Pre-LN versus Post-LN placement, RMSNorm simplification, gradient stability, and hybrid normalization layouts for... - [Mechanistic Interpretability](https://leetllm.com/learn/mechanistic-interpretability-sparse-autoencoders): Hard; 31 min; Transformer Architecture; Learn how sparse autoencoders decompose transformer activations into candidate interpretable features, support circuit tracing, and enable controlled... - [Decoding Strategies: Greedy to Nucleus](https://leetllm.com/learn/decoding-strategies-greedy-beam-nucleus): Hard; 34 min; NLP Fundamentals; Compare decoding strategies for text generation: greedy, beam search, top-k, nucleus (top-p), temperature, repetition controls, and newer variants like min-p. ## Curriculum: Advanced Training & Adaptation - [Scaling Laws & Compute-Optimal Training](https://leetllm.com/learn/scaling-laws-compute-optimal-training): Hard; 35 min; Fine-Tuning & Training; Learn the empirical power laws governing LLM performance, from Kaplan's parameter-heavy frontier through Chinchilla-optimal ratios to modern... - [Pre-training Data at Scale](https://leetllm.com/learn/pre-training-data-pipelines-scale): Hard; 35 min; Fine-Tuning & Training; Understand how web-scale pre-training data is extracted, filtered, deduplicated, mixed, tokenized, and packed into training-ready shards, including... - [Build GPT from Scratch Lab](https://leetllm.com/learn/build-gpt-from-scratch-lab): Hard; 22 min; Fine-Tuning & Training; Build and train a tiny GPT end to end on Shakespeare: tokenize with GPT-style subwords, remap active token IDs, run causal self-attention, track... - [JAX for PyTorch Researchers](https://leetllm.com/learn/jax-for-ml-research): Hard; 23 min; Fine-Tuning & Training; Read and modify JAX research code from a PyTorch foundation by making state, randomness, transformations, compilation, and timing explicit. - [Continued Pretraining for Domain Shift](https://leetllm.com/learn/continued-pretraining-domain-adaptation): Hard; 20 min; Fine-Tuning & Training; Learn when to keep the causal language-modeling objective and continue pretraining on domain text instead of jumping straight to SFT, and how to... - [Synthetic Data Pipelines](https://leetllm.com/learn/synthetic-data-generation-pipelines-llms): Hard; 25 min; Fine-Tuning & Training; Build synthetic post-training data pipelines with Self-Instruct, Evol-Instruct, calibrated judge signals, verifiers, preference pairs, diversity... - [Supervised Fine-Tuning Pipeline](https://leetllm.com/learn/supervised-fine-tuning-pipeline): Hard; 23 min; Fine-Tuning & Training; Run supervised fine-tuning as a real training system: choose the learning objective before the update surface, verify response-token loss and packing,... - [Mixed Precision Training](https://leetllm.com/learn/mixed-precision-training-loss-scaling): Hard; 22 min; Fine-Tuning & Training; Measure how FP16 and BF16 affect training range, update precision, memory, and release evidence before enabling faster low-precision compute. - [Distributed Training: FSDP & ZeRO](https://leetllm.com/learn/distributed-training-fsdp-deepspeed-zero): Hard; 40 min; Fine-Tuning & Training; Understand ZeRO stages, current FSDP1 vs FSDP2 guidance, and when native PyTorch or DeepSpeed is the right choice for large-model training. - [LoRA & Parameter-Efficient Tuning](https://leetllm.com/learn/lora-parameter-efficient-fine-tuning): Hard; 33 min; Fine-Tuning & Training; Understand the mathematics of Low-Rank Adaptation (LoRA), modern adapter targeting strategies, and the real memory tradeoffs compared to full... - [Reward Modeling from Preference Data](https://leetllm.com/learn/reward-modeling-from-preference-data): Hard; 18 min; Alignment & Safety; Train reward models as a first-class post-training stage: validate chosen/rejected pairs and splits, fit a scalar reward head with Bradley-Terry loss,... - [RLHF & DPO Alignment](https://leetllm.com/learn/rlhf-dpo-aligning-language-models): Hard; 33 min; Alignment & Safety; Understand the RLHF pipeline and DPO, including reward modeling, PPO mechanics, and the trade-offs between iterative reinforcement learning and direct... - [Constitutional AI & Red Teaming](https://leetllm.com/learn/constitutional-ai-automated-red-teaming): Hard; 30 min; Alignment & Safety; Understand how Constitutional AI reduces reliance on repeated human preference labeling through AI critique and ranking, and how automated red teaming... - [RLVR & Verifiable Rewards](https://leetllm.com/learn/rlvr-verifiable-rewards-training): Hard; 37 min; Fine-Tuning & Training; Understand RLVR, a post-training approach that uses programmatic verification instead of learned human-preference rewards to improve checked outcomes... - [Knowledge Distillation for LLMs](https://leetllm.com/learn/knowledge-distillation-llms): Hard; 25 min; Fine-Tuning & Training; Understand the main forms of knowledge distillation for LLMs, from logit matching and response-based supervision to on-policy KD. Learn when... - [Model Merging and Weight Interpolation](https://leetllm.com/learn/model-merging-weight-interpolation): Hard; 29 min; Fine-Tuning & Training; Learn model merging techniques, from simple weight averaging and task arithmetic to TIES-Merging and DARE, including practical guidance on tokenizer... ## Curriculum: Advanced Agents & Retrieval - [Vector DB Internals: HNSW & IVF](https://leetllm.com/learn/vector-database-internals-hnsw-ivf): Hard; 35 min; Embeddings & Vector Search; Learn how approximate nearest neighbor indexes use HNSW, IVF, and Product Quantization to balance speed, recall, and memory in production vector... - [Advanced RAG: HyDE & Self-RAG](https://leetllm.com/learn/advanced-rag-query-rewriting-self-rag): Hard; 33 min; RAG & Retrieval; Learn how query rewriting, HyDE, Self-RAG, and Corrective RAG change retrieval control, and how to evaluate their cost and evidence quality. - [GraphRAG & Knowledge Graphs](https://leetllm.com/learn/graphrag-knowledge-graphs-vector-search): Hard; 37 min; RAG & Retrieval; Learn how GraphRAG uses entity graphs, hierarchical community reports, and embeddings to retrieve evidence for relationship-heavy and corpus-level questions. - [RAG Security & Access Control](https://leetllm.com/learn/enterprise-rag-security-rbac-access-control): Hard; 37 min; RAG & Retrieval; Learn how document ACLs, tenant isolation, retrieval-time authorization, output checks, and audit logs reduce private-data leakage risk in enterprise RAG. - [Structured Output Generation](https://leetllm.com/learn/structured-output-constrained-generation): Hard; 37 min; LLM Agents & Tool Use; Build reliable LLM interfaces with JSON mode, structured outputs, schema validation, and grammar-guided decoding. - [ReAct & Plan-and-Execute](https://leetllm.com/learn/agentic-architectures-react-plan-execute): Hard; 34 min; LLM Agents & Tool Use; Compare ReAct for tightly coupled tool use with Plan-and-Execute for longer workflows with explicit planning and replanning. - [Guardrails & Safety Filters](https://leetllm.com/learn/guardrails-safety-filters-production): Hard; 40 min; Alignment & Safety; Build layered guardrails for prompt injection defense, sensitive-data controls, structured outputs, policy enforcement, and safe tool use. - [Code Generation & Sandboxing](https://leetllm.com/learn/code-generation-agents-sandboxing): Hard; 33 min; LLM Agents & Tool Use; Build code agents that test candidate patches inside bounded sandboxes with runtime evidence and defense-in-depth controls. - [Computer-Use / GUI / Browser Agents](https://leetllm.com/learn/computer-use-gui-browser-agents): Hard; 28 min; LLM Agents & Tool Use; Build browser and desktop agents whose proposed clicks and keystrokes remain behind host policy, approval, verification, and sandbox controls. - [Human-in-the-Loop Agent Architecture](https://leetllm.com/learn/human-in-the-loop-agent-architecture): Hard; 36 min; LLM Agents & Tool Use; Build approval gates, durable checkpoints, and guarded resumes for agent actions that change external state. - [AI Coding Workflow with Agents](https://leetllm.com/learn/ai-coding-workflow-with-agents): Hard; 26 min; LLM Agents & Tool Use; Scope coding-agent tasks, isolate execution, keep patches on branches, verify behavior, and preserve human merge ownership. - [Agent Memory & Persistence](https://leetllm.com/learn/agent-memory-persistence-patterns): Hard; 35 min; LLM Agents & Tool Use; Design agent memory systems with scoped storage, sourced recall, tenant isolation, and durable checkpoints without letting recalled context authorize... - [Agent Failure & Recovery](https://leetllm.com/learn/agent-failure-states-retry-fallback): Hard; 51 min; LLM Agents & Tool Use; Learn how to implement validation checks, retries, checkpointed recovery, state reconciliation, loop breakers, and graceful degradation when LLM agents... - [Recursive Language Models (RLM)](https://leetllm.com/learn/recursive-language-models-rlm): Hard; 40 min; Reasoning & Scaling; Learn Recursive Language Models (RLMs): keep long context in a programmable environment, delegate targeted sub-calls, and release the design only after... - [Multi-Agent Orchestration](https://leetllm.com/learn/multi-agent-dags-langgraph-orchestration): Hard; 43 min; LLM Agents & Tool Use; Master multi-agent orchestration with LangGraph, AutoGen teams, and OpenAI handoffs. Learn DAG-style routing, typed shared state, protocol boundaries,... - [Capstone: Production Agent](https://leetllm.com/learn/capstone-production-llm-agent): Hard; 21 min; LLM Agents & Tool Use; Assemble classifier intake, cited policy evidence, approval-gated actions, and episode release tests into a production agent. ## Curriculum: Inference & Production Scale - [Inference: TTFT, TPS & KV Cache](https://leetllm.com/learn/inference-mechanics-ttft-tps-kv-cache): Hard; 29 min; Inference Optimization; Understand the two-phase inference process (prefill vs decode), derive the KV cache memory formula, and learn production optimizations like chunked... - [Multi-Query & Grouped-Query Attention](https://leetllm.com/learn/multi-query-grouped-query-attention): Hard; 35 min; Inference Optimization; Compare MHA, MQA, and GQA architectures, calculate their KV cache footprint, and reason about memory-limited serving tradeoffs. - [KV Cache & PagedAttention](https://leetllm.com/learn/kv-cache-paged-attention-llm-serving): Hard; 34 min; Inference Optimization; Calculate KV cache capacity, trace paged block allocation, and separate memory packing from prefix reuse and scheduling tradeoffs. - [Prefix Caching and Prompt Caching](https://leetllm.com/learn/prefix-caching-prompt-caching): Hard; 19 min; Inference Optimization; Structure exact reusable prefixes, validate cache hits from usage fields, and enforce invalidation and tenant-isolation boundaries. - [FlashAttention & Memory Efficiency](https://leetllm.com/learn/flashattention-memory-efficient-attention): Hard; 32 min; Inference Optimization; Understand how FlashAttention cuts auxiliary attention memory from O(n²) to O(n) with tiling and online softmax, and analyze its IO complexity. - [Continuous Batching & Scheduling](https://leetllm.com/learn/continuous-batching-request-scheduling): Hard; 32 min; Inference Optimization; Understand how LLM schedulers use continuous batching, chunked prefill, and prefill-decode disaggregation to improve throughput without violating TTFT,... - [Scaling LLM Inference](https://leetllm.com/learn/scaling-llm-inference-batching-kv-cache): Hard; 40 min; Inference Optimization; Explains why decode-heavy LLM serving is often memory-bound and how KV-cache design, batching, PagedAttention, and speculative decoding improve scale. - [Model Parallelism for LLM Inference](https://leetllm.com/learn/model-parallelism-for-llm-inference): Hard; 22 min; Inference Optimization; Learn tensor parallelism, pipeline parallelism, context parallelism, and how multi-GPU serving trades memory capacity for communication overhead. - [Model Quantization: GPTQ, AWQ & GGUF](https://leetllm.com/learn/model-quantization-gptq-awq-gguf): Hard; 33 min; Inference Optimization; Understand how GPTQ, AWQ, and GGUF trade off accuracy, memory footprint, and portability when serving LLMs on GPUs or local hardware. - [Local LLM Deployment](https://leetllm.com/learn/local-llm-deployment-hardware-quantization): Hard; 22 min; Inference Optimization; Plan local LLM deployment with model size, quantization, pruning and sparsity trade-offs, Docker packaging, runtime choice, and hardware budgets. - [SLM Specialization & Edge Deployment](https://leetllm.com/learn/slm-specialization-edge-deployment): Hard; 26 min; Inference Optimization; Distill large teachers into compact SLMs using MobileLLM architectures and Phi-style data recipes. Compile and run them on-device with MLC LLM, ONNX... - [Speculative Decoding](https://leetllm.com/learn/speculative-decoding): Hard; 32 min; Inference Optimization; Reduce LLM inter-token latency by pairing cheap drafting with target-model verification. Learn the rejection-sampling proof, speedup model, method... - [Long Context Window Management](https://leetllm.com/learn/long-context-window-management): Hard; 34 min; Inference Optimization; Master long-context LLM engineering: KV-cache math, prefill-vs-decode bottlenecks, RoPE scaling, lost-in-the-middle behavior, and long-context vs. RAG... - [Mixture of Experts Architecture](https://leetllm.com/learn/mixture-of-experts-architecture): Hard; 35 min; Transformer Architecture; Master MoE routing, load balancing, and dense-vs-sparse serving tradeoffs from Mixtral through GLM-5.2 and DeepSeek V4 Flash 0731. - [Mamba & State Space Models](https://leetllm.com/learn/state-space-models-mamba-alternatives): Hard; 33 min; Inference Optimization; Master linear-time sequence modeling: from S4 and HiPPO to Mamba's selective recurrence, Mamba-2's SSD framework, Mamba-3's inference-first... - [Reasoning & Test-Time Compute](https://leetllm.com/learn/reasoning-models-test-time-compute): Hard; 38 min; Reasoning & Scaling; Understand how reasoning models trade extra inference compute for better answers, and what that means for search, verifiers, KV cache pressure, and... - [Advanced MLOps & DevOps for AI](https://leetllm.com/learn/advanced-mlops-devops-ai): Hard; 20 min; MLOps & Deployment; Master advanced MLOps and DevOps patterns for LLM systems: GitOps for prompts and models, feature stores for embedding features, automated rollback on eval... - [GPU Serving & Autoscaling](https://leetllm.com/learn/llm-deployment-gpu-serving-autoscaling): Hard; 50 min; MLOps & Deployment; Master the design of GPU serving infrastructure for LLMs with autoscaling, continuous batching, and cost optimization. - [A/B Testing for LLMs](https://leetllm.com/learn/ab-testing-online-evaluation-llms): Hard; 43 min; Evaluation & Benchmarks; Master the design of an A/B testing framework for LLM-powered features, including traffic routing, metric selection, sample sizing, and automated... ## Curriculum: System Design Capstones - [Content Moderation System](https://leetllm.com/learn/design-real-time-content-moderation-system): Hard; 40 min; System Design; Master the architecture of a real-time content moderation system using LLMs and specialized classifiers. - [Code Completion System](https://leetllm.com/learn/design-code-completion-system): Hard; 42 min; System Design; Design a real-time code completion path with context construction, measured serving latency, privacy controls, and stale-result suppression. - [Multi-Tenant LLM Platform](https://leetllm.com/learn/design-multi-tenant-llm-serving-platform): Hard; 34 min; System Design; Design a shared LLM platform with tenant-scoped state, quota enforcement, adapter routing, KV accounting, and measured GPU utilization. - [LLM-Powered Search Engine](https://leetllm.com/learn/design-llm-powered-search-engine): Hard; 35 min; System Design; Master the architecture of an end-to-end AI search engine, covering freshness routing, hybrid retrieval, evidence packing, citation verification, and streaming... - [Vision-Language Models & CLIP](https://leetllm.com/learn/vision-language-models-clip-beyond): Hard; 44 min; Multimodal Models; Design a visual inspection and search product while learning CLIP, zero-shot classification, visual token budgets, grounding, and modern VLM connectors. - [Multimodal LLM Architecture](https://leetllm.com/learn/multimodal-llms-architecture-design): Hard; 40 min; Multimodal Models; Design a multimodal incident-evidence copilot while learning encoders, connectors, fusion, token budgets, training, grounding, and serving constraints. - [Diffusion Models: Images & Text](https://leetllm.com/learn/image-generation-diffusion-models): Hard; 47 min; Multimodal Models; Design a governed image-generation service while learning DDPM noising, stochastic sampling, latent diffusion, classifier-free guidance, DiT backbones, and... - [Real-Time Voice AI Agent](https://leetllm.com/learn/design-real-time-voice-ai-agent): Hard; 38 min; System Design; Master real-time voice AI architecture: turn detection, streaming STT/LLM/TTS, native audio trade-offs, WebRTC transport, and barge-in state. - [Reasoning Agent System Design](https://leetllm.com/learn/design-reasoning-agent-test-time-compute): Hard; 45 min; System Design; Design a production reasoning agent that routes by difficulty, evaluates candidate work, requires evidence before release, and survives serving bottlenecks like... ## Curriculum: AI Lab Interviewing - [AI Lab Coding Interview: Python Systems](https://leetllm.com/learn/ai-lab-coding-interview-python-systems): Hard; 25 min; MLOps & Deployment; Practice production-shaped Python coding prompts: crawlers, in-memory stores, ledgers, schedulers, parsers, rate limiters, caches, and concurrency... - [AI Lab System Design Interview](https://leetllm.com/learn/ai-lab-system-design-interview): Hard; 26 min; System Design; Design AI lab systems with clear goals, scale math, APIs, data models, overload behavior, permissions, eval gates, and operational debugging paths. - [AI Lab Behavioral Interview](https://leetllm.com/learn/ai-lab-behavioral-interview): Hard; 17 min; MLOps & Deployment; Prepare behavioral answers for AI labs around judgment, humility, incident leadership, disagreement, safety mechanisms, ambiguity, and evidence of... - [AI Lab Technical Presentation](https://leetllm.com/learn/ai-lab-technical-presentation): Hard; 15 min; System Design; Prepare a technical project presentation that proves ownership, architecture taste, tradeoff judgment, rollout discipline, metrics, and depth under questioning. ## Curriculum: Project Deep Dives - [Deep Dive - vLLM](https://leetllm.com/learn/deep-dive-vllm): Hard; 20 min; Inference Optimization; Read vLLM as a living serving system: PagedAttention's memory idea, the V1 engine loop, block-pool caching, scheduling, kernels, APIs, and production... - [Deep Dive - SkyRL](https://leetllm.com/learn/deep-dive-skyrl): Hard; 33 min; Fine-Tuning & Training; Read SkyRL as a set of replaceable interfaces for training language-model agents: environments, generators, inference engines, trainers, weight... - [Deep Dive - FlashAttention](https://leetllm.com/learn/deep-dive-flashattention): Hard; 20 min; Inference Optimization; Read FlashAttention from first principles: exact tiled attention, online softmax, GPU memory traffic, kernel dispatch, version history, and production... - [Deep Dive - FlashInfer](https://leetllm.com/learn/deep-dive-flashinfer): Hard; 20 min; Inference Optimization; Trace FlashInfer from irregular KV-cache layouts through load-balanced attention kernels, composable state, and production serving boundaries. - [Deep Dive - DeepGEMM](https://leetllm.com/learn/deep-dive-deepgemm): Hard; 31 min; Inference Optimization; Read DeepGEMM as a GPU-kernel case study: tiled GEMMs, low-bit scaling, runtime JIT, the DeepSeek indexer, and Mega MoE communication-compute overlap. - [Deep Dive - NCCL](https://leetllm.com/learn/deep-dive-nccl): Hard; 21 min; Fine-Tuning & Training; Read NCCL as the communication engine beneath distributed AI: collective contracts, rings and trees, topology discovery, CUDA streams, transports,... - [Deep Dive - Megatron](https://leetllm.com/learn/deep-dive-megatron): Hard; 23 min; Fine-Tuning & Training; Read Megatron-LM and Megatron Core as a distributed training system: rank groups, parallel axes, optimizer sharding, MoE dispatch, low precision, and... - [Deep Dive - DeepSpeed](https://leetllm.com/learn/deep-dive-deepspeed): Hard; 20 min; Fine-Tuning & Training; Read DeepSpeed from its engine boundary through ZeRO state ownership, layer-time gathers, CPU and NVMe offload, pipeline limits, checkpoint recovery,... - [Deep Dive - Ray](https://leetllm.com/learn/deep-dive-ray): Hard; 16 min; MLOps & Deployment; Read Ray as a distributed execution substrate for LLM data, training, reinforcement learning, tuning, and serving: tasks, actors, objects, scheduling,... - [Deep Dive - MLflow](https://leetllm.com/learn/deep-dive-mlflow): Hard; 17 min; MLOps & Deployment; Read MLflow as an evidence and lineage system for models and LLM applications: tracking, artifacts, traces, evaluation datasets, prompts, registries,... - [Deep Dive - PyTorch](https://leetllm.com/learn/deep-dive-pytorch): Hard; 32 min; Fine-Tuning & Training; Read PyTorch from TensorImpl and dispatch keys through autograd, torch.compile, attention kernels, FSDP2, DTensor, and DeviceMesh, then debug the... - [Deep Dive - Transformers](https://leetllm.com/learn/deep-dive-transformers): Hard; 26 min; Model Architecture; Read Hugging Face Transformers as a model-definition boundary: Hub revisions, AutoClass dispatch, weights, tokenizers, multimodal processors, generation,... - [Deep Dive - SGLang](https://leetllm.com/learn/deep-dive-sglang): Hard; 26 min; Inference Optimization; Read SGLang from frontend program to GPU step: RadixAttention, scheduling, constrained decoding, speculative execution, parallelism, and production... - [Deep Dive - slime](https://leetllm.com/learn/deep-dive-slime): Hard; 30 min; Fine-Tuning & Training; Read slime as a SGLang-native reinforcement-learning post-training system: Ray placement, Megatron training, Data Buffer contracts, asynchronous... - [Deep Dive - DeepEP](https://leetllm.com/learn/deep-dive-deepep): Hard; 36 min; Inference Optimization; Read DeepEP from token routing to GPU and RDMA transport: V2's ElasticBuffer, NCCL Gin, hybrid topology, deterministic handles, low precision, and the... - [Deep Dive - Tinker](https://leetllm.com/learn/deep-dive-tinker): Hard; 36 min; Fine-Tuning & Training; Read Tinker as a hosted LoRA post-training service: local control loops, remote GPU workers, token-level RL contracts, pipelined clock cycles,... - [Deep Dive - Light-PEFT](https://leetllm.com/learn/deep-dive-light-peft): Hard; 35 min; Fine-Tuning & Training; Read Light-PEFT as an academic early-pruning research release: learn why frozen-backbone forward cost remains, how masks become physical structured... ## Blog Posts - [How We Built LeetLLM](https://leetllm.com/blog/how-we-built-leetllm): updated 2026-08-02; tags: LeetLLM, Content Engineering, AI Engineering, Production, Education; How LeetLLM turns research into curated lessons with research packets, article bundles,... - [Run Qwen3.6 Locally with Unsloth GGUF](https://leetllm.com/blog/run-qwen36-local-unsloth-gguf): updated 2026-08-08; tags: Local LLM, Qwen3.6, Unsloth, GGUF, llama.cpp, GPU Inference; Run Qwen3.6 locally with llama.cpp and Unsloth GGUF. Pick between 27B dense and 35B-A3B MoE, choose a... - [AI Engineer Portfolio Projects That Get Interviews](https://leetllm.com/blog/ai-engineer-portfolio-projects-interviews): updated 2026-06-11; tags: Career, Portfolio, Projects; Five portfolio projects that prove real AI engineering skill: shipped demos, eval reports, traces, cost notes, tests, and design docs. - [How to Become an AI Engineer from Zero in 2026](https://leetllm.com/blog/how-to-become-ai-engineer-from-zero-2026): updated 2026-06-12; tags: Career, AI Engineering, Roadmap; A practical path from beginner to hire-ready AI engineer: programming basics, LLM APIs, RAG, evals, agents, deployment, and... - [DeepSeek V4 and the US AI Lab Squeeze](https://leetllm.com/blog/deepseek-v4-us-ai-labs): updated 2026-08-08; tags: DeepSeek, Open Models, AI Infrastructure, Agentic Coding, Industry; DeepSeek V4 Flash 0731 pairs MIT-licensed weights, 1M text context, DSpark speculative... - [Best AI Plans for OpenClaw in 2026](https://leetllm.com/blog/best-ai-plan-openclaw-2026): updated 2026-07-31; tags: OpenClaw, AI Coding Plans, Cost Optimization, Agentic AI, API Pricing, Qwen Cloud, Comparison; OpenClaw plan selection is a routing and quota problem. This July... - [Run Gemma 4 Locally with Ollama](https://leetllm.com/blog/run-gemma4-local-ollama): updated 2026-07-11; tags: Local LLM, Ollama, Gemma 4, MoE, GPU Inference; Gemma 4 now has a 12B laptop lane, MTP drafters, and QAT checkpoints. Pick an Ollama tag, tune context, and keep... - [vLLM vs SGLang vs TensorRT-LLM vs Ollama: Choosing an Inference Engine in 2026](https://leetllm.com/blog/llm-inference-engine-comparison-2026): updated 2026-08-08; tags: Inference, vLLM, SGLang, TensorRT-LLM, Ollama, Production AI, Benchmarks; Raw throughput is only half the inference-engine decision. Read an H100 benchmark... - [50 Essential LLM Engineering Concepts for 2026](https://leetllm.com/blog/top-50-llm-interview-questions-2026): updated 2026-08-08; tags: AI Engineering, Deep Dive, Architecture, System Design; Fifty LLM engineering concepts, organized by system layer. Each answer focuses on mechanism, trade-off,... - [AI Engineer Salary Guide 2026](https://leetllm.com/blog/ai-engineer-salary-compensation-guide-2026): updated 2026-08-02; tags: Career, Compensation; AI engineering pay is not one market. Use public job postings, Levels.fyi snapshots, and H-1B base records to benchmark offers by role... - [Open-Weight vs Closed API LLMs in 2026: The Enterprise Decision Framework](https://leetllm.com/blog/open-source-vs-closed-source-llms-2026): updated 2026-08-08; tags: LLMs, AI Engineering, Deep Dive, Enterprise, Open Models; The real 2026 decision is whether you need control over weights and deployment, or the speed of a managed... - [The Million-Token Era: What 1M Context Windows Change](https://leetllm.com/blog/million-token-context-windows): updated 2026-07-14; tags: Context Windows, Long Context, Benchmarks, Infrastructure, Deep Dive; Long-context windows help when relationships across a bounded corpus matter. Know what fits,... - [How to Build an AI Agent from Scratch](https://leetllm.com/blog/how-to-build-ai-agent-from-scratch): updated 2026-08-02; tags: Agents, Deep Dive, Tutorial; Build a working AI agent from the raw loop: define tools, let the model choose one, execute it in Python, append the observation, and... - [RAG vs Fine-Tuning vs Prompting](https://leetllm.com/blog/rag-vs-fine-tuning-vs-prompt-engineering): updated 2026-08-02; tags: Research, Deep Dive, Industry; Every LLM project starts with the same architecture question: improve the prompt, add retrieval, or fine-tune the model. Use a... - [What Does an AI Engineer Actually Do?](https://leetllm.com/blog/what-does-an-ai-engineer-do): updated 2026-08-08; tags: Industry, Deep Dive; AI engineering sits between foundation models and product engineering. The day-to-day work behind useful LLM systems is prompts, RAG, tools,... - [Understanding SWE-bench](https://leetllm.com/blog/swe-bench-deep-dive): updated 2026-08-02; tags: Benchmarks, Evaluation, SWE-bench, Agents, Deep Dive; SWE-bench measures whether coding agents can fix real repository issues: task format, scoring, benchmark... - [How to Prepare for ML & LLM Engineering Interviews in 2026](https://leetllm.com/blog/how-to-prepare-for-ml-llm-interviews-2026): updated 2026-08-02; tags: Career, Interview Prep, 2026; A practical guide to ML and LLM engineering interview prep in 2026, covering classical ML filters, LLM systems design, evaluation,... ## Practice Problems - [Stack Trace Reconstructor](https://leetllm.com/practice/stack-trace-reconstructor): Medium; 30 min; Parsing; languages: python3, java; Reconstruct exclusive function runtimes from nested start/end profiler events. - [Same-host Crawler](https://leetllm.com/practice/same-host-crawler): Medium; 30 min; Graph Traversal; languages: python3, java; Traverse a web graph in breadth-first order while enforcing same-host and duplicate-visit invariants. - [Repository Hash Tree](https://leetllm.com/practice/repository-hash-tree): Medium; 45 min; Repository Systems; languages: python3, java; Build a deterministic repository hash tree that updates parent directories after file writes and deletes. - [Repository Diff](https://leetllm.com/practice/repository-diff): Easy; 30 min; Repository Systems; languages: python3, java; Compare two repository snapshots and produce deterministic added, removed, and modified file changes. - [Duplicate File Groups](https://leetllm.com/practice/duplicate-file-groups): Easy; 25 min; Repository Systems; languages: python3, java; Group duplicate repository files by content with deterministic output and path validation. - [Repository Ignore Filter](https://leetllm.com/practice/repo-ignore-filter): Easy; 30 min; Repository Systems; languages: python3, java; Filter repository paths with simple ordered ignore and re-include rules. - [TTL Key/Value Store](https://leetllm.com/practice/ttl-key-value-store): Medium; 35 min; Stateful Design; languages: python3, java; Build a deterministic in-memory store where every read path treats expired keys as missing. - [Transactional KV Store](https://leetllm.com/practice/transactional-kv-store): Medium; 40 min; Stateful Design; languages: python3, java; Implement nested in-memory transactions with read-your-writes, tombstones, commit, and rollback. - [Dependency Scheduler With Retries](https://leetllm.com/practice/dependency-scheduler-retries): Hard; 45 min; Scheduling; languages: python3, java; Run a dependency graph deterministically, retry transient failures, and reject invalid graphs. - [Patch Conflict Detector](https://leetllm.com/practice/patch-conflict-detector): Medium; 40 min; Editing Systems; languages: python3, java; Apply optimistic multi-file patches atomically while rejecting stale or overlapping edits. - [Workflow State Machine](https://leetllm.com/practice/workflow-state-machine): Medium; 40 min; Scheduling; languages: python3, java; Track dependent jobs through legal workflow states with retry and cancellation behavior. - [Token Bucket Rate Limiter](https://leetllm.com/practice/token-bucket-rate-limiter): Medium; 35 min; Rate Limiting; languages: python3, java; Model per-key API quotas with deterministic refill math and caller-friendly retry-after decisions. - [Idempotent Ledger](https://leetllm.com/practice/idempotent-ledger): Medium; 30 min; Stateful Design; languages: python3, java; Apply account mutations exactly once while preserving retry-safe request semantics. - [Notification Rate Limiter](https://leetllm.com/practice/notification-rate-limiter): Medium; 35 min; Rate Limiting; languages: python3, java; Throttle noisy notification channels while deduping repeated webhook or automation events. - [LRU TTL Cache](https://leetllm.com/practice/lru-ttl-cache): Medium; 35 min; Caching; languages: python3, java; Combine recency ordering and TTL cleanup without corrupting cache invariants. - [LFU TTL Cache](https://leetllm.com/practice/lfu-ttl-cache): Hard; 45 min; Caching; languages: python3, java; Combine frequency buckets, recency tie-breaks, and TTL cleanup without corrupting eviction state. - [Log Error Parser](https://leetllm.com/practice/log-error-parser): Easy; 20 min; Parsing; languages: python3, java; Summarize top error messages from noisy logs with deterministic tie-breaking. - [In-memory Filesystem](https://leetllm.com/practice/in-memory-filesystem): Medium; 35 min; Stateful Design; languages: python3, java; Model directory and file state with predictable path normalization and listing behavior. - [File Patch Applier](https://leetllm.com/practice/file-patch-applier): Easy; 30 min; Editing Systems; languages: python3, java; Apply sorted or unsorted text edits atomically while rejecting invalid and overlapping ranges. - [Longest-match Tokenizer](https://leetllm.com/practice/longest-match-tokenizer): Medium; 35 min; Parsing; languages: python3, java; Tokenize text by greedily choosing the longest vocabulary token at each position. - [Multi-tenant Job Scheduler](https://leetllm.com/practice/multi-tenant-job-scheduler): Medium; 45 min; Scheduling; languages: python3, java; Dispatch queued jobs by priority while preventing one tenant from monopolizing workers. - [Sliding Window Error Counter](https://leetllm.com/practice/sliding-window-error-counter): Easy; 25 min; Observability; languages: python3, java; Track top recent errors while every read path lazily removes expired events. - [Bounded Worker Queue](https://leetllm.com/practice/bounded-worker-queue): Medium; 35 min; Scheduling; languages: python3, java; Model a bounded FIFO work queue with explicit in-progress ownership and retry behavior. Items are non-null. - [Worker Lease Registry](https://leetllm.com/practice/worker-lease-registry): Medium; 40 min; Scheduling; languages: python3, java; Coordinate worker ownership with TTL leases, heartbeat extension, stale-worker rejection, and fencing tokens. - [Priority Deadline Scheduler](https://leetllm.com/practice/priority-deadline-scheduler): Medium; 40 min; Scheduling; languages: python3, java; Schedule due tasks by priority while skipping not-yet-ready and expired work deterministically. - [Circuit Breaker State](https://leetllm.com/practice/circuit-breaker-state): Medium; 35 min; Reliability; languages: python3, java; Model closed, open, and half-open breaker transitions with deterministic cooldown behavior. - [Context Window Packer](https://leetllm.com/practice/context-window-packer): Medium; 30 min; Context Management; languages: python3, java; Select system and recent conversation messages without exceeding a token budget. - [RAG Chunk Selector](https://leetllm.com/practice/rag-chunk-selector): Medium; 30 min; Retrieval; languages: python3, java; Rank retrieval chunks while deduplicating repeated text and limiting per-document dominance. - [Agent Event Timeline](https://leetllm.com/practice/agent-event-timeline): Medium; 35 min; Observability; languages: python3, java; Build a reviewable append-only timeline for cloud-agent run events. - [Command Log Classifier](https://leetllm.com/practice/command-log-classifier): Easy; 25 min; Observability; languages: python3, java; Summarize command outcomes from an agent run while preserving safe review logs. - [Codebase Symbol Index](https://leetllm.com/practice/codebase-symbol-index): Medium; 40 min; Retrieval; languages: python3, java; Build an incremental code symbol index with file replacement, deletion, case-insensitive prefix search, and stable ranking. - [Eval Failure Aggregator](https://leetllm.com/practice/eval-failure-aggregator): Easy; 25 min; Evaluation; languages: python3, java; Aggregate eval run rows into stable pass-rate and error diagnostics. - [SSE Event Parser](https://leetllm.com/practice/sse-event-parser): Medium; 35 min; Parsing; languages: python3, java; Parse chunked server-sent events while handling split lines, comments, and multiline data payloads. - [Streaming Markdown Parser](https://leetllm.com/practice/streaming-markdown-parser): Medium; 35 min; Parsing; languages: python3, java; Parse streamed Markdown chunks into text, inline-code, and fenced-code tokens while preserving delimiter state. - [Retry Backoff Planner](https://leetllm.com/practice/retry-backoff-planner): Easy; 25 min; Reliability; languages: python3, java; Plan retry delays with capped exponential backoff and explicit retryable error classes. - [Tool Call Schema Validator](https://leetllm.com/practice/tool-call-schema-validator): Medium; 35 min; Agents; languages: python3, java; Validate tool call names and arguments before runtime execution. - [Webhook Idempotency Receiver](https://leetllm.com/practice/webhook-idempotency-receiver): Medium; 35 min; Agents; languages: python3, java; Handle retried provider events with delivery dedupe, logical idempotency keys, and repo policy checks. - [Model Fallback Router](https://leetllm.com/practice/model-fallback-router): Medium; 30 min; Serving; languages: python3, java; Route requests to the cheapest eligible healthy model with capability and quota checks. - [PR Readiness Gate](https://leetllm.com/practice/pr-readiness-gate): Easy; 25 min; Agents; languages: python3, java; Decide whether an agent-created pull request has enough checks, approval, and proof artifacts to merge. - [Token Budget Ledger](https://leetllm.com/practice/token-budget-ledger): Medium; 35 min; Rate Limiting; languages: python3, java; Maintain rolling-window token spend per caller with expiry on every read and write path. - [Batch Inference Scheduler](https://leetllm.com/practice/batch-inference-scheduler): Medium; 35 min; Serving; languages: python3, java; Batch requests by size or oldest-wait deadline while preserving FIFO order. - [Streaming Token Assembler](https://leetllm.com/practice/streaming-token-assembler): Easy; 25 min; Streaming; languages: python3, java; Assemble interleaved streaming token deltas into complete message text. - [Document Permission Filter](https://leetllm.com/practice/document-permission-filter): Medium; 30 min; Retrieval; languages: python3, java; Filter retrieval chunks by public, tenant, group, and private permissions before ranking. - [Experiment Traffic Splitter](https://leetllm.com/practice/experiment-traffic-splitter): Medium; 30 min; Evaluation; languages: python3, java; Assign stable experiment variants using weighted deterministic buckets. - [Prompt Section Extractor](https://leetllm.com/practice/prompt-section-extractor): Medium; 35 min; Parsing; languages: python3, java; Parse explicit prompt boundary markers while detecting outside text, duplicates, and unterminated sections. - [Secret Redactor](https://leetllm.com/practice/secret-redactor): Easy; 25 min; Security; languages: python3, java; Redact common emails, bearer tokens, and API-key-shaped strings from log text. - [Canary Metric Judge](https://leetllm.com/practice/canary-metric-judge): Medium; 35 min; Evaluation; languages: python3, java; Judge canary promotion with traffic, error-rate, latency, and quality thresholds. ## Optional - [Privacy Policy](https://leetllm.com/privacy): Privacy policy for user accounts, progress, analytics, and site data. - [Terms](https://leetllm.com/terms): Terms for using LeetLLM.