LeetLLM
My PlanLearnGlossaryTracksPracticeBlog
LeetLLM

Your go-to resource for mastering AI & LLM systems.

Product

  • Learn
  • Glossary
  • Tracks
  • Practice
  • Blog
  • RSS

Legal

  • Terms of Service
  • Privacy Policy

© 2026 LeetLLM. All rights reserved.

All Topics
Your Progress
0%

0 of 158 articles completed

🛠️Computing Foundations0/9
Git, Shell, Linux for AIDocker for Reproducible AIPython for AI EngineeringNumPy and Tensor ShapesCUDA for ML TrainingMPS & Metal for ML on MacData Structures for AISQL and Data ModelingAlgorithms for ML Engineers
📊Math & Statistics0/8
Gradients and BackpropVectors, Matrices & TensorsLinear Algebra for MLAdam, Momentum, SchedulersProbability for Machine LearningStatistics and UncertaintyDistributions and SamplingHypothesis Tests, Intervals, and pass@k
📚Preparation & Prerequisites0/13
Neural Networks from ScratchCNNs from ScratchTraining & BackpropagationSoftmax, Cross-Entropy & OptimizationRNNs, LSTMs, GRUs, and Sequence ModelingAutoencoders and VAEsThe Transformer Architecture End-to-EndLanguage Modeling & Next TokensFrom GPT to Modern LLMsPrompt Engineering FundamentalsCalling LLM APIs in ProductionFirst AI App End-to-EndThe LLM Lifecycle
🧮ML Algorithms & Evaluation0/11
Linear Regression from ScratchLogistic Regression and MetricsDecision Trees, Forests, and BoostingReinforcement Learning BasicsValidation and LeakageClustering and PCACore Retrieval AlgorithmsDecoding AlgorithmsExperiment Design and A/B TestingPyTorch Training LoopsDataset Pipelines and Data Quality
📦Production ML Systems0/6
Feature Engineering for Production MLBatch and Streaming Feature PipelinesGradient Boosted Trees in ProductionRanking and Recommendation SystemsForecasting and Anomaly DetectionMonitoring Predictive Models
🧪Core LLM Foundations0/8
The Bitter Lesson & ComputeBPE, WordPiece, and SentencePieceStatic to Contextual EmbeddingsPerplexity & Model EvaluationFile Ingestion for AIChunking StrategiesLLM Benchmarks & LimitationsInstruction Tuning & Chat Templates
🧰Applied LLM Engineering0/24
Dimensionality Reduction for EmbeddingsCoT, ToT & Self-Consistency PromptingFunction Calling & Tool UseMCP & Tool Protocol StandardsContext EngineeringPrompt Injection DefenseResponsible AI GovernanceData Labeling and Human FeedbackEvaluating AI AgentsProduction RAG PipelinesHybrid Search: Dense + SparseReranking and Cross-Encoders for RAGRAG Evaluation for Reliable AnswersLLM-as-a-Judge EvaluationBias & Fairness in LLMsHallucination Detection & MitigationLLM Observability & MonitoringExperiment Tracking with MLflow and W&BPrompt Optimization with DSPyModel Versioning & DeploymentSemantic Caching & Cost OptimizationLLM Cost Engineering & Token EconomicsModel Gateways, Routing, and FallbacksDesign an Automated Support Agent
🎓Portfolio Capstones0/8
Capstone: Delivery ETA PredictionCapstone: Product RankingCapstone: Demand ForecastingCapstone: Image Damage ClassifierCapstone: Production ML PipelineCapstone: Document QACapstone: Eval DashboardCapstone: Fine-Tuned Classifier
🧠Transformer Deep Dives0/8
Sentence Embeddings & Contrastive LossEmbedding Similarity & QuantizationScaled Dot-Product AttentionVision Transformers and Image EncodersPositional Encoding: RoPE & ALiBiLayer Normalization: Pre-LN vs Post-LNMechanistic InterpretabilityDecoding Strategies: Greedy to Nucleus
🧬Advanced Training & Adaptation0/15
Scaling Laws & Compute-Optimal TrainingPre-training Data at ScaleBuild GPT from Scratch LabContinued Pretraining for Domain ShiftSynthetic Data PipelinesSupervised Fine-Tuning PipelineMixed Precision TrainingDistributed Training: FSDP & ZeROLoRA & Parameter-Efficient TuningReward Modeling from Preference DataRLHF & DPO AlignmentConstitutional AI & Red TeamingRLVR & Verifiable RewardsKnowledge Distillation for LLMsModel Merging and Weight Interpolation
🤖Advanced Agents & Retrieval0/16
Vector DB Internals: HNSW & IVFAdvanced RAG: HyDE & Self-RAGGraphRAG & Knowledge GraphsRAG Security & Access ControlStructured Output GenerationReAct & Plan-and-ExecuteGuardrails & Safety FiltersCode Generation & SandboxingComputer-Use / GUI / Browser AgentsHuman-in-the-Loop Agent ArchitectureAI Coding Workflow with AgentsAgent Memory & PersistenceAgent Failure & RecoveryRecursive Language Models (RLM)Multi-Agent OrchestrationCapstone: Production Agent
⚡Inference & Production Scale0/19
Inference: TTFT, TPS & KV CacheMulti-Query & Grouped-Query AttentionKV Cache & PagedAttentionPrefix Caching and Prompt CachingFlashAttention & Memory EfficiencyContinuous Batching & SchedulingScaling LLM InferenceModel Parallelism for LLM InferenceModel Quantization: GPTQ, AWQ & GGUFLocal LLM DeploymentSLM Specialization & Edge DeploymentSpeculative DecodingLong Context Window ManagementMixture of Experts ArchitectureMamba & State Space ModelsReasoning & Test-Time ComputeAdvanced MLOps & DevOps for AIGPU Serving & AutoscalingA/B Testing for LLMs
🏗️System Design Capstones0/9
Content Moderation SystemCode Completion SystemMulti-Tenant LLM PlatformLLM-Powered Search EngineVision-Language Models & CLIPMultimodal LLM ArchitectureDiffusion Models: Images & TextReal-Time Voice AI AgentReasoning & Test-Time Compute
🎤AI Lab Interviewing0/4
AI Lab Coding Interview: Python SystemsAI Lab System Design InterviewAI Lab Behavioral InterviewAI Lab Technical Presentation
Back to Topics
LearnAI Lab InterviewingAI Lab Coding Interview: Python Systems
⚙️HardMLOps & Deployment

AI Lab Coding Interview: Python Systems

Practice production-shaped Python coding prompts: crawlers, in-memory stores, ledgers, schedulers, parsers, rate limiters, caches, and concurrency follow-ups.

28 min read
Learning path
Step 155 of 158 in the full curriculum
Reasoning & Test-Time ComputeAI Lab System Design Interview

After the system design capstones, the final section turns architecture knowledge into interview execution. The first test is whether you can build small, correct systems while requirements change.

AI lab coding formats vary by team. Public frontier-lab guidance emphasizes well-designed solutions, tests, live coding, debugging, standard-library fluency, experience, motivation, communication, and tradeoff analysis.[1]Reference 1Interview guidehttps://openai.com/interview-guide/[2]Reference 2Careershttps://www.anthropic.com/careers Practice that bar with one base prompt followed by staged requirements: add TTLs, add concurrency, add cancellation, add rate limits, preserve deterministic output, or explain why your state model won't corrupt itself.

The final interview-prep section ends with a coding session. Build speed at practical Python: clear state, small APIs, local tests, and honest concurrency invariants.

Python systems coding interview loop with clarify, implement, test, extend, add concurrency, and explain invariants Python systems coding interview loop with clarify, implement, test, extend, add concurrency, and explain invariants
AI lab coding rounds reward staged implementation: ship a correct base version, test it, then extend the same small state model without losing invariants.

Four-round prep map

Treat the final prep section as one packet. Each round tests a different artifact, but the artifacts should agree with each other.

RoundPattern familiesProof artifact
Codingtraversal, stores, TTL, ordering, parsing, concurrency, validationpassing tests plus an invariant you can explain under follow-up
System designmodel gateway, retrieval, scheduler, coding agent, eval rolloutAPI, data model, scale math, overload plan, permission boundary
Behavioralmotivation, risk judgment, disagreement, ambiguity, failure, growthfive story bank with metrics, artifacts, and honest reflections
Technical presentationarchitecture narrative, tradeoffs, metrics, AI bridge, Q&A defense15-minute talk, 5-minute version, appendix, ownership boundary

Use the map to avoid uneven prep. A strong candidate doesn't only solve Python prompts; they can also explain why a design boundary exists, what failure changed their judgment, and which project artifact proves depth.

The operating model

Use this loop for every prompt:

  1. Restate input, output, and failure behavior.
  2. Ship version 1 with the smallest correct state model.
  3. Add table-driven tests before adding stage 2.
  4. Isolate shared mutable state before adding threads.
  5. End by naming complexity, race risks, and production hardening.

Good interview code isn't the most abstract code. It's code whose invariants can be defended while requirements change.

Python tools to know cold

Know these without documentation:

NeedPython building block
FIFO work queuecollections.deque, queue.Queue, asyncio.Queue
Counts and top errorscollections.Counter
LRU cachecollections.OrderedDict
Deadlines and TTLtime.monotonic, injected now function
Priority schedulingheapq, queue.PriorityQueue
Thread safetythreading.Lock, threading.RLock, threading.Condition, threading.Event
Worker fanoutconcurrent.futures.ThreadPoolExecutor, as_completed
Parsingsplitlines, re, explicit state machines

Don't wait for a test framework. Write a small run_tests() and use plain assert.

Prompt bank

Use these as drills. Don't memorize wording. Learn the patterns.

Prompt patternBase implementationFollow-ups
Same-host web crawlerBFS/DFS with visited setconcurrency, per-host rate limit, timeouts, cancellation
In-memory key/value DBset, get, delete, scanTTL, compare-and-set, transactions, snapshots
Banking ledgeraccounts, deposit, withdraw, transferidempotency, reversals, scheduled transfers, deadlock avoidance
Task schedulerdependency graph and ready queuecycle detection, retries, worker pool, cancellation
Log parsermultiline event groupingmalformed lines, rolling windows, top errors
Rate limiterfixed or sliding windowtoken bucket, multi-dimensional quotas, retry-after
LRU/TTL cachecapacity evictionTTL, thread safety, metrics, stale cleanup
LFU/cache policyfrequency buckets and recency tie-breakTTL, capacity 0, update semantics, thread safety
Stream assemblerchunks, sequence IDs, end markersout-of-order chunks, duplicate chunks, timeout, memory cap
Schema validatornested dict/list validationdefaults, unknown fields, coercion, error paths
Batch schedulerqueue by arrival and priorityfairness, deadlines, retries, starvation
Secret redactortoken scanning and replacementoverlapping matches, allowlists, multiline logs

Frontier live-practice set

Use this as the main circuit. Each linked drill carries a full prompt, clarification questions, sample tests, hidden tests, solution guide, and validated Python plus Java paths. Run each problem twice: first in Python for speed, then in Java to force explicit classes, maps, and boundary checks.

ArenaPractice drillsInterview signal
Repository and codebase systemsRepository Hash Tree, Repository Diff, Duplicate File Groups, Repository Ignore Filter, File Patch Applier, Patch Conflict Detector, Codebase Symbol Index, In-memory Filesystem, Stack Trace ReconstructorCode review agents, workspace snapshots, deterministic patches, symbol lookup, upload filtering, incremental state
Stateful servicesTTL Key/Value Store, Transactional KV Store, Idempotent Ledger, LRU TTL Cache, LFU TTL Cache, Token Budget LedgerDurable APIs, atomic updates, idempotency, eviction policy, TTL boundaries
Scheduling and workersDependency Scheduler With Retries, Workflow State Machine, Multi-tenant Job Scheduler, Bounded Worker Queue, Worker Lease Registry, Priority Deadline Scheduler, Batch Inference SchedulerRun orchestration, backpressure, retries, leases, fairness, cancellation follow-ups
Streaming and parsersStreaming Markdown Parser, SSE Event Parser, Streaming Token Assembler, Log Error Parser, Longest-match Tokenizer, Prompt Section Extractor, Secret RedactorChunked state, final flush, malformed records, overlapping matches, transcript safety
Agents and webhooksTool Call Schema Validator, Webhook Idempotency Receiver, PR Readiness Gate, Model Fallback Router, Retry Backoff Planner, Circuit Breaker StateTool execution, dedupe, provider fallback, proof artifacts, retry policy, outage behavior
Retrieval and contextRAG Chunk Selector, Context Window Packer, Document Permission FilterRanking, budget packing, access control, explainable inclusion
Evals and telemetryEval Failure Aggregator, Canary Metric Judge, Experiment Traffic Splitter, Agent Event Timeline, Command Log Classifier, Sliding Window Error CounterRollout judgment, failure clustering, reviewable proof, redacted logs, stable assignment, moving windows
Systems basics under frontier wordingSame-host Crawler, Token Bucket Rate Limiter, Notification Rate LimiterGraph traversal, duplicate suppression, host policy, quota math, noisy automation control

Live drill contract

For every practice run, write these artifacts before opening the guide:

  1. Problem proposition in your own words: API, input shape, output shape, failure behavior.
  2. Three clarification questions that can change tests.
  3. Invariant sentence: state owner, allowed transition, and cheap lookup.
  4. Public tests: happy path, duplicate input, missing input, boundary.
  5. Hidden tests: malformed input, ordering tie-break, expiry equality, retry exhaustion, or conflict.
  6. Python implementation with plain assert tests.
  7. Java implementation with explicit classes and deterministic collections.
  8. Follow-up answer: concurrency, persistence, scale, observability, or rollback.

That routine makes LeetLLM practice closer to a real frontier coding loop: define semantics, ship correct code, prove boundaries, then extend without rewriting the state model.

Question pattern taxonomy

Most prompts are one of these shapes. Classify the prompt before writing code.

PatternRecognition signalFirst state modelHigh-value tests
TraversalURLs, graph nodes, dependencies, neighborsvisited plus queue or stackcycle, duplicate edge, malformed node, deterministic order
Mutable storeset, get, delete, scan, inventory operationsdict from key to recordmissing key, overwrite, delete, scan ordering, expired key
Time boundaryTTL, deadline, rate limit, retry-afterinjected now, deadline fieldsequality boundary, refill edge, expired-on-read, no sleep
Ordering policypriority, deadline, LRU, LFU, ready queueheap, deque, OrderedDict, frequency bucketstie-break, stale heap entry, capacity 0, promotion
Idempotent writerequest ID, retry, ledger, external actionrequest key to stored resultduplicate success, duplicate failure, conflicting retry
Parserlogs, events, chunks, sections, streaming linesexplicit current record or buffermalformed line, continuation, empty input, final flush
Concurrencyworkers, thread-safe, fanout, cancellationlock-protected claim point plus queueduplicate claim, shutdown, exception, partial result
Validationschema, payload, permissions, filtersrecursive validator or rule tablenested failure, path reporting, unknown field, missing required

Use this sentence when stuck:

The invariant is X; this data structure makes X cheap to maintain; these tests prove the boundary cases.

Answer rhythm

Use the same rhythm for every timed solution:

  1. Clarify semantics that change tests: ordering, equality boundary, failure behavior, and concurrency expectation.
  2. Name invariant and data structures before code.
  3. Implement smallest single-threaded version.
  4. Add sample tests while code is still small.
  5. Add follow-up by changing one boundary: time, ordering, retries, or locking.
  6. Explain complexity and the first production hardening step.

For a frontier-lab coding round, communication is part of correctness. Say when you're making an assumption, say which invariant you're protecting, and say what test you'll write before you write it.

Pattern drills by week

SessionDrill setGoal
1crawler, filesystem, tokenizertraversal and parsing without losing order
2TTL store, token bucket, retry plannerdeterministic time and boundary tests
3LRU, LFU, priority schedulereviction, tie-breaks, and stale heap entries
4ledger, workflow state machine, schema validatoridempotency, transitions, and explainable errors
5worker queue, batch scheduler, stream assemblerconcurrency, backpressure, and partial results
6two random problems under 35 minutes eachspeed, tests, and clean explanation

Levelled prompt ladder

Many frontier coding rounds feel like one product-shaped problem that grows across levels. Practice each ladder by shipping level 1 quickly, then preserving the same invariant as requirements change.

Prompt familyLevel 1Level 2Level 3Level 4
GPU quota reservation systemadd, remove, lookupreserve/release capacityexpirations, over-allocation preventionconcurrency or audit log
Key/value storeset, get, deleteprefix scan and orderingTTL and compare-and-settransactions or snapshots
Crawlersame-host BFSURL normalizationretries and rate limitsworker fanout and cancellation
Schedulerdependencies and ready queuecycle detectionretries and deadlinesworkers, cancellation, fairness
Cacheget/put capacity evictionupdate semanticsTTL or LFUmetrics and thread safety
Chat or event routerregister handlersroute messagespriorities or filtersreplay, idempotency, backpressure
Log processorparse recordsmultiline eventstop errors and windowsmalformed input and streaming
Permission filterinclude/exclude resourcesgroups and inheritancedeny precedenceaudit why each item passed
Stream assemblerappend chunkssequence orderingduplicate/missing chunkstimeout and memory cap
Experiment splitterassign usersstable hashingramp percentagessticky overrides and rollback

Use this timing target:

TimeTarget
0-5 minclarify ordering, failure behavior, and mutable state
5-20 minlevel 1 complete with tests
20-32 minlevel 2 or 3 complete without rewriting
32-38 minedge tests and complexity
final minutesname next follow-up design, race risk, and production hardening

Coding readiness rubric

Score yourself after each drill.

SignalNot readyReadyStrong
Clarificationstarts coding before semanticsasks about order, missing inputs, equality boundarypredicts which answer changes tests
Data modelgrows incidental statenames records and invariantscan extend same model across levels
Testsonly happy pathmissing, duplicate, boundary, and sample teststests one future follow-up before code
Standard libraryreinvents queues, counters, heaps, cachesuses obvious standard toolexplains why that tool fits invariant
Debuggingstares at failurenarrows with print/assert/reprostates expected vs actual before editing
Follow-up handlingrewrites from scratchchanges one boundary at a timenames tradeoff and reversal signal
Communicationquiet implementationnarrates assumptions and complexitykeeps interviewer aligned through decisions

When a drill scores "not ready," repeat the same prompt family with different nouns. Train transfer, not memorization.

Pattern testcase checklist

State the tests before coding the follow-up. That keeps the interviewer aligned and prevents late rewrites.

PatternMinimum public testsPrivate-edge tests to rehearse
Crawler or graph traversalstart node, duplicate edge, off-host or invalid neighborcycles, relative links, malformed URL, empty graph, deterministic order
TTL store or rate limiterset/get, expiry, scan, refillequality boundary, zero TTL, time moving backward, cleanup during read
LRU or LFU cachecapacity eviction, update existing key, get promotioncapacity 0, tie-break by recency, expired entry, stale heap record
Schedulerindependent tasks, dependency release, retrycycle, missing dependency, permanent failure, blocked dependents
Ledgerdeposit, withdraw, transferduplicate idempotency key, conflicting retry, insufficient funds, lock order
Parserone record, multiline continuation, malformed lineorphan continuation, empty input, final flush, very large record
Validatorrequired field, unknown field, nested listerror path, default value, type coercion, repeated failure
Stream assemblerin-order chunks, out-of-order chunksduplicate chunk, missing end marker, timeout, memory cap

Use this sentence before running tests:

I expect these tests to fail if I lose the invariant. The invariant is X; the edge case most likely to break it: Y.

Debugging protocol

When a test fails, narrate the smallest useful investigation:

  1. Read the assertion and say expected versus actual.
  2. Reproduce with one smaller fixture.
  3. Print or inspect the state that owns the invariant.
  4. Fix the state transition, not the symptom.
  5. Add one regression test before moving to the next follow-up.

Common debug pivots:

SymptomFirst thing to inspect
Duplicate outputclaim point or visited insertion timing
Missing outputenqueue condition, prefix filter, or failure policy
Wrong orderqueue/heap tie-break and where sorting happens
Expired key returnedread path skipped cleanup
Retry ran too many timesattempt counter increment point
Blocked task marked faileddependency failure policy mixed with task execution failure
Threaded result flakesshared state changed outside the lock

Strong candidates aren't bug-free. They recover fast because they can name which invariant is broken.

Live coding narration bank

Practice short narration so you don't go silent while thinking. Use these sentences as scaffolding, then replace the placeholders with prompt-specific details.

MomentWhat to say
Before coding"I will first ship the single-threaded version, then add follow-up without changing the core invariant."
Choosing data structures"The cheap operation needs to be operation, so I will store state as structure."
Before tests"I want tests for happy path, duplicate input, boundary condition, and malformed input."
On failure"Expected X, got Y. The likely owner is state transition, so I will inspect that before editing."
Adding TTL or deadlines"I will inject time so expiry tests don't sleep and the equality boundary is explicit."
Adding concurrency"The shared state is X; the claim point is Y; work runs outside the lock."
Running out of time"The base invariant is correct. The next production hardening step is race/cleanup/backpressure."

Follow-up triage map

When the interviewer adds a follow-up, classify it before changing code.

Follow-up typeFirst moveCommon trap
Timeinject now, store deadline, clean on readsleeping in tests
Orderingchoose deque, heap, sort, or insertion ordermixing policy into unrelated state
Capacityevict by explicit rulehidden off-by-one at capacity 0 or 1
Retrystore attempt count and final stateretrying external writes without idempotency
Snapshotversion records or copy-on-writemutating data a snapshot should freeze
Compare-and-setlock compare and write togethercalling get then set as two operations
Thread safetyprotect claim point and shared mapsholding lock during slow fetch or execution
Cancellationdurable flag plus cooperative checksstopping new work while workers keep publishing children

Good follow-up handling sounds calm because the state boundary is still visible.

Mock coding prompts

Use these as timed practice. Read the prompt, write a small solution and tests, then open the solution guide.

Prompt 1: same-host crawler

You're given a starting URL and a function get_links(url) -> list[str]. Implement a crawler that visits only URLs on the same host as the start URL.

Requirements:

  • Return visited URLs in deterministic breadth-first order for the single-threaded version.
  • Don't visit the same URL twice.
  • Ignore malformed URLs and off-host URLs.
  • Follow-up: add worker fanout without corrupting visited.
  • Follow-up: add a per-host rate limit and cancellation event.

Clarifying questions to ask:

  • Should URLs be normalized for fragments, trailing slashes, query strings, and redirects?
  • Should failed fetches be retried, skipped, or returned separately?
  • Does deterministic order still matter after worker fanout, or only for the single-threaded version?
Solution guide

Use a FIFO work queue for the base version because the prompt asks for breadth-first order. Put URL normalization and host checks before enqueueing. The invariant is: a URL enters the queue only after it has been accepted into visited.

same-host-crawler.py
1from collections import deque 2from urllib.parse import urlparse, urljoin 3 4PAGES = { 5 "https://repo.example/start": ["/tests", "/ci", "https://other.example/x"], 6 "https://repo.example/tests": ["/ci"], 7 "https://repo.example/ci": [], 8} 9 10def crawl(start_url: str, get_links) -> list[str]: 11 start = urlparse(start_url) 12 if not start.scheme or not start.netloc: 13 return [] 14 15 visited = {start_url} 16 queue = deque([start_url]) 17 ordered: list[str] = [] 18 19 while queue: 20 url = queue.popleft() 21 ordered.append(url) 22 23 for raw_link in get_links(url): 24 candidate = urljoin(url, raw_link) 25 parsed = urlparse(candidate) 26 if parsed.scheme not in {"http", "https"}: 27 continue 28 if parsed.netloc != start.netloc: 29 continue 30 if candidate in visited: 31 continue 32 visited.add(candidate) 33 queue.append(candidate) 34 35 return ordered 36 37print(crawl("https://repo.example/start", lambda url: PAGES.get(url, [])))
Output
1['https://repo.example/start', 'https://repo.example/tests', 'https://repo.example/ci']

Follow-up guide

For worker fanout, say explicitly that deterministic return order becomes expensive unless the prompt still requires it. Protect the claim step, not the whole fetch:

crawler-claim-step.py
1from threading import Lock 2 3visited = {"https://repo.example/start"} 4visited_lock = Lock() 5 6def claim(candidate: str) -> bool: 7 with visited_lock: 8 if candidate in visited: 9 return False 10 visited.add(candidate) 11 return True 12 13print(claim("https://repo.example/tests")) 14print(claim("https://repo.example/tests")) 15print(sorted(visited))
Output
1True 2False 3['https://repo.example/start', 'https://repo.example/tests']

For rate limiting, add a per-host token bucket or next-allowed timestamp before fetch. For cancellation, check a threading.Event before scheduling new work, before fetching, and after each fetch before enqueueing children. The full answer should name all three behaviors: duplicate prevention, bounded fetch concurrency, and graceful stop.

Prompt 2: TTL key/value store

Implement an in-memory store with set(key, value, ttl=None), get(key), delete(key), and scan(prefix).

Requirements:

  • ttl is measured in seconds.
  • Expired keys behave as missing.
  • Tests must not call sleep.
  • Follow-up: add compare-and-set.
  • Follow-up: add thread safety.

Clarifying questions to ask:

  • Should scan(prefix) return keys, values, or key/value pairs?
  • Is a key expired when expires_at == now, or only when expires_at < now?
  • Should compare-and-set treat an expired key as missing?
Solution guide

Inject now so tests can advance time directly. Store expires_at beside each value. The invariant is: every public read path either returns a non-expired value or removes the expired key.

ttl-store.py
1from dataclasses import dataclass 2from typing import Callable 3 4@dataclass 5class Entry: 6 value: str 7 expires_at: float | None 8 9class Store: 10 def __init__(self, now: Callable[[], float]) -> None: 11 self.now = now 12 self.items: dict[str, Entry] = {} 13 14 def set(self, key: str, value: str, ttl: float | None = None) -> None: 15 expires_at = None if ttl is None else self.now() + ttl 16 self.items[key] = Entry(value, expires_at) 17 18 def get(self, key: str) -> str | None: 19 entry = self.items.get(key) 20 if entry is None: 21 return None 22 if entry.expires_at is not None and entry.expires_at <= self.now(): 23 self.items.pop(key, None) 24 return None 25 return entry.value 26 27 def delete(self, key: str) -> None: 28 self.items.pop(key, None) 29 30 def scan(self, prefix: str) -> list[str]: 31 return sorted(key for key in list(self.items) if key.startswith(prefix) and self.get(key) is not None) 32 33clock = {"now": 10.0} 34store = Store(lambda: clock["now"]) 35store.set("task:1", "running", ttl=5.0) 36store.set("task:2", "ready") 37print(store.get("task:1"), store.scan("task:")) 38clock["now"] = 15.0 39print(store.get("task:1"), store.scan("task:"))
Output
1running ['task:1', 'task:2'] 2None ['task:2']

Follow-up guide

For compare-and-set, clean any expired value and compare while holding the same lock. The compact example below omits TTL so the atomic boundary is easy to see:

compare-and-set.py
1from threading import RLock 2 3class AtomicStore: 4 def __init__(self) -> None: 5 self.items: dict[str, str] = {} 6 self.lock = RLock() 7 8 def compare_and_set(self, key: str, expected: str | None, value: str) -> bool: 9 with self.lock: 10 if self.items.get(key) != expected: 11 return False 12 self.items[key] = value 13 return True 14 15store = AtomicStore() 16print(store.compare_and_set("task:1", None, "running")) 17print(store.compare_and_set("task:1", None, "ready")) 18print(store.compare_and_set("task:1", "running", "ready"))
Output
1True 2False 3True

For the TTL store, wrap every public method with the same lock. If compare_and_set() calls get() and set() while holding that lock, use an RLock or split the internal helpers so the lock is acquired once. The important answer is atomicity: no other thread can change the key between compare and set.

Prompt 3: task scheduler with retries

Implement a scheduler for tasks with dependencies. A task becomes runnable when all dependencies have completed.

Requirements:

  • Detect dependency cycles before running.
  • Run ready tasks in priority order.
  • Retry failed tasks up to max_attempts.
  • Return completed tasks, permanently failed tasks, and blocked dependents separately.
  • Follow-up: add worker fanout.

Clarifying questions to ask:

  • Does a lower priority number run first, or does a higher number run first?
  • If a dependency permanently fails, should dependents be marked failed, skipped, or blocked?
  • Are retries immediate, delayed, or scheduled with backoff?
Solution guide

Represent the graph explicitly: dependents[task] points to tasks released by this task, and remaining[task] counts unmet dependencies. Use heapq for priority. The invariant is: a task enters the heap only when remaining[task] == 0.

dependency-scheduler.py
1import heapq 2from collections import defaultdict 3from dataclasses import dataclass 4 5@dataclass(frozen=True) 6class Task: 7 name: str 8 priority: int 9 deps: tuple[str, ...] = () 10 11def schedule(tasks: list[Task], run, max_attempts: int = 2) -> tuple[list[str], list[str], list[str]]: 12 if max_attempts <= 0: 13 raise ValueError("max_attempts must be positive") 14 by_name = {task.name: task for task in tasks} 15 if len(by_name) != len(tasks): 16 raise ValueError("duplicate task name") 17 dependents: dict[str, list[str]] = defaultdict(list) 18 remaining = {task.name: len(task.deps) for task in tasks} 19 20 for task in tasks: 21 for dep in task.deps: 22 if dep not in by_name: 23 raise ValueError(f"unknown dependency: {dep}") 24 dependents[dep].append(task.name) 25 26 state: dict[str, int] = {} 27 28 def visit(name: str) -> None: 29 marker = state.get(name, 0) 30 if marker == 1: 31 raise ValueError("cycle") 32 if marker == 2: 33 return 34 state[name] = 1 35 for dep in by_name[name].deps: 36 visit(dep) 37 state[name] = 2 38 39 for name in by_name: 40 visit(name) 41 42 ready = [(task.priority, task.name) for task in tasks if remaining[task.name] == 0] 43 heapq.heapify(ready) 44 attempts = defaultdict(int) 45 completed: list[str] = [] 46 failed: list[str] = [] 47 48 while ready: 49 _, name = heapq.heappop(ready) 50 attempts[name] += 1 51 if not run(name): 52 if attempts[name] < max_attempts: 53 heapq.heappush(ready, (by_name[name].priority, name)) 54 else: 55 failed.append(name) 56 continue 57 58 completed.append(name) 59 for child in dependents[name]: 60 remaining[child] -= 1 61 if remaining[child] == 0: 62 heapq.heappush(ready, (by_name[child].priority, child)) 63 64 blocked = sorted(set(by_name) - set(completed) - set(failed)) 65 return completed, failed, blocked 66 67tasks = [ 68 Task("pack", priority=2), 69 Task("ship", priority=3, deps=("pack",)), 70 Task("label", priority=1), 71] 72fail_once = {"pack"} 73 74def run_with_one_retry(name: str) -> bool: 75 if name in fail_once: 76 fail_once.remove(name) 77 return False 78 return True 79 80print(schedule(tasks, run_with_one_retry)) 81print(schedule([Task("pack", 1), Task("ship", 2, ("pack",))], lambda _: False, max_attempts=1))
Output
1(['label', 'pack', 'ship'], [], []) 2([], ['pack'], ['ship'])

Follow-up guide

Keep permanently failed tasks separate from blocked dependents. A blocked task never ran; callers may skip it, surface the failed prerequisite, or retry after repair.

For fanout, keep graph construction and cycle detection single-threaded. Then protect only shared scheduler state: ready heap, attempts, completed, failed, and remaining dependency counts. Worker threads can run tasks outside the lock, then reacquire the lock to publish success/failure and release dependents.

If asked about retries, say whether retries preserve priority or use backoff. A concise full answer: "Ready tasks are claimed under a condition variable, execution happens outside the lock, and completion updates notify workers when new tasks become ready."

Drill 1: token bucket with deterministic time

Rate limiters show up because they combine state, boundary conditions, and production behavior. Retry policies often need jitter to avoid synchronized retry storms, so this prompt is really about overload control as much as counters.[3]Reference 3Exponential Backoff And Jitterhttps://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/ A strong implementation injects time so tests don't sleep.

token-bucket.py
1import math 2from dataclasses import dataclass 3 4@dataclass 5class Bucket: 6 capacity: float 7 refill_per_second: float 8 tokens: float 9 updated_at: float 10 11class TokenBucketLimiter: 12 def __init__(self, capacity: int, refill_per_second: float) -> None: 13 if capacity <= 0: 14 raise ValueError("capacity must be positive") 15 if not math.isfinite(refill_per_second) or refill_per_second <= 0: 16 raise ValueError("refill_per_second must be positive and finite") 17 self.capacity = float(capacity) 18 self.refill_per_second = float(refill_per_second) 19 self._buckets: dict[str, Bucket] = {} 20 21 def allow(self, key: str, now: float, cost: float = 1.0) -> tuple[bool, float]: 22 if not math.isfinite(now): 23 raise ValueError("now must be finite") 24 if not math.isfinite(cost) or not 0 < cost <= self.capacity: 25 raise ValueError("cost must be greater than zero and no larger than capacity") 26 bucket = self._buckets.get(key) 27 if bucket is None: 28 bucket = Bucket(self.capacity, self.refill_per_second, self.capacity, now) 29 self._buckets[key] = bucket 30 if now < bucket.updated_at: 31 raise ValueError("now must not move backwards") 32 33 elapsed = max(0.0, now - bucket.updated_at) 34 bucket.tokens = min(bucket.capacity, bucket.tokens + elapsed * bucket.refill_per_second) 35 bucket.updated_at = now 36 37 if bucket.tokens >= cost: 38 bucket.tokens -= cost 39 return True, 0.0 40 41 missing = cost - bucket.tokens 42 retry_after = missing / bucket.refill_per_second 43 return False, retry_after 44 45limiter = TokenBucketLimiter(capacity=3, refill_per_second=1.0) 46print([limiter.allow("org-a", now=0.0)[0] for _ in range(4)]) 47print(limiter.allow("org-a", now=0.5)) 48print(limiter.allow("org-a", now=1.0)) 49try: 50 limiter.allow("org-a", now=2.0, cost=4.0) 51except ValueError as error: 52 print(error)
Output
1[True, True, True, False] 2(False, 0.5) 3(True, 0.0) 4cost must be greater than zero and no larger than capacity

What to say out loud:

  • The key can be a user, organization, endpoint, or model.
  • now is injected for deterministic tests.
  • Impossible costs fail immediately instead of returning retry advice that can never succeed.
  • Cleanup for idle buckets is a production memory concern, not a correctness requirement for the base prompt.
  • A thread-safe version needs a lock around _buckets and bucket mutation.

Drill 2: thread-safe same-host crawler shape

A crawler prompt tests graph traversal plus concurrency. The invariant is simple: each URL is claimed once before it's fetched or enqueued.

single-process-crawler-core.py
1from collections import deque 2from urllib.parse import urlparse, urljoin 3 4PAGES = { 5 "https://lab.example/start": ["/a", "/b", "https://other.example/x"], 6 "https://lab.example/a": ["/b", "/c"], 7 "https://lab.example/b": ["/c"], 8 "https://lab.example/c": [], 9} 10 11def get_urls(url: str) -> list[str]: 12 return PAGES.get(url, []) 13 14def same_host_crawl(start_url: str) -> list[str]: 15 start_host = urlparse(start_url).netloc 16 queue = deque([start_url]) 17 visited = {start_url} 18 ordered: list[str] = [] 19 20 while queue: 21 url = queue.popleft() 22 ordered.append(url) 23 for raw_link in get_urls(url): 24 link = urljoin(url, raw_link) 25 if urlparse(link).netloc != start_host: 26 continue 27 if link in visited: 28 continue 29 visited.add(link) 30 queue.append(link) 31 32 return ordered 33 34print(same_host_crawl("https://lab.example/start"))
Output
1['https://lab.example/start', 'https://lab.example/a', 'https://lab.example/b', 'https://lab.example/c']

Concurrency follow-up:

  • Protect visited with a lock.
  • Claim a URL while holding the lock, before scheduling a worker.
  • Keep output nondeterministic unless the prompt explicitly asks for deterministic order.
  • Add timeouts and failed-fetch handling without retry storms.

Drill 3: ledger with idempotency

Ledger prompts test whether you can keep money-like state consistent. Use append-only events when possible; if you maintain balances, update balance and event together.

ledger-idempotency.py
1from dataclasses import dataclass 2 3@dataclass(frozen=True) 4class Event: 5 idempotency_key: str 6 account: str 7 delta: int 8 balance_after: int 9 10class Ledger: 11 def __init__(self) -> None: 12 self.balance: dict[str, int] = {} 13 self.events: list[Event] = [] 14 self.results: dict[str, Event] = {} 15 16 def apply(self, key: str, account: str, delta: int) -> int: 17 if key in self.results: 18 prior = self.results[key] 19 if (prior.account, prior.delta) != (account, delta): 20 raise ValueError("idempotency key reused with different operation") 21 return prior.balance_after 22 new_balance = self.balance.get(account, 0) + delta 23 if new_balance < 0: 24 raise ValueError("insufficient funds") 25 self.balance[account] = new_balance 26 event = Event(key, account, delta, new_balance) 27 self.events.append(event) 28 self.results[key] = event 29 return new_balance 30 31ledger = Ledger() 32print(ledger.apply("deposit-1", "acct", 100)) 33print(ledger.apply("withdraw-1", "acct", -30)) 34print(ledger.apply("deposit-1", "acct", 100)) 35print(ledger.balance["acct"], len(ledger.events)) 36try: 37 ledger.apply("deposit-1", "acct", 200) 38except ValueError as error: 39 print(error)
Output
1100 270 3100 470 2 5idempotency key reused with different operation

A replay with the same key and operation returns the stored result, even if the account changed later. Reusing a key for a different operation fails loudly instead of silently returning an unrelated balance.

Transfer follow-up:

  • Use one idempotency key for the whole transfer.
  • Lock account IDs in sorted order to avoid deadlock.
  • Record both debit and credit events together.
  • Define whether external side effects happen before or after durable commit.

A reusable lock-order helper makes deadlock prevention concrete:

ordered-account-locks.py
1from contextlib import ExitStack 2from threading import Lock 3 4locks = {"acct-a": Lock(), "acct-b": Lock()} 5 6def lock_accounts(*account_ids: str) -> ExitStack: 7 stack = ExitStack() 8 for account_id in sorted(set(account_ids)): 9 stack.enter_context(locks[account_id]) 10 return stack 11 12with lock_accounts("acct-b", "acct-a"): 13 print("locked:", sorted({"acct-b", "acct-a"}))
Output
1locked: ['acct-a', 'acct-b']

Drill 4: multiline log parser

Log parsers test string handling and explicit malformed-input policy. Group indented continuation lines under the previous valid event. Ignore orphan continuations and malformed records in this base version; a production parser should count or collect rejects.

multiline-log-parser.py
1import json 2 3def parse_events(text: str) -> list[dict[str, str]]: 4 events: list[dict[str, str]] = [] 5 current: dict[str, str] | None = None 6 7 for line in text.splitlines(): 8 if line.startswith(" "): 9 if current is not None: 10 current["message"] += f" {line.strip()}" 11 continue 12 13 parts = line.split("|", maxsplit=2) 14 if len(parts) != 3: 15 current = None 16 continue 17 18 timestamp, level, message = parts 19 current = {"timestamp": timestamp, "level": level, "message": message} 20 events.append(current) 21 22 return events 23 24LOGS = """ orphan continuation 252026-06-02T12:00:00Z|ERROR|request failed 26 timeout while calling model 27malformed line 282026-06-02T12:01:00Z|INFO|retry queued""" 29 30print(json.dumps(parse_events(LOGS), indent=2))
Output
1[ 2 { 3 "timestamp": "2026-06-02T12:00:00Z", 4 "level": "ERROR", 5 "message": "request failed timeout while calling model" 6 }, 7 { 8 "timestamp": "2026-06-02T12:01:00Z", 9 "level": "INFO", 10 "message": "retry queued" 11 } 12]

Concurrency answer template

When asked to make a solution concurrent, say this before writing code:

  1. Shared state is X.
  2. The lock protects X.
  3. A work item is claimed at this point.
  4. Worker shutdown happens through this sentinel, event, or executor lifecycle.
  5. Failed work records an error and doesn't corrupt shared state.

This sounds mechanical because it should. Concurrency interview failures usually come from vague ownership.

Common pitfalls

  • Solving version 1, then adding TTL, transactions, or threads without restating the invariant.
  • Using wall-clock sleeps in tests instead of injected time.
  • Making crawler output order part of correctness after adding concurrent workers.
  • Treating idempotency as "retry the operation" instead of recording the request key and result semantics.
  • Adding a global lock around everything without explaining throughput, deadlock, and fairness tradeoffs.

Mastery checklist

  • Implement a rate limiter with deterministic time and boundary tests.
  • Implement a same-host crawler and explain where to place the visited lock.
  • Implement a ledger with idempotency and insufficient-funds behavior.
  • Explain how to add TTLs to a key/value store without sleeping in tests.
  • Build a dependency scheduler with cycle detection and a ready queue.
  • Parse multiline logs with malformed-line handling.
  • State complexity and memory growth for each solution.
Complete the lesson

Mastery Check

Answer every question, then check your score. Score 75% or higher to mark this lesson complete.

1.A key/value store gains TTL and compare-and-set. get removes expired entries, but scan reads the dictionary directly, and compare-and-set calls get outside the write lock before set. Which redesign preserves both invariants?
2.A TTL store treats a key as expired when expires_at <= now. You want to test a key set at time 10.0 with ttl=5.0. Which setup checks the boundary without making the interview test flaky?
3.A same-host crawler currently uses a visited set and a FIFO queue. You add worker fanout, and get_links(url) may be slow. Which locking plan preserves the crawler invariant without unnecessarily serializing fetches?
4.A token bucket has capacity 3 and refills at 1 token per second. A key starts full at time 0. Four cost-1 requests arrive at time 0, then another cost-1 request arrives at time 0.5. What should the time-0.5 call return?
5.A scheduler has three tasks: label has priority 1 and no dependencies, pack has priority 2 and no dependencies, and ship has priority 3 and depends on pack. Lower priority numbers run first. run(label) succeeds, run(pack) always fails, and max_attempts is 2. After pack exhausts its attempts, what should the scheduler return?
6.A ledger records balances, append-only events, and a results map from idempotency key to the stored event. Starting from an empty ledger, it applies ('deposit-1', 'acct', +100), then ('withdraw-1', 'acct', -30), then repeats ('deposit-1', 'acct', +100). What should the repeated deposit do?
7.A multiline log parser treats timestamp|level|message as a new event, appends indented lines to the current event's message, ignores orphan continuations, and clears the current event on malformed non-indented lines. For input orphan continuation\n2026-06-02T12:00:00Z|ERROR|request failed\n timeout while calling model\nmalformed line\n2026-06-02T12:01:00Z|INFO|retry queued, what should it return?
8.A transfer operation debits acct-b and credits acct-a, while another thread may transfer in the opposite direction. Each account has its own lock. Which plan prevents deadlock while keeping the transfer state consistent?
9.A scheduler must detect dependency cycles before invoking run. Its graph has A depending on B, B on C, C on A, and independent D. Which validation behavior is correct?

9 questions remaining.

Next Step
Continue to AI Lab System Design Interview

You'll turn the same building blocks into end-to-end AI/backend designs with scale, overload behavior, permissions, rollout gates, and observability.

PreviousReasoning & Test-Time Compute
Share this article
XFacebookLinkedInBlueskyRedditHacker NewsEmail
References

Interview guide

OpenAI · 2026

Careers

Frontier AI lab · 2026

Exponential Backoff And Jitter

Brooker, M. (AWS) · 2015

Discussion

Questions and insights from fellow learners.

Discussion loads when you reach this section.