You have 45 minutes left. A support assistant says, “INC-4829 was resolved yesterday.” The status record is current and the search found the right incident, but the instructions sent to the model omitted today’s date. If you answer “add a reranker,” you’ve named a component before locating the break. Start with one question: where did the evidence first go wrong?
That question changes how you study. A strong practice session produces something another engineer can inspect, under the same constraints your role uses. You make one attempt, preserve its trace, find the first failed boundary, and choose the smallest next drill.
Public hiring guides point in that direction. OpenAI says assessments vary by team and may include pair coding, take-home projects, or technical tests. Anthropic names live coding tools such as Colab and CodeSignal and expects candidates to be comfortable with basic syntax and standard libraries. Google DeepMind describes four interview stages and says invited candidates receive role-specific preparation.[1][2][3]
Process check: Public guides describe possible formats, not your personal schedule. Ask the recruiter which skills, tools, time limits, and interview policies apply to your loop.
If titles still blur together, What Does an AI Engineer Actually Do? separates applied, platform, and research work.
Start with the artifact the role produces
Titles overlap. Read the verbs and expected outputs in several current postings, then let those outputs choose your practice. Anthropic’s Applied AI role asks for rapid prototypes, user workflows, evaluations, and structured experiments. Its RL engineering role emphasizes training-infrastructure performance, robustness, and usability. Current research-engineering postings combine strong programming with experiments, distributed systems, and research direction.[4][5][6][7]
Research science asks for a different kind of evidence. When a posting asks candidates to choose research questions, develop new ideas, and own a research agenda, prepare to defend hypotheses, controls, interpretation, and disconfirming evidence. OpenAI’s current Research Scientist posting states that distinction directly.[8]
Postings can close or change. Use them as dated examples of work shape, not promises about your interview.
| Role family | Work to practice | Evidence to bring into follow-ups |
|---|---|---|
| Applied AI or product AI | Turn a user workflow into a measured model-backed product | Eval rows, traces, user feedback, latency and cost, failed ideas |
| Predictive / classical ML | Turn a decision into a validated model that survives a real split | Split contract, leakage check, calibration, error slices, offline-to-online gap |
| ML systems or platform | Make training or serving fast, reliable, and usable | Profiles, capacity math, failure recovery, benchmarks on a stated workload |
| Research engineer | Turn a method or hypothesis into working code and reproducible experiments | Baseline, implementation, ablations, run artifacts, negative results |
| Research scientist | Choose and test research questions, then interpret what the evidence supports | Hypothesis, experimental design, controls, uncertainty, research judgment |
These are centers of gravity, not walls. A small team may expect product, serving, and evaluation work from one person. A research engineer may coauthor papers, and a scientist may write a lot of systems code. Let the posting and recruiter resolve that overlap.
If you still need a project that produces this kind of evidence, AI Engineer Portfolio Projects for Interviews is the companion. Pick one artifact the target role would inspect, then make its evals, traces, and failures easy to open.
The artifact tells you what to make. The hiring process tells you the conditions under which you must make it.
Turn the hiring process into a practice contract
Make a one-page process map before you open a textbook. Keep facts separate from guesses.
| Evidence source | Extract | Preparation decision |
|---|---|---|
| Job description | Repeated tasks, required depth, expected artifacts | Pick representative work samples |
| Official hiring guide | Possible formats and general evaluation signals | Add coding, take-home, or design drills the guide names |
| Recruiter or team brief | Exact rounds, tools, timing, and policies | Reproduce real constraints |
| Your baseline attempt | Where code, reasoning, or communication breaks | Choose next drill |
Use the table to make a decision about one real posting. Neither example below is a universal loop.
Applied AI (prototype, users, evals). Reproduce a 45-minute debug of a wrong incident answer with a frozen eval row and prompt snapshot. Finish when you can name the first failed layer and the smallest next test.
RL engineering (speed, reliability, usability). Reproduce a step-time regression on a stated training workload. Finish when you can point at a profile, change one mechanism, and rerun the same job.
For each expected skill, write one observable exit condition. "Know RAG" is too vague. "Given a failed retrieval trace, separate source freshness, retrieval recall, context assembly, and unsupported generation, then propose one measurable next test" is something another person can review.
Follow the employer’s current tool policy. AI assistance can help with critique and rehearsal, but take-home and live-task rules vary. Anthropic’s candidate guidance allows AI for preparation while requiring candidates to work without it during take-homes and live interviews unless explicitly told otherwise. That page still isn’t a substitute for the rules your recruiter sends. Don’t infer permission from the fact that a company builds AI products.[9]
Policy check: Ask what documentation, web search, AI tools, local editors, and test runners are allowed. Practice in that environment once before interview day.
Practice loop: Choose the artifact, write the constraints, produce it, preserve the trace, find the first broken contract, change one layer, and rerun the same case. Stop when the exit condition holds twice.
With the conditions pinned down, choose the shared floor your role-specific work will build on.
Shared skills and role-specific depth
Role-specific preparation still sits on a common floor.
Correct code starts with explicit inputs and outputs. Choose data structures, handle edge cases, test invariants, and explain complexity before you optimize.
ML reasoning starts with a claim you can test. State the objective, check leakage, choose a valid split, inspect calibration, and study error slices.
System reasoning makes choices accountable. Trace a request or training path, estimate its bottleneck, assign failure ownership, and plan observability and recovery.
Communication makes the reasoning inspectable. State the assumptions, tradeoffs, evidence, uncertainty, and ownership behind each decision.
Use curriculum links as repairs, not as a second syllabus:
| Gap exposed by practice | Focused path | Proof to produce |
|---|---|---|
| Stateful coding breaks under follow-ups | AI Lab Coding Interview: Python Systems | Working code, deterministic tests, defended invariants |
| Validation or experiment claims are weak | Validation and Leakage → Experiment Design and A/B Testing | Baseline, held-out result, error slices, limits |
| Applied LLM design is hand-wavy | Capstone: Document QA → AI Lab System Design Interview | Request trace, retrieval eval, capacity estimate, rollback path |
| Training or serving depth is thin | Distributed Training: FSDP & ZeRO → Inference: TTFT, TPS & KV Cache | Profile, memory estimate, bottleneck explanation |
| Research reasoning lacks evidence | Capstone: Reproducible ML Study | Baseline, controlled change, uncertainty, negative result |
Someone interviewing for classical ML should spend more time on data quality, leakage, objective choice, calibration, and offline-to-online evaluation. Someone interviewing for LLM systems should add transformer mechanics, retrieval, context assembly, inference, tool boundaries, and model evaluation. Neither path benefits from skipping experimental discipline.
That floor is a starting point, not proof. To find out what holds under pressure, stop reading and make a representative attempt.
Practice with a feedback trace
Reading creates vocabulary. A representative attempt exposes whether you can use it.
Save a small receipt after every attempt. Fill it with facts from that attempt, not with the topic you meant to study. This one records the incident-assistant failure we’ll inspect next:
1Role signal: Applied AI; 45-minute debug; no AI tools unless recruiter said so
2Prompt and constraints: "Was INC-4829 resolved yesterday?" Frozen eval row. Inspect before changing code.
3Artifact produced: diagnosis note plus failing row
4Evidence captured: source ID match; prompt snapshot missing trusted_today
5First failed contract: context assembly
6Feedback received: jumped to a reranker before opening the prompt
7Next drill: add trusted clock, freeze the row, rerun the same fixture
8Exit condition: separate source, retrieval, assembly, and claim failures on this row
The receipt turns a vague feeling into a decision. “I need more RAG” becomes “the source ID matched, the prompt omitted trusted_today, and the next drill is a frozen replay with a trusted clock.”
The artifact changes by role:
- Applied AI: build or debug one bounded workflow, then inspect exact failed rows and traces.
- Predictive ML: fix one split or leakage bug, then show the metric movement on a held-out slice.
- ML systems: profile one bottleneck, estimate capacity, change one mechanism, and rerun the same workload.
- Research engineering: reproduce a result, keep the baseline fixed, add one ablation, and preserve the run manifest.
- Research science: state a falsifiable claim, choose controls, predict outcomes, and explain what would change the research direction.
Feedback needs a source. Use a peer, hiring manager, mentor, recorded self-review, test suite, profiler, or frozen eval set. A model can generate skeptical follow-ups, but it can’t certify experience it didn’t observe. Keep every answer tied to work and evidence you can defend.
Now run that loop end to end on one small failure.
Debug one complete LLM failure
Use this as a frozen fixture, not a live incident. A user asks, “Was INC-4829 resolved yesterday?” Status service output is incident_id=INC-4829, version=v12, resolved_at=2026-03-15T18:02Z. The prompt should carry trusted date 2026-03-20.
Pause here. On March 20, “yesterday” means March 19. The record says March 15, so the answer can’t be true. The model still replies, “Yes. INC-4829 was resolved yesterday.” A larger model wouldn’t repair a missing input.
Retrieval-Augmented Generation (RAG) supplies external evidence to a model at request time.[10] The name matters less than the boundaries: source state, retrieval, context assembly, and final claims each make a different promise.
| Boundary | Evidence to inspect | Smallest useful next test |
|---|---|---|
| Source freshness | Incident ID, source version, updated_at | Replay with current and stale records |
| Retrieval | Expected source ID and rank | Measure recall on exact-ID and semantic cases |
| Context assembly | Returned fields, current date, ordering, truncation | Snapshot the exact model input |
| Answer grounding | Support for each date and status claim | Require claim-to-field citations or abstention |
A fluent wrong answer tells you only where the failure became visible. Walk backward until the first contract breaks:
The tempting move is to jump from symptom to “use a reranker” or “fine-tune the model.” Resist it. A stale record needs a freshness fix. A missing date needs a context fix. Correct inputs with an unsupported chronology need a generation or validation fix. Freeze the failed case, change one layer, and rerun the same row.
Translate that triage into a contract checker you can inspect in code:
1import datetime
2from dataclasses import dataclass
3
4@dataclass(frozen=True)
5class EvalFixture:
6 incident_id: str
7 target_id: str
8 retrieval_rank: int
9 source_status: dict[str, str]
10 prompt_snapshot: dict[str, str]
11 model_answer: str
12
13def triage_failure(fixture: EvalFixture) -> tuple[str, str, str]:
14 """Isolate the first broken contract before jumping to rerankers or fine-tuning."""
15 # Contract 1: Source validity
16 if fixture.source_status.get("incident_id") != fixture.target_id:
17 return "source", "stale or mismatched source record", "refresh status database record"
18
19 # Contract 2: Retrieval rank and ID
20 if fixture.incident_id != fixture.target_id or fixture.retrieval_rank != 1:
21 return "retrieval", "wrong document or rank > 1", "measure retrieval recall and rank loss"
22
23 # Contract 3: Context assembly (required fields present in prompt)
24 if not fixture.prompt_snapshot.get("trusted_today"):
25 return (
26 "context_assembly",
27 "missing trusted_today in prompt template",
28 "pass trusted clock to context builder",
29 )
30
31 # Contract 4: Grounding and claim verification
32 resolved_date = datetime.date.fromisoformat(fixture.source_status["resolved_at"][:10])
33 today = datetime.date.fromisoformat(fixture.prompt_snapshot["trusted_today"])
34 yesterday = today - datetime.timedelta(days=1)
35
36 if "yesterday" in fixture.model_answer.lower() and resolved_date != yesterday:
37 return (
38 "claims",
39 f"unsupported yesterday claim (resolved {resolved_date}, yesterday was {yesterday})",
40 "add claim-to-field citation gate",
41 )
42
43 return "pass", "all contracts satisfied", "none"
44
45# INC-4829 fixture: source and retrieval pass, context assembly omits trusted date
46inc_4829_fixture = EvalFixture(
47 incident_id="INC-4829",
48 target_id="INC-4829",
49 retrieval_rank=1,
50 source_status={"incident_id": "INC-4829", "version": "v12", "resolved_at": "2026-03-15T18:02Z"},
51 prompt_snapshot={"incident_id": "INC-4829", "resolved_at": "2026-03-15T18:02Z"},
52 model_answer="Yes. INC-4829 was resolved yesterday.",
53)
54
55layer, cause, fix = triage_failure(inc_4829_fixture)
56print(f"First broken contract: {layer}")
57print(f"Root cause: {cause}")
58print(f"Smallest next test: {fix}")1First broken contract: context_assembly
2Root cause: missing trusted_today in prompt template
3Smallest next test: pass trusted clock to context builderThat drill transfers beyond RAG. Classical ML uses the same habit: verify label and feature snapshots, inspect split leakage, check the objective and threshold, then look at the decision shown to the user.
💡 Key insight: The first failed contract is the next drill. A fluent answer that skips that inspection isn't practice.
Rehearse communication from the same evidence
Once you can locate the first break, use the same artifact to practice speaking. An incident trace, benchmark report, experiment run, or code review can support four forms:
- A 90-second mechanism explanation.
- A design answer with one bottleneck estimate and one failure path.
- A project story that separates personal ownership from team outcome.
- A skeptical follow-up about tradeoffs, uncertainty, or what failed.
The AI Lab Behavioral Interview lesson turns project evidence into stories about judgment, disagreement, incidents, and changed decisions. The AI Lab Technical Presentation lesson turns the same evidence into architecture, metrics, ownership boundaries, and Q&A defense.
Don’t polish one script until it sounds memorized. Rehearse the same idea at different lengths and let another person interrupt. If an explanation falls apart under one “why?”, return to the implementation or evidence instead of adding smoother wording.
The speaking drill is another rerun of the same case: explain the mechanism, quantify one trade-off, name the failure path, then answer what would change your decision.
Use the first failure to pick the next drill
Fixed four-week or eight-week plans pretend everyone starts from the same place. Let an observed gap choose the next drill instead.
| Evidence from an attempt | Next drill | Exit condition |
|---|---|---|
| Code works only for happy path | Add staged requirements and table-driven tests | Preserve invariant without rewriting from scratch |
| Design has boxes but no constraints | Calculate memory, traffic, queue, or cost bottleneck | Explain which assumption would change architecture |
| ML answer names a metric but no dataset contract | Define split, baseline, slices, and failure cost | State what score proves and what it doesn't |
| Research result has no strong comparison | Add baseline, ablation, seed or uncertainty check | Defend conclusion and one plausible alternative explanation |
| Project story sounds broader than personal work | Mark owned decisions, artifacts, and team interfaces | Answer depth questions without borrowing credit |
| Explanation is fluent but shallow | Implement or trace mechanism with concrete values | Survive two levels of "why" and one failure question |
Read the table as a loop, not a syllabus. Produce representative work under stated constraints, inspect its evidence, change one thing, and recover from a follow-up without losing the underlying model. Readiness is role-specific evidence, not a guarantee of an offer.
Only after this loop is working should current topics change your plan.
Keep current topics in proportion
Model Context Protocol, reasoning models, Reinforcement Learning from Verifiable Rewards, Mixture of Experts, speculative decoding, and new serving stacks may matter for one role and be noise for another. Study one when the posting names it, your project depends on it, or you need it to explain a measured tradeoff. These are recognition topics, not a replacement for the mechanics above.[11][12][13][14]
For broad recall, use 50 LLM Interview Questions for 2026 as a diagnostic. Cover each answer and explain its mechanism, tradeoff, failure signal, and measurement. Open the linked curriculum lesson when one of those four parts is missing. Don't treat the list as the study plan.
Leave evidence behind
Interview preparation gets stronger when each cycle leaves an artifact: tested code, a design note, an eval report, a reproducible run, a failure trace, or a project explanation with honest ownership. Pick one target role, run one representative baseline, and let evidence choose what you study next. The question list can wait until you know which contract keeps breaking.