Take a support incident. At 09:17, an assistant tells a customer that an old cancellation policy still applies. The service is up, the response is fluent, and the latency dashboard is green. The customer is still wronged.
Before anyone rewrites a prompt, inspect one request. Did the current policy reach the model? Did the application authorize any action? Did the answer match its evidence? Did the customer see the validated result? Each "no" points to a different fix.
The engineer who traces that chain is doing AI engineering: building reliable software around a model whose output isn't guaranteed. A model is one component. The feature also needs data boundaries, trusted actions, evaluation, rollback, and someone watching it after release.
The title isn't standardized either. U.S. labor statistics place adjacent work under software developers, data scientists, and computer and information research scientists rather than one AI-engineer occupation.[1][2][3] Read the artifact a role has to ship before trusting its title.
Start with artifact, not title
In 2023, swyx used "AI engineer" for software built on foundation models, general-purpose models reused across products, distinct from training those models.[4] The distinction remains useful, but teams now use the title for several kinds of ownership.
Current postings make that spread concrete. Anthropic's Staff Software Engineer, Labs: Applied AI role asks for full-stack prototypes, user workflows, evaluations, and structured experiments. It also expects the engineer to stop a project when user evidence says it isn't working.[5]
Anthropic's Research Engineer, RL Engineering role sits closer to reinforcement-learning (RL) systems. Its representative work includes profiling training pipelines, launching test jobs, diagnosing slowdowns, and implementing a researcher's algorithm so the training system can use it reliably.[6]
OpenAI's Research Engineer role combines strong programming and large distributed systems with building machine-learning code and the science behind algorithms. Anthropic's Research Engineer, Machine Learning (Reinforcement Learning) role combines novel methods, environments, evaluations, scaling, and research direction.[7][8]
The same company can use "Research Engineer" for infrastructure work and method-plus-direction work. The shipped artifact tells you which one a posting means.
| Role family | Primary artifact | Work that often dominates | Not implied by title |
|---|---|---|---|
| Applied AI or product AI engineer | User-facing model-backed workflow | Product integration, retrieval, tool use, evaluations, experiments, and operations | Training a foundation model |
| Machine-learning or platform engineer | Repeatable train, deploy, or serve path | Data and training pipelines, serving, performance, reliability, and monitoring | Owning the user interface |
| Research engineer | Experimental result backed by working code | Implementing methods, building environments and evaluations, scaling runs, and analyzing failures | Being first author on every research claim |
| Research scientist | Supported research claim or new method | Hypotheses, experiment design, interpretation, and publication | Owning production infrastructure end to end |
| Data scientist | Analysis, predictive model, or decision evidence | Data preparation, statistics, experimentation, modeling, and communication | Operating a model-backed product service |
These are centers of gravity, not walls. A small team might combine several rows; a larger team might split retrieval, evaluation, serving, product integration, and research tooling across people. Follow the same support assistant and the boundary becomes visible: applied work ships a measured user workflow, platform work ships a reliable train-or-serve path, and research engineering ships a reproducible experimental result.
Scope check: In a job description, underline the artifact, production owner, evaluation method, and failure budget. Those details predict the work better than the words "AI engineer" do.
That artifact view gives us a boundary to follow. Start with the applied-AI path, where a user request must become a safe, measured product behavior.
Follow one request to its release
Applied AI starts with a user task, not a model name. For the support assistant, the contract might require an answer grounded in the user's permitted policy documents, claim-level citations, an abstention when evidence is missing, and a latency budget.
The model is one stage inside that contract. The surrounding work is easier to see when each failure has an owner:
| Workstream | Concrete artifact | Failure that exposes it |
|---|---|---|
| Task and evaluation contract | Supported inputs, expected outputs, refusal rules, and fixed evaluation cases | Team can't say whether one answer is correct |
| Context and data | Parsed sources, versioned index, access policy, and retrieval tests | Correct or permitted evidence never reaches the model |
| Tool boundary | Schemas, authorization, idempotency, approval, and audit events | Model proposes an invalid or duplicate side effect |
| Model boundary | Versioned prompt and model config, response schema, timeout, and fallback | Output can't be parsed, or behavior changes silently |
| Release evidence | Baseline-versus-candidate report, failed rows, and rollback target | Demo improves while an important slice regresses |
| Operations | Joined traces, latency, errors, usage, and cost | Team sees a bad answer but can't locate its failing stage |
Evidence before generation
Start with the policy, not the model. Retrieval-augmented generation (RAG) supplies selected external evidence at request time. A dependable path parses and versions sources, preserves lineage while chunking, searches with keyword and dense retrieval, applies reranking, and enforces access control. Dense retrieval can find semantic matches, while BM25 protects exact identifiers such as plan codes and policy names.[9][10]
Permission checks belong in retrieval. A filter after generation can't make the model forget a private chunk that already reached its context. The Document QA capstone builds this boundary from ingestion through citations and abstention.
Trusted tools and actions
A model can propose a tool call, but trusted application code decides whether to execute it. That code validates arguments, authenticates the caller, checks authorization, applies rate limits and idempotency, and records the result. A fluent response isn't proof that a side effect was permitted or completed.
This is ordinary backend engineering around an unusual decision-maker. If the assistant may open a billing ticket, the runtime owns duplicate submissions, missing fields, unauthorized callers, and tool timeouts. Function Calling & Tool Use develops that boundary through read authorization, write approval, retries, and trajectory evaluation.
Model and serving choices
A hosted API keeps accelerator scheduling outside the product team's boundary, but the application still owns model configuration, timeouts, usage, fallbacks, privacy, and release checks. Self-hosting adds batching, routing, GPU memory, deployment, and cache management. PagedAttention, for example, stores the key-value (KV) cache in blocks to reduce wasted VRAM under variable request lengths.[11][12]
The next choice is adaptation. A missing or changing fact calls for better retrieval. An output-shape failure may yield to structured output.[13] A durable behavior gap with representative training and evaluation data may justify Low-Rank Adaptation (LoRA) or another tuning method.[14][15]
Inference: TTFT, TPS & KV Cache explains the serving path. LoRA & Parameter-Efficient Tuning covers the training and memory tradeoffs after a measured behavior gap makes tuning reasonable.
Evaluation and release
Evaluation starts before release. It defines what the team is trying to preserve while prompts, retrieval, tools, models, and infrastructure change.
Use direct assertions for schemas, citations, permissions, tool arguments, and abstentions. Use human review or a calibrated model-based judge for qualities that require language judgment. Public benchmarks such as Massive Multitask Language Understanding (MMLU), HumanEval, and SWE-bench describe behavior under their own datasets and protocols. They don't replace cases from your product.[16][17][18]
Release rule: Compare baseline and candidate on the same versioned cases. Inspect failed rows and required slices before accepting an aggregate score.
The LLM Benchmarks & Limitations lesson turns these checks into a release decision with reproducible evidence.
The request path now has an artifact and a proof obligation. A week of work is the sequence of decisions needed to keep both intact.
A week is a sequence of decisions
A fixed hourly calendar would hide the difference between role families. The same support assistant creates different work depending on which artifact is at risk:
- Applied AI: Sit with support agents, prototype the cited-answer path, and run a structured experiment on abstention versus guessing. The next decision is whether user evidence supports another iteration or a stop.[5]
- Platform engineering: Profile why an embedding or training job stalled, add instrumentation, and keep the policy index and serve path inside latency and error budgets. A researcher's algorithm still needs a stable implementation before anyone can trust its result.[6]
- Research engineering: Implement a hybrid retriever or a new evaluation, run a paired comparison on policy-ID queries, and debug a distributed run. Failed cases decide which method gets the next experiment.[7][8]
Design reviews, code review, tests, incident diagnosis, documentation, and coordination fill the gaps in every version of that week. Model expertise changes the failure modes. It doesn't remove software discipline.
On a small team, one person may move through all three lanes before lunch. At a larger organization, the handoff itself becomes part of the job: product owns the user outcome, platform owns a reliable path, and research owns a defensible result. The artifact stays connected only when traces and eval rows cross those boundaries.
An incident becomes useful career evidence when it shows that you can locate a failure before proposing a fix.
Trace the failing layer
Suppose the assistant cites an obsolete cancellation policy. Don't change anything yet. Reconstruct one trace: user and access scope, normalized query, retrieved source IDs and versions, reranked context, model response, validation result, and rendered output.
If the allowed policy never entered context, inspect ingestion, filters, retrieval, and reranking. A prompt change can't recover evidence that was never returned.
If the right policy did enter context but the answer contradicts it, inspect the response contract, abstention behavior, and model choice. Hold the evidence packet steady while comparing generation changes.
If validation passed but the customer saw different text, inspect parsing, caching, state, and rendering. The model can be correct while an ordinary application layer is wrong.
Whichever branch you take, keep the failed request as a fixed eval row and rerun the relevant slice before release or rollback.
Common mistake: Editing the prompt first when the current policy never entered context. Prompt work can't recover evidence the retriever never returned.
The trace gives you a concrete proof artifact. Career growth follows the same pattern: the surface area of decisions expands, and so does the evidence you must defend.
Grow by widening the proof
Early-career work usually owns one bounded path: a retrieval endpoint, a tool contract, an eval harness, or a serving change. The proof is a reproducible test and a clear failure diagnosis.
Mid-level work connects those pieces end to end. You choose trade-offs across quality, latency, cost, permissions, and rollback, then make the result legible to the teams that depend on it.
Senior and staff work widens the boundary again. You set shared contracts, reliability budgets, evaluation practice, or serving architecture so several teams can ship without losing the evidence that made one feature safe. Titles vary; decision surface and proof are harder to fake.
Match study to artifact
Don't build a study plan from a fast-changing tool catalog. Choose stable mechanisms and the artifact your target role has to produce.
| Target | Curriculum path | Evidence to finish with |
|---|---|---|
| Applied AI | First AI App End-to-End → File Ingestion for AI → Function Calling & Tool Use → LLM Benchmarks & Limitations | Deployed workflow with traces, eval rows, failure handling, and rollback |
| Machine-learning or platform engineering | PyTorch Training Loops → Dataset Pipelines and Data Quality → Model Versioning & Deployment → Inference: TTFT, TPS & KV Cache | Reproducible train-or-serve path with reliability and performance evidence |
| Research engineering | The Transformer Architecture End-to-End → Distributed Training: FSDP & ZeRO → Capstone: Reproducible ML Study | Baseline, implementation, controlled experiments, uncertainty, and artifacts |
| Data science moving toward AI systems | Probability for Machine Learning → Experiment Design and A/B Testing → PyTorch Training Loops | Analysis or model tied to honest validation and a decision |
Framework names will change. The durable skills are task definition, data and permission boundaries, model behavior, evaluation, systems performance, and evidence-backed communication.
For a step-by-step transition plan, continue to How to Become an AI Engineer from Zero in 2026. For project packaging and interview evidence, use AI Engineer Portfolio Projects for Interviews.