On this page
AI engineering turns models into software people can use. Depending on the team, the job can mean shipping a product feature, operating a training or serving platform, or building experiments for researchers. The engineer writes the surrounding code and checks whether the resulting system actually solves the task.
Consider a fictional support incident. An assistant tells a customer that cancellation requires 60 days' notice, but the current policy says 30. Availability and latency dashboards are green. Customers still receive the wrong answer.
Before rewriting a prompt, inspect one request. Did the current policy reach the model? Did the answer match that evidence? Did the customer see the checked response? Each "no" points to different code. Tracing that chain, fixing the fault, and preventing its return is a useful picture of the job.
The title alone won't tell you which part you would own. The U.S. Bureau of Labor Statistics describes related work in its software developer, data scientist, and computer research scientist profiles; those occupations overlap with the work discussed here but are not interchangeable with an employer's AI-engineer title.[1][2][3]
Read the job behind the title
In 2023, swyx used "AI engineer" for software built on foundation models, general-purpose models reused across products, distinct from training those models.[4] The distinction remains useful, but teams now use the title for several kinds of ownership.
Three job descriptions checked on October 10, 2026 illustrate the spread. They are examples from frontier labs, not a representative survey of the job market. Openings and responsibilities can change.
OpenAI's Applied AI Engineer, API Platform, Digital Natives role combines customer-facing architecture, hands-on prototypes, evaluations, integrations, and production launch. Workshops and customer relationships are part of this particular job, not just implementation.[5]
Anthropic's Research Engineer, RL Engineering role owns reinforcement-learning (RL) training systems across orchestration, environments, training, inference, and evaluation. The posting combines operating those systems with studying training behavior at scale and working with researchers on new methods. Profiling and debugging distributed runs support that work.[6]
Anthropic's Research Engineer, Machine Learning (Reinforcement Learning) role combines novel methods, environments, evaluations, scaling, and research direction.[7]
Both Anthropic postings mix systems and research. One emphasizes ownership of the RL training system; the other explicitly includes shaping research direction. Ask what artifact you would own and how the team would evaluate it, rather than inferring a clean boundary from the title.
| Role family | Primary artifact | Work that often dominates | Not implied by title |
|---|---|---|---|
| Applied AI or product AI engineer | User-facing model-backed workflow | Product integration, retrieval, tool use, evaluations, experiments, and operations | Training a foundation model |
| Machine-learning or platform engineer | Repeatable train, deploy, or serve path | Data and training pipelines, serving, performance, reliability, and monitoring | Owning the user interface |
| Research engineer | Experimental result backed by working code | Implementing methods, building environments and evaluations, scaling runs, and analyzing failures | Being first author on every research claim |
| Research scientist | Supported research claim or new method | Hypotheses, experiment design, interpretation, and publication | Owning production infrastructure end to end |
| Data scientist | Analysis, predictive model, or decision evidence | Data preparation, statistics, experimentation, modeling, and communication | Operating a model-backed product service |
These are common emphases, not exclusive assignments. A small team may combine several rows. In an interview, ask what you would deliver in the first three months, who uses it, how success is measured, and who handles failures after launch. A job that mainly helps customers integrate an API can look very different from one that owns an internal training cluster.
Follow one request to its release
Applied AI starts with a user task, not a model name. For the support assistant, the contract might require an answer grounded in the user's permitted policy documents, claim-level citations, an abstention when evidence is missing, and a latency budget.
An abstention states that the available evidence doesn't support an answer. It keeps a missing policy from becoming a guessed policy.
The contract can also define a response schema: the expected output fields and their types. Application code checks that shape before using the response.
The model is one stage inside that contract. The surrounding work is easier to see when each failure has an owner:
| Workstream | Concrete artifact | Failure that exposes it |
|---|---|---|
| Task and evaluation contract | Supported inputs, expected outputs, refusal rules, and fixed evaluation cases | Team can't say whether one answer is correct |
| Context and data | Parsed sources, versioned index, access policy, and retrieval tests | Correct or permitted evidence never reaches the model |
| Tool boundary | Schemas, authorization, idempotency, approval, and audit events | Model proposes an invalid or duplicate side effect |
| Model boundary | Versioned prompt and model config, response schema, timeout, and fallback | Output can't be parsed, or behavior changes silently |
| Release evidence | Baseline-versus-candidate report, failed rows, and rollback target | Demo improves while an important slice regresses |
| Operations | Joined traces, latency, errors, usage, and cost | Team sees a bad answer but can't locate its failing stage |
Evidence before generation
Retrieval-augmented generation (RAG) supplies selected external evidence at request time. For this assistant, every policy passage needs a source, version, and access scope. Splitting documents into passages is called chunking.
Search design depends on the documents. Dense retrieval can find semantic matches; lexical methods such as BM25 score term matches.[8][9] Neither guarantees an exact identifier survives tokenization or that a highly ranked policy is current. An exact-ID lookup may be simpler for a known plan code. Add hybrid search or a reranker when measured retrieval failures justify the extra stage.
Enforce document permissions before sending evidence to the model or another service not authorized to see it. Check them again when a link or tool reads the source. A filter after generation is too late to prevent disclosure through the model request. Scope alone is also insufficient: A-v1 is allowed but stale. The Document QA capstone builds the ingestion, retrieval, citation, and abstention path.
Trusted tools and actions
A model can propose a tool call, but application code decides whether to execute it. That code uses the authenticated caller's identity, validates arguments, checks authorization, applies rate limits, and records the result. A model-supplied account ID is not permission to act on that account.
If the assistant may open a billing ticket, the runtime must also handle duplicate submissions and ambiguous timeouts. A local request log can't by itself prevent a remote ticket from being created twice; the downstream service needs an idempotency contract or a way to reconcile the result. Function Calling & Tool Use develops these boundaries through read authorization, write approval, retries, and trajectory evaluation.
Model and serving choices
A hosted API keeps accelerator scheduling outside the product team's boundary, but the application still owns model configuration, timeouts, usage, fallbacks, privacy, and release checks. Self-hosting adds batching, routing, GPU memory, deployment, and cache management.
The key-value (KV) cache stores attention data from previous tokens. PagedAttention organizes that cache in blocks to reduce wasted GPU memory when request lengths vary.[10][11]
Diagnosis also guides adaptation. If the correct policy never arrives, investigate the data path. If fields are malformed, test structured output, while still handling refusals and incomplete responses; schema compliance doesn't establish truth.[12] A persistent behavior gap with representative training and evaluation data may justify Low-Rank Adaptation (LoRA) or another tuning method.[13][14] These are hypotheses to test, not a rule that every failure requires a larger model or a training run.
Inference: TTFT, TPS & KV Cache explains the serving path. LoRA & Parameter-Efficient Tuning covers the training and memory tradeoffs after a measured behavior gap makes tuning reasonable.
Evaluation and release
Evaluation starts before release. It defines what the team is trying to preserve while prompts, retrieval, tools, models, and infrastructure change.
Use direct assertions for schema validity, permitted source IDs, tool arguments, and labeled abstention cases. A citation's presence can be checked mechanically; whether it supports the claim may require human review or a model-based judge checked against human labels. Public benchmarks such as Massive Multitask Language Understanding (MMLU), HumanEval, and SWE-bench describe behavior under their own datasets and protocols. They don't replace cases from your product.[15][16][17]
Release rule: Compare baseline and candidate on the same versioned cases. Inspect failed rows and required slices before accepting an aggregate score.
The LLM Benchmarks & Limitations lesson turns these checks into a release decision with reproducible evidence.
Trace the failing layer
Return to the 60-day answer. Reconstruct the request: caller access scope, search query, returned source IDs and versions, final context, model response, validation result, and rendered output. Avoid copying sensitive request contents into unrestricted logs.
Read the trace in order. Stop at the first failed condition, repair that layer, then replay the whole path to check for another fault.
| Trace condition | If it fails, inspect | Regression evidence |
|---|---|---|
| Current permitted policy A-v2 reaches context | Ingestion, effective-version filters, retrieval, and reranking | A-v2 enters; stale A-v1 and forbidden B-v4 don't |
| The answer is supported by A-v2 | Response contract, abstention, and model choice | The answer says 30 days and cites A-v2 |
| The user sees the validated response | Parser, cache, persisted state, and rendering | The displayed answer remains 30 days for this account |
| All conditions hold but the case is still labeled wrong | Evaluation label and product expectation | The policy owner confirms the expected behavior |
If A-v2 never entered context, inspect ingestion, version selection, filters, retrieval, and any reranking. A prompt can't supply the missing source, although it can make the assistant abstain or request another lookup instead of guessing.
If the right policy did enter context but the answer contradicts it, inspect the response contract, abstention behavior, and model choice. Hold the evidence packet steady while comparing generation changes.
If validation passed but the customer saw different text, inspect parsing, caching, state, and rendering. The model can be correct while an ordinary application layer is wrong.
The trace contains A-v2, the model returns 30 days, and validation passes. The customer sees 60 days. Which layer would you inspect first, and which tempting change would fail to address the evidence?
Answer
Inspect the response path after validation: cache keys, persisted state, parsing, and rendering. Check whether a stale response was reused across policy versions or accounts. Replacing the model or rewriting its prompt doesn't explain why a checked 30-day response became a displayed 60-day response. Add a regression case that checks the customer-visible result, not only the raw model output.
For this fictional incident, suppose the trace contains A-v1 and no A-v2. An illustrative day's work might look like this:
| Work | Result to inspect |
|---|---|
| Confirm the intended policy with its owner | A-v2 is effective for this account; 30 days is the expected answer |
| Reproduce the bad lookup locally | A-v1 still qualifies because the query omits the effective-version condition |
| Write the failing case and repair the query | A-v2 is included; A-v1 and another account's policy are excluded |
| Run the retrieval and answer evaluations | Inspect failures, abstentions, latency, and unchanged-account cases, not only the original request |
| Review and release the patch | Save the tested revisions, monitor the affected slice, and retain a known-good rollback path |
That's a query fix, a test, a review, and a rollout. No new model was needed. A different trace could lead to a prompt experiment, an API integration fix, or an investigation with the model provider. During an active incident, contain harm first: a safe fallback or rollback may be necessary before the full repair is ready.
Choose work you want to own
One way to plan growth is to widen the responsibility you can handle. Start with a bounded path: a retrieval endpoint, a tool contract, an eval harness, or a serving change. Make it reproducible and explain a failure you diagnosed.
Then connect those pieces end to end. Choose trade-offs across quality, latency, cost, permissions, and rollback, and explain them to the people who use or operate the system.
Broader roles may set shared interfaces, reliability budgets, evaluation practice, or serving architecture across teams. This is a way to assess scope, not a universal leveling ladder. Ask the employer how its levels map to actual responsibility.
Match study to artifact
Choose a study path that matches the work you want to show. These routes suggest practice projects, not hiring guarantees; prerequisites still matter.
| Target | Curriculum path | Evidence to finish with |
|---|---|---|
| Applied AI | First AI App End-to-End → File Ingestion for AI → Function Calling & Tool Use → LLM Benchmarks & Limitations | Deployed workflow with traces, eval rows, failure handling, and rollback |
| Machine-learning or platform engineering | PyTorch Training Loops → Dataset Pipelines and Data Quality → Model Versioning & Deployment → Inference: TTFT, TPS & KV Cache | Reproducible train-or-serve path with reliability and performance evidence |
| Research engineering | The Transformer Architecture End-to-End → Distributed Training: FSDP & ZeRO → Capstone: Reproducible ML Study | Baseline, implementation, controlled experiments, uncertainty, and artifacts |
| Data science moving toward AI systems | Probability for Machine Learning → Experiment Design and A/B Testing → PyTorch Training Loops | Analysis or model tied to honest validation and a decision |
Framework names will change. The durable skills are task definition, data and permission boundaries, model behavior, evaluation, systems performance, and evidence-backed communication.
For a step-by-step transition plan, continue to How to Become an AI Engineer from Zero in 2026. For project packaging and interview evidence, use AI Engineer Portfolio Projects for Interviews.