Prompt Engineering Fundamentals

You already know that a modern large language model (LLM) predicts the next token from its input context. That doesn't give it a live copy of your merchant's return policy. If support agent Luna needs an answer about damaged order A10234, your application must supply the applicable policy or use tool calling to retrieve it from a trusted source.

Prompt engineering is the practice of packaging instructions, context, examples, and output constraints so a model response can be checked. It isn't a hunt for secret phrases. The useful work is ordinary engineering: state the task, supply trusted facts, isolate untrusted input, define the response shape, and evaluate failures.

Key idea: Prompts don't make an answer trustworthy on their own. A grounded task needs trusted context, a response contract, and a test that catches unsupported outputs.

---

What a prompt actually controls

A common beginner mistake is treating a plain LLM call like a database lookup. Without attached retrieval or tools, a generation request doesn't fetch a live policy during inference. Transformer layers process the supplied tokens and the decoder generates one continuation token after another.

Think of it like a very well-read autocomplete. The prompt is the text on the screen so far, and the model's job is to continue it in a way that looks coherent. At the model-input level, prompting is probability shaping: you arrange the input so useful continuations become more likely.

This has two practical consequences:

1. Grounded facts need a path into the request. If the damaged-item rule isn't in supplied context or a tool result, the response isn't grounded in your actual rule.
2. The context window has a budget. Longer inputs require more processing and, for metered hosted APIs, can add billable input tokens. Keep evidence that matters and measure cost and latency in the next chapter.

The work-order analogy

One way to think about prompting is writing a work order for a fulfillment system. The model has broad capability, but it needs a task, constraints, and relevant context. Stable instructions say it must answer only from supplied return-policy lines. The current request carries order facts and the customer's message. If the work order is vague, the model may fill gaps. If it has evidence and explicit checks, a bad output is easier to catch.

---

Message roles in chat-style requests

Chat-style APIs structure a conversation into roles. Exact role names vary by provider and API, but the idea is the same: separate application instructions, user requests, and prior assistant outputs.

A provider's chat-style request might represent a grounded support turn like this:

1. Application instructions: Higher-priority rules for the active request or managed conversation. APIs may call this layer developer, system, or expose it separately. It holds stable behavior and format constraints.
2. User message: The specific request or question from the human.
3. Assistant message: The model's previous replies. In a stateless request pattern, you pass these back in so the model can use the prior conversation.

Why roles help

Roles aren't security boundaries. They're structure.

Without roles, the model receives a blob of text and has to infer which parts are instructions, which parts are user data, and which parts are prior answers.

Roles make that separation explicit:

If an API is stateless, it won't remember old turns unless your application sends them again. Browser chat apps usually manage that history for you. API clients often make you manage it yourself.

Build the message list in code so you can test where dynamic data goes. The assertion prevents order details from leaking into stable application instructions.

From vague to engineered

Here is how a prompt evolves from a vague request to a testable one. Imagine you're building a support tool that extracts return eligibility from merchant policy text.

Vague

Improved

Engineered

The engineered version does four useful things:

1. Names the task and the role.
2. Separates rules from source data.
3. Puts the untrusted or changeable text inside a delimiter.
4. States the expected output shape so your application knows what to parse and validate.

The delimiter doesn't make prompt injection impossible. It gives the model a clearer layout and gives your application a place to apply extra checks.

Now render the same contract from variables rather than pasting facts into an untestable string:

---

Building prompts in layers

Reliable prompts are built in layers. You put stable rules first, add trusted context and examples, isolate untrusted user data, then check the output shape.

At each layer, you choose a prompting pattern based on what the model needs to know:

---

Teaching the model with examples

A zero-shot prompt asks the model to do something without examples. It relies on the model's training, instruction tuning, and the instruction text in the prompt.

User: Classify this ticket as refund, shipping, or catalog: "Where is my parcel?"

A few-shot prompt provides examples in the prompt to teach the model the exact format or logic you want. This uses the model's in-context learning ability, made famous by GPT-3.^[1]

User: Ticket: "The mug arrived cracked." Label: refund Ticket: "The carrier scan hasn't moved." Label: shipping Ticket: "Is the mug available in blue?" Label: catalog Ticket: "The box arrived, but the lamp is broken." Label:

Few-shot prompting is useful when you need a specific JSON shape, classification boundary, or unusual style.

One-shot vs few-shot

You will also hear one-shot, which sits between zero-shot and few-shot.

Zero-shot prompt

One-shot prompt

Few-shot prompt

The few-shot version teaches a boundary: a delivered package can still be a refund issue when the item is damaged, rather than a shipping issue.

Bad examples teach bad behavior

On classification and multiple-choice tasks tested by Min et al., randomly replacing demonstration labels barely reduced performance across the evaluated models; label space, input distribution, and format were important signals.^[2] Don't misread that result as permission to use wrong policy examples. A product prompt still needs correct examples and held-out tests because a wrong boundary can create a wrong business action.

If a prompt feels unreliable, inspect the examples before blaming the model. Many prompt bugs are data bugs in miniature.

Render each shot strategy from the same label rule, then keep the target ticket out of its demonstrations:

Examples used in the prompt aren't evaluation evidence. Reserve separate gold cases so you can see whether a revised prompt generalizes beyond the demonstrations:

---

Reasoning prompts

One important prompt engineering result is chain-of-thought prompting: Wei et al. supplied worked intermediate steps as few-shot exemplars and reported improvements on arithmetic, commonsense, and symbolic reasoning benchmarks for sufficiently large models.^[3] We'll return to advanced reasoning patterns later.

If you ask an LLM a multi-step inventory question and demand only the final number, it can fail silently. A short, checkable calculation gives a reviewer intermediate values to verify.

Standard prompt

Q: A warehouse has 5 pallets, ships 2 to a store, receives 3 more, and splits every pallet into half-pallet units. How many half-pallet units are there? A:

For models and tasks where it helps, ask for useful intermediate work: a short calculation, cited policy lines, or a few-shot example with checkable steps. The goal isn't to expose unrestricted deliberation. It's to request evidence a reviewer or test can verify.

Worked-reasoning prompt

Q: A warehouse has 5 pallets, ships 2 to a store, receives 3 more, and splits every pallet into half-pallet units. How many half-pallet units are there? A: Short solution: 5 pallets - 2 shipped = 3. Then 3 + 3 received = 6 pallets. Each pallet becomes 2 half-pallet units, so 6 x 2 = 12. The answer is 12.

By generating intermediate steps, the model adds those steps to its context window. It can use its visible output as a scratchpad, making the final answer easier to predict.

For applicable OpenAI reasoning models, current API documentation recommends giving the task, constraints, and desired output format, then treating reasoning.effort as a tuning knob rather than the primary way to recover quality.^[4] Don't assume visible chain-of-thought prompts help every provider or model. Compare prompt variants on held-out cases and measure answer quality, tokens, and latency.

When visible reasoning helps

Use visible reasoning when:

1. The task has multiple steps.
2. The answer can be checked.
3. The model benefits from writing intermediate values.
4. You aren't asking for sensitive hidden deliberation.

Avoid long visible reasoning when:

1. The task is simple.
2. You need a short production answer.
3. A reasoning-specific model already has an internal reasoning interface.
4. The reasoning could expose private deliberation you don't want to store or display.

For production apps, ask for structured evidence instead of unrestricted reasoning.

• Avoid:

Think step by step and show all your reasoning.

• Prefer:

Return:

• answer
• 2-sentence explanation
• source line IDs used
• missing facts, if any

That gives users something useful to inspect without turning every response into a long scratchpad.

For a return decision, a compact evidence contract is easier to check than an open-ended monologue:

---

Locking the output shape

Prompts become much easier to test when the output has a shape.

For example, a support triage tool might need this object:

You can ask for JSON in plain text, but production systems should use a provider's schema-constrained output facility when it supports the needed schema. In OpenAI's API, Structured Outputs with a supported strict JSON schema is distinct from JSON mode: Structured Outputs enforces supported schema adherence, while JSON mode targets valid JSON and still has documented edge cases that callers must handle.^[5] Regardless of provider, your application must still handle refusals or incomplete output and validate business facts and permissions.

Minimal schema prompt

If you are using a plain prompt, still define the shape clearly:

Then validate the result in code. A prompt isn't a parser.

This is the bridge from prompt engineering to system design: prompts are only one layer. Schemas, validators, tests, and fallback paths make the behavior reliable.

You can start with a validator before you ever call a model. One candidate is correct; the other contains a plausible but unsupported window:

---

Anti-patterns that break prompts

Even good prompts fail when common mistakes slip in. Here are four anti-patterns to watch for.

The wall of text

Without visible boundaries, instruction drift is easier. When instructions, user data, and examples are all mashed together in one paragraph, the model has to guess where one ends and the other begins. Instead, use headers, XML tags, or triple backticks to create visible boundaries.

The hallucination trap

Asking a model for facts it doesn't have, without providing a knowledge base, invites fabrication. If your prompt asks for a merchant's return rule but the rule isn't supplied, the model may emit a plausible window. Retrieve trusted evidence (or use a tool), then require source line IDs.

Negative constraints

Positive constraints are usually more actionable than negative-only instructions. Instead of "Don't make up details," say "Only use facts from the provided policy. If something is missing, say 'Not specified.'"

Token waste

Including pages of unrelated policy when only one rule is relevant adds input work and distractors. Before sending a long document, ask: what is the smallest evidence slice needed for this question? Retrieve relevant chunks and place decisive evidence close to the final request; long-context studies have found lower performance when relevant information is placed in the middle of long inputs.^[6]

This toy keyword retriever isn't a production search engine, but it proves the prompt-building idea: carry only lines that can support the answer and keep their IDs.

Delimiters don't grant permissions

Putting customer text in <ticket> tags helps identify it as data. It can't stop the model from proposing an unauthorized refund or tool action. Authorization belongs in code outside the prompt.

---

When a prompt fails

When a prompt fails, don't rewrite the whole thing first. Debug it like a program.

A tiny prompt test

Here is a prompt you can test by hand:

Expected output

You should not trust that shape by inspection alone. Parse it and check the fields you care about:

If the model returns 90 days, it contradicts supplied evidence. If it omits source_line_ids, it failed the traceability contract.

That's how prompt engineering becomes engineering: define expected behavior, run examples, and check failures.

Once a failing case exists, turn it into a regression test for candidate prompt versions. Here, outputs are fixtures standing in for results collected from two prompt versions; the test logic is real.

---

Choosing the right pattern

Before writing a prompt, answer three questions:

1. What information must the model know?
2. What output shape must the app consume?
3. How will you tell whether the response is wrong?

If you can't answer those, the prompt isn't ready for production.

Here is a quick decision table for common tasks:

---

Carry this into the API boundary

By now you should be able to:

1. Explain why a model can't retrieve a live fact that isn't supplied through context or attached tools.
2. Transform a vague request into a multi-part structured prompt with role, task, rules, context, and format.
3. Choose between zero-shot, one-shot, and few-shot prompting based on whether the model needs to learn a new pattern.
4. Ask for checkable evidence instead of unrestricted reasoning.
5. Constrain a model's output with a schema when supported, parse it, and validate evidence against supplied policy lines.
6. Keep tool authorization outside the prompt and preserve failing cases as regression tests.

Mastery check

Key concepts

• Chat-style API roles and message structure
• Prompt anatomy: stable rules, trusted context, examples, user task, output shape
• Zero-shot vs One-shot vs Few-shot prompting
• Checkable explanations and when reasoning prompts help or hurt
• Clear instructions, delimiters, output schemas, and regression tests

Evaluation rubric

• Foundational: Explains the difference between application instructions (such as developer, system, or platform rules), user messages, and assistant messages
• Foundational: Separates stable instructions, changing context, examples, untrusted user data, and output constraints
• Foundational: Differentiates between Zero-shot and Few-shot prompting
• Intermediate: Designs checkable evidence fields for a multi-step answer without demanding unrestricted reasoning
• Intermediate: Explains how schemas and validators turn prompts into testable application behavior
• Intermediate: Diagnoses ignored instructions, malformed JSON, missing source context, and unauthorized actions with regression tests

Follow-up questions

Common pitfalls