Python for AI Engineering

The Docker image from the previous lesson printed a passing evaluation report. Two predictions matched their expected labels; one didn't:

Eval rows: 3
Exact-match accuracy on tiny fixture: 0.667 (2/3)
Gate passed. You may commit.

Git stored eval/access_requests.jsonl and scripts/score_access_requests.py. Docker supplied the operating system files and Python runtime, but neither one explains how a line of text becomes 2/3.

That path is ordinary Python: a file becomes rows, rows become counts, and counts become a printed score. Rebuild it with each boundary visible.^{[1]Reference 1The Python Tutorial.https://docs.python.org/3/tutorial/}

View larger

Pin a local Python 3.12 project

The image already pins one runtime path. Before changing the scorer, give your checkout a matching interpreter and a separate place for its tools. Otherwise a passing test can depend on whichever Python and packages happen to be first on your PATH.

A virtual environment provides a project-local package directory separate from the operating system's Python. Local work here uses uv's project workflow for that isolation.

Keep the lab on Python 3.12 because that's what the Docker image already pins, not because 3.12 is the newest CPython release.

Confirm uv --version works. If it doesn't, use the official uv installation guide for your operating system. Run these commands from your earlier lab repository, not from an unrelated project. Skip uv init if pyproject.toml already exists:

uv python install 3.12
uv init --bare --python 3.12
uv python pin 3.12
uv sync
uv run python --version

Read the files after running these commands. uv init --bare writes only pyproject.toml; its requires-python field declares compatible Python versions. A 3.12 pin selects that minor release family, not one exact patch release.

uv python pin 3.12 writes .python-version. The first uv sync or uv run creates .venv and uv.lock.^{[2]Reference 2Locking and Syncinghttps://docs.astral.sh/uv/concepts/projects/sync/}

Each path answers a different reproducibility question. Keep the portable inputs in Git and recreate machine-local output:

uv run uses the project environment without shell activation. If you activate it on macOS or Linux, run source .venv/bin/activate; on Windows PowerShell, run .venv\Scripts\Activate.ps1. Activation changes which python your shell finds.

uv sync creates the environment and resolves its dependencies. Activation is a shell convenience, not dependency management.

Python also ships venv. In a project that doesn't use uv, python3.12 -m venv .venv creates the same kind of isolation. Pick one environment workflow per project so package state stays understandable. Keep Docker's requirements.txt; don't replace the image install path used by the earlier scorer.

One row is ordinary Python data

Start with request 102, the row the scorer should mark wrong. Before writing a loop, predict what one row must contain and what the comparison should return:

{"prompt": "Access request 102 status?", "expected": "blocked", "prediction": "escalated"}

After parsing, this JSON object becomes a Python dictionary. A dictionary maps keys such as "expected" to values such as "blocked", so row["expected"] retrieves one named field. Assigning the mapping to row gives later operations a stable name.

The next cell reads both labels, normalizes outer whitespace and case, then stores the comparison as a Boolean. Its output lets you check the type and the decision before any batch logic appears.

row = {
    "prompt": "Access request 102 status?",
    "expected": "blocked",
    "prediction": "escalated",
}

expected = row["expected"].strip().lower()
prediction = row["prediction"].strip().lower()
is_correct = prediction == expected

print("prompt:", row["prompt"])
print("expected type:", type(expected).__name__)
print("is correct:", is_correct)

prompt: Access request 102 status?
expected type: str
is correct: False

Python evaluates the right side of = first, then binds that value to the name on the left. A single = assigns; == compares.

strip() removes whitespace from both ends, and lower() returns a lowercase string. Neither method edits the original string. Strings are immutable: operations return results instead of changing their contents in place.

Here, exact match means equality after these two normalization steps. It's appropriate for the fixture's short status labels. It won't recognize two differently worded explanations as equivalent, and it doesn't remove spaces inside a label.

Built-in values used in the scorer

You don't need every Python type before writing useful code. These eight cover the values moving through this evaluator:

Python is dynamically typed: a name can be rebound to a value of another type. That flexibility doesn't make types interchangeable. "2" is text, 2 is an integer, and 2.0 is a floating-point number. Convert with int(), float(), or str() only when the data contract says the conversion is valid. A malformed label shouldn't become a value that happens to score.

A list turns one row into a batch

The protected evaluation fixture contains three dictionaries in order. Before running the batch, predict the receipt: two labels match, one differs, and the denominator stays at three.

A list keeps those rows together, and a for loop visits each row once. Indentation marks the statements that belong to the loop. The code records each Boolean before it computes an aggregate, so a surprising score still has a per-row trail.

examples = [
    {"prompt": "Access request 101 status?", "expected": "approved", "prediction": "approved"},
    {"prompt": "Access request 102 status?", "expected": "blocked", "prediction": "escalated"},
    {"prompt": "Access request 103 status?", "expected": "restored", "prediction": "restored"},
]

row_results = []

for position, row in enumerate(examples, start=1):
    is_correct = row["prediction"].strip().lower() == row["expected"].strip().lower()
    row_results.append(is_correct)
    print(position, row["prediction"], is_correct)

matches = sum(row_results)
total = len(row_results)
score = matches / total

print("matches:", matches)
print("score:", f"{score:.3f}")

1 approved True
2 escalated False
3 restored True
matches: 2
score: 0.667

enumerate(..., start=1) supplies a readable position with each row, while append() keeps one Boolean for that row.

Then len() counts all results and sum() treats True as 1 and False as 0, producing the correct-row count. An f-string such as f"{score:.3f}" formats the fraction for display after the calculation is done.

Check the arithmetic by hand before trusting the output:

If the code prints anything else for these exact three rows, inspect the per-row Booleans first. A mismatch there points to normalization; matching Booleans with a wrong total points to aggregation.

A second name isn't a second dataset

A loop variable refers to the actual row dictionary in the list. Changing row["prediction"] inside the loop changes that row for later callers. Assigning trial = baseline shares the entire list; even baseline.copy() creates only a new outer list, with the same dictionaries inside.^{[3]Reference 3Built-in Typeshttps://docs.python.org/3/library/stdtypes.html}

Predict which changes reach the baseline:

baseline = [{"expected": "blocked", "prediction": "escalated"}]
trial = baseline.copy()
trial[0]["prediction"] = "blocked"
print("after list copy:", baseline[0]["prediction"])

baseline = [{"expected": "blocked", "prediction": "escalated"}]
trial = [row.copy() for row in baseline]
trial[0]["prediction"] = "blocked"
print("after row copies:", baseline[0]["prediction"])

after list copy: blocked
after row copies: escalated

The list comprehension, [row.copy() for row in baseline], builds a new list containing a new dictionary for each row. That is enough here because the values are immutable strings. Nested lists or dictionaries would still be shared by a shallow row copy. Our scorer only reads its input rows and creates separate results; normalizing a string doesn't mutate the dictionary that held it.

Functions give the comparison a name

The loop works, but comparison, aggregation, and printing are tangled together. Pull the comparison out first. A function packages one operation behind a name: inputs arrive as parameters, and return sends a value back to the caller.

Ask what should stay true if another caller needs the same rule. Both labels should be stripped and lowercased, and the function should return only the Boolean decision. This function owns that normalization rule:

def exact_match(expected: str, prediction: str) -> bool:
    normalized_expected = expected.strip().lower()
    normalized_prediction = prediction.strip().lower()
    return normalized_prediction == normalized_expected

print(exact_match("blocked", "escalated"))
print(exact_match("approved", " APPROVED "))

False
True

expected: str and -> bool are type hints. They document intended inputs and output for readers, editors, and static type checkers. Python doesn't enforce those hints when the function runs.^{[1]Reference 1The Python Tutorial.https://docs.python.org/3/tutorial/} Calling exact_match("approved", 7) still enters the function, then raises AttributeError because an integer has no strip() method.

That failure points to a boundary: editor-visible annotations aren't a parser. Runtime data needs runtime checks as soon as values come from a file instead of a literal in the editor.

JSONL moves the same rows to disk

The three dictionaries still live in the editor. The receipt from the Docker lesson came from a file, so those same rows have to survive a trip to disk. Store them with JSONL (JSON Lines), where each line contains one complete JSON value.

Save the stable fixture at eval/access_requests.jsonl:

{"prompt": "Access request 101 status?", "expected": "approved", "prediction": "approved"}
{"prompt": "Access request 102 status?", "expected": "blocked", "prediction": "escalated"}
{"prompt": "Access request 103 status?", "expected": "restored", "prediction": "restored"}

JSON and Python use similar punctuation, but they aren't the same language. When json.loads parses a line, Python's standard-library json module maps JSON values into Python values:

Path from the standard-library pathlib module represents a filesystem path. Opening it with encoding="utf-8" makes text decoding explicit.

Iterating over the handle yields one line at a time instead of reading one giant string.^{[1]Reference 1The Python Tutorial.https://docs.python.org/3/tutorial/} Our loader will still collect the parsed dictionaries in a list, so its memory use grows with the dataset. Streaming the scoring itself is a separate change.

Create the same fixture in the next helper cell so every marked example can run in isolation. If you followed the Git and Docker lessons, that file already exists, and running this cell recreates its three lines.

from pathlib import Path

fixture = Path("eval/access_requests.jsonl")
fixture.parent.mkdir(exist_ok=True)
fixture.write_text(
    "\n".join(
        [
            '{"prompt": "Access request 101 status?", "expected": "approved", "prediction": "approved"}',
            '{"prompt": "Access request 102 status?", "expected": "blocked", "prediction": "escalated"}',
            '{"prompt": "Access request 103 status?", "expected": "restored", "prediction": "restored"}',
        ]
    )
    + "\n",
    encoding="utf-8",
)
print("fixture rows:", len(fixture.read_text(encoding="utf-8").splitlines()))

fixture rows: 3

Read the file, then print the same receipt

Now connect the file to the receipt. Follow one row through JSON decoding, field checks, comparison, aggregation, and the 2/3 eval gate. If any boundary changes the contract, the receipt should stop matching the Git and Docker chapters.

The Git scorer rejected non-string fields before normalizing labels with .strip().lower(). Replacing that type check with str(label) would turn JSON number 7 into string "7", counting a producer bug as a wrong prediction against "approved". Refusing that row keeps invalid data out of the mismatch count.

The parser needs two kinds of checks. isinstance(value, str) checks a value's type; not value.strip() rejects a string containing no visible text. raise stops normal execution with an exception, and try/except handles a named exception. Around json.loads, that handler adds our file's line number; from error preserves the original decoding error as its cause.

View larger

Save these definitions in scripts/score_access_requests.py, replacing the earlier scorer. Square brackets in a hint describe contents: dict[str, str] has string keys and values, and list[dict[str, str]] is a list of those dictionaries.

import json
import sys
from pathlib import Path

REQUIRED_FIELDS = ("prompt", "expected", "prediction")
DEFAULT_INPUT = Path("eval/access_requests.jsonl")

def parse_example(line: str, line_number: int) -> dict[str, str]:
    try:
        row = json.loads(line)
    except json.JSONDecodeError as error:
        raise ValueError(f"line {line_number}: invalid JSON") from error

    if not isinstance(row, dict):
        raise TypeError(f"line {line_number}: expected a JSON object")

    example: dict[str, str] = {}
    for key in REQUIRED_FIELDS:
        if key not in row:
            raise KeyError(f"line {line_number}: missing {key}")

        value = row[key]
        if not isinstance(value, str):
            raise TypeError(f"line {line_number}: {key} must be a string")
        if not value.strip():
            raise ValueError(f"line {line_number}: {key} must not be empty")

        example[key] = value

    return example

def load_examples(path: Path) -> list[dict[str, str]]:
    examples = []

    with path.open("r", encoding="utf-8") as handle:
        for line_number, line in enumerate(handle, start=1):
            if line.strip():
                examples.append(parse_example(line, line_number))

    if not examples:
        raise ValueError("input file has no examples")

    return examples

def exact_match(expected: str, prediction: str) -> bool:
    return expected.strip().lower() == prediction.strip().lower()

def score_examples(examples: list[dict[str, str]]) -> tuple[int, int, float]:
    if not examples:
        raise ValueError("examples must not be empty")

    matches = 0
    for example in examples:
        if exact_match(example["expected"], example["prediction"]):
            matches += 1

    total = len(examples)
    return matches, total, matches / total

def passes_gate(matches: int, total: int) -> bool:
    return matches * 3 >= total * 2

def main(path: Path) -> None:
    examples = load_examples(path)
    matches, total, score = score_examples(examples)
    print(f"Eval rows: {total}")
    print(f"Exact-match accuracy on tiny fixture: {score:.3f} ({matches}/{total})")
    if not passes_gate(matches, total):
        print("Gate failed: score regressed below 2/3", file=sys.stderr)
        raise SystemExit(1)
    print("Gate passed. You may commit.")

Read the loader from the outside in. if runs its indented block only when the condition is True, and matches += 1 means matches = matches + 1.

The three values after return form a tuple; matches, total, score = ... unpacks it into three names. A with block closes the file even if parsing raises an exception.

The loader deliberately ignores blank lines, but never skips a malformed nonblank row. It also ignores extra object fields after copying the three required strings. These are choices in this evaluator, not rules imposed by JSON. if not examples rejects an empty list before division could use a zero denominator.

This validates fields in the decoded row. The default decoder also accepts repeated object keys and keeps the last value. A line containing two prediction keys can therefore pass these checks. If duplicate fields must be rejected, enforce that during decoding with object_pairs_hook, before a dictionary loses the earlier value.^{[4]Reference 4json: JSON encoder and decoderhttps://docs.python.org/3/library/json.html}

The file handle is an iterator that advances as the loop consumes lines. A second loop over the same handle at end-of-file sees no rows. load_examples opens a fresh handle on each call and returns a list, so the scorer can visit those loaded rows again. Don't delete or append rows while iterating to score them; build a separate result or filtered list when that is the intended operation.^{[1]Reference 1The Python Tutorial.https://docs.python.org/3/tutorial/}

passes_gate keeps the rule in integer counts, the same form Git already used: fail when matches * 3 < total * 2. For a positive total, multiplying matches / total >= 2 / 3 by 3 * total gives that exact count comparison. The caller supplies valid counts from a nonempty list; this helper doesn't validate arbitrary counts.

The saved module also needs a small command-line wrapper. sys.argv[0] is the script name, and sys.argv[1] is the optional path. Add it at the bottom so direct execution and imports have different behavior:

if __name__ == "__main__":
    if len(sys.argv) > 2:
        raise SystemExit("usage: uv run scripts/score_access_requests.py [examples.jsonl]")

    input_path = Path(sys.argv[1]) if len(sys.argv) == 2 else DEFAULT_INPUT
    main(input_path)

The if __name__ == "__main__" guard calls main when you run the file directly. Importing the functions in a test doesn't score a file as a side effect.

Run the default path once against the stable fixture. Before you do, predict the same three lines from the earlier receipt:

main(DEFAULT_INPUT)

Eval rows: 3
Exact-match accuracy on tiny fixture: 0.667 (2/3)
Gate passed. You may commit.

From the shell, both commands should print the same three-line receipt. The first uses DEFAULT_INPUT; the second proves that an explicit path follows the same route:

uv run scripts/score_access_requests.py
uv run scripts/score_access_requests.py eval/access_requests.jsonl

Trace request 102 through the boundaries

When the score surprises you, follow one value instead of staring at the final number. Request 102 is valid input, yet its labels differ. Its trace should end in one zero, not an input error.

That order gives each failure a location. A malformed line should fail while parsing, not disappear from the denominator or turn into a wrong prediction.

Errors should name the broken boundary

Suppose request 102 suddenly disappears from a run. Before changing the metric, ask whether Python ever accepted that row.

A syntax error means Python couldn't parse the program; Python reports it as a SyntaxError exception. Other exceptions, such as TypeError and FileNotFoundError, can arise while executing code that parsed successfully. The final line of a traceback gives the exception type and message; lines above it show the call path.^{[5]Reference 5Python Built-in Exceptionshttps://docs.python.org/3/library/exceptions.html}

Start at the final exception line, then work upward to the call that failed:

Catch an exception only when your code can add context, recover, or turn it into a deliberate command-line exit. Continuing after every exception would let broken rows produce a misleading metric.

The next cell deliberately sends a number where prediction must be text. Predict the boundary first: JSON decoding should succeed, field validation should reject the value, and scoring should never run.

bad_line = '{"prompt": "Access request 104 status?", "expected": "approved", "prediction": 7}'

try:
    parse_example(bad_line, line_number=4)
except TypeError as error:
    print(type(error).__name__)
    print(error)

TypeError
line 4: prediction must be a string

The exception is expected here, so catching it makes the output visible. In the real loading path, let it stop the run. That preserves the distinction between an invalid input row and a valid row the model got wrong.

Fail fast is this lab's policy: one malformed nonblank row stops the run before main prints any metric. A larger ingestion job may instead quarantine rejected rows for repair. That requires a separate record of each rejected line and reason, plus counts such as attempted=100, accepted=99, rejected=1. A score on the 99 accepted rows describes that subset. It can't be compared as though all 100 original rows were evaluated. Set a coverage rule alongside the score gate if partial evaluation is allowed.

If the row later gains a numeric confidence

The current parser accepts only the three required strings and ignores extra fields. Adding a confidence field to a future version requires a different contract: a number between zero and one. Reusing isinstance(value, (int, float)) alone has two traps. bool is a subclass of int, so it accepts True. A float may also be infinity or NaN (not a number).^{[3]Reference 3Built-in Typeshttps://docs.python.org/3/library/stdtypes.html}

Python's default JSON decoder accepts the nonstandard tokens NaN and Infinity; even the valid JSON number 1e400 becomes an infinite float. Decoding successfully doesn't establish a finite probability.^{[4]Reference 4json: JSON encoder and decoderhttps://docs.python.org/3/library/json.html}

Try this separate boundary experiment before extending the scorer. object in the hint allows any incoming value; the function checks its type, range, and finiteness before returning a trusted float. The range check comes before float conversion, so an enormous integer is rejected without overflowing the conversion.

import json
import math

def validate_confidence(value: object) -> float:
    if isinstance(value, bool) or not isinstance(value, (int, float)):
        raise TypeError("confidence must be a number, not a Boolean or string")
    if not 0 <= value <= 1:
        raise ValueError("confidence must be finite and between 0 and 1")
    confidence = float(value)
    if not math.isfinite(confidence):
        raise ValueError("confidence must be finite and between 0 and 1")
    return confidence

print("True passes an int check:", isinstance(True, int))
for raw in ("0", "0.75", "true", '"0.75"', "NaN", "Infinity", "1e400", "1.1"):
    try:
        print(raw, "->", validate_confidence(json.loads(raw)))
    except (TypeError, ValueError) as error:
        print(raw, "->", type(error).__name__)

True passes an int check: True
0 -> 0.0
0.75 -> 0.75
true -> TypeError
"0.75" -> TypeError
NaN -> ValueError
Infinity -> ValueError
1e400 -> ValueError
1.1 -> ValueError

math.isfinite excludes infinities and NaN.^{[6]Reference 6math: Mathematical functionshttps://docs.python.org/3/library/math.html} For this bounded probability, 0 <= value <= 1 already rejects them, including NaN because its ordered comparisons are false. The explicit finite check also states what the returned float promises. A future unbounded numeric field would need a finite check without this probability range. Keep rejection ahead of aggregation rather than coercing true to 1.0 or a quoted number to a float.

Keep the fraction separate from its display

The receipt displays 0.667, while the scorer stores 2 / 3. Those values look interchangeable until a gate compares them.

Python's float type uses binary floating-point arithmetic. Many fractions, including 2 / 3, have no exact finite representation. Formatting to three decimal places produces a readable report; it doesn't change the stored value.^{[1]Reference 1The Python Tutorial.https://docs.python.org/3/tutorial/}

import math

matches = 2
total = 3
score = matches / total

print("stored:", score)
print("displayed:", f"{score:.3f}")
print("equal to 0.667:", score == 0.667)
print("close to 2/3:", math.isclose(score, 2 / 3))
print("count floor:", matches * 3 >= total * 2)

stored: 0.6666666666666666
displayed: 0.667
equal to 0.667: False
close to 2/3: True
count floor: True

Keep matches and total for exact checks on this discrete metric. Use math.isclose when two independently computed floating-point results should be approximately equal. Never compare the stored score with its rounded display string.

The count floor stays exact in integer form: 2 * 3 >= 3 * 2 doesn't care that 2 / 3 isn't the literal 0.667.

Cover the contracts with pytest

The manual run proved one path once. A test suite lets you rerun the boundary decisions after every edit. Add pytest as a development dependency; uv updates pyproject.toml and uv.lock.^{[7]Reference 7pytest Documentationhttps://docs.pytest.org/}

uv add --dev pytest

Add this section to pyproject.toml, preserving the existing project fields, so pytest can import scripts.score_access_requests from the project root:

[tool.pytest.ini_options]
pythonpath = ["."]

Create a tests/ directory and save these checks as tests/test_score_access_requests.py. pytest discovers functions whose names begin with test_. An assert fails if its condition is false; pytest.raises instead expects the enclosed operation to raise the named exception and can check its message. A missing or different exception fails the test.^{[7]Reference 7pytest Documentationhttps://docs.pytest.org/}

import json
from pathlib import Path

import pytest

from scripts.score_access_requests import (
    exact_match,
    load_examples,
    parse_example,
    passes_gate,
    score_examples,
)

def test_exact_match_normalizes_case_and_whitespace():
    assert exact_match("approved", " APPROVED ")

@pytest.mark.parametrize("prediction", [7, True, None, [], {}])
def test_parse_example_rejects_non_string_prediction(prediction):
    line = json.dumps({"prompt": "Q", "expected": "approved", "prediction": prediction})
    with pytest.raises(TypeError, match="line 4: prediction must be a string"):
        parse_example(line, line_number=4)

def test_loader_preserves_physical_line_numbers(tmp_path: Path):
    path = tmp_path / "bad.jsonl"
    path.write_text('\n{"prompt": "Q", "expected": "approved"}\n', encoding="utf-8")
    with pytest.raises(KeyError, match="line 2: missing prediction"):
        load_examples(path)

def test_load_examples_rejects_empty_file(tmp_path: Path):
    empty_file = tmp_path / "empty.jsonl"
    empty_file.write_text("", encoding="utf-8")
    with pytest.raises(ValueError, match="input file has no examples"):
        load_examples(empty_file)

def test_score_keeps_exact_counts():
    examples = [
        {"prompt": "101", "expected": "approved", "prediction": "approved"},
        {"prompt": "102", "expected": "blocked", "prediction": "escalated"},
        {"prompt": "103", "expected": "restored", "prediction": "restored"},
    ]
    matches, total, score = score_examples(examples)
    assert (matches, total) == (2, 3)
    assert score == matches / total

def test_gate_uses_counts_not_the_display():
    assert passes_gate(2, 3)
    assert not passes_gate(1, 3)
    assert passes_gate(2, 2)

tmp_path is a pytest fixture: a test that requests this parameter gets its own temporary directory as a pathlib.Path.^{[7]Reference 7pytest Documentationhttps://docs.pytest.org/}

@pytest.mark.parametrize runs the same rejection check for five values, including a Boolean and null. The loader test includes a blank first line: ignoring it must not renumber the missing-field error from line 2 to line 1.

The last test is the trap. Two correct rows out of two produce 2 / 2, and the 2/3 floor still passes because 2 * 3 >= 2 * 2. Deleting the hard row improved the metric without failing the gate.

Run only this file while learning:

uv run python -m pytest tests/test_score_access_requests.py -q

Expected summary:

..........
10 passed

Each test protects a different contract. Normalization covers a valid edge case, while the parser and empty-file tests exercise deliberate failures. Exact counts stay attached to the displayed float. The gate test exposes a limit of an accuracy floor: it doesn't freeze the fixture itself.

A lockfile doesn't lock the metric

The environment can be identical while the metric changes. Remove request 102 and neither Python nor pytest needs to change for the score to jump.

View larger

On a clean checkout, verify the lockfile and then run the scorer against the committed rows:^{[2]Reference 2Locking and Syncinghttps://docs.astral.sh/uv/concepts/projects/sync/}

uv sync --locked
uv run --locked python -m pytest tests/test_score_access_requests.py -q
uv run --locked scripts/score_access_requests.py

--locked asks uv to fail instead of updating an out-of-date lockfile. A successful clean sync confirms that declared project state can create the environment.

Plain uv run can update the lockfile when project metadata changes, so use --locked on the verification commands too. By default, uv sync removes packages absent from the lockfile, while uv run leaves extra installed packages in place. Verifying in a fresh environment or syncing before the run avoids accidentally relying on a manually installed tool.^{[2]Reference 2Locking and Syncinghttps://docs.astral.sh/uv/concepts/projects/sync/}

It says nothing about whether the JSONL denominator changed. Git records the reviewed fixture, but doesn't prevent a later edit. Use git diff -- eval/access_requests.jsonl to inspect local changes, and compare against the same committed fixture when evaluating two versions of a scorer. Data, random state, hardware, and external model behavior still need their own controls.

This scorer has no randomness, which is ideal for a first lab. Later simulations and training code should create and seed their random-number generators explicitly, record the seed, and still avoid promising bit-for-bit equality across every device and library implementation.

Practice by changing one contract at a time

Use the same fixture for each exercise. Predict the failure or receipt first, make one edit, run the scorer and tests, then restore the file before the next change. Each edit should answer one question about a boundary.

1. Add request 104 with expected: "approved" and prediction: "approved".
2. Put spaces around request 101's prediction.
3. Change request 103's prediction to the JSON number 7.
4. Delete request 102's prediction key.
5. Remove a closing brace from request 101.
6. Delete request 102 entirely and leave 101 and 103 in place.

Compare your observation with this feedback. Notice which changes stop before scoring and which ones alter the denominator:

Adding a correct row raises the measured fraction to 3 / 4; it doesn't demonstrate that the model improved because the evaluation set changed. Deleting request 102 also clears the floor, but removes evidence of a mistake. Compare model versions on the same rows before attributing a score change to the model.

To see the gate fail, add a fourth row whose labels differ: 2 / 4 is below 2 / 3, so the receipt should print Gate failed.

Same habit, next on arrays

One small program now exposes every boundary. JSON text becomes a checked dictionary; a loop visits rows; a function decides exact match; exact counts become a displayed score; and a count floor decides whether the receipt may say the gate passed. Failures point to syntax, file access, JSON decoding, field validation, or metric logic instead of collapsing into one bad number.

The next lesson replaces row dictionaries with numerical arrays. Field names such as expected were the contract here. Axis names and sizes become the contract there, so indexing, broadcasting, and reductions need the same explicit checks before you trust their output.