Python for AI Engineering

The Docker image from the previous lesson printed a passing evaluation report. Two predictions matched their expected labels; one didn't:

Git stored eval/access_requests.jsonl and scripts/score_access_requests.py. Docker supplied the operating system files and Python runtime, but neither one explains how a line of text becomes 2/3.

That path is ordinary Python: a file becomes rows, rows become counts, and counts become a printed score. Rebuild it with each boundary visible.^{[1]Reference 1The Python Tutorial.https://docs.python.org/3/tutorial/}

= 3 * 2." />

Pin a local Python 3.12 project

The image already pins one runtime path. Before changing the scorer, give your checkout a matching interpreter and a separate place for its tools. Otherwise a passing test can depend on whichever Python and packages happen to be first on your PATH.

A virtual environment provides a project-local package directory separate from the operating system's Python. Local work here uses uv's project workflow for that isolation.

Keep the lab on Python 3.12 because that's what the Docker image already pins, not because 3.12 is the newest CPython release.

Confirm uv --version works. If it doesn't, use the official uv installation guide for your operating system. Run these commands from your earlier lab repository, not from an unrelated project. Skip uv init if pyproject.toml already exists:

Read the files after running these commands. uv init --bare writes only pyproject.toml; its requires-python field declares compatible Python versions. A 3.12 pin selects that minor release family, not one exact patch release.

uv python pin 3.12 writes .python-version. The first uv sync or uv run creates .venv and uv.lock.^{[2]Reference 2Locking and Syncinghttps://docs.astral.sh/uv/concepts/projects/sync/}

Each path answers a different reproducibility question. Keep the portable inputs in Git and recreate machine-local output:

uv run uses the project environment without shell activation. If you activate it on macOS or Linux, run source .venv/bin/activate; on Windows PowerShell, run .venv\Scripts\Activate.ps1. Activation changes which python your shell finds.

uv sync creates the environment and resolves its dependencies. Activation is a shell convenience, not dependency management.

Python also ships venv. In a project that doesn't use uv, python3.12 -m venv .venv creates the same kind of isolation. Pick one environment workflow per project so package state stays understandable. Keep Docker's requirements.txt; don't replace the image install path used by the earlier scorer.

Why shouldn't .venv/ be committed even though uv.lock should be?

Answer

The lockfile is portable input that records resolved dependencies. .venv/ contains machine-specific installed files and can be recreated from the project files with uv sync.

One row is ordinary Python data

Start with request 102, the row the scorer should mark wrong. Before writing a loop, predict what one row must contain and what the comparison should return:

After parsing, this JSON object becomes a Python dictionary. A dictionary maps keys such as "expected" to values such as "blocked", so row["expected"] retrieves one named field. Assigning the mapping to row gives later operations a stable name.

The next cell reads both labels, normalizes outer whitespace and case, then stores the comparison as a Boolean. Its output lets you check the type and the decision before any batch logic appears.

Python evaluates the right side of = first, then binds that value to the name on the left. A single = assigns; == compares.

strip() removes whitespace from both ends, and lower() returns a lowercase string. Neither method edits the original string. Strings are immutable: operations return results instead of changing their contents in place.

Here, exact match means equality after these two normalization steps. It's appropriate for the fixture's short status labels. It won't recognize two differently worded explanations as equivalent, and it doesn't remove spaces inside a label.

Built-in values used in the scorer

You don't need every Python type before writing useful code. These eight cover the values moving through this evaluator:

Python is dynamically typed: a name can be rebound to a value of another type. That flexibility doesn't make types interchangeable. "2" is text, 2 is an integer, and 2.0 is a floating-point number. Convert with int(), float(), or str() only when the data contract says the conversion is valid. A malformed label shouldn't become a value that happens to score.

A list turns one row into a batch

The protected evaluation fixture contains three dictionaries in order. Before running the batch, predict the receipt: two labels match, one differs, and the denominator stays at three.

A list keeps those rows together, and a for loop visits each row once. Indentation marks the statements that belong to the loop. The code records each Boolean before it computes an aggregate, so a surprising score still has a per-row trail.

enumerate(..., start=1) supplies a readable position with each row, while append() keeps one Boolean for that row.

Then len() counts all results and sum() treats True as 1 and False as 0, producing the correct-row count. An f-string such as f"{score:.3f}" formats the fraction for display after the calculation is done.

Check the arithmetic by hand before trusting the output:

$$\text{accuracy} = \frac{\text{correct rows}}{\text{all rows}} = \frac{2}{3} \approx 0.667$$

If the code prints anything else for these exact three rows, inspect the per-row Booleans first. A mismatch there points to normalization; matching Booleans with a wrong total points to aggregation.

After adding a fourth row whose prediction matches its expected label, what should the score become?

Answer

There are now three correct rows out of four, so the score becomes 3 / 4 = 0.75.

Functions give the comparison a name

The loop works, but comparison, aggregation, and printing are tangled together. Pull the comparison out first. A function packages one operation behind a name: inputs arrive as parameters, and return sends a value back to the caller.

Ask what should stay true if another caller needs the same rule. Both labels should be stripped and lowercased, and the function should return only the Boolean decision. This function owns that normalization rule:

expected: str and -> bool are type hints. They document intended inputs and output for readers, editors, and static type checkers. Python doesn't enforce those hints when the function runs.^{[1]Reference 1The Python Tutorial.https://docs.python.org/3/tutorial/} Calling exact_match("approved", 7) still enters the function, then raises AttributeError because an integer has no strip() method.

That failure points to a boundary: editor-visible annotations aren't a parser. Runtime data needs runtime checks as soon as values come from a file instead of a literal in the editor.

JSONL moves the same rows to disk

Hardcoded lists are useful for learning, but an evaluator must read the rows it was given. Store the same three requests in a file using JSONL (JSON Lines), where each line contains one complete JSON value.

Save the stable fixture at eval/access_requests.jsonl:

JSON and Python use similar punctuation, but they aren't the same language. When json.loads parses a line, Python's standard-library json module maps JSON values into Python values:

Path from the standard-library pathlib module represents a filesystem path. Opening it with encoding="utf-8" makes text decoding explicit.

Iterating over the handle yields one line at a time instead of reading one giant string.^{[1]Reference 1The Python Tutorial.https://docs.python.org/3/tutorial/} Our loader will still collect the parsed dictionaries in a list, so its memory use grows with the dataset. Streaming the scoring itself is a separate change.

Create the same fixture in the next helper cell so every marked example can run in isolation. If you followed the Git and Docker lessons, that file already exists, and running this cell recreates its three lines.

Read the file, then print the same receipt

Now connect the file to the receipt. Follow one row through JSON decoding, field checks, comparison, aggregation, and the 2/3 eval gate. If any boundary changes the contract, the receipt should stop matching the Git and Docker chapters.

The Git scorer rejected non-string fields before normalizing labels with .strip().lower(). Replacing that type check with str(label) would turn JSON number 7 into string "7", counting a producer bug as a wrong prediction against "approved". Refusing that row keeps invalid data out of the mismatch count.

The parser needs two kinds of checks. isinstance(value, str) checks a value's type; not value.strip() rejects a string containing no visible text. raise stops normal execution with an exception, and try/except handles a named exception. Around json.loads, that handler adds our file's line number; from error preserves the original decoding error as its cause.

Save these definitions in scripts/score_access_requests.py, replacing the earlier scorer. Square brackets in a hint describe contents: dict[str, str] has string keys and values, and list[dict[str, str]] is a list of those dictionaries.

Read the loader from the outside in. if runs its indented block only when the condition is True, and matches += 1 means matches = matches + 1.

The three values after return form a tuple; matches, total, score = ... unpacks it into three names. A with block closes the file even if parsing raises an exception.

The loader deliberately ignores blank lines, but never skips a malformed nonblank row. It also ignores extra object fields after copying the three required strings. These are choices in this evaluator, not rules imposed by JSON. if not examples rejects an empty list before division could use a zero denominator.

passes_gate keeps the rule in integer counts, the same form Git already used: fail when matches * 3 < total * 2. For a positive total, multiplying matches / total >= 2 / 3 by 3 * total gives that exact count comparison. The caller supplies valid counts from a nonempty list; this helper doesn't validate arbitrary counts.

The saved module also needs a small command-line wrapper. sys.argv[0] is the script name, and sys.argv[1] is the optional path. Add it at the bottom so direct execution and imports have different behavior:

The if __name__ == "__main__" guard calls main when you run the file directly. Importing the functions in a test doesn't score a file as a side effect.

Run the default path once against the stable fixture. Before you do, predict the same three lines from the earlier receipt:

From the shell, both commands should print the same three-line receipt. The first uses DEFAULT_INPUT; the second proves that an explicit path follows the same route:

Trace request 102 through the boundaries

When the score surprises you, follow one value instead of staring at the final number. Request 102 is valid input, yet its labels differ. Its trace should end in one zero, not an input error.

That order gives each failure a location. A malformed line should fail while parsing, not disappear from the denominator or turn into a wrong prediction.

Errors should name the broken boundary

Suppose request 102 suddenly disappears from a run. Before changing the metric, ask whether Python ever accepted that row.

A syntax error means Python couldn't parse the program. An exception means valid Python hit a problem while running. The final line of a traceback gives the exception type and message; lines above it show the call path.^{[1]Reference 1The Python Tutorial.https://docs.python.org/3/tutorial/}

Start at the final exception line, then work upward to the call that failed:

Catch an exception only when your code can add context, recover, or turn it into a deliberate command-line exit. Continuing after every exception would let broken rows produce a misleading metric.

The next cell deliberately sends a number where prediction must be text. Predict the boundary first: JSON decoding should succeed, field validation should reject the value, and scoring should never run.

The exception is expected here, so catching it makes the output visible. In the real loading path, let it stop the run. That preserves the distinction between an invalid input row and a valid row the model got wrong.

Keep the fraction separate from its display

The receipt displays 0.667, while the scorer stores 2 / 3. Those values look interchangeable until a gate compares them.

Python's float type uses binary floating-point arithmetic. Many fractions, including 2 / 3, have no exact finite representation. Formatting to three decimal places produces a readable report; it doesn't change the stored value.^{[1]Reference 1The Python Tutorial.https://docs.python.org/3/tutorial/}

Keep matches and total for exact checks on this discrete metric. Use math.isclose when two independently computed floating-point results should be approximately equal. Never compare the stored score with its rounded display string.

The count floor stays exact in integer form: 2 * 3 >= 3 * 2 doesn't care that 2 / 3 isn't the literal 0.667.

Cover the contracts with pytest

The manual run proved one path once. A test suite lets you rerun the boundary decisions after every edit. Add pytest as a development dependency; uv updates pyproject.toml and uv.lock.^{[3]Reference 3pytest Documentationhttps://docs.pytest.org/}

Add this section to pyproject.toml, preserving the existing project fields, so pytest can import scripts.score_access_requests from the project root:

Create a tests/ directory and save these checks as tests/test_score_access_requests.py. pytest discovers functions whose names begin with test_. An assert fails if its condition is false; pytest.raises instead expects the enclosed operation to raise the named exception and can check its message. A missing or different exception fails the test.^{[3]Reference 3pytest Documentationhttps://docs.pytest.org/}

tmp_path is a pytest fixture: a test that requests this parameter gets its own temporary directory as a pathlib.Path.^{[3]Reference 3pytest Documentationhttps://docs.pytest.org/}

The last test is the trap. Two correct rows out of two produce 2 / 2, and the 2/3 floor still passes because 2 * 3 >= 2 * 2. Deleting the hard row improved the metric without failing the gate.

Run only this file while learning:

Expected summary:

Each test protects a different contract. Normalization covers a valid edge case, while the parser and empty-file tests exercise deliberate failures. Exact counts stay attached to the displayed float. The gate test exposes a limit of an accuracy floor: it doesn't freeze the fixture itself.

A lockfile doesn't lock the metric

The environment can be identical while the metric changes. Remove request 102 and neither Python nor pytest needs to change for the score to jump.

On a clean checkout, verify the lockfile and then run the scorer against the committed rows:^{[2]Reference 2Locking and Syncinghttps://docs.astral.sh/uv/concepts/projects/sync/}

--locked asks uv to fail instead of updating an out-of-date lockfile. A successful clean sync confirms that declared project state can create the environment.

It says nothing about whether the JSONL denominator changed. Git records the reviewed fixture, but doesn't prevent a later edit. Use git diff -- eval/access_requests.jsonl to inspect local changes, and compare against the same committed fixture when evaluating two versions of a scorer. Data, random state, hardware, and external model behavior still need their own controls.

This scorer has no randomness, which is ideal for a first lab. Later simulations and training code should create and seed their random-number generators explicitly, record the seed, and still avoid promising bit-for-bit equality across every device and library implementation.

Practice by changing one contract at a time

Use the same fixture for each exercise. Predict the failure or receipt first, make one edit, run the scorer and tests, then restore the file before the next change. Each edit should answer one question about a boundary.

1. Add request 104 with expected: "approved" and prediction: "approved".
2. Put spaces around request 101's prediction.
3. Change request 103's prediction to the JSON number 7.
4. Delete request 102's prediction key.
5. Remove a closing brace from request 101.
6. Delete request 102 entirely and leave 101 and 103 in place.

Compare your observation with this feedback. Notice which changes stop before scoring and which ones alter the denominator:

Adding a correct row raises the measured fraction to 3 / 4; it doesn't demonstrate that the model improved because the evaluation set changed. Deleting request 102 also clears the floor, but removes evidence of a mistake. Compare model versions on the same rows before attributing a score change to the model.

To see the gate fail, add a fourth row whose labels differ: 2 / 4 is below 2 / 3, so the receipt should print Gate failed.

Why is silently skipping a malformed row worse than stopping the run?

Answer

Skipping changes the denominator and can make the metric look better without disclosing missing data. Stopping keeps input failure separate from model quality.

Same habit, next on arrays

One small program now exposes every boundary. JSON text becomes a checked dictionary; a loop visits rows; a function decides exact match; exact counts become a displayed score; and a count floor decides whether the receipt may say the gate passed. Failures point to syntax, file access, JSON decoding, field validation, or metric logic instead of collapsing into one bad number.

The next lesson replaces row dictionaries with numerical arrays. Field names such as expected were the contract here. Axis names and sizes become the contract there, so indexing, broadcasting, and reductions need the same explicit checks before you trust their output.