Docker for Reproducible AI

Git carries the scorer and fixture between machines, but it can't pin Python, OS packages, runtime users, filesystem layout, or how secrets arrive. Docker records those assumptions in an image and a runtime command.^{[1]Reference 1Docker Documentation.https://docs.docker.com/}

The repo from the previous chapter already contains eval/access_requests.jsonl and a scorer that must print 0.667. This lesson packages that exact command so another laptop or CI worker uses the same version-constrained environment instead of discovering a different Python version, missing dependency, leaked .env, or volume permission error.

Check your reasoning: if two people run the same files with different Python versions and different installed packages, Git did its job but the project still isn't reproducible.

What a container must guarantee

A useful Docker setup answers six questions before a teammate has to ask them.

Git made the files reproducible. Docker makes the environment portable.

These boundaries become a small lifecycle: build inputs become image layers, then run-time data and credentials join only when the scorer starts.

The solid path is the artifact lifecycle. The dotted inputs stay outside image layers and arrive only for a specific run.

Start with .dockerignore

Create .dockerignore at the root of access-rag/, next to .gitignore:

Docker reads this file before sending the build context to the daemon. Without it, a broad COPY . /app can accidentally send .env, cached model files, local indexes, notebooks, and virtualenvs into the build. That makes images slower to build and easier to leak.

The filter is simple in principle: if a path matches .dockerignore, it never enters the build context.

Reuse the scorer you already tested

The previous chapter committed scripts/score_access_requests.py and eval/access_requests.jsonl. Keep those as the single scorer and fixture instead of creating a second implementation for Docker. The image also expects requirements.txt; it can stay empty until the scorer gains external dependencies.

Build the smallest runnable scorer image

Start with a CPU image. That choice is deliberate. The three-row scorer doesn't need a GPU, and a beginner should be able to prove the container contract on a normal laptop before adding NVIDIA runtime setup.

Create this Dockerfile:

Dependencies install into /opt/venv in the first stage. A clean second stage copies the virtualenv, scorer, and fixture, then runs as appuser instead of root. Its entrypoint calls the same scorer the Git gate used.

The official Python image manifest maps python:3.12-slim-trixie to the Python 3.12 line on Debian 13 (trixie).^{[2]Reference 2Official Python Image Manifesthttps://github.com/docker-library/official-images/blob/master/library/python[3]Reference 3Debian Trixie Release Informationhttps://www.debian.org/releases/trixie/} This constrains the Python minor line and Debian release, but it doesn't pin exact bytes. Image tags are mutable, so a later rebuild can pick up a new Python patch release or rebuilt OS packages.^{[4]Reference 4Building Best Practiceshttps://docs.docker.com/build/building/best-practices/#pin-base-image-versions} Use a digest (python:3.12-slim-trixie@sha256:<verified-digest>) when the same base-image bytes must resolve on every machine and in CI. Lock application dependencies, including any build tools you add, and use hashes when the project requires repeatable package artifacts.

Later, when you add dependencies, package changes invalidate the dependency layer instead of the code layer. Editing scripts/score_access_requests.py won't force a full package reinstall.

Docker layer caching follows the same order rule: copy requirements.txt and install dependencies before application code so routine code edits don't bust the dependency layer.

Why does the Dockerfile copy requirements.txt before the scorer source?

Answer

Dependency installation stays in an earlier cacheable layer. Editing scorer code can then reuse the dependency layer instead of reinstalling every package.

Build and run it

From the project root:

Run the image without any host mount first:

Expected output:

The printed score is rounded for humans. Keep the gate comparison on the exact fraction, as this tiny check does:

Verify copied and mounted data

The image ID is not the proof. The proof is that the same three rows produce the same 0.667 after moving into the container. Exact rebuilds also require a base-image digest and locked dependency artifacts.

What proves more than the image ID when checking this containerized scorer?

Answer

Running the image against the three-row fixture and observing 0.667 (2/3). That receipt checks the command, code, data, and runtime together.

Run the image against host data:

A bind mount replaces the image path for that run; Docker doesn't merge the host and image directories. If the score changes, compare the mounted fixture with the committed one before changing the image.

Practice: break one thing on purpose

After the first successful run, make one small mistake and predict the symptom before you rerun the command.

These failures are useful. Each one proves which part of the contract you were relying on.

Compose keeps the local command stable

For one service, docker run is fine. As soon as the project adds a vector database, API, worker, or model cache, you want one checked-in Compose file so each engineer starts the same stack.

Start with a runnable docker-compose.yml for the scorer:

Validate the file before running it:

Docker Desktop includes Compose v2. Some Linux installs need the docker-compose-plugin package first.^{[1]Reference 1Docker Documentation.https://docs.docker.com/} If docker compose version is unknown, the Dockerfile is still usable through docker run, but the Compose workflow isn't installed yet. Note two modern conventions: the old top-level version: key is obsolete and Compose v2 ignores it (omit it, as above), and the preferred filename is now compose.yaml, though docker-compose.yml still works. Always use the hyphen-free docker compose command; the legacy docker-compose v1 binary is end-of-life.

When the RAG stack grows, add vector-db, api, and worker services to this same file. Don't add fake services before they exist. A Compose file that starts today is better than an impressive YAML file that fails on the first command.

Secrets belong at runtime

Don't pass real secrets with ARG:

Use runtime environment instead:

Compose uses the same idea:

The secret is available to the process when the container runs. It isn't copied into the image, pushed to the registry, or shown by docker history.

Build context and runtime env are separate channels. A secret should appear only in the runtime channel.

Why should an API key enter through runtime environment configuration instead of COPY or a Docker build argument?

Answer

Runtime injection keeps the key out of image layers, registry artifacts, and docker history. .dockerignore also prevents the local secret file from entering the build context.

When the GPU enters the story

The CPU-first image is the right base contract here. It works on Linux, macOS, CI runners, and most developer laptops. A CUDA image is different: it targets Linux hosts with NVIDIA drivers and the NVIDIA Container Toolkit, which installs a runtime hook so the container can see the host GPU. Configure the runtime, restart Docker so the daemon loads the change, then pass --gpus at run time.^{[5]Reference 5NVIDIA Container Toolkit Installation Guidehttps://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html}

When a later PyTorch or inference chapter needs GPU acceleration, keep the same contracts and change the platform-specific pieces:

Don't promise that one CUDA image gives Apple Silicon and NVIDIA parity. On Apple Silicon, use the CPU image for this foundation scorer or a separate Metal/MPS path for PyTorch. For production GPU serving, test the Linux NVIDIA image on a host that has the NVIDIA runtime.

The first GPU smoke test is:

If that fails, the Dockerfile isn't the problem yet. The host can't expose the GPU to containers.

That CUDA tag is illustrative; NVIDIA publishes new CUDA versions often, so pin to whatever release your PyTorch wheel targets rather than copying the number here. One subtle trap: the container's CUDA runtime must not be newer than the host's NVIDIA driver supports, or the container fails with CUDA driver version is insufficient for CUDA runtime version. That mismatch lives on the host, not in your Dockerfile.

The gate that protects the contract

Add a local or CI gate that proves the image still builds and the score still matches:

This catches broken COPY paths, missing files, invalid Compose syntax, and scorer drift before a teammate pulls the repo. If your machine doesn't have the Compose plugin yet, keep the docker build and docker run lines as the mandatory gate, then install Compose before using the Compose part.

Failure modes to keep ready

The important habit isn't memorizing each Docker flag. It's making the repo contain the diagnosis, the command, and the prevention.

The runtime contract

The runtime contract is explicit:

1. git clone brings the code, the three-row eval, the scorer, the Dockerfile, .dockerignore, and docker-compose.yml.
2. docker build -t access-rag:local . produces a version-constrained Python runtime. Add a verified base-image digest and locked dependency artifacts when exact rebuilds are required.
3. docker run --rm access-rag:local returns the expected 0.667 from the starter scorer.
4. docker compose run --rm scorer gives teammates one stable local command.
5. Later GPU images, vector databases, API services, workers, and Cloud Run deployments must preserve the same habit: explicit runtime constraints, mounted data, runtime secrets, and a gate that proves the expected output.

The next chapter opens the Python scorer instead of leaving it hidden behind a command. Docker matters there because code bugs are hard enough without also wondering whether two machines are running different Python environments.

Reproducibility release checklist

Before publishing the image, build from a clean context, run the baked fixture and read-only mounted fixture, confirm both print 0.667 (2/3), inspect that no .env entered image history, and validate Compose configuration. For exact rebuilds, also pin the base image by digest and lock dependency artifacts.