Docker for Reproducible AI

Most AI engineering still fails at the boundary between "my laptop" and "any other machine". Git can keep files in sync, but it can't pin Python version, OS packages, runtime users, filesystem layout, or how secrets arrive. Docker matters because it turns those machine assumptions into one portable runtime contract that teammates, CI, and production can all run the same way.

You finished the Git chapter. The repo now has a tiny starter shape: eval/access_requests.jsonl, shell guards around repo hygiene, and a visible rule: three access-request rows go in, exact-match accuracy should come out as 0.667. You don't need to understand each line of the scorer yet. The later Python chapter will build that code carefully from scratch. Here, the job is narrower: make sure the same command runs in the same environment on each machine.

• Git keeps files aligned.
• Docker keeps runtime aligned.

Files plus shell guards still leave one gap: runtime drift. Another engineer can clone the same repo and hit a different Python version, a missing dependency, a leaked .env, or a volume permission error. Later, the same project will add embeddings, a vector database, and a serving API. If the runtime isn't pinned early, later chapters get harder to reproduce.

Docker adds the next contract: the repo now states both the files and the runtime that runs them.^{[1]Reference 1Docker Documentation.https://docs.docker.com/[2]Reference 2NVIDIA PyTorch Container Release Noteshttps://docs.nvidia.com/deeplearning/frameworks/pytorch-release-notes/rel-25-06.html}

Check your reasoning: if two people run the same files with different Python versions and different installed packages, Git did its job but the project still isn't reproducible.

Git locked the files. Docker locks Python version, packages, users, mounts, and secrets so the same command runs the same way elsewhere. The later Python chapter will harden the implementation and the checks around it.

What a container must guarantee

A useful Docker setup answers six questions before a teammate has to ask them.

This is the same idea as the previous chapter. Git made the files reproducible. Docker makes the environment portable. The later Python chapter will make the implementation and tests explicit.

Start with .dockerignore

Create .dockerignore at the root of access-rag/, next to .gitignore:

Docker reads this file before sending the build context to the daemon. Without it, a broad COPY . /app can accidentally send .env, cached model files, local indexes, notebooks, and virtualenvs into the build. That makes images slower to build and easier to leak.

Add the tiny files the image expects

The Dockerfile below expects a dependency file and an importable scorer module. Create both before you build the image.

For now, requirements.txt can be empty. The file still matters because the Docker layer cache will treat dependency changes separately from code changes.

Add the three-row fixture if it isn't already present from the previous chapter:

Now add a small starter scorer. The Python chapter later in the course will turn this into a cleaner typed module; this version exists so Docker has something real to run today.

Build the smallest runnable scorer image

Start with a CPU image. That choice is deliberate. The three-row scorer doesn't need a GPU, and a beginner should be able to prove the container contract on a normal laptop before adding NVIDIA runtime setup.

Create this Dockerfile:

The first stage installs dependencies into /opt/venv. The second stage starts from a clean Python image, copies the virtualenv plus the tiny app files, and runs as appuser instead of root.

The -slim-trixie suffix pins both the Python version and the OS: slim drops build tooling and docs for a smaller surface, and trixie names Debian 13, the current base for the official Python images since its release in August 2025.^{[1]Reference 1Docker Documentation.https://docs.docker.com/} Pin the distribution explicitly rather than relying on a bare python:3.12-slim; a floating tag silently follows whatever Debian release is newest and can shift system libraries (OpenSSL, glibc, compilers) under your code. For reproducibility that must survive months, go one step further and pin the image digest (python:3.12-slim-trixie@sha256:...) so the same bytes resolve on every machine and in CI.

Later, when you add FastAPI, Chroma, tokenizers, or PyTorch, dependency changes invalidate the dependency layer instead of the code layer. Editing scripts/score.py won't force a full package reinstall.

Build and run it

From the project root:

Run the image without any host mount first:

Expected output:

Worked reasoning: why the score survives the image

The image ID isn't the evidence. The evidence is that the same three rows produce the same 0.667 after the code has moved into a pinned container that any machine can rebuild.

Work the tiny eval by hand before trusting the tool:

That gives 2 correct rows out of 3. The arithmetic is:

This small distinction matters. A bad gate might compare the raw float to 0.667 and fail because 2 / 3 is mathematically 0.666.... A better gate checks the count (correct >= 2) or compares against the exact fraction (accuracy >= 2 / 3). The container doesn't fix bad scoring logic. It makes bad scoring logic easier to reproduce.

Now run it with the eval directory mounted with :ro. The same image should score data supplied at runtime:

The output should still be 0.667. If it isn't, you have a real reproducibility bug: either the mounted file differs from the copied fixture, the scorer depends on host state, or the command points at the wrong path.

Practice: break one thing on purpose

After the first successful run, make one small mistake and predict the symptom before you rerun the command.

These failures are useful. Each one proves which part of the contract you were relying on.

Compose keeps the local command stable

For one service, docker run is fine. As soon as the project adds a vector database, API, worker, or model cache, you want one checked-in Compose file so each engineer starts the same stack.

Start with a runnable docker-compose.yml for the scorer:

Validate the file before running it:

Docker Desktop includes Compose v2. Some Linux installs need the docker-compose-plugin package first.^{[1]Reference 1Docker Documentation.https://docs.docker.com/} If docker compose version is unknown, the Dockerfile is still usable through docker run, but the Compose workflow isn't installed yet. Note two modern conventions: the old top-level version: key is obsolete and Compose v2 ignores it (omit it, as above), and the preferred filename is now compose.yaml, though docker-compose.yml still works. Always use the hyphen-free docker compose command; the legacy docker-compose v1 binary is end-of-life.

When the RAG stack grows, add vector-db, api, and worker services to this same file. Don't add fake services before they exist. A Compose file that starts today is better than an impressive YAML file that fails on the first command.

Secrets belong at runtime

Don't pass real secrets with ARG:

Use runtime environment instead:

Compose uses the same idea:

The secret is available to the process when the container runs. It isn't copied into the image, pushed to the registry, or shown by docker history.

When the GPU enters the story

The CPU-first image is the right base contract here. It works on Linux, macOS, CI runners, and most developer laptops. A CUDA image is different: it targets Linux hosts with NVIDIA drivers and the NVIDIA Container Toolkit, which installs a runtime hook so the container can see the host GPU. You configure it once with sudo nvidia-ctk runtime configure --runtime=docker, then pass --gpus at run time.^{[2]Reference 2NVIDIA PyTorch Container Release Noteshttps://docs.nvidia.com/deeplearning/frameworks/pytorch-release-notes/rel-25-06.html}

When a later PyTorch or inference chapter needs GPU acceleration, keep the same contracts and change the platform-specific pieces:

Don't promise that one CUDA image gives Apple Silicon and NVIDIA parity. On Apple Silicon, use the CPU image for this foundation scorer or a separate Metal/MPS path for PyTorch. For production GPU serving, test the Linux NVIDIA image on a host that has the NVIDIA runtime.

The first GPU smoke test is:

If that fails, the Dockerfile isn't the problem yet. The host can't expose the GPU to containers.

That CUDA tag is illustrative; NVIDIA publishes new CUDA versions often, so pin to whatever release your PyTorch wheel targets rather than copying the number here. One subtle trap: the container's CUDA runtime must not be newer than the host's NVIDIA driver supports, or the container fails with CUDA driver version is insufficient for CUDA runtime version. That mismatch lives on the host, not in your Dockerfile.

The gate that protects the contract

Add a local or CI gate that proves the image still builds and the score still matches:

This catches broken COPY paths, missing files, invalid Compose syntax, and scorer drift before a teammate pulls the repo. If your machine doesn't have the Compose plugin yet, keep the docker build and docker run lines as the mandatory gate, then install Compose before using the Compose part.

Failure modes worth memorizing

The important habit isn't memorizing each Docker flag. It's making the repo contain the diagnosis, the command, and the prevention.

The runtime contract

The runtime contract is explicit:

1. git clone brings the code, the three-row eval, the scorer, the Dockerfile, .dockerignore, and docker-compose.yml.
2. docker build -t access-rag:local . produces a pinned Python runtime.
3. docker run --rm access-rag:local returns the expected 0.667 from the starter scorer.
4. docker compose run --rm scorer gives teammates one stable local command.
5. Later GPU images, vector databases, API services, workers, and Cloud Run deployments must preserve the same habit: pinned runtime, mounted data, runtime secrets, and a gate that proves the expected output.

The next chapter opens the Python scorer instead of leaving it hidden behind a command. Docker matters there because code bugs are hard enough without also wondering whether two machines are running different Python environments.

Mastery check

Key concepts

• multi-stage Dockerfile (builder vs runtime) for a slim Python image
• .dockerignore patterns for AI projects
• volume mounts (-v) and bind mounts for eval fixtures
• docker compose for a stable local command
• secrets management (env_file, --env-file, no ARG or baked secrets)
• non-root users and layer cache optimization
• reproducible build across laptops and CI
• GPU container failures as a later platform-specific extension

Evaluation rubric

• Foundational: Writes a correct multi-stage Dockerfile and .dockerignore that builds a slim runtime image containing the three-row eval scorer and produces 0.667 when run with or without the eval volume mount
• Intermediate: Authors a docker-compose.yml that validates with docker compose config --quiet, runs the scorer through one stable local command, and keeps secrets in runtime env files
• Advanced: Diagnoses and prevents common 'container worked on my laptop' failures: missing dependencies, secret leaks, volume permission problems, layer cache thrash, and premature GPU assumptions

Follow-up questions

Common pitfalls

• Using a single-stage Dockerfile that leaves build tools and the entire context in the final image. The container is larger than necessary and harder to inspect.
• COPY . /app before installing requirements.txt so each code change invalidates the dependency layer; teammates wait for package installs on each rebuild.
• Passing secrets with --build-arg OPENAI_API_KEY or baking .env into the image; the key now lives in each layer and in the registry, and docker history exposes it.
• Running the container as root and mounting host directories; permission denied errors or root-owned output files appear on teammate machines.
• Making the first Docker lesson require NVIDIA runtime setup even though the scorer runs on CPU; the beginner doesn't prove the basic container contract.