Gradients and Backprop

Measuring engineering work turns vague performance into numbers. Training builds on the same instinct, but with a different problem: the system begins with poor numbers and must learn better ones from examples.

Suppose a coding assistant predicts response latency from prompt length. A request used 200 input tokens and finished in 5 seconds, but the current rule predicts 6. Which stored number should change, in which direction, and by how much? Gradients answer that question.

A model with one adjustable number

A parameter is a number a model is allowed to change while it learns. Start with one parameter, w, representing predicted seconds per 100 input tokens.

For one completed request:

The loss is one non-negative number. Zero means the prediction matches this row. Larger numbers mean the miss is worse.

That code evaluates a model. It doesn't yet learn. Learning requires knowing whether to move w upward or downward.

Nudge the weight to estimate a slope

Change w by a tiny amount and recompute the loss:

Loss increased by 0.004004 when the weight increased by 0.001. Divide those changes:

$$\text{slope estimate} = \frac{1.004004 - 1.000000}{0.001} = 4.004$$

A positive slope means increasing w makes loss worse near this point. To lower the loss, move w down.

This is a finite difference: a practical approximation to a derivative. It's slow if you repeat it for millions of parameters, but it's an excellent check for a small formula.

The derivative gives the exact local slope

A derivative is the slope as the nudge becomes infinitesimally small. Here the loss is:

$$L(w) = (2w - 5)^2$$

Apply the chain rule in two pieces:

$$\frac{dL}{dw} = 2(2w - 5) \times 2$$

The first factor differentiates the square. The final 2 comes from the prediction 2w: changing w by one changes the prediction by two seconds for this row.

At w = 3:

$$\frac{dL}{dw} = 2(6 - 5) \times 2 = 4$$

The finite-difference estimate was 4.004; the derivative is exactly 4 at this point.

A centered finite difference nudges left and right, so it's usually a better numerical check than a one-sided nudge.

Gradient descent takes a downhill step

The derivative tells you uphill. Gradient descent moves the opposite way. A learning rate is a small chosen step scale:

$$w_{\text{new}} = w - \text{learning rate} \times \frac{dL}{dw}$$

With w = 3, derivative 4, and learning rate 0.1:

$$w_{\text{new}} = 3 - 0.1 \times 4 = 2.6$$

Now the prediction is 2.6 * 2 = 5.2, and loss is (5.2 - 5)² = 0.04. One update reduced the loss from 1.0 to 0.04.

A correct gradient can still take a bad step

Calculus supplies direction. It doesn't choose a safe learning rate for you.

From the same starting point, a learning rate of 1.0 produces:

$$w_{\text{new}} = 3 - 1.0 \times 4 = -1$$

The new prediction is -2 seconds and loss becomes (-2 - 5)² = 49. The derivative was right; the step jumped straight past the good region.

In a real training loop, watch for non-finite loss values such as NaN or infinity. They can signal an unstable step or invalid input. Later optimization chapters will teach adaptive optimizers and schedules; first you need a backward calculation you can trust.

Several parameters mean several partial derivatives

Response latency doesn't depend on prompt length alone. Add a second parameter for queue delay:

$$\hat{y} = w_p x_p + w_q x_q$$

Here x_p is prompt length in hundreds of tokens, and x_q is queue delay in seconds. For one request:

A partial derivative changes one parameter while holding the other fixed:

$$\frac{\partial L}{\partial w_p} = 2(\hat{y} - y)x_p = 2 \times 1 \times 2 = 4$$ $$\frac{\partial L}{\partial w_q} = 2(\hat{y} - y)x_q = 2 \times 1 \times 1 = 2$$

Collect those slopes in order and you have the gradient: [4, 2]. The next article gives that list its full vector-and-shape language. For now, read it as one correction signal per stored parameter.

One row can illustrate an update, but it can't uniquely determine what prompt length and queue delay should contribute across all future requests. Learning useful weights requires several observed requests.

Backpropagation is chain-rule bookkeeping

The two-parameter gradient came from a chain of small operations:

1. Predict: prediction = w_p * x_p + w_q * x_q
2. Subtract target: error = prediction - y
3. Square error: loss = error ** 2

During the forward pass, these operations produce prediction = 6, error = 1, and loss = 1. During the backward pass, begin at the scalar loss and move backward:

Multiply along each path. Prompt length gets 2 * 1 * 2 = 4; queue delay gets 2 * 1 * 1 = 2.

Backpropagation efficiently repeats this reverse chain-rule calculation through a layered model. Rumelhart, Hinton, and Williams used back-propagated error derivatives to learn internal representations in multilayer networks in their 1986 Nature paper.^{[1]Reference 1Learning Representations by Back-Propagating Errors.https://doi.org/10.1038/323533a0}

When paths meet, add their signals

The example above sends the loss signal to two different parameters. A computation graph can also use the same parameter more than once. When backward paths meet at one parameter, add their contributions.

Use an intentionally simple latency rule to isolate that idea. The same weight w scales both the prompt-length estimate and the queue-delay estimate:

$$\hat{y} = w x_p + w x_q$$

With w = 2, x_p = 2, x_q = 1, and y = 5, prediction is 6 and error is 1. The prompt path contributes 2 * 1 * 2 = 4. The queue path contributes 2 * 1 * 1 = 2. Both paths reach w, so its gradient is 4 + 2 = 6.

This scalar rule is deliberately small. Larger models reuse parameters across many rows or token positions, and branches can merge inside a graph. The next batch example combines row contributions too, then divides by the number of rows because its loss is a mean.

Build it: learn from a small latency table

Now use four completed requests rather than one. Each row has prompt length and recorded queue delay; the target is observed response seconds. You already used NumPy arrays earlier in the curriculum, so X @ weights computes all four predictions at once.

The finite-value guard doesn't make training stable by itself. It stops the loop before a NaN or infinity update silently corrupts the weights. If it fires, inspect the learning rate and input scale first.

This is still a tiny model, not a production latency service. Its value is transparency: every prediction, loss, derivative, and update fits into code you can explain.

Gradient checking catches backward bugs

A manual backward formula can silently omit a factor, transpose the wrong array, or reuse the wrong value. Compare it to centered finite differences before trusting it.

For each parameter, nudge only that one value by eps, compute the two losses, and compare the numerical slope to the formula from your backward pass.

Finite differences are far too expensive for normal training because they need repeated forward computations per parameter. They are ideal for testing a small backward formula.

PyTorch computes the same backward pass

Reverse-mode automatic differentiation propagates chain-rule products backward from outputs to inputs. It's especially useful when one scalar loss depends on many parameters, which is the training shape we have here.^{[2]Reference 2Automatic Differentiation in Machine Learning: a Surveyhttps://www.jmlr.org/papers/v18/17-468.html}

PyTorch autograd records operations involving tensors that request gradients, then exposes the reverse pass through a call to backward().^{[3]Reference 3PyTorch: An Imperative Style, High-Performance Deep Learning Library.https://arxiv.org/abs/1912.01703} This cell repeats the small latency table and checks PyTorch against the manual NumPy derivative. It also runs backward twice to expose one important default: PyTorch adds each new result into the leaf tensor's .grad buffer.

The second backward pass doubles the stored numbers because it adds the same gradient again. A training loop normally calls optimizer.zero_grad() before the next batch, or sets buffers to None, unless accumulating across several batches is intentional.^{[4]Reference 4Optimizing Model Parameters.https://docs.pytorch.org/tutorials/beginner/basics/optimization_tutorial.html} Frameworks save you from writing every derivative. They don't remove your responsibility to understand why a gradient has its sign, shape, and size.

Practice with pencil before code

Use one latency row with x_prompt = 3, x_queue = 1, actual seconds y = 7, and weights [w_prompt, w_queue] = [2, 2].

1. Compute prediction, error, and squared loss.
2. Compute both partial derivatives by multiplying 2 * error by the relevant feature.
3. Apply one gradient-descent update with learning rate 0.05. What is the new prediction and loss?
4. If a centered finite-difference check for the prompt parameter returns 5.99999 while your manual derivative says 6, is that evidence of a bug?
5. If loss becomes NaN after a very large update, what two things should you inspect first?

Solution sketches

1. Prediction is 2*3 + 2*1 = 8, error is 8 - 7 = 1, and loss is 1.
2. The prompt partial is 2*1*3 = 6; the queue partial is 2*1*1 = 2. The gradient is [6, 2].
3. New weights are [2 - 0.05*6, 2 - 0.05*2] = [1.7, 1.9]. Prediction is 1.7*3 + 1.9 = 7.0, so loss is 0.
4. No. A numerical derivative is approximate, and agreement within a small tolerance with the same sign is a passing check.
5. Inspect the learning rate and the scale or validity of input values. A huge step or non-finite data can make the forward loss invalid before a later optimizer can help.

What you're ready for next

Training should now read as a concrete process:

We deliberately stopped before optimizer algorithms such as momentum or Adam. They control how to use gradients over many steps; a later chapter teaches them once vectors, matrices, and deeper linear algebra are available.

Mastery check

Key concepts

• A scalar loss turns prediction error into a number training can minimize.
• A derivative measures local loss sensitivity to one parameter.
• A gradient collects one partial derivative per parameter.
• Gradient descent subtracts the gradient scaled by a learning rate.
• Backpropagation applies the chain rule backward through stored operations.
• When several backward paths reach one parameter, their gradient contributions add.
• Centered finite differences and PyTorch autograd can verify a manual gradient.

Evaluation rubric

• Foundational: Calculates prediction, squared loss, and one derivative for the single-feature latency rule.
• Intermediate: Traces local chain-rule factors for the prompt-and-queue model and implements its NumPy update.
• Advanced: Diagnoses an overshooting update, validates a manual batch gradient numerically, and confirms it with PyTorch autograd.

Follow-up questions

Common pitfalls

Updating in the uphill direction

• Symptom: Loss increases immediately even with a small learning rate.
• Cause: Code adds learning_rate * gradient instead of subtracting it.
• Fix: Print one derivative and verify its sign with a tiny finite-difference nudge.

Forgetting a local chain-rule factor

• Symptom: Your derivative has the right sign but disagrees with a numerical check by a constant multiplier.
• Cause: The backward formula omitted a feature value or the 2 * error term from squared loss.
• Fix: Write the forward computation as three operations and multiply one local derivative per edge.

Trusting a step that's too large

• Symptom: Correct gradients produce exploding or non-finite loss.
• Cause: Learning rate or input scale turns a useful direction into an unsafe jump.
• Fix: Reduce step size, inspect input magnitude, and fail when loss isn't finite.

Using autograd without understanding the result

• Symptom: backward() runs, but you can't explain a zero, huge, or wrong-shaped gradient.
• Cause: The framework is hiding derivative details before the local derivatives are understood.
• Fix: Reproduce one tiny row by hand and compare its derivative with autograd before debugging a large model.

Carrying stale gradients into the next batch

• Symptom: A repeated PyTorch backward pass doubles .grad even though the batch didn't change.
• Cause: PyTorch accumulates gradients in leaf buffers until you clear them.
• Fix: Call optimizer.zero_grad() before each fresh batch, unless summing across batches is intentional.