Probability for Machine Learning

An abuse detector flags an API signup for manual review. It catches 95 percent of abusive signups, but that doesn't mean a flagged signup is 95 percent likely to be abusive. You also need the abuse rate in the traffic population and the detector's false-alarm rate.

Probability makes those quantities precise: name the event, population, and evidence, then count inside the right denominator.^{[1]Reference 1Machine Learning: A Probabilistic Perspective.https://probml.github.io/pml-book/book0.html[2]Reference 2Pattern Recognition and Machine Learning.https://www.microsoft.com/en-us/research/publication/pattern-recognition-machine-learning/[3]Reference 3Deep Learning.https://www.deeplearningbook.org/} The same discipline later applies to score calibration and token probabilities.

Before the detector runs, you need a world to count in.

If any of those pieces are missing, the number is floating. Floating numbers are how teams turn model scores into bad product decisions.

Start with the population

Use 10,000 recent signups from the same API signup surface.

The probability of an abusive signup is a fraction:

$$P(\text{abusive}) = \frac{100}{10000} = 0.01$$

Read that as: before the detector says anything, 1 percent of signups are abusive.

That starting probability is the prior. A prior isn't a guess pulled from nowhere. In an engineering system, it should usually come from a measured population: production logs, labeled eval data, reviewed tickets, or another concrete sample.

In probability language, a random variable is a number that depends on which case you happened to pick. If we let F = 1 when a signup is abusive and F = 0 when it's clean, the expectation E[F] is the long-run average value, each outcome weighted by its probability:

$$E[F] = 1 \times 0.01 + 0 \times 0.99 = 0.01$$

That 0.01 is the base rate written as an average instead of a fraction. A 0/1 variable like this is called a Bernoulli variable, and its expectation is always just the probability of the 1 outcome.

Expectation alone hides how much outcomes move around. Variance measures the average squared distance from the expectation, Var[F] = E[(F - E[F])^2]. For a Bernoulli variable with success probability p, it simplifies to p(1 - p):

$$\text{Var}[F] = 0.01 \times (1 - 0.01) = 0.0099$$

Variance names outcome-level spread around the average. Later statistics chapters separate that spread from uncertainty in an estimated rate and from variation across product slices or repeated training runs.

The count table can be written as a Bernoulli indicator. 1 means an abusive signup and 0 means a clean signup. The mean of that indicator is the prior.

Evidence changes the question

Now add the detector.

That top row is strong. If a signup is abusive, the detector catches it 95 percent of the time.

But product teams need a different decision after seeing a flag:

Given that this signup was flagged, how likely is it abusive?

That's a different question. Probability notation makes the difference visible:

Those two lines aren't interchangeable. Reversing them is one of the most common probability mistakes in ML systems.

Count the flagged pile

Work from the 10,000 signups. Don't start with Bayes rule yet.

Abusive signups:

$$100 \times 0.95 = 95$$

Clean signups:

$$9900 \times 0.05 = 495$$

Now put the flagged signups into one pile.

The false-positive rate is small, but it acts on the huge clean pile. Five percent of 9,900 is larger than 95 percent of 100.

That's why the posterior is surprising:

$$P(\text{abusive} \mid \text{flagged}) = \frac{95}{590} \approx 0.161$$

A flagged signup is about 16 percent likely to be abusive, not 95 percent likely.

The detector didn't become bad. The question changed. We stopped asking how often abusive signups are caught and started asking what lives inside the flagged pile.

Turn the arithmetic into a confusion-table calculation. Notice that the program prints counts before it prints the posterior.

Conditioning means narrowing the world

Conditional probability always narrows the world first.

For $P(\text{abusive} \mid \text{flagged})$, the denominator isn't all 10,000 signups. The denominator is only the 590 flagged signups.

That's the whole mental move. Ask "among which cases?" before you divide.

This is the same flow as the diagram:

When a probability problem feels abstract, draw that flow. The formula should summarize the drawing, not replace it.

Joint and marginal probability

Two more words tie the counts together.

A joint probability is the chance that two things happen together. In the population, 95 signups are both abusive and flagged, so:

$$P(\text{abusive and flagged}) = \frac{95}{10000} = 0.0095$$

A marginal probability is the chance of one event by itself, ignoring the other. The marginal probability of a flag adds up every way a flag can happen:

$$P(\text{flagged}) = \frac{95 + 495}{10000} = \frac{590}{10000} = 0.059$$

Joint, conditional, and marginal probabilities are linked by one identity. When $P(B) > 0$, conditional probability is the joint divided by the world you conditioned on:

$$P(A \mid B) = \frac{P(A \text{ and } B)}{P(B)}$$

Rearranged, that gives the multiplication rule: a joint probability is a conditional times a marginal.

$$P(A \text{ and } B) = P(A \mid B)\,P(B) = P(B \mid A)\,P(A)$$

Check it against the counts: $P(\text{abusive} \mid \text{flagged}) = 0.0095 / 0.059 \approx 0.161$, the same posterior as before. This identity is also where Bayes rule comes from. The next section just reads it from the other direction.

Bayes rule after the counts

Bayes rule is the formula version of the flagged-pile count.

For events $A$ and $B$ with $P(B) > 0$:

$$P(A \mid B) = \frac{P(B \mid A)P(A)}{P(B)}$$

For this example:

When $B$ is the observed evidence, $P(B \mid A)$ is its likelihood under hypothesis $A$. Here it asks how likely a flag would be if the signup really were abusive. Bayes rule combines that likelihood with the prior to obtain the posterior.

The denominator $P(B)$ means "how often does a flag happen at all?"

It includes true flags and false alarms:

$$P(B) = P(B \mid A)P(A) + P(B \mid \text{not }A)P(\text{not }A)$$

The denominator $P(B)$ is the marginal probability of a flag. It averages over both kinds of signups, weighted by how common each kind is.

Plug in the numbers:

$$P(B) = 0.95 \times 0.01 + 0.05 \times 0.99 = 0.059$$

Now compute the posterior:

$$P(A \mid B) = \frac{0.95 \times 0.01}{0.059} \approx 0.161$$

Same answer as the count table. Bayes rule is the compact form of the same accounting: track where the flagged signups came from before you divide.

Independence means no update

Evidence helps when it changes the event rate.

When $P(B) > 0$, seeing $B$ leaves the probability of $A$ unchanged if the events are independent:

$$P(A \mid B) = P(A)$$

Start with a broken abuse detector:

Out of 10,000 signups, this detector produces:

The flagged pile is still 1 percent abusive:

$$\frac{20}{2000} = 0.01$$

The flag created work, but it didn't create information. Useful ML signals are useful because they change the rate of the event you care about.

Code makes the "no update" claim testable. If both groups are flagged at the same rate, that rate cancels from Bayes rule.

Same detector, different world

Keep the detector fixed:

Now change only the population.

The model didn't change. The world around the model changed.

The same classifier can behave differently across products, countries, languages, traffic sources, or time periods. A score without a population is like a map without a scale. The score may look precise, but it isn't enough to act carefully.

Build it: compute posterior risk

The code should read like the table:

1. Check that each input is a valid probability.
2. Count true flags as true_positive * prior.
3. Count false flags as false_positive * (1 - prior).
4. Divide true flags by all flags.

Put this in probability_demo.py. It's small enough to audit line by line, but already exposes the input validation and evidence guard that production code needs.

The detector stayed fixed. The prior changed, so the posterior changed.

A threshold changes two probabilities at once

A deployed detector rarely emits only flag or not flag. It emits a score, and a threshold creates the flag. Raising a threshold normally sends fewer signups to reviewers and may make the flagged pile cleaner, but it can also miss more abusive signups.

Use an illustrative measurement table from the same 1 percent abuse population. These rates would need to be measured on labeled data in a real system.

The following code models this trade-off by simulating the review queues for both policies:

The stricter policy reduces review load and improves the fraction of reviewed signups that are abuse, which is precision, but it catches fewer abuse cases, which lowers recall. This is a precision-recall tradeoff. Probability exposes the tradeoff; product cost and safety policy choose among the options. A threshold can also send no signups to review. In that case, queue precision is undefined because no flagged pile exists, so code should reject or explicitly represent the empty queue instead of dividing by zero.

Probability in ML work

The abuse example is one surface.

The same habit works everywhere:

1. Name the event.
2. Name the population.
3. Measure the base rate.
4. Name the evidence.
5. Ask how the evidence changes the event rate.

Skip those steps and a score starts pretending to be a conclusion.

A score isn't automatically a probability

So far, the evidence was a thresholded flag and all rates were given. A model may instead emit a score such as 0.80. That score earns the interpretation "80 percent probability of abuse" only if similarly scored signups are abusive about 80 percent of the time. That property is calibration. Modern neural classifiers can be accurate while still producing poorly calibrated confidence scores, which is why score calibration is measured rather than assumed.^{[4]Reference 4On Calibration of Modern Neural Networkshttps://arxiv.org/abs/1706.04599}

This toy bucket demonstrates the check. Every signup was assigned a score near 0.80, but only half of these labeled signups were abusive.

Six signups can't prove how a deployed model is calibrated. The calculation names the question. Statistics will teach how much evidence you need before trusting the measured gap.

A token model multiplies probabilities

We used one binary event, but an LLM produces a categorical distribution over the next token. For a known target token, training cares about the probability assigned to that target. With a one-hot target, that token's cross-entropy contribution is its negative log probability, -log(p).^{[3]Reference 3Deep Learning.https://www.deeplearningbook.org/}

This tiny snippet prints the loss for varying target probabilities:

Low probability for the observed token costs more because the observation was more surprising under the model.

For a sequence, the model's joint probability follows the chain rule: multiply the conditional probability of each next token given the previous tokens.

$$P(t_1, t_2, \ldots, t_n) = P(t_1) \prod_{i=2}^{n} P(t_i \mid t_1, \ldots, t_{i-1})$$

Here, $t_i$ is the token at position $i$, and the expression to the right of the bar is the preceding context. Multiplying many small probabilities eventually underflows in floating-point arithmetic. Logs convert the product into a stable sum.

We can verify this underflow by multiplying 200 token probabilities in Python:

The 0.0 doesn't mean the sequence was impossible. It means ordinary floating-point multiplication lost a representable nonzero number. Later language-modeling chapters use this same log-space habit for cross-entropy and perplexity.

Common mistakes

Most probability bugs are question bugs.

The debugging question is short:

Among which cases am I counting?

If you can answer that, the formula usually becomes much easier.

Try it yourself

Use the same 10,000-signup population.

1. Recompute $P(\text{abusive})$ from the first table.
2. Recompute the 495 false flags by hand.
3. Change the false-positive rate from 0.05 to 0.01.
4. Compute the new posterior.
5. Explain why the posterior changed even though the true-positive rate stayed 0.95.
6. Compare the broad and stricter review policies: which catches more abuse, and which sends a cleaner queue to reviewers?
7. Compute -log(0.80) and -log(0.10). Which observed token is more surprising to a model?

Then translate the lesson to product search:

Don't memorize the abuse numbers. Carry the counting habit into any model score.

Solution checks

Check your work after you try the practice.

If your explanation starts with Bayes rule, translate it back into piles. A good answer can move between counts, words, code, and notation.

A complete probability claim

Probability turns model scores into named claims.

For an API platform, a useful probability statement sounds like this:

Event: API-key request is abusive. Population: new workspace key requests in the last 30 days. Evidence: abuse detector score above threshold. Decision: send to manual review when posterior risk is above 20 percent.

That sentence is longer than "score is high," but it's much safer.