Probabilistic Generative Models

A latency dashboard reports a mean of 60 milliseconds. Half its requests finish near 30 milliseconds, while the other half wait near 90. No request in that sample finishes near the mean, and a hard cluster assignment would force an in-between request into one story.

Clustering, PCA, and Representation Learning grouped examples by geometric similarity. That helps assign points, but it doesn't describe how likely each group is. A Gaussian mixture model asks how hidden populations could generate the observed values and keeps uncertain membership visible.^{[1]Reference 1Pattern Recognition and Machine Learning.https://www.microsoft.com/en-us/research/publication/pattern-recognition-machine-learning/[2]Reference 2Maximum Likelihood from Incomplete Data via the EM Algorithmhttps://doi.org/10.1111/j.2517-6161.1977.tb01600.x}

Describe a population with hidden components

Start with six observed request latencies: [28, 30, 32, 86, 90, 94]. Their single mean is 60 milliseconds. Before naming a model, ask whether that average is a useful description of any request in the sample.

A generative model asks how an observed latency might have been produced. One possible story first chooses a component, then samples from that component's probability distribution.

That component is a latent variable: the request log records 28 or 90 milliseconds, not the hidden label that supposedly generated it. The diagram makes the two-stage story explicit:

The two components are useful statistical descriptions, not proof that a specific queue, GPU, or tenant caused a latency. Causal attribution requires measurements such as queue state or device traces outside this mixture.

Compute soft responsibilities

The six values suggest two groups, but the model still needs to decide how much each group explains each request. For $K$ components, the mixture density is:

$$p(x)=\sum_{k=1}^{K}\pi_k\,\mathcal N(x\mid\mu_k,\sigma_k^2)$$

Here $\pi_k$ is a component weight, $\mu_k$ its mean, and $\sigma_k^2$ its variance. The weights sum to one. Once those parameters are fixed, an observation's responsibility for component $k$ is:

$$r_{ik}=\frac{\pi_k\mathcal N(x_i\mid\mu_k,\sigma_k^2)}{\sum_j\pi_j\mathcal N(x_i\mid\mu_j,\sigma_j^2)}$$

Responsibilities sum to one across components. A request near both component centers can receive fractional membership instead of a forced hard label.

Use equal weights, centers at 30 and 90 milliseconds, and a common standard deviation of 20. For a 50-millisecond request, which component should receive more responsibility? The next cell computes the normalized values.

The 50-millisecond request is closer to the fast center, so the fast component gets responsibility 0.818 and the slow component gets 0.182. Those values are conditional on the assumed weights, means, and variances.

They don't certify that a real fast queue handled the request. The model only saw latency, not queue identity.

A 50-millisecond request receives fast-component responsibility 0.818. What happens to the remaining 0.182, and does the larger number identify the physical queue that handled the request?

Answer

The remaining 0.182 is assigned to the slow component, so both responsibilities sum to one. The 0.818 value is a probability inside the assumed mixture model; no physical queue identity was observed, so it doesn't establish infrastructure ownership or causation.

Fit the mixture with expectation-maximization

Responsibilities need parameters, but the parameters are exactly what we want to learn. Expectation-maximization (EM) handles that loop by alternating two operations:^{[2]Reference 2Maximum Likelihood from Incomplete Data via the EM Algorithmhttps://doi.org/10.1111/j.2517-6161.1977.tb01600.x}

1. In the E-step, estimate each observation's fractional component membership using the current parameters.
2. In the M-step, recompute component weights, means, and variances from those fractional memberships.

The M-step's weighted mean update is:

$$\mu_k^{\text{new}}=\frac{\sum_i r_{ik}x_i}{\sum_i r_{ik}}$$

With exact E-steps and M-steps, each completed EM iteration doesn't decrease the observed-data log likelihood:

$$\log p(x_1,\ldots,x_n)=\sum_{i=1}^{n}\log\left(\sum_{k=1}^{K}\pi_k\mathcal N(x_i\mid\mu_k,\sigma_k^2)\right)$$

The outer sum runs over observed requests. The inner sum adds the possible hidden component assignments for each request. A higher value means the observed latencies fit the stated model better; it doesn't guarantee a global optimum or identify a real infrastructure cause.

The next cell starts with provisional means and runs eight one-dimensional EM iterations. It adds a variance floor, checks the observed-data log likelihood after each update, and prints the fitted means. Before running it, predict whether the two groups will settle near 30 and 90 milliseconds.

The floor prevents a component from collapsing onto one observation with nearly zero variance. Because it changes the optimization problem, the code measures the constrained fit instead of assuming that an unconstrained proof transfers automatically. Larger mixtures should compute densities in log space with a log-sum-exp operation to avoid numerical underflow.

The fitted model's observed-data log likelihood rises from -24.438 to -17.694. What does that improvement establish, and which conclusions remain unsupported?

Answer

The observed latencies receive higher likelihood under the updated two-component model, and the recorded EM iterations didn't decrease that objective. The increase doesn't prove global optimality, establish that two is the correct number of components, or identify either component with a real serving-system cause.

Why is a responsibility of 0.82 different from observing that a request definitely came from the fast queue?

Answer

The responsibility is a conditional probability inside the assumed mixture model. The true queue identity wasn't observed, and the mixture components may not correspond to actual infrastructure causes.

Compare hard clusters with probabilistic models

The responsibility calculation answers a different question from a hard cluster label. Put the two views side by side:

Maximum likelihood compares how compatible the observations are with candidate parameters. EM can improve the observed-data likelihood, but a better likelihood alone doesn't prove that the number of components is correct or that the labels describe real causes.^{[1]Reference 1Pattern Recognition and Machine Learning.https://www.microsoft.com/en-us/research/publication/pattern-recognition-machine-learning/}

That limitation changes how you validate a fit. Run multiple initializations, inspect held-out likelihood, check component sizes, and compare the generated distribution with actual traffic. A component containing one request with near-zero variance is a collapse warning, not a new scientific discovery.

Practice receipt: For this six-point fixture, record each component's weight, fitted mean, variance, effective responsibility total, and observed-data log likelihood after every EM iteration. The expected receipt has weights of 0.500, means near 30 and 90 milliseconds, responsibilities summing to one for every request, and likelihood increasing from -24.438 to -17.694.

Then try several starting means. Flag a collapsed variance, a tiny component, or a lower held-out likelihood instead of hiding the run that produced it. Draw samples from the fitted mixture and compare their latency shape with held-out traffic. That posterior predictive check asks whether the model can reproduce both modes, not only whether it scored the six training values well.

Finally, keep the causal boundary visible: a statistical component doesn't name a real queue until separate infrastructure evidence confirms it.

Create a mixture-fit scorecard for each initialization. Record weights, means, variances, responsibilities, held-out likelihood, and posterior-predictive checks; compare the scorecards and flag collapsed components or a poor traffic match. Keep the selected scorecard with the model handoff, including the evidence that still can't identify an infrastructure cause.