Dimensionality Reduction for Embeddings

An instruction-tuned incident assistant still needs evidence before it answers. Suppose its retrieval index stores runbook chunks for canary failures, latency spikes, rollback approvals, and access escalations. Each chunk is represented by a long embedding, and a query such as "the canary started returning 500s" should land near the canary-failure runbook.

That retrieval system creates two different questions:

1. Can we store and search shorter vectors while returning the same useful runbook chunks?
2. Can we draw a two-dimensional map that helps an engineer inspect confusing clusters?

Both questions use dimensionality reduction. They don't have the same success criterion. A production vector is approved by retrieval quality and cost. A plot is approved by what it lets you investigate, with important conclusions checked back in the original embedding space.

Start with the budget, not an algorithm

An embedding is an array of numbers. With float32 storage, every dimension costs four bytes. Storing ten million 1,536-dimensional runbook or document vectors therefore costs:

$10{,}000{,}000 \times 1{,}536 \times 4 = 61{,}440{,}000{,}000 \text{ bytes}$

That's 61.44 GB using decimal gigabytes, before the index graph, metadata, replicas, or backups. Cutting the vector to 256 dimensions makes the raw array six times smaller. It does not prove that search remains good.

Run the arithmetic first. It turns "embeddings feel large" into a concrete engineering constraint.

The first applied habit is simple: separate a cost metric from a quality metric.

Build a retrieval baseline

Before compressing anything, define what "the same useful neighbors" means. The small fixture below represents six incident documents. Its coordinates are hand-written so you can see the experiment without needing an API key: the first two coordinates concern canary failures, the next two concern latency, and the last two concern rollback.

Cosine similarity compares directions rather than raw lengths. We L2-normalize each vector, score one canary-failure query, and save the top results as the baseline that reduction must try to preserve.

In a real evaluation set, each query has judged relevant documents. For a query with two relevant runbook chunks, recall@2 = 1.0 means both appear in the top two returned results. That's the comparison target for every compressed representation here.

PCA: learn a shorter linear coordinate system

Principal Component Analysis (PCA) fits a linear transformation to a corpus. Center the embedding matrix, find directions with the most variation, and keep the first k directions. If $X$ is the centered document matrix and $V_k$ holds the chosen directions, the reduced vectors are:

$Z = X V_k$

The rows are still documents, now with fewer columns. In implementations, Singular Value Decomposition (SVD) is a practical way to find principal directions without explicitly constructing a covariance matrix.^{[1]Reference 1Numerical Linear Algebra.https://people.maths.ox.ac.uk/trefethen/books.html}

There is one failure mode to understand before using the library: fitting PCA separately on document vectors and live queries creates different axes. Their dot products no longer compare like with like. Fit on representative document data, persist the fitted transform, and call transform for both indexed documents and future queries.

This example adds small variations around the three runbook themes, fits a two-dimensional PCA model, and retrieves with a query transformed by the same model.

PCA optimizes variance, not relevance. A direction that rarely changes in the corpus can still distinguish a rollback-approved incident from a rollback-blocked incident. Explained variance is useful diagnostic information; retrieval recall is the release metric.

Sweep dimensions against neighbor recall

On a small data set, you can provide the relevance judgments yourself. This lab creates twenty incident intents in a noisy 24-dimensional space. Each query has three acceptable runbook chunks. It then tries PCA dimensions 2, 4, 8, and 12.

The numbers belong to this generated fixture, not to all embedding models. Notice that variance and relevant-chunk recall answer different questions. Copy the experiment pattern, then replace its documents, queries, and judgments with data from your own product.

A two-dimensional map is an investigation tool

A two-dimensional projection answers questions such as:

• Are failed canary queries scattered into the rollback cluster?
• Are documents from a new runbook version isolated from older versions?
• Which points should an engineer inspect in the original data?

It doesn't become a search index just because the plot looks separated. Compressing a high-dimensional semantic vector to two coordinates forces distortion.

Compare a linear map and a neighborhood map

PCA offers a deterministic linear view. t-SNE and UMAP focus more strongly on neighborhood relationships, using nonlinear objectives. The visual difference matters because a layout can make clusters appear convincingly separated even when a production retrieval test would disagree.

The next lab demonstrates the evaluation idea without asking you to trust a picture. It measures how many of each point's original nearest neighbors remain nearby after a two-dimensional PCA view and after a t-SNE view.

This local-overlap measurement still isn't a retrieval release gate. It checks whether a map is useful for investigating local cases. The retrieval gate remains based on queries and relevant documents.

t-SNE: excellent local pictures, unsafe search coordinates

t-SNE converts high-dimensional neighbor relationships into probabilities $p_{ij}$, creates corresponding low-dimensional probabilities $q_{ij}$ using a heavy-tailed Student-t distribution, and minimizes:

$D_{KL}(P \parallel Q) = \sum_{i \ne j} p_{ij} \log \frac{p_{ij}}{q_{ij}}$

The heavy tail lets moderately distant points spread in the map instead of crowding into the center. This is the central construction in van der Maaten and Hinton's t-SNE paper.^{[2]Reference 2Visualizing Data using t-SNE.https://jmlr.org/papers/v9/vandermaaten08a.html}

perplexity controls the rough neighborhood scale considered by the map. Trying several values is useful for investigation. It isn't acceptable to pick whichever rendering tells the cleanest story.

UMAP: optimize a weighted neighbor graph

UMAP first constructs a weighted nearest-neighbor graph, represented as a fuzzy simplicial set, then optimizes a lower-dimensional graph using a cross-entropy objective.^{[3]Reference 3UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction.https://arxiv.org/abs/1802.03426} Parameters such as n_neighbors and min_dist change the view: one layout can expose small pockets of runbook confusion while another reveals broader groupings.

Common UMAP implementations can place later samples into a fitted map with an approximate transform. That's convenient for a diagnostic dashboard, but it isn't the same as PCA's fixed linear transform. If incoming incident traffic changes, re-check the map before using it to explain failures.

Random projection: a no-fit serving baseline

PCA learns from the document corpus. A random projection samples a fixed matrix once and applies it to every document and query:

$z = xR, \qquad R \in \mathbb{R}^{d \times k}$

The Johnson-Lindenstrauss lemma establishes that a finite set of points can be projected into a lower-dimensional space while approximately preserving pairwise distances, given enough target dimensions.^{[4]Reference 4Extensions of Lipschitz mappings into a Hilbert space.https://web.stanford.edu/class/cs114/readings/JL-Johnson.pdf} The bound is a worst-case guarantee, not a claim that a specific 128-dimensional incident index will meet your recall requirement.

This experiment compares original cosine neighbors with a Gaussian random projection. A fixed seed makes the transform reproducible.

Random projection is worth benchmarking when fitting a data-dependent reducer is inconvenient or expensive. It isn't automatically better or worse than PCA. Let the same labeled retrieval set judge both.

Compression after reduction: quantization

Dimension reduction stores fewer coordinates. Quantization stores each coordinate, or each subvector, with fewer bits. You can use them independently or together.

For example, a 256-dimensional float32 vector occupies 256 × 4 = 1,024 bytes. Storing one signed byte per dimension uses 256 bytes. Storing one bit per dimension uses 32 bytes. Lower memory comes with progressively more approximation.

Scalar and binary codes

Scalar quantization maps each value onto an integer grid. Binary quantization records only a sign or threshold result. Vector databases often pair aggressive compressed candidate search with rescoring of a larger candidate set in a more accurate representation; the available choices and configuration depend on the database implementation.^{[5]Reference 5Qdrant Quantization Guidehttps://qdrant.tech/documentation/guides/quantization/}

The next lab makes the compression visible. It quantizes four normalized vectors to int8 and converts their signs to bits. The bit count is exact; the quality decision still needs retrieval measurements.

Binary codes are tiny, but "same sign" is a much weaker statement than "same semantic evidence." For an incident application, retrieve more candidates with a compressed representation and rerank before final evidence selection if benchmarks justify that design.

Product quantization: code sub-vectors

Product Quantization (PQ) splits a vector into sub-vectors, learns a codebook for each subspace, and stores only the selected centroid IDs.^{[6]Reference 6Product Quantization for Nearest Neighbor Search.https://dblp.org/rec/journals/pami/JegouDS11} In the common case of 256 possible centroids, each ID fits in one byte. A 128-dimensional float32 vector split into eight subspaces can therefore be represented by eight code bytes, plus shared codebooks.

The original PQ paper describes asymmetric distance computation (ADC): keep a query unquantized, precompute its distances to codebook centroids, and score each stored code through table lookups.^{[6]Reference 6Product Quantization for Nearest Neighbor Search.https://dblp.org/rec/journals/pami/JegouDS11} This miniature implementation uses only two subspaces and four centroids so that every shape remains visible.

PQ isn't an interchangeable synonym for PCA. PCA shortens a coordinate system; PQ encodes sub-vectors with learned IDs. An index may use a shortened embedding and then PQ-code it when memory is tight.

Native shortening: train prefixes to be useful

PCA and random projection operate after an encoder produces its vector. Matryoshka Representation Learning (MRL) changes training itself: the model is optimized so prefixes of its representation remain useful at several selected lengths.^{[7]Reference 7Matryoshka Representation Learning.https://arxiv.org/abs/2205.13147}

For an embedding $e$ and prefix lengths in a set $\mathcal{D}$, a simplified training view is:

$\mathcal{L}_{MRL} = \sum_{d \in \mathcal{D}} w_d \mathcal{L}(e_{:d})$

Here, $e_{:d}$ is the first d coordinates. Unlike PCA, serving a shorter MRL-trained representation doesn't require a separately fitted rotation. It still requires matching document and query lengths and testing retrieval quality.

This lab illustrates the mechanics with deliberately ordered vectors. It isn't pretending to train an embedding model. The first coordinates carry runbook theme information; later coordinates add detail and noise. Truncating a standard, unordered representation wouldn't have this promise.

One hosted API example is OpenAI's text-embedding-3 family. Its documentation lists default lengths of 1,536 for text-embedding-3-small and 3,072 for text-embedding-3-large, and documents a dimensions request parameter for shorter outputs.^{[8]Reference 8Vector embeddingshttps://developers.openai.com/api/docs/guides/embeddings} A request shape looks like this:

The provider feature makes generation of shorter vectors convenient. It doesn't decide whether 256 dimensions meet your canary-failure retrieval requirement.

Choose a deployment experiment

You have three tool families:

A release gate should turn this table into a decision that someone else can audit. This lab accepts only configurations that meet both a recall floor and a raw-vector memory budget, then chooses the smallest accepted representation.

The values above are example measurements for the gate, not benchmark results for a real model. A serious report would include:

1. A frozen set of realistic incident queries and judged runbook chunks.
2. Recall or nDCG at the candidate count sent to generation or reranking.
3. Index RAM, build time, and query-latency percentiles for each candidate.
4. A failure list: which queries lost relevant evidence, and what risk that creates.
5. Re-evaluation when embedding models, runbook documents, or traffic distributions change.

What you've built

The long-embedding budget question leaves four concrete skills:

• calculate the raw cost of storing a vector representation;
• establish cosine-retrieval neighbors before changing representations;
• fit PCA once, apply it consistently, and evaluate shortened retrieval;
• use t-SNE and UMAP as diagnostic maps rather than search coordinates;
• benchmark random projection as a no-fit alternative;
• distinguish fewer dimensions from lower-precision or PQ-coded storage; and
• evaluate model-supported prefixes such as MRL-style shortening through the same retrieval gate.

The recurring idea is more important than any preferred method: compression is an experiment with a quality constraint. Storage savings are real immediately. Semantic safety must be measured.

Mastery check

Key concepts

• PCA fits variance-aligned linear coordinates and reuses one transform for documents and queries.
• t-SNE and UMAP make investigation maps; neither plot distance should approve retrieval.
• Random projection supplies a fixed, data-independent serving baseline.
• MRL trains selected prefixes to remain useful, so shortening can happen at model output.
• Quantization reduces storage precision or stores centroid codes; PQ is distinct from dimensionality reduction.
• Recall@K or an equivalent downstream metric decides whether reduced vectors remain useful.

Evaluation rubric

• Foundational: Calculates float32 vector storage and explains why it isn't a quality metric.
• Intermediate: Fits a PCA transform once, normalizes reduced vectors for cosine comparison, and computes neighbor recall.
• Intermediate: Explains why a 2D t-SNE or UMAP map is useful for debugging but unsafe as a retrieval representation.
• Advanced: Designs a comparison among native shortening, linear projection, and quantization under RAM and recall constraints.

Common pitfalls

• Refitting a reducer for query batches: documents and queries end up in different coordinate systems. Fit and version one serving transform.
• Choosing dimensions by explained variance only: retained variance isn't relevance. Use labeled retrieval or grounded-answer evaluations.
• Reading plot gaps as semantic distance: nonlinear map layout distorts separation. Check specific claims in original vectors.
• Truncating an arbitrary embedding: prefix shortening is justified only when the model documents or was trained for it.
• Compressing without reranking analysis: aggressive codes can lose required evidence. Evaluate oversampling and rescoring when using them.

Practice extension

Create a small runbook corpus with four query categories: canary failure, latency spike, rollback eligibility, and access escalation. Assign each query two acceptable runbook chunks. Replace the generated embeddings in pca-recall-sweep.py with embeddings from a model you can run or an API you can audit. Compare full vectors, one shorter representation, and one quantized candidate. Your artifact is a table of recall, bytes per vector, and the exact failed queries, followed by a release recommendation.