LearnApplied LLM EngineeringDimensionality Reduction for Embeddings

📐MediumEmbeddings & Vector Search

Dimensionality Reduction for Embeddings

Shrink and inspect embedding indexes without guessing: measure recall while testing PCA, projections, native shortening, and quantization.

26 min read

Learning path

Step 61 of 196 in the full curriculum

Instruction Tuning & Chat Templates CoT, ToT & Self-Consistency Prompting

Personalize this lesson

Adapt explanations and teaching visuals to your background and preferred voice.

The last chapter taught an access-policy assistant to answer with the same chat template at train and serve time. Now consider a failure before generation: the assistant receives the right question but retrieves the wrong policy clause. "My service-account key is 18 days old. Can I keep using it?" should land near "Keys older than 14 days require a rotation ticket," not near an auto-merge rule.

Each clause is stored as a long embedding. An engineer proposes shortening every vector to save memory. What evidence would make that safe? A cleaner-looking plot isn't enough; the rotation clause still has to survive the top- $K$ retrieval that supplies the answer.

Keep two jobs separate:

Can we store and search shorter vectors while still returning the same useful policy chunks?
Can we draw a two-dimensional map that helps an engineer inspect confusing clusters?

Both jobs use dimensionality reduction, but they don't share a success criterion. A production vector is approved by retrieval quality and cost. A plot is approved by what it lets you investigate, with important conclusions checked back in the original embedding space.

The article keeps one policy index in view while changing one representation at a time. First establish cost and original neighbors. Then project, shorten, or quantize and ask which neighbors survived.

A shared 1536-dimensional policy embedding space splits into two paths: a 256-dimensional serving vector checked by Recall@K and a separate 2D map used to inspect clusters and outliers. — Reduction has two different jobs: serving candidates must preserve relevant neighbors under a cost budget, while 2D maps help investigate records without becoming the production index.

Diagram showing Long embedding, Fewer dims or bits, Recall@K + RAM, and Serve. — Long embedding, Fewer dims or bits, Recall@K + RAM, and Serve.

Start with the budget, not an algorithm

Before choosing a reducer, put the storage pressure on paper. An embedding is an array of numbers, and float32 storage costs four bytes per dimension. Storing ten million 1,536-dimensional policy or document vectors therefore costs:

$10{,}000{,}000 \times 1{,}536 \times 4 = 61{,}440{,}000{,}000 \text{ bytes}$

That's 61.44 GB using decimal gigabytes, before the index graph, metadata, replicas, or backups. Cutting the vector to 256 dimensions makes the raw array six times smaller. The arithmetic proves a cost change, not a safe search change.

Run the arithmetic first. It turns "embeddings feel large" into a concrete engineering constraint.

estimate-vector-storage.py

def decimal_gb(byte_count: int) -> float:
    return byte_count / 1_000_000_000

documents = 10_000_000
bytes_per_float32 = 4
full_dim = 1_536
candidate_dim = 256

full_bytes = documents * full_dim * bytes_per_float32
candidate_bytes = documents * candidate_dim * bytes_per_float32

print(f"Full float32 array: {decimal_gb(full_bytes):.2f} GB")
print(f"256-d float32 array: {decimal_gb(candidate_bytes):.2f} GB")
print(f"Raw-vector reduction: {full_bytes / candidate_bytes:.1f}x")

Output

Full float32 array: 61.44 GB
256-d float32 array: 10.24 GB
Raw-vector reduction: 6.0x

The first applied habit is simple: separate a cost metric from a quality metric.

Question	Useful metric
Will vectors fit in memory?	Bytes per vector, total RAM, index overhead
Will useful evidence still be retrieved?	Recall@K, nDCG@K, answer success on grounded evaluations
Will an engineer understand failures?	A labeled plot plus checks in the original space

Build a retrieval baseline

Storage has a number now, but quality still needs a reference. Before compressing anything, define what "the same useful neighbors" means. The small fixture below represents six policy clauses. Its coordinates are hand-written so you can see the experiment without needing an API key: the first two coordinates concern key rotation, the next two concern merge gates, and the last two concern access escalations.

Cosine similarity compares directions rather than raw lengths. We L2-normalize each vector, score one 18-day-key query, and save the top results as the baseline that reduction must try to preserve.

Before running the cell, predict its first two clauses. The query points toward the first two coordinates, so the two rotation clauses should lead; the later merge and access clauses are controls for a false neighbor.

baseline-policy-retrieval.py

import math

documents = [
    "Keys older than 14 days require a rotation ticket before continued use.",
    "Rotate a stale service-account key and open a ticket before reuse.",
    "Auto-merge only when tests, security scan, and reviewer approval all pass.",
    "A failed security scan blocks merge even if tests are green.",
    "Access escalations need an owner and an expiry; never grant standing admin.",
    "Standing production admin is forbidden; use a time-bounded escalation.",
]

vectors = [
    [1.00, 0.92, 0.04, 0.00, 0.02, 0.00],
    [0.92, 0.80, 0.04, 0.01, 0.02, 0.00],
    [0.02, 0.00, 1.00, 0.88, 0.04, 0.00],
    [0.02, 0.02, 0.91, 0.98, 0.00, 0.00],
    [0.00, 0.00, 0.04, 0.02, 1.00, 0.89],
    [0.02, 0.00, 0.00, 0.02, 0.91, 1.00],
]
query = [0.98, 1.00, 0.03, 0.00, 0.01, 0.00]

def dot(left: list[float], right: list[float]) -> float:
    return sum(a * b for a, b in zip(left, right))

def normalize(vector: list[float]) -> list[float]:
    scale = math.sqrt(dot(vector, vector))
    return [value / scale for value in vector]

query_hat = normalize(query)
scores = [dot(normalize(vector), query_hat) for vector in vectors]
ranking = sorted(range(len(scores)), key=lambda index: -scores[index])

for rank, index in enumerate(ranking[:3], start=1):
    print(f"{rank}. {documents[index]}  score={scores[index]:.3f}")

Output

Keys older than 14 days require a rotation ticket before continued use.  score=0.999
Rotate a stale service-account key and open a ticket before reuse.  score=0.997
A failed security scan blocks merge even if tests are green.  score=0.036

In a real evaluation set, each query has judged relevant documents. For a query with two relevant rotation clauses, recall@2 = 1.0 means both appear in the top two returned results. That's the comparison target for every compressed representation here.

PCA: learn a shorter linear coordinate system

The first post-hoc candidate changes coordinates without retraining the encoder. Principal component analysis (PCA) fits a linear transformation to a corpus: center the embedding matrix, find directions with the most variation, and keep the first k directions. If $X$ is the centered document matrix and $V_k$ holds the chosen directions, the reduced vectors are:

$Z = X V_k$

The rows are still documents, now with fewer columns. In implementations, singular value decomposition (SVD) is a practical way to find principal directions without explicitly constructing a covariance matrix.^{[1]Reference 1Numerical Linear Algebra.https://people.maths.ox.ac.uk/trefethen/books.html} The labs use power iteration so you can run them with the standard library; the geometry is the same idea.

A centered embedding cloud shows the first principal axis capturing the largest spread, while a single fitted mean and basis transform both document and query matrices into comparable reduced coordinates. — PCA learns variance-aligned axes from representative documents. Persist the mean and basis, then project stored documents and incoming queries through that same coordinate system.

Follow the right-hand path in the figure before reading the code: one fitted mean and basis must transform both sides of the search. Fitting PCA separately on document vectors and live queries creates different axes, so their dot products no longer compare like with like. Persist the transform fitted on representative documents and reuse it for indexed documents and future queries.

The next cell adds small variations around the three policy themes, fits a two-dimensional PCA model, and retrieves with a query transformed by that same model. Predict the final label first: the query still describes rotation, even though its coordinates will change.

fit-one-pca-transform.py

import math
import random

centers = [
    [1.0, 0.9, 0.0, 0.0, 0.0, 0.0],  # rotation
    [0.0, 0.0, 1.0, 0.9, 0.0, 0.0],  # merge
    [0.0, 0.0, 0.0, 0.0, 1.0, 0.9],  # access
]
labels = ["rotation"] * 20 + ["merge"] * 20 + ["access"] * 20
rng = random.Random(4)
documents = [
    [center[j] + rng.gauss(0, 0.05) for j in range(6)]
    for center in centers
    for _ in range(20)
]
query = [[0.96, 0.91, 0.02, 0.01, 0.00, 0.00]]

def dot(left: list[float], right: list[float]) -> float:
    return sum(a * b for a, b in zip(left, right))

def normalize(vector: list[float]) -> list[float]:
    scale = math.sqrt(dot(vector, vector))
    return [value / scale for value in vector]

def mean_col(rows: list[list[float]]) -> list[float]:
    n_rows, n_cols = len(rows), len(rows[0])
    return [sum(row[j] for row in rows) / n_rows for j in range(n_cols)]

def sub(left: list[float], right: list[float]) -> list[float]:
    return [a - b for a, b in zip(left, right)]

def fit_pca(rows: list[list[float]], k: int, seed: int = 0) -> tuple[list[float], list[list[float]]]:
    mu = mean_col(rows)
    centered = [sub(row, mu) for row in rows]
    n_cols = len(rows[0])
    local = random.Random(seed)
    components: list[list[float]] = []
    for _ in range(k):
        vector = normalize([local.gauss(0, 1) for _ in range(n_cols)])
        for _ in range(80):
            projected = [dot(row, vector) for row in centered]
            vector = [
                sum(centered[i][j] * projected[i] for i in range(len(centered)))
                for j in range(n_cols)
            ]
            for previous in components:
                coeff = dot(vector, previous)
                vector = [value - coeff * axis for value, axis in zip(vector, previous)]
            vector = normalize(vector)
        components.append(vector)
    return mu, components

def transform(rows: list[list[float]], mu: list[float], components: list[list[float]]) -> list[list[float]]:
    return [[dot(sub(row, mu), axis) for axis in components] for row in rows]

def explained_ratio(rows: list[list[float]], mu: list[float], components: list[list[float]]) -> float:
    centered = [sub(row, mu) for row in rows]
    total = sum(dot(row, row) for row in centered)
    kept = 0.0
    for axis in components:
        scores = [dot(row, axis) for row in centered]
        kept += sum(score * score for score in scores)
    return kept / total

mu, components = fit_pca(documents, k=2)
reduced_documents = [normalize(row) for row in transform(documents, mu, components)]
reduced_query = normalize(transform(query, mu, components)[0])
best_index = max(range(len(reduced_documents)), key=lambda i: dot(reduced_documents[i], reduced_query))

print(f"Shape: ({len(documents)}, {len(documents[0])}) -> ({len(reduced_documents)}, {len(reduced_documents[0])})")
print(f"Explained variance kept: {explained_ratio(documents, mu, components):.3f}")
print(f"Closest policy theme: {labels[best_index]}")

Output

Shape: (60, 6) -> (60, 2)
Explained variance kept: 0.992
Closest policy theme: rotation

PCA optimizes variance, not relevance. A direction that rarely changes in the corpus can still distinguish "auto-merge allowed" from "security scan failed." Explained variance helps diagnose what the projection retained; retrieval recall decides whether the index is safe to ship.

Sweep dimensions against neighbor recall

One successful query doesn't establish a dimension. On a small data set, you can provide the relevance judgments yourself. This lab creates twenty policy intents in a noisy 24-dimensional space, gives each query three acceptable chunks, and tries PCA dimensions 2, 4, 8, and 12. Predict the tradeoff: the first cuts should save more coordinates, but may lose neighbors.

pca-recall-sweep.py

import math
import random

rng = random.Random(21)
dimensions = 24
intent_count = 20
acceptable_chunks = 3

def dot(left: list[float], right: list[float]) -> float:
    return sum(a * b for a, b in zip(left, right))

def normalize(vector: list[float]) -> list[float]:
    scale = math.sqrt(dot(vector, vector))
    return [value / scale for value in vector]

def mean_col(rows: list[list[float]]) -> list[float]:
    n_rows, n_cols = len(rows), len(rows[0])
    return [sum(row[j] for row in rows) / n_rows for j in range(n_cols)]

def sub(left: list[float], right: list[float]) -> list[float]:
    return [a - b for a, b in zip(left, right)]

def fit_pca(rows: list[list[float]], k: int, seed: int = 0) -> tuple[list[float], list[list[float]]]:
    mu = mean_col(rows)
    centered = [sub(row, mu) for row in rows]
    n_cols = len(rows[0])
    local = random.Random(seed)
    components: list[list[float]] = []
    for _ in range(k):
        vector = normalize([local.gauss(0, 1) for _ in range(n_cols)])
        for _ in range(60):
            projected = [dot(row, vector) for row in centered]
            vector = [
                sum(centered[i][j] * projected[i] for i in range(len(centered)))
                for j in range(n_cols)
            ]
            for previous in components:
                coeff = dot(vector, previous)
                vector = [value - coeff * axis for value, axis in zip(vector, previous)]
            vector = normalize(vector)
        components.append(vector)
    return mu, components

def transform(rows: list[list[float]], mu: list[float], components: list[list[float]]) -> list[list[float]]:
    return [[dot(sub(row, mu), axis) for axis in components] for row in rows]

def explained_ratio(rows: list[list[float]], mu: list[float], components: list[list[float]]) -> float:
    centered = [sub(row, mu) for row in rows]
    total = sum(dot(row, row) for row in centered)
    kept = 0.0
    for axis in components:
        scores = [dot(row, axis) for row in centered]
        kept += sum(score * score for score in scores)
    return kept / total

def top_k(query_rows: list[list[float]], doc_rows: list[list[float]], k: int) -> list[list[int]]:
    doc_hat = [normalize(row) for row in doc_rows]
    ranking: list[list[int]] = []
    for query in query_rows:
        query_hat = normalize(query)
        scores = [dot(doc, query_hat) for doc in doc_hat]
        order = sorted(range(len(scores)), key=lambda index: -scores[index])
        ranking.append(order[:k])
    return ranking

intent_vectors = [[rng.gauss(0, 1) for _ in range(dimensions)] for _ in range(intent_count)]
center = mean_col(intent_vectors)
intent_vectors = [normalize(sub(vector, center)) for vector in intent_vectors]

documents: list[list[float]] = []
queries: list[list[float]] = []
relevant: list[set[int]] = []
for intent in intent_vectors:
    first_chunk = len(documents)
    for _ in range(acceptable_chunks):
        documents.append([intent[j] + rng.gauss(0, 0.11) for j in range(dimensions)])
    queries.append([intent[j] + rng.gauss(0, 0.03) for j in range(dimensions)])
    relevant.append(set(range(first_chunk, first_chunk + acceptable_chunks)))

for target_dim in [2, 4, 8, 12]:
    mu, components = fit_pca(documents, k=target_dim)
    short_docs = transform(documents, mu, components)
    short_queries = transform(queries, mu, components)
    returned = top_k(short_queries, short_docs, k=3)
    recall = sum(
        len(expected & set(found)) / acceptable_chunks
        for expected, found in zip(relevant, returned)
    ) / len(relevant)
    print(
        f"{target_dim:>2} dims: recall@3={recall:.3f}, "
        f"variance={explained_ratio(documents, mu, components):.3f}"
    )

Output

dims: recall@3=0.383, variance=0.294
dims: recall@3=0.867, variance=0.495
dims: recall@3=1.000, variance=0.727
dims: recall@3=1.000, variance=0.871

These numbers belong to this generated fixture, not to all embedding models. The two curves answer different questions: variance describes the projection, while relevant-chunk recall describes the product. Keep the experiment pattern, then replace its documents, queries, and judgments with data from your own retrieval workload.

A two-dimensional map is an investigation tool

The baseline and PCA experiment protect serving quality with judged neighbors. Debugging needs a different view. A two-dimensional projection can help an engineer ask:

Are 18-day-key queries scattered into the merge-gate cluster?
Are clauses from a new policy version isolated from older versions?
Which points should an engineer inspect in the original data?

It doesn't become a search index just because the plot looks separated. Compressing a high-dimensional semantic vector to two coordinates forces distortion, so use the map to choose cases for inspection and return to the original vectors for a decision.

Compare a linear map and a prettier picture

PCA already gave us a deterministic linear view. T-distributed stochastic neighbor embedding (t-SNE) and Uniform Manifold Approximation and Projection (UMAP) focus more strongly on neighborhood relationships, using nonlinear objectives. The visual difference matters because a layout can make clusters appear convincingly separated even when a production retrieval test would disagree.

The same three policy groups form an overlapping diagonal under PCA, separated local islands under t-SNE, and a UMAP neighbor-graph view, with one anchor point keeping its nearest neighbors in each map. — Each method tells a different 2D story. Treat local neighborhoods as investigation leads, not proof that cluster gaps or production retrieval are faithful.

The next lab uses the same six policy vectors as the baseline. Before running it, predict what a pretty island map could hide: a merge clause might look close to a rotation clause even when their original vectors are far apart. The cell reports each clause's original 1-nearest neighbor, then compares PCA with a cleaned-up island picture that an engineer might be tempted to trust.

pretty-maps-can-lie.py

import math

names = ["rot-a", "rot-b", "merge-a", "merge-b", "access-a", "access-b"]
vectors = [
    [1.00, 0.92, 0.04, 0.00, 0.02, 0.00],
    [0.92, 0.80, 0.04, 0.01, 0.02, 0.00],
    [0.02, 0.00, 1.00, 0.88, 0.04, 0.00],
    [0.02, 0.02, 0.91, 0.98, 0.00, 0.00],
    [0.00, 0.00, 0.04, 0.02, 1.00, 0.89],
    [0.02, 0.00, 0.00, 0.02, 0.91, 1.00],
]
pretty_map = [
    [0.00, 0.00],
    [0.20, 0.10],
    [0.30, 0.05],  # merge-a pulled into the rotation island
    [2.00, 2.00],
    [0.00, 2.00],
    [0.20, 2.10],
]

def dot(left: list[float], right: list[float]) -> float:
    return sum(a * b for a, b in zip(left, right))

def mean_col(rows: list[list[float]]) -> list[float]:
    n_rows, n_cols = len(rows), len(rows[0])
    return [sum(row[j] for row in rows) / n_rows for j in range(n_cols)]

def sub(left: list[float], right: list[float]) -> list[float]:
    return [a - b for a, b in zip(left, right)]

def normalize(vector: list[float]) -> list[float]:
    scale = math.sqrt(dot(vector, vector))
    return [value / scale for value in vector]

def fit_pca(rows: list[list[float]], k: int) -> tuple[list[float], list[list[float]]]:
    mu = mean_col(rows)
    centered = [sub(row, mu) for row in rows]
    n_cols = len(rows[0])
    components: list[list[float]] = []
    seed = [1.0] * n_cols
    for _ in range(k):
        vector = normalize(seed)
        for _ in range(80):
            projected = [dot(row, vector) for row in centered]
            vector = [
                sum(centered[i][j] * projected[i] for i in range(len(centered)))
                for j in range(n_cols)
            ]
            for previous in components:
                coeff = dot(vector, previous)
                vector = [value - coeff * axis for value, axis in zip(vector, previous)]
            vector = normalize(vector)
        components.append(vector)
        seed = [1.0 if j == len(components) else 0.25 for j in range(n_cols)]
    return mu, components

def transform(rows: list[list[float]], mu: list[float], components: list[list[float]]) -> list[list[float]]:
    return [[dot(sub(row, mu), axis) for axis in components] for row in rows]

def nearest_names(rows: list[list[float]]) -> list[str]:
    neighbors: list[str] = []
    for i, left in enumerate(rows):
        best_j, best_d = -1, float("inf")
        for j, right in enumerate(rows):
            if i == j:
                continue
            dist = sum((a - b) ** 2 for a, b in zip(left, right))
            if dist < best_d:
                best_j, best_d = j, dist
        neighbors.append(names[best_j])
    return neighbors

def overlap(left: list[str], right: list[str]) -> float:
    return sum(a == b for a, b in zip(left, right)) / len(left)

original = nearest_names(vectors)
mu, components = fit_pca(vectors, k=2)
pca_map = transform(vectors, mu, components)
pca_neighbors = nearest_names(pca_map)
pretty_neighbors = nearest_names(pretty_map)

print("Original 1-NN:", list(zip(names, original)))
print(f"PCA 1-NN overlap:           {overlap(original, pca_neighbors):.2f}")
print(f"Pretty-island 1-NN overlap: {overlap(original, pretty_neighbors):.2f}")
print(f"Pretty-island neighbor of merge-a: {pretty_neighbors[2]}")

Output

Original 1-NN: [('rot-a', 'rot-b'), ('rot-b', 'rot-a'), ('merge-a', 'merge-b'), ('merge-b', 'merge-a'), ('access-a', 'access-b'), ('access-b', 'access-a')]
PCA 1-NN overlap:           1.00
Pretty-island 1-NN overlap: 0.50
Pretty-island neighbor of merge-a: rot-b

PCA kept the original pairs on this tiny set because the three themes already sit on high-variance axes. The prettier island map still "looks clustered," but it moved merge-a next to a rotation clause. Local-overlap checks whether a map is useful for investigating nearby cases; it doesn't become a retrieval release gate. That gate remains based on queries and relevant documents.

t-SNE: excellent local pictures, unsafe search coordinates

Start with one question: which points should stay neighbors after the map gets smaller? t-SNE converts high-dimensional neighbor relationships into probabilities $p_{ij}$ , creates corresponding low-dimensional probabilities $q_{ij}$ using a heavy-tailed Student-t distribution with one degree of freedom, and minimizes:

$D_{KL}(P \parallel Q) = \sum_{i \ne j} p_{ij} \log \frac{p_{ij}}{q_{ij}}$

The heavy tail lets moderately distant points spread in the map instead of crowding into the center. This is the central construction in van der Maaten and Hinton's t-SNE paper.^{[2]Reference 2Visualizing Data using t-SNE.https://jmlr.org/papers/v9/vandermaaten08a.html}

A t-SNE visual compares a high-dimensional neighbor matrix P with crowded and optimized 2D states Q; point movement makes local high-probability neighborhoods agree while global gaps stay uninterpretable. — t-SNE optimizes local probability agreement. The final islands help identify nearby records, but the empty space between islands isn't a semantic distance scale.

Because the map is built around neighborhoods, perplexity controls the rough scale considered by t-SNE. Trying several values is useful for investigation. Picking whichever rendering tells the cleanest story isn't a reliable analysis.

UMAP: optimize a weighted neighbor graph

UMAP starts with the same neighbor question but makes the intermediate object explicit: a weighted nearest-neighbor graph. It represents that graph as a fuzzy simplicial set, then optimizes a lower-dimensional graph with a cross-entropy objective.^{[3]Reference 3UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction.https://arxiv.org/abs/1802.03426} Parameters such as n_neighbors and min_dist change the view.

One layout can expose small pockets of policy confusion while another reveals broader groupings. Read both as views of a chosen graph, not as a new semantic coordinate system.

UMAP turns directed nearest-neighbor scores into a weighted graph and then lays that graph out in 2D; edge pairs attract, sampled non-edges repel, and n_neighbors and min_dist change the visible map. — UMAP's output follows its weighted graph and chosen parameters. Use the map to investigate labeled cases, then verify important neighbors in source vectors.

Common UMAP implementations can place later samples into a fitted map with an approximate transform. That's convenient for a diagnostic dashboard, but it isn't the same as PCA's fixed linear transform. If incoming traffic changes, re-check the map before using it to explain failures.

Goal	Appropriate first measurement	Suitable methods to test
Inspect clusters or outliers	Neighbor overlap plus human review	PCA, t-SNE, UMAP
Shorten vectors for retrieval	Recall@K or nDCG@K plus RAM and latency	Native shortening, PCA, random projection
Reduce memory further	Recall after compressed search and reranking	Scalar, PQ, binary quantization

Random projection: a no-fit serving baseline

Suppose policy documents change often and fitting a corpus-dependent transform is inconvenient. PCA learns from the document corpus; a random projection samples a fixed matrix once and applies it to every document and query:

$z = xR, \qquad R \in \mathbb{R}^{d \times k}$

The Johnson-Lindenstrauss lemma establishes that a finite set of points can be projected into a lower-dimensional space while approximately preserving pairwise distances, given enough target dimensions.^{[4]Reference 4Extensions of Lipschitz mappings into a Hilbert space.https://web.stanford.edu/class/cs114/readings/JL-Johnson.pdf} That bound is a worst-case guarantee, not a claim that a specific 256-dimensional policy index will meet your recall requirement. Save the matrix, then let judged retrieval decide.

A fixed random matrix maps a 1536-dimensional vector to 256 dimensions, keeps example pairwise distances close, and is reused for documents and queries before a Recall@K check. — Random projection is one persisted matrix multiplication, not a fitted corpus model. Its approximate geometry still needs a judged retrieval gate.

The experiment compares original cosine neighbors with a Gaussian random projection. A fixed seed makes the transform reproducible. The Gaussian entries are scaled by $1/\sqrt{k}$ , matching the usual construction. Predict the result before running it: fewer bytes can come with a noticeable neighbor loss on this intentionally small target dimension.

random-projection-recall.py

import math
import random

rng = random.Random(33)
documents = [[rng.gauss(0, 1) for _ in range(48)] for _ in range(180)]
queries = [
    [documents[i][j] + rng.gauss(0, 0.03) for j in range(48)]
    for i in range(12)
]

def dot(left: list[float], right: list[float]) -> float:
    return sum(a * b for a, b in zip(left, right))

def normalize(vector: list[float]) -> list[float]:
    scale = math.sqrt(dot(vector, vector))
    return [value / scale for value in vector]

def neighbors(query_rows: list[list[float]], doc_rows: list[list[float]], k: int = 5) -> list[list[int]]:
    doc_hat = [normalize(row) for row in doc_rows]
    ranking: list[list[int]] = []
    for query in query_rows:
        query_hat = normalize(query)
        scores = [dot(doc, query_hat) for doc in doc_hat]
        order = sorted(range(len(scores)), key=lambda index: -scores[index])
        ranking.append(order[:k])
    return ranking

def apply_R(rows: list[list[float]], matrix: list[list[float]]) -> list[list[float]]:
    n_out = len(matrix[0])
    return [
        [sum(row[i] * matrix[i][j] for i in range(len(row))) for j in range(n_out)]
        for row in rows
    ]

gold = neighbors(queries, documents)
n_out = 16
proj_rng = random.Random(7)
scale = 1.0 / math.sqrt(n_out)
projection = [[proj_rng.gauss(0, scale) for _ in range(n_out)] for _ in range(48)]
short_docs = apply_R(documents, projection)
short_queries = apply_R(queries, projection)
predicted = neighbors(short_queries, short_docs)
recall = sum(
    len(set(left) & set(right)) / len(left)
    for left, right in zip(gold, predicted)
) / len(gold)

print(f"Bytes/vector: {48 * 4} -> {n_out * 4}")
print(f"Neighbor recall@5: {recall:.3f}")

Output

Bytes/vector: 192 -> 64
Neighbor recall@5: 0.317

Random projection is worth benchmarking when fitting a data-dependent reducer is inconvenient or expensive. It isn't automatically better or worse than PCA. Let the same labeled retrieval set judge both.

Compression after reduction: quantization

Projection changes how many coordinates a vector has. Quantization changes how precisely those coordinates are stored. You can use the two levers independently or together.

For example, a 256-dimensional float32 vector occupies 256 × 4 = 1,024 bytes. Storing one signed byte per dimension uses 256 bytes. Storing one bit per dimension uses 32 bytes. The smaller representations buy memory by accepting more approximation.

Scalar and binary codes

Scalar quantization maps each value onto an integer grid. Binary quantization records only a sign or threshold result. A compressed index can search cheaply, then rescore a larger candidate set with a more accurate representation. Qdrant's quantization guide documents this as rescore plus oversampling: if oversampling is 2.4 and the limit is 100, the compressed index preselects 240 candidates, then the original vectors rescore that shortlist down to 100.^{[5]Reference 5Qdrant Quantization Guidehttps://qdrant.tech/documentation/guides/quantization/} The same guide currently treats 4-bit TurboQuant (a random rotation followed by a learned quantizer) as a strong default at 8× compression, and it recommends enabling rescoring for binary codes.

Before trusting compressed candidates, separate the mechanical win from the quality question. The next lab quantizes four normalized vectors to int8 and converts their signs to bits. The bit count is exact; the quality decision still needs retrieval measurements.

scalar-and-binary-codes.py

vectors = [
    [0.90, 0.80, -0.10, -0.20, 0.04, 0.02, -0.05, 0.08],
    [0.85, 0.74, -0.04, -0.16, 0.01, 0.04, -0.02, 0.05],
    [-0.02, 0.04, 0.92, 0.81, -0.06, 0.01, 0.02, -0.04],
    [-0.05, 0.03, 0.88, 0.79, -0.03, 0.00, 0.06, -0.02],
]

peak = max(abs(value) for row in vectors for value in row)
scale = 127 / peak
int8_vectors = [
    [max(-128, min(127, int(round(value * scale)))) for value in row]
    for row in vectors
]
binary_vectors = [[value > 0 for value in row] for row in vectors]
query_bits = binary_vectors[0]
hamming = [
    sum(left != right for left, right in zip(row, query_bits))
    for row in binary_vectors
]

print(f"float32 bytes/vector: {len(vectors[0]) * 4}")
print(f"int8 bytes/vector:    {len(int8_vectors[0])}")
print(f"binary bits/vector:   {len(binary_vectors[0])}")
print(f"Binary Hamming distances from row 0: {hamming}")

Output

float32 bytes/vector: 32
int8 bytes/vector:    8
binary bits/vector:   8
Binary Hamming distances from row 0: [0, 0, 6, 7]

Binary codes are tiny, but "same sign" is a much weaker statement than "same semantic evidence." For a policy assistant, retrieve more candidates with a compressed representation and rerank before final evidence selection if benchmarks justify that design. Measure the oversample factor against final recall and end-to-end latency; too small and true neighbors drop before rerank, too large and you pay second-stage cost without quality gain.

Product quantization: code sub-vectors

Binary codes have shown the precision tradeoff. Product Quantization (PQ) changes the storage unit again: it splits a vector into sub-vectors, learns a codebook for each subspace, and stores only the selected centroid IDs.^{[6]Reference 6Product Quantization for Nearest Neighbor Search.https://dblp.org/rec/journals/pami/JegouDS11} In the common case of 256 possible centroids, each ID fits in one byte. A 128-dimensional float32 vector split into eight subspaces can therefore be represented by eight code bytes, plus shared codebooks.

Product quantization partitions a 128-dimensional vector into eight subspaces, replaces each block with a learned centroid ID, stores eight bytes, and scores a full-precision query through lookup-table sums. — PQ stores eight centroid choices instead of 128 floats. ADC keeps the query uncompressed, builds query-to-centroid lookup tables, and lets each stored byte select one distance to add.

The original PQ paper describes asymmetric distance computation (ADC): keep a query unquantized, precompute its distances to codebook centroids, and score each stored code through table lookups.^{[6]Reference 6Product Quantization for Nearest Neighbor Search.https://dblp.org/rec/journals/pami/JegouDS11} The miniature implementation uses only two subspaces and four centroids so that every shape remains visible. Predict what shrinks: the stored code gets shorter, while the shared codebooks stay available for scoring.

mini-product-quantization.py

import random

rng = random.Random(12)
documents = (
    [[rng.gauss(1, 0.08), rng.gauss(1, 0.08), rng.gauss(0, 0.08), rng.gauss(0, 0.08)] for _ in range(20)]
    + [[rng.gauss(0, 0.08), rng.gauss(0, 0.08), rng.gauss(1, 0.08), rng.gauss(1, 0.08)] for _ in range(20)]
)
query = [0.94, 1.02, 0.01, -0.02]

def mean_col(rows: list[list[float]]) -> list[float]:
    n_rows, n_cols = len(rows), len(rows[0])
    return [sum(row[j] for row in rows) / n_rows for j in range(n_cols)]

def kmeans(rows: list[list[float]], k: int, seed: int) -> tuple[list[list[float]], list[int]]:
    local = random.Random(seed)
    centroids = [rows[local.randrange(len(rows))][:] ]
    while len(centroids) < k:
        weights = []
        for row in rows:
            nearest = min(
                sum((row[j] - center[j]) ** 2 for j in range(len(row)))
                for center in centroids
            )
            weights.append(nearest)
        pick = local.random() * (sum(weights) or 1.0)
        acc = 0.0
        chosen = rows[-1]
        for row, weight in zip(rows, weights):
            acc += weight
            if acc >= pick:
                chosen = row
                break
        centroids.append(chosen[:])
    labels = [0] * len(rows)
    for _ in range(15):
        for i, row in enumerate(rows):
            labels[i] = min(
                range(k),
                key=lambda c: sum((row[j] - centroids[c][j]) ** 2 for j in range(len(row))),
            )
        for cluster in range(k):
            members = [rows[i] for i, label in enumerate(labels) if label == cluster]
            if members:
                centroids[cluster] = mean_col(members)
    return centroids, labels

subspace_width = 2
codebooks: list[list[list[float]]] = []
codes: list[list[int]] = []
for start in range(0, len(documents[0]), subspace_width):
    block = [row[start:start + subspace_width] for row in documents]
    centroids, labels = kmeans(block, k=4, seed=start)
    codebooks.append(centroids)
    codes.append(labels)

adc_scores = [0.0] * len(documents)
for subspace, start in enumerate(range(0, len(documents[0]), subspace_width)):
    query_block = query[start:start + subspace_width]
    table = [
        sum((center[j] - query_block[j]) ** 2 for j in range(subspace_width))
        for center in codebooks[subspace]
    ]
    for i, code in enumerate(codes[subspace]):
        adc_scores[i] += table[code]

best = min(range(len(adc_scores)), key=lambda i: adc_scores[i])
print(f"Document shape: ({len(documents)}, {len(documents[0])})")
print(f"Stored code shape: ({len(documents)}, {len(codes)})")
print(f"Raw bytes/vector: {len(documents[0]) * 8} (float64 in this lab)")
print(f"Code bytes/vector with <=256 centroids: {len(codes)}")
print(f"Nearest PQ-coded document group: {'rotation' if best < 20 else 'merge'}")

Output

Document shape: (40, 4)
Stored code shape: (40, 2)
Raw bytes/vector: 32 (float64 in this lab)
Code bytes/vector with <=256 centroids: 2
Nearest PQ-coded document group: rotation

PQ isn't an interchangeable synonym for PCA. PCA shortens a coordinate system; PQ encodes sub-vectors with learned IDs. An index may use a shortened embedding and then PQ-code it when memory is tight.

Native shortening: train prefixes to be useful

PCA and random projection operate after an encoder produces its vector. Native shortening moves the decision into training: Matryoshka Representation Learning (MRL) optimizes the model so prefixes of its representation remain useful at several selected lengths.^{[7]Reference 7Matryoshka Representation Learning.https://arxiv.org/abs/2205.13147}

For an embedding $e$ and prefix lengths in a set $\mathcal{M}$ , a simplified view of the original nested objective is a weighted sum of task losses, each computed from a prefix $e_{:m}$ through its own linear head $W^{(m)}$ :

$\min \sum_{m \in \mathcal{M}} c_m \,\mathcal{L}\bigl(W^{(m)} e_{:m}\bigr)$

Unlike PCA, serving a shorter MRL-trained representation doesn't require a separately fitted rotation. It still requires matching document and query lengths and testing retrieval quality. The original paper also notes that intermediate lengths between the trained cuts often interpolate reasonably, but that still isn't a substitute for a product recall gate.

A Matryoshka embedding exposes nested prefixes at 64, 128, 256, and 1024 dimensions; an illustrative recall curve clears its gate at 256 dimensions, which is then shared by documents and queries. — MRL trains selected prefixes to carry task signal. Production then benchmarks supported cuts and applies one chosen length identically to document and query embeddings. The recall curve in the figure is schematic, not a product benchmark.

The lab makes the promise concrete with deliberately ordered vectors. It isn't pretending to train an embedding model: the first coordinates carry policy-theme information, while later coordinates add detail and noise. Truncating a standard, unordered representation wouldn't have this promise. Predict which prefix should recover all three themes before running it.

evaluate-ordered-prefixes.py

import math
import random

rng = random.Random(15)
centers = [
    [1.0, 0.9, 0.3, 0.2, 0.0, 0.0, 0.0, 0.0],
    [0.0, 0.0, 1.0, 0.9, 0.3, 0.2, 0.0, 0.0],
    [0.0, 0.0, 0.0, 0.0, 1.0, 0.9, 0.3, 0.2],
]
documents = [
    [center[j] + rng.gauss(0, 0.04) for j in range(8)]
    for center in centers
    for _ in range(18)
]
queries = [
    [center[j] + rng.gauss(0, 0.02) for j in range(8)]
    for center in centers
    for _ in range(3)
]

def dot(left: list[float], right: list[float]) -> float:
    return sum(a * b for a, b in zip(left, right))

def normalize(vector: list[float]) -> list[float]:
    scale = math.sqrt(dot(vector, vector))
    return [value / scale for value in vector]

def top3(rows: list[list[float]], corpus: list[list[float]]) -> list[list[int]]:
    corpus_hat = [normalize(row) for row in corpus]
    ranking: list[list[int]] = []
    for query in rows:
        query_hat = normalize(query)
        scores = [dot(doc, query_hat) for doc in corpus_hat]
        order = sorted(range(len(scores)), key=lambda index: -scores[index])
        ranking.append(order[:3])
    return ranking

gold = top3(queries, documents)
for prefix in [2, 4, 8]:
    shortened = top3(
        [query[:prefix] for query in queries],
        [doc[:prefix] for doc in documents],
    )
    recall = sum(
        len(set(expected) & set(found)) / 3
        for expected, found in zip(gold, shortened)
    ) / len(gold)
    print(f"Prefix {prefix}: recall@3={recall:.3f}")

Output

Prefix 2: recall@3=0.185
Prefix 4: recall@3=0.407
Prefix 8: recall@3=1.000

One hosted API example is OpenAI's text-embedding-3 family. Its documentation lists default lengths of 1,536 for text-embedding-3-small and 3,072 for text-embedding-3-large, and documents a dimensions request parameter for shorter outputs.^{[8]Reference 8Vector embeddingshttps://developers.openai.com/api/docs/guides/embeddings} On MTEB, that page reports that a 256-dimensional text-embedding-3-large vector still outperforms an unshortened 1,536-dimensional text-embedding-ada-002 vector. That's a public benchmark comparison, not a promise about your rotation-ticket queries.

A request shape looks like this:

embedding-request.json

{
  "model": "text-embedding-3-large",
  "input": "My service-account key is 18 days old. Can I keep using it?",
  "dimensions": 256
}

⚠️ Common mistake: If you slice a returned vector yourself, L2-normalize the prefix again. OpenAI's dimensions parameter handles that on their side; a manual cut doesn't.

The provider feature makes generation of shorter vectors convenient. It doesn't decide whether 256 dimensions meet your 18-day-key retrieval requirement.

Choose a deployment experiment

The methods change different parts of the representation. Before choosing one, name the knob you need to turn: coordinates after encoding, coordinates learned into the encoder, or bits used to store each coordinate. The map methods stay on the investigation side.

These methods fall into three serving families and one diagnostic family:

Tool family	What changes	Best first question
PCA or random projection	Number of served coordinates after encoding	How much recall survives at a lower dimension?
MRL-style native shortening	Number of coordinates emitted from a trained model	Does a supported prefix beat post-hoc alternatives at the same cost?
Scalar, binary, or product quantization	Precision or code used to store/search vectors	Can compressed candidate search plus reranking fit the RAM budget?
t-SNE or UMAP	Coordinates used for inspection maps	What failures should an engineer examine next?

The table becomes useful when it turns into a decision someone else can audit. This lab accepts only configurations that meet both a recall floor and a raw-vector memory budget, then chooses the smallest accepted representation. Predict the winner from the two thresholds before reading the output.

select-a-serving-candidate.py

measurements = [
    {"method": "float32-full", "bytes_per_vector": 6144, "recall_at_10": 1.000},
    {"method": "native-256", "bytes_per_vector": 1024, "recall_at_10": 0.984},
    {"method": "pca-256", "bytes_per_vector": 1024, "recall_at_10": 0.971},
    {"method": "pq-16-code", "bytes_per_vector": 16, "recall_at_10": 0.939},
]

required_recall = 0.975
maximum_bytes = 1200
accepted = [
    row for row in measurements
    if row["recall_at_10"] >= required_recall
    and row["bytes_per_vector"] <= maximum_bytes
]
choice = min(accepted, key=lambda row: row["bytes_per_vector"])

for row in measurements:
    status = "pass" if row in accepted else "reject"
    print(f"{row['method']:<14} recall={row['recall_at_10']:.3f} "
          f"bytes={row['bytes_per_vector']:<4} {status}")
print(f"Selected representation: {choice['method']}")

Output

float32-full   recall=1.000 bytes=6144 reject
native-256     recall=0.984 bytes=1024 pass
pca-256        recall=0.971 bytes=1024 reject
pq-16-code     recall=0.939 bytes=16   reject
Selected representation: native-256

The values above are example measurements for the gate, not benchmark results for a real model. A serious report would include:

A frozen set of realistic policy queries and judged clauses.
Recall or nDCG at the candidate count sent to generation or reranking.
Index RAM, build time, and query-latency percentiles for each candidate.
A failure list: which queries lost relevant evidence, and what risk that creates.
Re-evaluation when embedding models, policy documents, or traffic distributions change.

Carry the decision rule forward

You can now calculate the raw cost of a vector representation, establish cosine-retrieval neighbors before changing it, fit PCA once and reuse its transform, and benchmark random projection as a no-fit alternative. For inspection, t-SNE and UMAP can surface cases without becoming search coordinates. For storage, fewer dimensions, lower precision, PQ codes, and model-supported prefixes such as MRL-style shortening are different levers, so each one goes through the same retrieval gate.

No method wins by name alone. Compression is an experiment with a quality constraint: storage savings are immediate, while semantic safety has to be measured.

Next Step

Continue to CoT, ToT & Self-Consistency Prompting

Compact embeddings can put the right policy clause in context. Next you'll treat extra inference-time work as a controller: one reasoning path, several votes, or a bounded tree search over those retrieved facts.

PreviousInstruction Tuning & Chat Templates

Share this article