What Is Metric Learning? Complete Guide 2026
- 23 hours ago
- 36 min read
Every recommendation engine, face-verification system, and product-search bar depends on one quiet question: how do you tell a computer that two things are "similar"? Metric learning is the branch of machine learning built to answer that question properly, and getting it wrong quietly wrecks search results, recommendation quality, and fraud detection long before anyone notices why.
TL;DR
Metric learning trains a model to produce a distance or similarity score that reflects real, meaningful relationships between items, rather than relying on a fixed formula like raw Euclidean distance.
It can learn a Mahalanobis matrix, a low-dimensional projection, or — in deep learning — a full neural network that maps raw data into an embedding space.
Contrastive loss and triplet loss are two well-known training objectives, but metric learning is a broader field that also includes proxy-based, classification-style, and pair-mining approaches.
It shines in open-set problems — face verification, product search, duplicate detection — where new categories appear after training and a fixed classifier output layer breaks down.
Success depends as much on how training pairs and batches are constructed as on which loss function is chosen.
What Is Metric Learning?
Metric learning is a branch of machine learning that trains a model to compute a distance or similarity score reflecting real semantic relationships between data points, so that similar items end up close together and dissimilar items end up far apart in a learned space, rather than relying on a fixed, hand-picked distance formula.
Table of Contents
Definition and Intuition
In one sentence: metric learning is the process of training a model to produce a distance or similarity score that captures real, task-relevant relationships between data points, instead of using a fixed formula applied to raw features.
A slightly deeper version: rather than hand-coding "distance" as, say, the straight-line difference between two vectors of pixel values, metric learning lets an algorithm — sometimes a linear transformation, sometimes a full neural network — decide what "close" and "far" should mean, using examples of things that are known to be alike or different.
A useful everyday analogy: a wine novice judges two wines by color and price. A sommelier judges them by tannin structure, acidity, and finish — traits invisible to a casual glance. Metric learning is the process of training a system to develop the sommelier's sense of similarity instead of the novice's, based on examples of wines that experts agree are alike.
A concrete machine-learning example: imagine a photo app that must recognize a person it has never seen labeled before. A standard image classification model can only choose among the classes it was trained on. Metric learning instead trains an encoder to place two photos of the same person close together in a vector space, and photos of different people far apart, so the system can compare any two faces — including ones from brand-new identities — using nothing but distance.
The core goal is often summarized as "pull together, push apart": pull semantically similar points closer, and push dissimilar points farther apart in the learned space. That shorthand is useful, but it has limits. It says nothing about how much closer or farther, it can ignore intra-class diversity (a bulldog and a greyhound are still both "dog"), and pushing too hard on negatives can distort the geometry of the whole space. Good metric learning balances local separation with a globally sensible geometry, not just maximal push and pull.
Why Fixed Distance Functions Fall Short
Raw features rarely encode similarity in a way that a simple distance formula can exploit correctly. Several problems recur:
Scale differences. If one feature is measured in dollars (0–1,000,000) and another in years (0–100), an unweighted Euclidean distance will be dominated by the dollar feature purely because of its numeric range, not because it matters more.
Irrelevant dimensions. Background lighting, camera angle, and JPEG compression artifacts all change pixel values without changing who is in a photo. A distance function that treats every pixel equally treats noise and identity as equally important.
Correlated features. When features move together, ordinary Euclidean distance effectively double-counts the same signal, distorting the notion of "how different" two points really are.
Semantic vs. surface similarity. Two product photos can look nearly identical (similar surface pixels) yet be different SKUs, while two photos of the same shoe taken from different angles can look quite different at the pixel level yet represent the same item. Surface-level distance gets both cases backwards.
High dimensionality. In high-dimensional spaces, distances tend to concentrate — the ratio between the nearest and farthest points shrinks — a phenomenon well documented in the distance-concentration literature on high-dimensional geometry [1]. That makes raw-feature nearest-neighbor search increasingly unreliable as dimensionality grows.
A simple numeric illustration: consider two customers described by (income in dollars, age in years). Customer A is (52,000, 34) and Customer B is (54,000, 60). The raw Euclidean distance is dominated almost entirely by the 2,000-dollar gap; the 26-year age gap, which may matter far more for the task at hand, barely registers. A learned or rescaled metric can correct this imbalance directly.
What a Metric-Learning System Actually Learns
Depending on the approach, a metric-learning system may learn one of several things:
A linear transformation of the original feature space, so that distances computed after the transformation reflect task-relevant similarity.
A Mahalanobis matrix that reweights and rotates the input space according to learned feature importance and correlation structure.
A low-dimensional projection, useful for visualization, indexing, or removing redundant dimensions — conceptually related to classical techniques like Principal Component Analysis and Linear Discriminant Analysis, though those are typically unsupervised or class-mean based rather than optimized directly for pairwise or triplet relationships.
A neural embedding function, commonly a deep neural network that turns raw images, audio, or text into a fixed-length vector embedding.
A learned similarity function that is not a strict distance at all, but a scoring function trained to rank items by relevance.
A combination: an encoder that produces embeddings, paired with a comparison rule (often plain Euclidean distance or cosine similarity) applied afterward.
Picture the process as two stages, described rather than drawn: raw inputs enter an encoder box, which outputs points scattered somewhere in a high-dimensional space; a comparison rule then measures the distance between any two of those output points. Training adjusts the encoder — not the comparison rule — so that the resulting arrangement of points reflects true similarity.
Mathematical Foundations
Metric axioms
A function $d(x, y)$ is a true mathematical metric only if it satisfies four properties: non-negativity ($d(x,y) \geq 0$), identity of indiscernibles ($d(x,y) = 0$ if and only if $x = y$), symmetry ($d(x,y) = d(y,x)$), and the triangle inequality ($d(x,z) \leq d(x,y) + d(y,z)$).
Many learned comparison functions used in practice do not satisfy all four. A learned similarity score can be asymmetric (relevance of a query to a document need not equal the reverse), and some learned functions violate the triangle inequality. Practitioners often use "metric learning" loosely to describe learning any comparison function — a pseudometric, a semimetric, or a plain similarity score — because the practical goal (useful ranking and retrieval) does not always require strict metric properties. The distinction matters in practice: if the triangle inequality fails, transitive reasoning like "A is near B, B is near C, so A is roughly near C" can silently break, which affects clustering and indexing algorithms that assume it holds.
Distance and embedding notation
A data point is represented as a feature vector $x \in \mathbb{R}^d$. An embedding function maps it into a new space: $z = f_\theta(x)$, where $\theta$ represents the learnable parameters (a matrix, or the weights of a neural network). Similarity or distance is then computed on the embeddings $z_1, z_2$, not on the raw inputs.
Euclidean distance:
$$d(z_1, z_2) = \sqrt{\sum_{i=1}^{k} (z_{1i} - z_{2i})^2}$$
This measures straight-line distance in the embedding space. It is easy to interpret but sensitive to vector magnitude.
Manhattan distance, briefly, sums absolute differences instead of squared differences: $d(z_1, z_2) = \sum_i |z_{1i} - z_{2i}|$. It is less sensitive to outliers along any single dimension and occasionally preferred for sparse or count-based features.
Cosine similarity measures the angle between two vectors rather than their magnitude:
$$\text{cos}(z_1, z_2) = \frac{z_1 \cdot z_2}{|z_1| |z_2|}$$
A value near 1 means the vectors point in almost the same direction (very similar); near 0 means they are unrelated; near −1 means opposite. Cosine similarity is popular in text and image retrieval because it ignores overall vector length, which often reflects incidental factors like document length or image brightness rather than meaning.
Mahalanobis distance generalizes Euclidean distance by incorporating a learned matrix $M$:
$$d_M(x_1, x_2) = \sqrt{(x_1 - x_2)^\top M (x_1 - x_2)}$$
For this to behave like a valid distance, $M$ must be positive semidefinite — informally, it must never produce a negative squared distance no matter which direction is measured. This constraint is commonly satisfied by factoring $M = L^\top L$ for some matrix $L$, which guarantees positive semidefiniteness automatically. Under this factorization, the Mahalanobis distance can be rewritten as an ordinary Euclidean distance after applying the linear transformation $L$: $d_M(x_1, x_2) = |Lx_1 - Lx_2|_2$. This is the conceptual bridge between classical Mahalanobis metric learning and deep metric learning — both ultimately apply a transformation and then measure Euclidean or cosine distance; deep methods simply replace the linear $L$ with a nonlinear neural network.
Normalization
L2 normalization rescales every embedding to unit length: $\hat{z} = z / |z|_2$. Once vectors are unit-length, Euclidean distance and cosine similarity become deterministic functions of each other:
$$|\hat{z}_1 - \hat{z}_2|_2^2 = 2 - 2\cos(\hat{z}_1, \hat{z}_2)$$
This is why many deep metric-learning systems normalize embeddings before computing distance — it lets practitioners reason in terms of angles while still using simple Euclidean operations, and it bounds the possible distance values, which stabilizes training.
Pairwise vs. ranking objectives
A pairwise objective looks at two points at a time and asks whether they should be close or far. A ranking objective — used by triplet, listwise, and many retrieval losses — looks at relative order: is this positive closer than that negative, regardless of the exact distances involved? Ranking objectives are often more forgiving because they only need relative geometry to be correct, not absolute distances.
Local vs. global geometry
Classical methods like Neighborhood Components Analysis focus on local neighborhood structure — who is nearby a given point — which can produce excellent local separation while sacrificing a clean global layout. Deep methods with proxy or classification-style objectives tend to impose more global structure (one region per class) at some cost to fine local detail. Neither is universally superior; the right balance depends on whether the deployed task cares more about nearest-neighbor accuracy or about overall cluster structure.
Classical Distance Metric Learning
Before deep learning, metric learning largely meant learning a linear transformation or Mahalanobis matrix from labeled or weakly labeled data.
Relevant Component Analysis (RCA), briefly, uses "chunklets" of points known to belong together (without needing full class labels) to compute a transformation that reduces within-chunklet variance, effectively whitening the noise directions shared by similar points.
Neighborhood Components Analysis (NCA), introduced by Goldberger, Roweis, Hinton, and Salakhutdinov at NeurIPS 2004, learns a linear (or low-rank) transformation that directly maximizes a stochastic approximation of leave-one-out k-nearest-neighbor accuracy [2]. It uses class-label supervision and optimizes a smooth, probabilistic version of the KNN decision rule. Its strength is that it optimizes the actual downstream task (nearest-neighbor classification) rather than a proxy; its limitation is that the objective is non-convex and can be sensitive to initialization, and it scales less gracefully to very high-dimensional or very large datasets than later deep approaches.
Large Margin Nearest Neighbor (LMNN), proposed by Weinberger and Saul, learns a Mahalanobis metric with the explicit goal that each point's same-class neighbors lie within a margin closer than its different-class neighbors, formulated as a convex optimization problem solvable with semidefinite programming [3]. It uses class labels and a "target neighbors" structure defined ahead of training. Its strength is a convex, globally solvable objective with strong guarantees for k-NN classification; its limitation is that a fixed neighborhood structure, chosen before optimization, can be suboptimal if the initial (Euclidean) neighbors are already poor.
Information-Theoretic Metric Learning (ITML), from Davis, Kulis, Jain, Sra, and Dhillon, frames Mahalanobis metric learning as minimizing the differential relative entropy (KL divergence) between two multivariate Gaussians, subject to pairwise distance constraints [4]. It uses similarity/dissimilarity pair supervision rather than full class labels, which makes it flexible for weakly supervised settings. Its strength is an elegant, regularized formulation with an efficient Bregman-projection optimization; its limitation is that, like other classical Mahalanobis methods, it is restricted to a linear transformation of the input space, which caps its expressiveness on complex data like raw images.
Mahalanobis metric learning more broadly covers the family of approaches above: they differ in optimization strategy and supervision but share the same mathematical object — a positive-semidefinite matrix reweighting the input space. Their common strength is interpretability and strong theoretical guarantees on modest-sized, modestly-dimensional data; their common limitation is that a purely linear transformation cannot capture the nonlinear structure present in raw pixels, audio waveforms, or free text, which is exactly the gap deep metric learning was built to close.
What Is Deep Metric Learning?
Deep metric learning replaces (or extends) the linear transformation of classical methods with a neural network encoder, allowing the system to learn nonlinear feature representations directly from raw data such as images, audio, or text.
The typical architecture has a backbone (a convolutional network, transformer, or similar) that extracts general-purpose features, followed by an embedding head — usually one or a few fully connected layers — that projects those features into the final embedding space, often followed by L2 normalization. The embedding dimensionality (commonly ranging from 64 to 512 in published work, though this varies widely by task) is a design choice with real tradeoffs, discussed later.
"Siamese" describes an architectural arrangement — two or more branches of the network sharing the same weights, processing different inputs (for example, an anchor and a positive) in parallel — not a specific loss function. The term originates from a 1994 IEEE paper on signature verification using a "Siamese" time-delay neural network [5], long before modern deep metric learning existed, and later underpins the dimensionality-reduction and face-verification work discussed below. A Siamese pair, a Siamese triplet, and a Siamese network trained with an N-pair loss are all "Siamese" in structure but use different loss functions.
During training, the network typically processes pairs or triplets of shared-weight branches and computes a loss over the resulting embeddings. At inference time, retrieval typically works differently: embeddings for an entire corpus are computed once and stored in an index, and a query embedding is compared against that index — no paired forward passes are needed at query time. This asymmetry between training-time comparison and inference-time retrieval is a key production consideration covered later.
Major Loss-Function Families
Pair-based losses
Contrastive loss, formalized for dimensionality reduction by Hadsell, Chopra, and LeCun in "Dimensionality Reduction by Learning an Invariant Mapping" (CVPR 2006), operates on pairs labeled as similar or dissimilar [6]. For a similar pair, it penalizes distance directly; for a dissimilar pair, it penalizes the shortfall below a margin $m$:
$$L = y \cdot d^2 + (1-y) \cdot \max(0, m - d)^2$$
where $y = 1$ for a positive pair and $y = 0$ for a negative pair. Its benefit is simplicity and interpretability; its limitation is pair imbalance — random pair sampling produces vastly more easy negatives than informative ones, and performance depends heavily on how pairs are selected, not just on the formula itself.
Triplet-based losses
Triplet loss, popularized at scale by Schroff, Kalenichenko, and Philbin's FaceNet paper (CVPR 2015) [7], operates on an anchor $a$, a positive $p$ (same class as the anchor), and a negative $n$ (different class):
$$L = \max(0, d(a,p) - d(a,n) + m)$$
The margin $m$ enforces that the negative must be farther from the anchor than the positive by at least that amount, not merely farther. Triplets are commonly categorized by difficulty: an easy negative is already farther than the margin requires and contributes zero gradient; a semi-hard negative is farther than the positive but still inside the margin, contributing a useful, stable gradient; a hard negative is closer to the anchor than the positive, contributing a large gradient that can also destabilize training if mined too aggressively. FaceNet's authors found that naive random triplet sampling wastes most training steps on uninformative easy triplets, which is why mining strategy — not just the loss formula — became central to the field.
Walkthrough example (labeled as simplified): Consider an anchor photo of "Person X" smiling. A positive is a second photo of Person X frowning — same identity, different expression. An easy negative is a photo of a young child, already embedded far away due to obviously different facial structure. A hard negative is a photo of Person X's sibling, who shares similar bone structure and embeds suspiciously close to Person X before training. Before training, in an untrained or poorly trained embedding space, the sibling (hard negative) may actually sit closer to the anchor than the true positive does — a wrong ordering. As triplet loss training proceeds, gradients pull the anchor and true positive closer together while pushing the anchor and sibling apart, gradually correcting the ordering so that, after sufficient training, the true positive is reliably closer to the anchor than the sibling is, even though both faces remain visually similar to a casual human observer.
Multi-sample and ranking losses
N-pair loss, introduced by Sohn (NeurIPS 2016), generalizes triplet loss by comparing one positive against multiple negatives simultaneously within a batch, which produces richer gradients per update and reduces the need for expensive explicit triplet construction [8]. Lifted-structure losses similarly exploit all pairwise relationships within a batch rather than isolated triplets. Multi-similarity loss, from Wang, Han, Huang, Dong, and Scott (CVPR 2019), weights pairs according to both self-similarity and relative similarity to other samples in the batch, aiming to capture more of the informative structure that basic pair or triplet sampling misses [9]. These batch-based objectives generally trade some conceptual simplicity for better sample efficiency, since a single batch forward pass yields gradient signal from many more relationships than one triplet at a time.
Proxy-based losses
Proxy-based methods, most notably Proxy-NCA from Movshovitz-Attias, Toshev, Leung, Ioffe, and Singh (ICCV 2017), introduce a small set of learned "proxy" points — one or more per class — and train each real example to be close to its class proxy and far from other proxies, rather than comparing directly against other real examples [10]. This converts an $O(n^3)$ or $O(n^2)$ triplet/pair problem into something closer to $O(n)$, since only proxy comparisons are needed at each step, greatly improving training efficiency and convergence speed on large datasets. The limitation is that a single proxy per class assumes each class occupies one coherent region in embedding space; classes with genuinely multimodal internal structure (a "chair" category spanning armchairs, office chairs, and folding chairs) can be poorly served by one static proxy.
Angular and cosine-margin objectives
A family of losses reformulates classification-style objectives around the angle between an embedding and its class weight vector rather than raw dot products, adding an angular or cosine margin to force greater separation between classes on the unit hypersphere. These objectives grew largely out of face-recognition research and vary meaningfully in their exact margin formulation and geometric assumptions — they should not be treated as one interchangeable method, since the choice of margin type and additive vs. multiplicative placement changes both the optimization behavior and the resulting embedding geometry.
Supervised contrastive learning
Supervised contrastive learning, formalized by Khosla and colleagues at NeurIPS 2020, extends the self-supervised contrastive framework by using label information to define multiple positives per anchor within a batch (all other same-class examples), rather than relying solely on data augmentation to generate a single positive per anchor [11]. It is batch-dependent — its effectiveness relies on large batches containing enough same-class examples — and sits conceptually between classification and pure metric learning. It is not synonymous with all of metric learning: it is one specific batch-contrastive formulation among many pair-, triplet-, and proxy-based alternatives, and it is best understood as a bridge between self-supervised contrastive representation learning and supervised deep metric learning, rather than as the field's definition.
Loss family | Training unit | Main advantage | Main limitation | Common use case |
Contrastive | Pair | Simple, interpretable | Needs careful pair sampling | Verification, duplicate detection |
Triplet | Anchor-positive-negative | Directly enforces relative ranking | Many triplets give near-zero gradient | Face and product retrieval |
N-pair / Multi-similarity | Batch of many samples | Richer gradient per step | More complex batch construction | Large-scale image retrieval |
Proxy-based | Sample vs. learned proxy | Fast convergence, scales well | Struggles with multimodal classes | Large closed or semi-open catalogs |
Supervised contrastive | Batch with multiple positives | Strong representations, robust to noise | Needs large batches | Pretraining before fine-tuning |
Pair, Triplet, and Batch Construction
Data construction often matters as much as loss selection, because the loss function only ever sees the pairs, triplets, or batches it is handed.
Random sampling is simple but, as FaceNet's authors observed, produces overwhelmingly easy examples that contribute little useful gradient once training has progressed past the earliest stages. Class-balanced sampling — drawing a fixed number of samples per class into each batch — ensures enough same-class examples exist for meaningful positive pairs, which is essential for both triplet mining and supervised contrastive batches. Online mining selects hard or semi-hard examples from within the current batch during training, adapting as the embedding space evolves; offline mining precomputes difficult examples using a snapshot of the model, which is more stable but can go stale as the model changes.
Hard-negative mining deliberately selects negatives that are currently close to the anchor, producing strong gradients — but mining too aggressively toward the very hardest examples risks selecting false negatives: two items that are actually the same identity or category but were mislabeled or under-annotated as different. Semi-hard mining, FaceNet's compromise, selects negatives that violate the margin but are still farther than the positive, avoiding the instability of the very hardest (and possibly mislabeled) examples.
In-batch negatives treat every other example in a batch as an implicit negative for a given anchor, which is computationally efficient but limits the negative pool to whatever happens to be in that batch. Cross-batch memory or memory banks address this by storing embeddings from recent batches, effectively expanding the negative pool without expanding the batch size itself, at the cost of using slightly stale embeddings.
Label noise — mislabeled pairs or triplets — is especially damaging to hard-mining strategies, since noisy labels are disproportionately likely to look "hard" precisely because they are wrong. Curriculum strategies that start training on easier examples and gradually introduce harder ones can improve stability, echoing the broader idea behind curriculum learning.
Practical recommendations: favor class-balanced batches over pure random sampling; use semi-hard rather than pure hardest-negative mining as a default starting point; audit a sample of "hard" negatives manually early in a project to check for label noise before trusting the mining pipeline; and increase batch size when GPU memory allows, since larger batches generally provide a richer, more representative pool of both positives and negatives.
A Complete Training Workflow
1. Define what "similar" means for the application. Same person, same product SKU, same document topic, and same customer intent are all different definitions of "similar," and the definition shapes everything downstream.
2. Choose the supervision source. This can be hard class labels, weak pairwise judgments (from click logs or human raters), or self-supervised augmentation-based positives.
3. Create train, validation, and test splits without identity or entity leakage. In verification and retrieval tasks, splits must be made at the identity or entity level — never split individual photos of the same person across train and test, since that leaks identity information and inflates evaluation scores.
4. Select an encoder appropriate to the input modality — a convolutional or transformer backbone for images, a transformer for text, a specialized architecture for audio.
5. Choose embedding dimensionality. Larger dimensions can capture more nuance but cost more storage and search latency; smaller dimensions are cheaper but risk losing discriminative detail (discussed further below).
6. Select normalization and similarity functions, typically L2-normalized embeddings compared via cosine similarity, or unnormalized embeddings compared via Euclidean distance.
7. Choose a loss appropriate to the data volume and class structure, using the comparison table above as a starting filter.
8. Design a sampler or miner that produces class-balanced batches and, if using triplet-style losses, an appropriate mining strategy.
9. Train and monitor, watching not just loss curves but embedding-space diagnostics such as intra-class and inter-class distance distributions.
10. Evaluate on the actual retrieval or verification task, not merely on training loss, using metrics detailed below.
11. Select operating thresholds if required — for verification tasks, this means choosing a distance cutoff that balances false accepts against false rejects for the specific application's risk tolerance.
12. Build and deploy an index for efficient nearest-neighbor search over the full corpus of embeddings.
13. Monitor drift and quality after deployment, since real-world data distributions shift and embeddings computed by an older model version may no longer be directly comparable to embeddings from a newer one.
Compact Implementation Example
The example below demonstrates a minimal triplet-based training step in PyTorch. It is intentionally simplified and is not production-ready.
import torch
import torch.nn as nn
import torch.nn.functional as F
class EmbeddingNet(nn.Module):
"""A small shared encoder producing L2-normalized embeddings."""
def __init__(self, embedding_dim=128):
super().__init__()
self.backbone = nn.Sequential(
nn.Conv2d(3, 32, 3, stride=2, padding=1), nn.ReLU(),
nn.Conv2d(32, 64, 3, stride=2, padding=1), nn.ReLU(),
nn.AdaptiveAvgPool2d(1),
)
self.head = nn.Linear(64, embedding_dim)
def forward(self, x):
# x shape: (batch_size, 3, 224, 224)
features = self.backbone(x).flatten(1) # (batch_size, 64)
embedding = self.head(features) # (batch_size, embedding_dim)
return F.normalize(embedding, p=2, dim=1) # L2 normalization
def triplet_loss(anchor, positive, negative, margin=0.2):
"""Basic triplet loss on L2-normalized embeddings."""
d_pos = (anchor - positive).pow(2).sum(dim=1)
d_neg = (anchor - negative).pow(2).sum(dim=1)
losses = F.relu(d_pos - d_neg + margin)
return losses.mean()
# Example training step
model = EmbeddingNet(embedding_dim=128)
optimizer = torch.optim.Adam(model.parameters(), lr=1e-4)
# anchor_imgs, positive_imgs, negative_imgs: (batch_size, 3, 224, 224)
anchor_emb = model(anchor_imgs)
positive_emb = model(positive_imgs)
negative_emb = model(negative_imgs)
loss = triplet_loss(anchor_emb, positive_emb, negative_emb)
optimizer.zero_grad()
loss.backward()
optimizer.step()The code shares one encoder across all three branches (a Siamese arrangement), produces L2-normalized embeddings, and computes a standard margin-based triplet loss. It omits data augmentation, an actual miner for selecting semi-hard or hard negatives, learning-rate scheduling, validation-based checkpointing, and any handling of label noise or class imbalance. For production use, the triplet-construction step shown here as pre-supplied anchor_imgs, positive_imgs, negative_imgs tensors would instead come from a proper sampler that draws class-balanced batches and applies an online mining strategy — computing all pairwise distances within a batch and selecting semi-hard negatives dynamically — rather than the naive fixed triplets implied here.
How Metric-Learning Models Are Evaluated
The correct evaluation metric depends heavily on the task.
Recall@K measures whether at least one correct match appears among the top K retrieved results — standard for image and product retrieval. Precision@K measures what fraction of the top K results are correct. Mean Average Precision (mAP) averages precision across all correct matches for a query, rewarding systems that rank all correct results highly, not just the first one. Mean Reciprocal Rank (MRR) is useful when there is exactly one correct answer per query and only its rank matters.
For verification tasks (is this the same person, yes or no), ROC curves and ROC-AUC summarize the tradeoff between true-accept rate and false-accept rate across all possible distance thresholds. The Equal Error Rate (EER) is the single threshold where the false-accept rate equals the false-reject rate, commonly reported for biometric systems. Verification accuracy at a fixed threshold is easier to communicate but hides how the system behaves at other operating points.
Clustering measures such as Normalized Mutual Information (NMI) can assess whether the embedding space naturally groups same-class points together, though NMI depends on the clustering algorithm used to define clusters over the embeddings, so it should be read as one signal among several rather than a definitive score. k-nearest-neighbor accuracy in the embedding space is a simple, widely reported proxy for embedding quality.
Calibration and threshold selection matter separately from ranking quality: two systems can produce identical Recall@K results while requiring very different distance thresholds to hit the same false-accept rate, so an operating threshold should always be chosen using validation data from the deployment distribution, not borrowed from a published benchmark.
Latency, memory use, and index size are evaluation dimensions of their own — a slightly less accurate model with 5x lower query latency may be the correct production choice.
Evaluation on unseen classes or identities is essential. The Musgrave, Maa, and Lipton paper "A Metric Learning Reality Check" (ECCV 2020) found that under carefully controlled, standardized comparisons, many published deep metric learning methods performed far more similarly to one another than the original papers suggested, largely because of inconsistent evaluation protocols, and that a well-tuned classic contrastive or triplet baseline was frequently competitive with newer proposed methods [12]. Roth, Milbich, Sinha, Gupta, Ommer, and Cohen's "Revisiting Training Strategies and Generalization Performance in Deep Metric Learning" (CVPR 2020) similarly emphasized that training and evaluation protocol choices — not just the loss function — heavily determine reported performance and real-world generalization [13]. Both papers are a caution against trusting classification accuracy alone: a model can classify seen categories perfectly while generalizing poorly to unseen ones, which is precisely the scenario metric learning is meant to address.
Data leakage risk is high in this field specifically because identity- or entity-based splits are easy to get wrong: if the same person, product, or document appears in both train and test sets (even in different photos or instances), reported performance will look far better than true generalization warrants.
Metric Learning vs. Related Concepts
Approach | Primary objective | Typical supervision | Output | Handles unseen classes | Typical use case |
Metric learning | Learn a meaningful distance/similarity | Labels, pairs, or triplets | Distance/similarity function | Yes, by design | Verification, retrieval |
Standard classification | Predict a fixed category | Class labels | Class probabilities | No (fixed label set) | Closed-set categorization |
Contrastive learning (self-supervised) | Learn general representations via augmentation | Usually unlabeled + augmentations | Embeddings | Yes | Pretraining, transfer learning |
Representation learning | Learn useful general features | Varies (supervised to unsupervised) | Feature vectors | Often yes | Broad downstream reuse |
Dimensionality reduction | Compress data, preserve structure | Usually unsupervised | Lower-dim vectors | Not the focus | Visualization, preprocessing |
Clustering | Group unlabeled points | Unsupervised | Cluster assignments | Yes, but unlabeled | Segmentation, exploration |
k-nearest neighbors | Classify/predict via neighbor votes | Labels (for classification) | Prediction or neighbor list | Depends on distance used | Simple baselines, retrieval |
A few nuances the table cannot capture. Standard classification is efficient and interpretable when the full set of categories is known ahead of time and stable, but its output layer has a fixed size — it structurally cannot recognize a brand-new person, product, or document category without retraining, which is exactly the scenario where metric learning is preferred. Representation learning is the broader umbrella that includes metric learning as one specialization: metric learning specifically optimizes for distance-based relationships, whereas representation learning more generally aims for features useful across many downstream tasks, not necessarily distance-based ones. Self-supervised contrastive learning and metric learning overlap heavily in mechanism (both push and pull embeddings) but differ in supervision source: self-supervised contrastive learning typically manufactures positives through data augmentation of the same unlabeled image, while supervised metric learning uses genuine label or identity information to define positives. Clustering and metric learning are complementary — a well-trained metric-learning embedding space often makes downstream clustering far more effective, since clustering algorithms depend heavily on the distance function operating over a meaningful space. K-nearest neighbors is a decision rule, not a representation method; it can be applied on raw features or on learned metric-learning embeddings, and its quality depends entirely on the distance function it is given.
Real-World Applications
Face or identity verification defines positives as multiple photos of the same enrolled identity and negatives as photos of different people; a common failure is degraded accuracy across demographic groups underrepresented in training data, discussed further in the ethics section.
Image and product retrieval, common in ecommerce, defines positives as different photos of the same exact SKU and negatives as visually similar but distinct products; a common failure is confusing near-duplicate products (different colorways of the same design) that a business genuinely wants distinguished.
Semantic search over documents defines positives as query-document or document-document pairs judged relevant by human raters or click behavior, and negatives as irrelevant pairs; a common failure is treating superficial keyword overlap as semantic relevance when the underlying intent differs.
Recommendation systems can use metric learning to place users and items in a shared embedding space, where proximity implies relevance; a common failure is popularity bias, where frequently interacted-with items dominate the geometry and crowd out genuinely relevant but less popular items.
Few-shot classification relies on a metric-learning embedding trained on a broad set of base classes, then classifies brand-new classes at test time using only a handful of labeled examples per new class, typically via nearest-neighbor or nearest-prototype comparison in the embedding space [14]; a common failure is poor generalization when the new classes are visually or semantically very different from the base training classes.
Speaker verification treats short audio segments from the same speaker as positives and different speakers as negatives; a common failure is sensitivity to microphone quality and background noise being mistaken for speaker identity signal.
Signature verification, historically one of the earliest applications of Siamese architectures [5], defines positives as genuine signatures from the same person and negatives as forgeries or other people's signatures; a common failure is skilled forgeries that closely mimic stroke dynamics.
Medical-image retrieval defines positives as images showing the same condition or tissue pattern, often across different patients, and negatives as images of differing pathology; a common failure is conflating visual similarity with clinical similarity when the two do not align, which is a serious concern in any decision-support context and should never replace qualified clinical judgment.
Anomaly detection can use metric learning to define "normal" as a tight cluster in embedding space, flagging points that fall far from that cluster as anomalies; a common failure is a shifting definition of "normal" over time (concept drift) that the model was never updated to reflect.
Duplicate detection, across text, images, or transaction records, defines positives as literal or near-literal duplicates; a common failure is distinguishing "near-duplicate but meaningfully different" content, such as a corrected version of a document, from a true duplicate.
Multimodal image-text retrieval trains encoders for two different modalities into a shared embedding space so that a text query can retrieve relevant images or vice versa; a common failure is a shortcut where the model latches onto spurious correlations between text and image style rather than genuine semantic content.
None of these applications is automatically solved by adopting metric learning; each requires careful definition of positives and negatives, appropriate evaluation, and ongoing monitoring for the specific ways it can go wrong.
Choosing the Right Approach
A practical decision framework starts with the nature of supervision available: are labels organized as discrete classes, as pairwise judgments, as rankings, or only as weak signals like clicks? It also depends on whether the system will encounter unseen classes after deployment, whether the task is fundamentally verification, retrieval, ranking, or clustering, how many classes exist and whether they are internally diverse, whether false positives or false negatives are more costly for the specific business or safety context, whether online inference latency is a hard constraint, how noisy or imbalanced the available data is, whether an existing pretrained embedding model already performs adequately without fine-tuning, and — if not — whether full fine-tuning or lighter adaptation is more appropriate.
Use metric learning when | Avoid or reconsider when |
New categories will appear after deployment (verification, open-set retrieval) | The category set is small, fixed, and unlikely to change |
The task is fundamentally about ranking or similarity, not fixed labels | Simple classification already meets accuracy and latency needs |
You need a reusable embedding space for retrieval or clustering | You need calibrated class probabilities, not distances |
You have meaningful pair, triplet, or identity-level supervision | Labels are noisy, sparse, or only available as raw class tags with tiny sample counts |
When class counts are large but stable and no open-set requirement exists, a well-tuned standard classifier is often simpler to train and evaluate. A hybrid approach — training with a classification-style objective (or a proxy-based loss, which behaves similarly) and only later evaluating the learned embeddings for retrieval — is a reasonable middle ground supported by the reality-check literature discussed earlier, which found that simple, well-tuned approaches often rival more exotic methods [12].
Common Failure Modes and Troubleshooting
Embedding collapse — when many or all inputs map to nearly the same point in the embedding space — often shows up as suspiciously low training loss combined with poor retrieval performance. Likely causes include a learning rate that is too aggressive, a margin set too small, or a loss that is too easy to satisfy trivially. Remedies include lowering the learning rate, verifying the margin is neither trivially satisfiable nor impossibly strict, and confirming normalization is applied consistently.
Poorly separated classes typically stem from inadequate hard-negative exposure during training; introducing harder negatives via mining, rather than relying on purely random sampling, is the standard remedy.
Overly easy negatives waste most of the training signal; symptoms include loss that plateaus quickly at a nonzero value with little further improvement. The remedy is semi-hard or hard mining rather than uniform random negative sampling.
Destructive hard-negative mining occurs when mining is too aggressive and repeatedly selects mislabeled or ambiguous examples, destabilizing training; symptoms include erratic loss spikes. The remedy is semi-hard mining as a gentler default, plus manual auditing of the hardest mined examples for label errors.
False negatives — genuinely same-class pairs mislabeled as different — quietly corrupt both loss computation and evaluation; the remedy is systematic label auditing, especially of any pairs a miner repeatedly flags as difficult.
Overfitting to known classes shows up as strong performance on seen classes but poor generalization to new ones at evaluation time; the remedy is evaluating consistently on held-out, disjoint identities or classes throughout development, not only at the very end.
Train-test identity leakage inflates every reported metric; the remedy is enforcing identity- or entity-level splits from the very start of a project, verified programmatically rather than assumed.
Inappropriate evaluation metrics, such as reporting only classification accuracy for a system meant to generalize to unseen classes, mask the real failure mode; the remedy is using Recall@K, mAP, or verification-specific metrics matched to the deployment task.
Sensitivity to batch composition and inadequate batch size produce unstable or inconsistent training, particularly for batch-based losses like N-pair or multi-similarity; the remedy is class-balanced sampling and, where memory allows, larger batches or a cross-batch memory mechanism.
Bad margin selection — too small trivializes the loss, too large destabilizes training — is best addressed with a modest search over a small margin range on validation data, rather than assuming one default value transfers across tasks.
Embedding dimension mis-sizing: too small a dimension bottlenecks the model's ability to represent genuinely diverse classes; unnecessarily large dimensions increase storage and search cost without corresponding accuracy gains and can make overfitting on small datasets more likely.
Hubness, a documented phenomenon in high-dimensional nearest-neighbor search where a small number of points recur disproportionately often as nearest neighbors to many other points, degrades retrieval quality and is a known challenge in high-dimensional similarity search literature [1].
Domain shift, shortcut learning, and biased data all describe variations on the same underlying risk: the model has learned spurious, dataset-specific correlations rather than the intended semantic relationship, and it fails when deployed on data that differs from training conditions.
Threshold drift and index staleness are deployment-stage issues: a verification threshold tuned on old data can become miscalibrated as the population shifts, and a retrieval index built from an older model version may not remain directly comparable once the model is updated — both are addressed further in the next section.
Production Deployment
A typical production pipeline precomputes corpus embeddings — running every catalog item, document, or enrolled identity through the encoder once, offline — and stores the results in a vector index for fast lookup. At query time, only the new query item needs a fresh embedding computation; this asymmetry between offline corpus indexing and online query embedding is what makes large-scale retrieval systems fast enough for real-time use.
Exact nearest-neighbor search guarantees the true closest matches but scales poorly with corpus size; approximate nearest-neighbor (ANN) search trades a small amount of accuracy for large gains in speed and memory efficiency, and is the standard choice at meaningful production scale. Specific index structures and libraries vary in their tradeoffs between build time, query latency, memory footprint, and recall, and the right choice depends on corpus size and query volume rather than any single universally best option.
Similarity thresholds for verification tasks must be selected using validation data that reflects the actual deployment population, not benchmark data, and should be re-validated periodically as the population shifts. Embedding versioning matters because embeddings from different model versions are not guaranteed to be comparable in the same space — mixing old and new embeddings in one index without re-indexing can silently corrupt retrieval quality. Re-indexing after model updates is therefore often necessary, and backward compatibility techniques (training new models to remain roughly compatible with older embedding spaces) exist specifically to reduce the cost of full re-indexing, though they add training complexity.
Ongoing operations should monitor retrieval quality using sampled human evaluation or proxy metrics, monitor score distributions for unexpected shifts that might indicate drift or data quality issues, and detect drift by comparing current query and corpus statistics against training-time baselines. Human review for high-risk matches — particularly in biometric verification, fraud, or medical contexts — remains an important safeguard rather than fully automating high-stakes decisions. Finally, privacy and security of biometric or behavioral embeddings deserve explicit engineering attention, since embeddings can sometimes be partially inverted or used to infer sensitive attributes, a concern explored further below.
Limitations, Ethics, and Responsible Use
Metric-learning systems inherit and can amplify bias present in their training data: if training pairs are drawn disproportionately from certain demographic groups, verification accuracy for underrepresented groups can be measurably worse, a well-documented concern in face-recognition research and independent evaluations of commercial systems.
Learned embeddings can unintentionally encode sensitive attributes — such as inferred age, gender, or ethnicity — even when the system was never explicitly trained to predict them, simply because those attributes correlate with the target similarity signal. This raises biometric surveillance risk: face or gait embeddings, if deployed without safeguards, can enable tracking that individuals did not consent to and may not be aware of.
Privacy leakage and membership or inversion risks are active research concerns: under some conditions, it may be possible to infer whether a specific individual's data was part of a training set, or to partially reconstruct sensitive input characteristics from an embedding, a risk profile broadly comparable to concerns raised in the wider privacy-preserving machine learning literature. Unequal error rates across demographic groups are a direct fairness concern in verification systems specifically, since a system with an acceptable overall false-accept rate can still perform poorly for specific subpopulations.
Dataset consent matters directly: face and voice datasets scraped without clear consent raise both ethical and, in a growing number of jurisdictions, legal concerns. Misleading notions of human or semantic similarity are a subtler risk — a model's learned notion of "similar" reflects statistical patterns in its training data, which may not match human intuitions of fairness, relevance, or meaning, especially for sensitive categories.
Explainability limitations are real: distance in a 128- or 512-dimensional embedding space is not inherently interpretable to a human reviewer, which complicates due-process concerns in high-stakes verification decisions. Security against adversarial or spoofing attacks — such as crafted inputs designed to trigger a false match, or presentation attacks against biometric sensors — requires dedicated countermeasures beyond the core metric-learning model itself.
Responsible deployment calls for application-specific governance: clear consent processes, documented and audited error rates across relevant subpopulations, human review pathways for high-stakes decisions, and periodic re-evaluation as both the model and the deployed population evolve. This overview is general guidance, not legal or compliance advice; organizations deploying biometric or similarity-based systems should consult qualified legal and privacy counsel for their specific jurisdiction and use case.
Current Directions and Future Outlook
Several directions are well established in current research and practice. Better generalization to unseen classes remains a central, ongoing goal, motivating continued comparison between proxy-based, classification-style, and traditional pair/triplet objectives, following the more rigorous evaluation protocols established by the reality-check and training-strategy studies discussed earlier [12, 13]. Large pretrained encoders, originally developed for general-purpose representation learning, are increasingly used as a starting point for metric-learning fine-tuning rather than training encoders from scratch, since they already capture broadly useful visual or linguistic structure. Multimodal embeddings, which place different data types (image, text, audio) into a shared comparable space, continue to expand the practical reach of retrieval systems beyond single-modality search.
Efficient retrieval at ever-larger corpus scale remains an active systems-engineering concern, closely tied to advances in approximate nearest-neighbor indexing. Uncertainty-aware similarity — producing not just a distance but a confidence estimate around it — is an emerging research direction relevant to high-stakes verification use cases. Robustness to adversarial inputs and distribution shift, and interpretable embeddings that offer some human-understandable structure rather than an opaque vector, are both recognized as open challenges rather than solved problems.
Self-supervised and weakly supervised signals continue to reduce reliance on expensive, fully labeled pair or triplet data, echoing the broader trend across machine learning toward learning from cheaper or naturally occurring supervision. Metric learning for few-shot and open-world systems remains a natural fit given the field's original motivation — supporting new, previously unseen categories — and continues to be an active area of applied research.
It is worth stating plainly what remains genuinely uncertain: no single loss function, distance metric, or architecture has been established as universally superior across tasks, and rigorous, standardized evaluation — rather than headline benchmark numbers alone — is the most reliable way to compare methods for a specific application.
FAQ
Is metric learning supervised or unsupervised?
It can be either. Classical and most deep metric-learning methods use supervision in the form of class labels, similarity/dissimilarity pairs, or rankings, but self-supervised and weakly supervised variants also exist, using data augmentation or weak signals like click behavior instead of hand-labeled pairs.
What is the difference between metric learning and contrastive learning?
Contrastive learning is a training mechanism — pulling positives together and pushing negatives apart — that appears both in self-supervised representation learning (positives from augmented views of unlabeled data) and in supervised metric learning (positives from labeled same-class or same-identity pairs). Metric learning is the broader field concerned with learning any meaningful distance or similarity function, of which contrastive-style training is one common technique, alongside triplet, proxy-based, and classification-style objectives.
Is triplet loss a form of metric learning?
Yes. Triplet loss is one specific loss function within the broader field of metric learning, not a synonym for the field itself. Metric learning also includes contrastive, N-pair, proxy-based, and classification-style approaches, among others.
Is a Siamese network the same as metric learning?
No. "Siamese network" describes a shared-weight architectural arrangement with two or more branches, which can be trained with many different loss functions, including but not limited to contrastive or triplet loss. Metric learning describes the broader goal; a Siamese network is one common architectural pattern used to pursue it.
What distance function is best?
There is no universally best distance function. Cosine similarity is popular for high-dimensional embeddings, especially text and normalized deep embeddings, because it ignores vector magnitude; Euclidean distance is intuitive and works well with L2-normalized embeddings; Mahalanobis distance is useful in classical, lower-dimensional settings. The right choice depends on the embedding space, normalization strategy, and task.
How many dimensions should an embedding have?
There is no fixed rule. Common published choices range roughly from 64 to 512 dimensions, but the right size depends on class diversity, dataset size, and storage or latency constraints. Larger dimensions can represent more nuance but risk overfitting on small datasets and increase index storage and search cost; smaller dimensions are cheaper but may bottleneck representational capacity.
How is metric learning evaluated?
Evaluation depends on the task: Recall@K, Precision@K, and mean Average Precision for retrieval; ROC-AUC, Equal Error Rate, and verification accuracy for verification tasks; k-nearest-neighbor accuracy and clustering measures like NMI for general embedding quality. Classification accuracy alone is usually insufficient, particularly for open-set tasks.
Can metric learning handle unseen classes?
Yes — this is one of its primary motivations. Because the learned function measures similarity rather than predicting a fixed set of class labels, it can compare and rank previously unseen categories, identities, or items without retraining, provided the new categories are reasonably similar in kind to those seen during training.
What is hard-negative mining?
Hard-negative mining is the practice of deliberately selecting negative examples that are currently close to the anchor in the embedding space, producing stronger and more informative training gradients than randomly sampled negatives, which are often already easy to distinguish once training has progressed.
What causes embedding collapse?
Embedding collapse, where many inputs map to nearly identical points, is commonly caused by a learning rate that is too high, a margin set too small to meaningfully separate classes, or a loss objective that can be trivially satisfied without learning genuine structure.
Does metric learning replace classification?
No. Standard classification remains preferable when the category set is fixed, known in advance, and unlikely to change, since it is often simpler to train and evaluate. Metric learning is preferred specifically when new categories will appear after deployment or when the task is fundamentally about ranking and similarity rather than fixed labels.
Can metric learning be used for text?
Yes. Text embeddings trained with metric-learning objectives support semantic search, duplicate question detection, and document retrieval, using the same underlying principles of pulling semantically related text together and pushing unrelated text apart in embedding space.
Is metric learning useful with small datasets?
It can be, particularly through few-shot learning approaches that rely on a metric-learning embedding trained on a broader set of base classes and then applied to new classes with only a handful of examples each. Fine-tuning a strong pretrained encoder is generally more effective on small datasets than training an encoder from scratch.
When should metric learning not be used?
It is often unnecessary when the category set is small, fixed, and unlikely to grow, when a standard classifier already meets accuracy and latency requirements, or when calibrated class probabilities matter more than relative distance rankings for the task at hand.
Key Takeaways
A learned distance function corrects for scale imbalance, irrelevant dimensions, and high-dimensional distance concentration that fixed formulas like raw Euclidean distance cannot handle on their own.
Classical Mahalanobis methods (NCA, LMNN, ITML) and deep metric learning share the same underlying idea — transform, then measure distance — but differ in whether that transformation is linear or a full neural network.
"Siamese" describes a shared-weight architecture, not a loss function; the same Siamese arrangement can be trained with contrastive, triplet, N-pair, proxy-based, or supervised contrastive losses.
Rigorous, standardized evaluation studies have shown that well-tuned classic losses often perform comparably to newer proposed methods, so evaluation protocol matters as much as the loss function chosen.
Triplet mining strategy — easy, semi-hard, or hard negatives — often determines training success or failure more than the loss formula itself.
Metric learning is specifically suited to open-set problems where new identities, products, or categories appear after deployment, unlike standard classifiers with a fixed output layer.
Production deployment introduces distinct challenges — embedding versioning, index staleness, and threshold drift — that do not exist during offline model development.
Bias, privacy, and demographic error-rate disparities are serious, well-documented risks in similarity systems, particularly biometric verification, and require explicit governance rather than being treated as edge cases.
Actionable Next Steps
Write a one-paragraph definition of what "similar" means for your specific application, including concrete examples of true positive and true negative pairs.
Audit your available supervision — class labels, pairwise judgments, or weak signals — and confirm you can construct identity- or entity-level train/validation/test splits without leakage.
Select a baseline architecture and loss (a pretrained encoder fine-tuned with contrastive or triplet loss is a reasonable default) rather than starting with the most novel published method.
Build a class-balanced batch sampler with semi-hard negative mining, and manually inspect a sample of mined "hard" negatives for label noise before trusting the pipeline.
Evaluate using task-appropriate metrics (Recall@K or mAP for retrieval; ROC-AUC and EER for verification) on a held-out set of genuinely unseen classes or identities, not just training-set accuracy.
Build a small-scale approximate nearest-neighbor index and measure real query latency and memory footprint before committing to a full production rollout.
Establish embedding versioning, re-indexing procedures, and ongoing score-distribution monitoring before shipping, so drift and index staleness are caught early rather than discovered in production.
Glossary
Anchor — The reference example in a triplet, compared against a positive and a negative example.
Approximate nearest neighbor (ANN) — A search technique that trades a small amount of accuracy for large gains in speed and memory efficiency when finding closest matches in a large corpus.
Contrastive loss — A pair-based loss that pulls similar pairs together and pushes dissimilar pairs apart beyond a margin.
Cosine similarity — A similarity measure based on the angle between two vectors, ignoring their magnitude.
Distance metric — A function measuring how far apart two points are; a true mathematical metric satisfies non-negativity, identity of indiscernibles, symmetry, and the triangle inequality.
Embedding — A vector representation of raw data (an image, word, or sound) produced by a learned encoder.
Embedding collapse — A training failure where many or all inputs map to nearly identical embeddings.
Equal Error Rate (EER) — The verification threshold at which the false-accept rate equals the false-reject rate.
Hard negative — A negative example that is currently close to the anchor in embedding space, producing a strong training gradient.
Hubness — A phenomenon in high-dimensional spaces where a small number of points recur disproportionately often as nearest neighbors to many others.
Mahalanobis distance — A generalized distance that reweights and correlates input dimensions using a learned matrix.
Margin — A minimum required gap between positive and negative distances in contrastive or triplet losses.
Mean Average Precision (mAP) — A retrieval metric averaging precision across all relevant results for a query.
Metric learning — The field of training models to produce distance or similarity scores that reflect meaningful relationships between data points.
N-pair loss — A loss comparing one anchor-positive pair against multiple negatives simultaneously within a batch.
Open-set recognition — The task of correctly handling categories not seen during training, as opposed to closed-set classification.
Positive pair — Two examples labeled or assumed to be similar, such as two photos of the same person.
Proxy — A learned representative point standing in for a class, used to reduce the cost of comparing every real example against every other.
Recall@K — A retrieval metric measuring whether a correct match appears among the top K results.
Representation learning — The broader field of learning useful feature representations from data, of which metric learning is one specialization.
ROC-AUC — The area under the receiver operating characteristic curve, summarizing verification performance across all thresholds.
Semi-hard negative — A negative example farther from the anchor than the positive, but still within the margin.
Siamese network — A shared-weight architectural arrangement processing two or more inputs in parallel branches.
Supervised contrastive learning — A batch-based contrastive method using labels to define multiple positives per anchor.
Triangle inequality — The metric axiom stating that the direct distance between two points cannot exceed the sum of distances through any third point.
Triplet loss — A ranking loss enforcing that an anchor is closer to its positive than to its negative by at least a margin.
Vector index — A data structure enabling efficient nearest-neighbor search over a large collection of embeddings.
Sources & References
Beyer, K., Goldstein, J., Ramakrishnan, R., & Shaft, U. "When Is 'Nearest Neighbor' Meaningful?" International Conference on Database Theory (ICDT), 1999. Link
Goldberger, J., Roweis, S., Hinton, G., & Salakhutdinov, R. "Neighbourhood Components Analysis." Advances in Neural Information Processing Systems 17 (NeurIPS 2004). Link
Weinberger, K. Q., & Saul, L. K. "Distance Metric Learning for Large Margin Nearest Neighbor Classification." Journal of Machine Learning Research, 2009. Link
Davis, J. V., Kulis, B., Jain, P., Sra, S., & Dhillon, I. S. "Information-Theoretic Metric Learning." Proceedings of the 24th International Conference on Machine Learning (ICML 2007). Link
Bromley, J., Guyon, I., LeCun, Y., Säckinger, E., & Shah, R. "Signature Verification Using a 'Siamese' Time Delay Neural Network." Advances in Neural Information Processing Systems 6 (NeurIPS 1993). Link
Hadsell, R., Chopra, S., & LeCun, Y. "Dimensionality Reduction by Learning an Invariant Mapping." IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2006). Link
Schroff, F., Kalenichenko, D., & Philbin, J. "FaceNet: A Unified Embedding for Face Recognition and Clustering." IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2015). Link
Sohn, K. "Improved Deep Metric Learning with Multi-class N-pair Loss Objective." Advances in Neural Information Processing Systems 29 (NeurIPS 2016). Link
Wang, X., Han, X., Huang, W., Dong, D., & Scott, M. R. "Multi-Similarity Loss with General Pair Weighting for Deep Metric Learning." IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2019). Link
Movshovitz-Attias, Y., Toshev, A., Leung, T. K., Ioffe, S., & Singh, S. "No Fuss Distance Metric Learning Using Proxies." IEEE International Conference on Computer Vision (ICCV 2017). Link
Khosla, P., Teterwak, P., Wang, C., Sarna, A., Tian, Y., Isola, P., Maschinot, A., Liu, C., & Krishnan, D. "Supervised Contrastive Learning." Advances in Neural Information Processing Systems 33 (NeurIPS 2020). Link
Musgrave, K., Belongie, S., & Lim, S. N. "A Metric Learning Reality Check." European Conference on Computer Vision (ECCV 2020). Link
Roth, K., Milbich, T., Sinha, S., Gupta, P., Ommer, B., & Cohen, J. P. "Revisiting Training Strategies and Generalization Performance in Deep Metric Learning." International Conference on Machine Learning (ICML 2020). Link
Snell, J., Swersky, K., & Zemel, R. "Prototypical Networks for Few-shot Learning." Advances in Neural Information Processing Systems 30 (NeurIPS 2017). Link
