Despite significant recent advances in the field of face recognition, implementing face verification and recognition efficiently at scale presents serious challenges to current approaches. In this paper we present a system, called FaceNet, that directly learns a mapping from face images to a compact Euclidean space where distances directly correspond to a measure of face similarity. Once this space has been produced, tasks such as face recognition, verification and clustering can be easily implemented using standard techniques with FaceNet embeddings as feature vectors. Our method uses a deep convolutional network trained to directly optimize the embedding itself, rather than an intermediate bottleneck layer as in previous deep learning approaches. We use a novel online triplet mining method to train. On the widely used Labeled Faces in the Wild (LFW) dataset, our system achieves a new record accuracy of 99.63%. On YouTube Faces DB it achieves 95.12%.

We present a unified system for face verification (is this the same person?), recognition (who is this person?), and clustering (find common people among these faces). Our approach is based on learning a Euclidean embedding per image using a deep convolutional network. The network is trained such that the squared L2 distances in the embedding space directly correspond to face similarity: faces of the same person have small distances and faces of different people have large distances. Once this embedding is established, the aforementioned tasks become straightforward: face verification becomes thresholding the distance, recognition becomes a k-NN classification problem, and clustering becomes a standard clustering problem like k-means.

Previous face recognition systems typically involve a multi-stage pipeline: face detection, alignment, feature extraction, and classification. Learned features have largely replaced hand-crafted ones, with DeepFace and DeepID demonstrating impressive results using deep neural networks followed by a classification layer. However, these approaches learn an intermediate representation through a softmax classifier, which does not directly optimize for the embedding quality. The resulting embedding is an indirect byproduct rather than the primary learning objective. In contrast, our approach directly trains the embedding using a triplet-based loss function, ensuring that the learned representation is optimized for the similarity metric that matters most.

The triplet loss operates on triplets of training samples: an anchor, a positive (same identity as anchor), and a negative (different identity). The loss function ensures that the anchor is closer to the positive than to the negative by at least a margin alpha: ||f(a) - f(p)||^2 + alpha < ||f(a) - f(n)||^2. Here, f(x) is the embedding of image x, normalized to live on the d-dimensional hypersphere. The embedding is constrained to ||f(x)||_2 = 1. This loss directly encourages the network to map images of the same person to nearby points and images of different people to distant points in the embedding space, without requiring an explicit classification layer.

Choosing the right triplets is crucial for achieving good performance and fast convergence. Given a training set of N embeddings, there are O(N^3) possible triplets, most of which are trivially satisfied and provide no useful gradient. We need to select hard triplets: for a given anchor-positive pair, we want to find the hardest negative (closest to the anchor but of different identity), and for a given anchor-negative pair, the hardest positive (farthest from the anchor but of same identity). Computing the exact hardest examples across the entire dataset is infeasible. Instead, we use online semi-hard negative mining within mini-batches, selecting negatives that are farther from the anchor than the positive but still within the margin. This avoids collapsed models from the hardest negatives early in training while still providing informative gradients.

We train FaceNet using a private dataset of roughly 200 million face thumbnails from 8 million different identities. Two network architectures are explored: a Zeiler&Fergus-based model with 22 million parameters and an Inception-based model with 7.5 million parameters. On LFW, our best model achieves 99.63% accuracy, establishing a new state-of-the-art. On YouTube Faces DB, we achieve 95.12%. We also evaluate on an internal dataset of one million faces, showing that the 128-dimensional embedding achieves near-perfect performance with embeddings as small as 128 bytes per face. The compact embedding enables large-scale face clustering and retrieval with minimal storage and computation.

We have presented FaceNet, a system that directly learns an embedding into a Euclidean space for face verification, recognition, and clustering. Our method uses a triplet loss function that directly reflects the desired properties of the embedding, combined with an effective online triplet mining strategy. The resulting 128-dimensional embedding is compact enough for large-scale deployment while providing state-of-the-art accuracy. FaceNet demonstrates that end-to-end learning of embeddings is a powerful paradigm for face representation, and we believe this approach generalizes to other visual similarity tasks.