This paper proposes Deep hidden IDentity features (DeepID), learned with deep convolutional neural networks for face verification. The features are taken from the last hidden layer neuron activations of deep convolutional networks trained as classifiers to predict approximately 10,000 face identity classes. The learned features on each face region are complementary to each other, and their combination achieves 97.45% verification accuracy on the LFW dataset using weakly aligned faces. The high-level identity features learned through the multi-class face identification task generalize well to the face verification task and to identities unseen during training.

Face verification — determining whether two face images belong to the same person — remains one of the most challenging problems in computer vision due to large intra-personal variations caused by pose, illumination, expression, age, and occlusion. Traditional approaches rely on hand-crafted features such as LBP, Gabor filters, and Fisher vectors, followed by metric learning to reduce intra-class variance. While effective in constrained settings, these methods struggle with unconstrained face images "in the wild".

Recent advances in deep learning have demonstrated remarkable performance in visual recognition tasks. The key insight of this work is that face identification (classifying among many identities) and face verification (comparing two faces) are closely related tasks: a network trained for identification with sufficiently many identity classes must learn highly discriminative and generalizable features that transfer directly to verification. We hypothesize that increasing the number of training identities forces the network to learn more compact and abstract representations that capture identity-specific information rather than superficial appearance cues.

Prior to deep learning approaches, face verification research centered on designing discriminative feature descriptors — LBP histograms, Gabor magnitude features, and high-dimensional Fisher vectors — and learning distance metrics such as Joint Bayesian, KISSME, and large-margin nearest neighbor (LMNN). The concurrent work of DeepFace by Taigman et al. also applies deep learning to face verification, using a large-scale private dataset of 4.4 million faces and 3D face alignment. In contrast, our method uses publicly available training data and achieves competitive results with a much simpler alignment procedure.

The DeepID network consists of four convolutional layers followed by the DeepID layer (fully-connected) and a softmax output layer. The first three convolutional layers each employ max-pooling for spatial downsampling. A key architectural feature is that the DeepID layer receives inputs from both the third and fourth convolutional layers, combining multi-scale features — local low-level details from the third layer and global high-level semantics from the fourth layer. The DeepID layer produces a 160-dimensional feature vector that serves as the face representation. The softmax layer then classifies this representation into approximately 10,000 identity classes during training.

Rather than using a single holistic face image, we extract DeepID features from multiple face patches at different positions and scales. In total, 60 face patches are defined, including 10 regions (around eyes, nose, mouth, forehead, and cheeks) at multiple scales with horizontal flipping. Each patch is fed through its own trained DeepID network, and the resulting 160-dimensional features are concatenated to form a high-dimensional representation. The rationale is that different face regions carry complementary identity information: the eye region may be discriminative for some identity pairs, while the mouth region is more informative for others.

For verification, the concatenated DeepID features are first reduced by PCA, then a Joint Bayesian model is applied to compute the log-likelihood ratio of two face representations belonging to the same person versus different persons. The Joint Bayesian model decomposes each face representation into an identity component and an intra-personal variation component, providing a principled probabilistic framework for verification. The combination of deep learning features with the Joint Bayesian model proves highly effective: the deep features provide discriminative representations, and the Bayesian model captures the residual intra-personal variations.

Experiments are conducted on the Labeled Faces in the Wild (LFW) dataset, the standard benchmark for unconstrained face verification. The training set includes approximately 10,000 identities from CelebFaces and WDRef datasets. Using 60 face patches, DeepID achieves 97.45% accuracy on the standard unrestricted protocol with weakly aligned faces. With stronger alignment, the accuracy improves further. Analysis shows that verification accuracy consistently improves as the number of training identities increases, confirming the hypothesis that large-scale identification training produces more generalizable features. Even with a single face patch, DeepID outperforms most existing methods, and the combination of 60 patches yields complementary information that significantly boosts performance.

We have demonstrated that deep convolutional networks trained for large-scale face identification learn features that generalize effectively to face verification. The DeepID representation, extracted from multiple face patches and combined with Joint Bayesian modeling, achieves 97.45% accuracy on LFW, advancing the state of the art. The key finding is that the richness of the identification task — predicting among 10,000 classes — forces the network to learn highly discriminative, identity-preserving features rather than memorizing training samples. Future work will explore jointly optimizing identification and verification objectives to further improve feature quality.