This paper addresses dense visual correspondence across different object instances without requiring direct ground-truth correspondence labels. The key innovation is exploiting cycle consistency as a "meta-supervision" signal. During training, the authors use 3D CAD models to establish correspondence 4-cycles — connecting pairs of real images through synthetic rendered views. A CNN is trained to predict "synthetic-to-real, real-to-real and real-to-synthetic correspondences that are cycle-consistent". At test time, no CAD models are required, and the approach outperforms SIFT flow on correspondence tasks.

Establishing dense correspondence between images is a fundamental problem in computer vision, underlying tasks such as 3D reconstruction, pose estimation, and image editing. While correspondence within the same instance (e.g., stereo matching) is well-studied, cross-instance correspondence — matching semantically equivalent parts across different object instances — remains challenging. The core difficulty is that ground-truth cross-instance correspondences are extremely expensive to annotate. Existing methods either rely on hand-crafted features like SIFT that are not tuned for semantic matching, or require dense manual annotations that scale poorly.

Traditional dense correspondence methods include SIFT flow, which computes dense pixel-wise matching using hand-crafted features. Recent deep learning approaches by Long et al. learn correspondence features but require direct supervision from manually annotated keypoints. FlowNet learns optical flow in a supervised manner but cannot handle cross-instance matching where appearance differs significantly. The use of 3D models as bridges for 2D tasks has been explored in recognition but not previously applied to learning dense correspondence in an end-to-end fashion.

The core idea uses cycle consistency as a training signal. For each training quartet <s1, s2, r1, r2>, where s represents synthetic rendered views and r represents real images of the same object category, the network learns four correspondence flows forming a 4-cycle. The training objective minimizes L_flow = distance(F_gt(s1,s2), F(s1,r1) composed with F(r1,r2) composed with F(r2,s2)), comparing the ground-truth synthetic-to-synthetic flow with the composed flow along the cycle. Flow composition uses bilinear interpolation for differentiability, and a truncated Euclidean loss with threshold T=15 pixels provides robustness to outliers.

Beyond flow prediction, the network also outputs matchability probability maps indicating whether a valid correspondence exists at each pixel. A separate decoder branch produces these maps using 9 fractionally-strided convolution layers. To prevent the trivial solution of predicting zero matchability everywhere, the authors fix M(s1,r1) = 1 and M(r2,s2) = 1, constraining the network to predict matchability only for the real-to-real flow. The matchability loss uses cross-entropy with lambda = 100 weighting in the combined objective.

Experiments evaluate keypoint transfer and matchability prediction on PASCAL3D+ and PASCAL-Part datasets. For keypoint transfer (PCK at alpha = 0.1), the method achieves 24.0% mean accuracy, compared to 19.6% for SIFT flow and 18.5% for Long et al. Particularly strong gains appear on "bottle" (40.3% vs. 28.3%) and "car" (33.3% vs. 22.4%) categories. For matchability prediction, the method reaches 67.8% mean accuracy on PASCAL-Part vs. 57.1% for SIFT flow. The network is trained on approximately 80,000 training quartets per category from PASCAL3D+ and ShapeNet, using Adam optimizer with learning rate 0.001 and a two-stage training strategy: initialization with SIFT flow mimicry, then consistency fine-tuning for 200,000 iterations.

This paper introduces a "general learning framework for tasks without direct labels through cycle consistency as an example of meta-supervision". By leveraging 3D CAD models as bridges between real images, the method learns dense cross-instance correspondence without manual keypoint annotations. The first end-to-end trained ConvNet supervised by cycle-consistency outperforms state-of-the-art pairwise matching methods. Feature analysis reveals that the network develops implicit viewpoint sensitivity despite no explicit viewpoint supervision, suggesting that 3D understanding emerges naturally from the cycle consistency objective.