We propose a method to learn 3D deformable object categories from raw single-view images, without external supervision. The method employs an autoencoder that factorizes each input image into depth, albedo, viewpoint, and illumination. The key insight is that many object categories have an approximate reflective symmetry, which we model with a symmetry probability map, learned end-to-end, accounting for the fact that not all object parts are symmetric. Our method achieves results that outperform or are comparable to state-of-the-art supervised and non-supervised methods for single-image 3D reconstruction of human faces, cat faces, and cars.

Humans can effortlessly perceive 3D shape and appearance of objects from a single image. Achieving the same with machines is a fundamental goal of computer vision. Most existing approaches rely on expensive supervision such as 3D ground truth, multi-view images, or manually annotated keypoints, limiting their scalability. We ask: can we learn to reconstruct 3D objects from single images without any supervision?

Our key idea is to exploit the fact that many natural object categories possess an approximate bilateral symmetry. This symmetry prior provides a powerful self-supervisory signal: given a predicted 3D shape, we can flip the reconstruction and check consistency with the original image. However, objects are not perfectly symmetric — hair, expressions, and accessories break symmetry. We therefore introduce a learned confidence map that assigns a probability of symmetry to each pixel.

Previous works on single-image 3D reconstruction include 3D Morphable Models (3DMM) that fit a parametric shape model to image observations, but they require pre-computed shape templates from 3D scans. Learning-based approaches using CNNs have shown promise, but typically require 3D ground truth, multi-view supervision, or keypoint annotations. Recent differentiable rendering techniques enable end-to-end learning of 3D representations from 2D images, which we leverage in our framework.

Our model is based on an image formation autoencoder that decomposes an input image into four components: depth d, albedo a, viewpoint w, and illumination l. An encoder maps the input image to these latent factors, and a differentiable renderer reconstructs the image from the predicted factors. The model is trained with a photometric reconstruction loss that measures how well the predicted factors explain the observed image. This approach does not require any 3D annotations, keypoints, or multi-view data.

The core innovation lies in modeling symmetry probabilistically. We learn a symmetry probability map sigma that assigns to each pixel the probability that it has a symmetric counterpart. During training, the reconstruction loss is weighted by sigma: symmetric regions contribute more to the loss, while asymmetric regions (e.g., hair, accessories) are down-weighted. This allows the model to "automatically discover which parts of the object are symmetric and which are not", without any manual annotation. The map sigma is predicted by the encoder and learned end-to-end together with all other factors.

The total loss combines several terms: a photometric loss measuring RGB reconstruction error, a perceptual loss using VGG features for higher-level similarity, and a regularization on the depth map to encourage smoothness. Crucially, the symmetry-weighted flip consistency loss reconstructs the image from a horizontally flipped version of the predicted depth and albedo, enforcing that symmetric parts of the object should produce consistent reconstructions regardless of the viewing direction.

We evaluate on three object categories: human faces (CelebA), cat faces (Cats dataset), and cars (CompCars). For human faces, the method achieves state-of-the-art 3D reconstruction quality with SIDE 0.793 and MAD 15.4 degrees on BFM benchmark, outperforming supervised baselines including 3DMM-CNN. On cat faces, the model successfully learns plausible 3D geometry despite the high variability in cat face shapes. For cars, reconstructions show accurate depth and shape recovery from single viewpoints. The symmetry probability map learned by the model correctly identifies asymmetric regions such as hair and earrings in human faces.

Ablation studies demonstrate the importance of each component. Removing the symmetry prior leads to significant degradation in reconstruction quality, with SIDE increasing from 0.793 to 0.897. Without the perceptual loss, results become blurry and lose fine details. The probabilistic symmetry map proves essential: using a hard symmetry constraint (sigma = 1 everywhere) performs worse because it cannot accommodate inherently asymmetric regions.

We have presented a method for learning 3D deformable object reconstruction from single images without any external supervision. The key enabler is a probabilistic symmetry model that automatically learns which parts of an object are symmetric, providing a powerful self-supervisory signal for 3D learning. Our results demonstrate that this approach achieves state-of-the-art performance on multiple object categories, challenging the assumption that 3D reconstruction requires explicit 3D supervision.