This paper presents a method to learn detailed depth of dressed humans from social media dance videos. The key insight is that social media dance videos provide natural multi-view supervision through diverse poses and viewpoints, enabling self-supervised learning of fine-grained human geometry. The proposed self-supervised framework recovers detailed geometry including clothing wrinkles and surface deformations without requiring any depth sensor data or 3D annotations. The method produces high-fidelity depth maps that surpass methods trained on synthetic data, demonstrating the power of leveraging abundantly available in-the-wild video data.

Reconstructing detailed 3D geometry of dressed humans is a fundamental challenge in computer vision, with applications in virtual try-on, telepresence, and digital content creation. Existing approaches rely on either expensive depth sensors (e.g., structured light, LiDAR) that are impractical for casual capture, or synthetic training data that fails to capture the diversity and complexity of real-world clothing. Methods trained on parametric body models like SMPL produce smooth body shapes but miss fine surface details such as clothing wrinkles, accessories, and hair.

The authors observe that social media platforms contain an enormous volume of dance videos where individuals perform diverse movements in front of relatively static cameras. These videos naturally provide multi-view observations of the same person from different angles as the dancer rotates and moves through various poses. This observation motivates the key idea: instead of collecting multi-view data with calibrated cameras or generating synthetic data, we can leverage the natural multi-view information embedded in freely available dance videos to supervise depth learning.

Monocular human depth estimation methods have evolved from template-based fitting of parametric models (SMPL, SCAPE) to learning-based approaches using implicit functions (PIFu, PIFuHD). While PIFuHD produces impressive results, it relies on synthetic training data rendered from 3D scans, limiting its generalization to in-the-wild scenarios. Self-supervised depth estimation from video has been explored for general scenes, using photometric consistency as supervision, but these methods struggle with non-rigid human motion and typically produce coarse depth maps.

The framework processes dance videos by first estimating the 3D pose (using an off-the-shelf SMPL estimator) and camera parameters for each frame. For any pair of frames showing the same person in different poses, the method establishes dense correspondences by mapping pixels through the estimated body model — warping from the source frame's surface to the canonical pose, then to the target frame's surface. This body-mediated correspondence enables photometric supervision: the depth prediction should be consistent with the warped appearance from other viewpoints.

The depth estimation network takes a single RGB image as input and predicts a detailed depth map of the visible human. The network architecture employs a coarse-to-fine strategy: first predicting a base depth aligned with the SMPL body, then refining it with a detail network that adds high-frequency surface information. The training loss combines photometric consistency across frames, perceptual similarity, and a smoothness regularization term. The detail network specifically learns the residual between the smooth body model and the actual clothed surface, capturing wrinkles, folds, and accessories that parametric models cannot represent.

The method is trained on approximately 3,600 TikTok dance videos collected from social media. Evaluation on the BUFF dataset (which provides ground-truth 3D scans of clothed humans) shows that the method achieves depth error of 5.7mm on average, outperforming PIFuHD (7.2mm) which is trained on synthetic 3D scan data. Qualitative results demonstrate the method's ability to recover fine-grained details including clothing wrinkles, hood contours, and skirt folds that other methods miss. The framework generalizes to diverse body types, clothing styles, and poses not seen during training, showing strong in-the-wild robustness. Ablation studies confirm that the multi-view supervision from dance videos is essential — models trained without it show significantly degraded detail recovery.

This work demonstrates that social media dance videos are a surprisingly powerful source of supervision for learning detailed human depth estimation. The self-supervised framework leveraging natural multi-view information through body-mediated correspondences produces high-fidelity depth maps that capture fine clothing details, surpassing methods trained on synthetic data. This paradigm of repurposing abundantly available in-the-wild videos as structured training data opens new possibilities for 3D human understanding without expensive data collection infrastructure.