This work introduces a novel network architecture for the task of human pose estimation. Features are processed across all scales and consolidated to best capture the various spatial relationships associated with the body. We show how repeated bottom-up, top-down processing used in conjunction with intermediate supervision is critical to improving the performance of the network. We refer to the architecture as a "stacked hourglass" network based on the successive steps of pooling and upsampling that are done to produce a final set of predictions. State-of-the-art results are achieved on the FLIC and MPII Human Pose benchmarks outperforming all recent methods.

A key component of understanding people in images and video is pose estimation. Achieving an understanding of a person's posture and limb articulation is useful for higher level tasks like action recognition. The challenge is that the appearance and position of body parts can vary dramatically due to changes in clothing, body shape, lighting, and occlusion. Furthermore, the need to capture information at different scales is essential — local evidence is critical for identifying body parts, but the final pose estimate requires a coherent understanding of the full body. Prior methods have addressed multi-scale processing through various means, but we propose a more systematic approach.

The design of our network is motivated by the need to capture information at every scale. We use a single pipeline with skip connections to preserve spatial information at each resolution and pass it along for upsampling further in the network. The hourglass is a simple, minimal design that has the capacity to capture features across scales. The network reaches its lowest resolution at 4x4 pixels before beginning the top-down upsampling with nearest neighbor interpolation and the combination of features across scales. At each scale, a residual module performs convolutions. The full architecture involves repeated pooling at each layer from the highest to lowest resolution, and subsequent upsampling and combination of features across all scales.

A single hourglass already captures multi-scale information, but we hypothesize that further performance gains can be achieved by stacking multiple hourglasses together, allowing for repeated bottom-up and top-down inference. After each hourglass, we apply intermediate supervision — the network produces an intermediate set of heatmaps that are compared against the ground truth. This encourages each hourglass to produce a reasonable prediction on its own, and allows the subsequent hourglass to refine and correct the predictions of the previous stage. We find that using 8 stacked hourglasses provides the best trade-off between performance and computational cost.

We evaluate our approach on the MPII Human Pose and FLIC datasets. On MPII, our 8-stack hourglass network achieves 90.9% PCKh@0.5, significantly outperforming prior methods. The improvement is particularly notable for difficult joints like wrists (83.7%) and ankles (80.2%), where multi-scale reasoning is crucial. On FLIC, we achieve 99.0% accuracy for elbows and 97.0% for wrists at PCK@0.2. Ablation studies confirm that both stacking and intermediate supervision contribute significantly: removing intermediate supervision drops performance by 1.2%, and reducing from 8 to 2 hourglasses drops by 2.3%.

We have introduced the stacked hourglass network, a novel architecture for human pose estimation that captures and consolidates information across all scales of the image through repeated bottom-up, top-down processing with intermediate supervision. The design is modular and elegant, achieving state-of-the-art results on multiple benchmarks. We believe the hourglass design will find broad applications beyond pose estimation in tasks requiring dense prediction at multiple scales.