Deformable part models (DPMs) and convolutional neural networks (CNNs) are two dominant approaches to visual recognition that have been developed largely in isolation. In this paper, we show that DPMs can be formulated as equivalent CNNs by unrolling the DPM inference algorithm and mapping each step to an equivalent CNN layer. This insight leads to DeepPyramid DPM, which replaces HOG features with learned deep convolutional features. The resulting model significantly outperforms HOG-based DPMs and slightly exceeds R-CNN while running approximately 20x faster. Our work provides a unified theoretical framework connecting two paradigms that have been regarded as fundamentally different approaches to object detection.

For nearly a decade, deformable part models (DPMs) have been one of the most successful approaches to object detection. DPMs represent objects as collections of parts arranged in a deformable spatial configuration, scored using HOG (Histogram of Oriented Gradients) features and latent SVM training. Meanwhile, CNNs have achieved dramatic improvements on image classification and, through R-CNN, on object detection. These two lines of work appear fundamentally different: DPMs use hand-crafted features with explicit part-based structure, while CNNs use learned features with implicit representations. We reveal that this apparent dichotomy is false — DPMs are a special case of CNNs.

The DPM framework, introduced by Felzenszwalb et al., models objects as a root filter plus deformable part filters scored on HOG feature pyramids. R-CNN by Girshick et al. demonstrated that CNN features dramatically outperform HOG for detection, but operates as a region-based approach requiring expensive selective search proposals. OverFeat applied CNNs in a sliding-window fashion but lacked explicit part-based reasoning. Several works have attempted to combine parts and deep features, but without establishing the fundamental mathematical equivalence between DPMs and CNNs. Our work is the first to derive this equivalence formally, enabling DPM inference to be implemented as standard CNN forward passes.

The key insight is that every step of DPM inference corresponds to a standard CNN operation. The root and part filter convolutions in DPM are equivalent to convolutional layers in a CNN. The distance transform used to compute the optimal part placement corresponds to a novel pooling operation we call "distance transform pooling" (DT-pooling) — a generalization of max pooling. The combination of part scores with the root score is equivalent to another convolutional layer encoding object geometry. Finally, the multi-component competition in DPM (selecting the best component model) corresponds to a maxout nonlinearity.

Distance transform pooling is the key novel component. Standard max pooling selects the maximum activation within a fixed spatial window. DT-pooling generalizes this by incorporating learnable quadratic deformation costs: the pooled value at each location is the maximum of (filter response minus a quadratic penalty for displacement from the anchor position). Mathematically, for a part filter response map R, the DT-pooled output at location p is: max_q [ R(q) - a(q_x - p_x)^2 - b(q_y - p_y)^2 ] where a, b are learned deformation cost parameters. When a = b = 0, this reduces to global max pooling; when a, b are very large, it reduces to fixed-location pooling. This operation can be computed in linear time using the generalized distance transform algorithm.

We evaluate DeepPyramid DPM on PASCAL VOC 2007. The feature pyramid front-end uses a truncated SuperVision CNN (ending at conv5) applied to an image pyramid, generating features at 1/16th spatial resolution. DeepPyramid DPM achieves 45.2% mAP, dramatically outperforming HOG-DPM at 33.7% mAP (an 11.5 point improvement) and slightly exceeding R-CNN pool5 at 44.2% mAP. Critically, DeepPyramid DPM runs approximately 20x faster than R-CNN variants because it processes the entire image in a single forward pass through the feature pyramid, rather than extracting features from thousands of region proposals. The model shows particularly strong performance on deformable object categories where explicit part modeling provides clear advantages.

We have shown that deformable part models are convolutional neural networks — not merely analogous, but mathematically equivalent when each inference step is mapped to its CNN counterpart. This unification reveals that the success of DPMs and CNNs stems from shared computational principles, with the key difference being the feature representation (learned vs. hand-crafted). DeepPyramid DPM demonstrates the practical value of this insight: sliding-window detectors on deep feature pyramids significantly outperform equivalent models on HOG while maintaining computational efficiency. We believe this synthesis suggests fruitful directions for future work, including end-to-end learning of deformable part models within deep network architectures.