Pedestrian detection is a key problem for automotive safety and surveillance. Most state-of-the-art systems rely on hand-crafted features such as HOG (Histogram of Oriented Gradients) combined with linear or kernel-based classifiers. We propose an approach that replaces hand-crafted features with features learned in an unsupervised manner using a multi-stage convolutional architecture. Each stage consists of convolutional filtering, non-linear activation, and spatial pooling, with filters learned via unsupervised methods (PSD — Predictive Sparse Decomposition). Our multi-stage system significantly outperforms HOG-based methods on the INRIA and Caltech pedestrian benchmarks, demonstrating that learned features can surpass carefully engineered ones for pedestrian detection.

Pedestrian detection has been extensively studied due to its critical role in autonomous driving, advanced driver assistance systems (ADAS), and intelligent surveillance. The dominant paradigm involves sliding window detection with HOG features and a linear SVM classifier, as proposed by Dalal and Triggs. While numerous refinements have been proposed — multi-scale HOG, HOG+LBP combinations, channel features — the core limitation remains: the features are hand-designed and may not capture the most discriminative patterns for pedestrian/non-pedestrian classification.

Deep learning approaches, particularly convolutional neural networks (ConvNets), have recently shown remarkable success in image classification (Krizhevsky et al., 2012). However, these successes rely on large-scale supervised training data, which may not always be available. We explore an alternative: using unsupervised feature learning to train convolutional filters, which are then combined with a supervised classifier for the final detection task. This two-phase approach (unsupervised pre-training + supervised fine-tuning) reduces the dependency on labeled data while still leveraging the representational power of deep architectures.

The HOG detector by Dalal and Triggs has been the foundation of pedestrian detection for nearly a decade. Key extensions include deformable part models (DPM) by Felzenszwalb et al., which model pedestrians as collections of parts, and channel features by Dollar et al., which aggregate multiple feature channels (gradient magnitude, color channels) efficiently. On the learning side, sparse coding and autoencoders have been used for unsupervised feature learning in various recognition tasks but have not been systematically applied to pedestrian detection with multi-scale architectures.

Our feature learning approach uses Predictive Sparse Decomposition (PSD), which learns a dictionary of convolutional filters by jointly minimizing a reconstruction error and a sparsity penalty. Given a set of unlabeled image patches extracted from natural images, PSD learns filters that produce sparse feature maps when convolved with input images. Unlike standard sparse coding (which requires iterative inference at test time), PSD trains an encoder network that directly predicts the sparse codes in a single forward pass, making it suitable for real-time applications.

A key contribution is our multi-stage, multi-scale convolutional architecture. The first stage applies 64 learned 7x7 filters to the input image, followed by absolute value rectification and contrast normalization. The output is passed to a 2x2 average pooling layer. The second stage applies another set of learned filters to the pooled features, with the same non-linearity and pooling. To capture both fine details and coarse structure, we extract features from multiple stages and combine them via concatenation before the final classifier. This "multi-scale feature" strategy allows the classifier to leverage both low-level edges and higher-level shape information.

The final detection system combines the multi-stage convolutional features with a linear SVM classifier. We also investigate boosting and neural network classifiers as alternatives. The detection pipeline follows the standard sliding window approach with multi-scale image pyramid, applying the learned feature extractor and classifier at each window position and scale. Non-maximum suppression is used to merge overlapping detections. While the feature extraction is more expensive than HOG computation, the system can be accelerated using GPU implementation.

We evaluate on the INRIA Person dataset and the Caltech Pedestrian benchmark. On INRIA, our multi-stage system with unsupervised features achieves a miss rate of 13.1% at 1 FPPI (false positives per image), compared to 23.1% for the standard HOG+SVM baseline. This represents a relative improvement of 43%. On the more challenging Caltech benchmark, we achieve significant improvements over HOG across all occlusion levels and pedestrian scales. The two-stage architecture outperforms the single-stage variant by approximately 5% in miss rate, confirming the value of hierarchical feature learning.

We further analyze the learned filters and find they exhibit meaningful structure: first-stage filters resemble Gabor-like edge detectors at various orientations and frequencies, while second-stage filters capture more complex patterns like corners and junctions. This hierarchical emergence of feature complexity — from edges to parts — mirrors the design philosophy of hand-crafted feature hierarchies but arises automatically from the data. We also show that features learned from general natural images transfer well to the pedestrian detection task, suggesting strong generalizability.

We have demonstrated that unsupervised multi-stage feature learning can significantly improve pedestrian detection compared to traditional hand-crafted features. Our convolutional architecture with PSD-learned filters achieves state-of-the-art results on standard benchmarks while requiring no labeled data for the feature learning stage. The learned features exhibit interpretable hierarchical structure and transfer across tasks. Our work provides strong evidence that the era of hand-crafted features in pedestrian detection may be drawing to a close, and that learned representations — whether unsupervised or supervised — are the path forward.