This paper proposes a Fast Region-based Convolutional Network method (Fast R-CNN) for object detection. Fast R-CNN trains VGG16 networks 9x faster than R-CNN and is 213x faster at test time while achieving higher mAP on PASCAL VOC 2012. Compared to SPPnet, Fast R-CNN trains 3x faster, tests 10x faster, and has higher accuracy. Fast R-CNN is implemented in Python and C++ (Caffe) and the code is made available.

Object detection requires accurate localization of objects while processing numerous candidate locations, making it computationally demanding. R-CNN achieved state-of-the-art results but suffers from multi-stage training (CNN fine-tuning, SVM training, bounding-box regression), expensive feature caching to disk, and slow inference due to per-proposal CNN evaluation. SPPnet accelerated inference by sharing computation but still requires multi-stage training and cannot update convolutional layers during fine-tuning. We present a single-stage training algorithm that jointly learns to classify object proposals and refine their spatial locations, achieving 66% mAP on VOC 2012 at 0.3 seconds per image.

The Fast R-CNN network takes as input an entire image and a set of object proposals. The network first processes the image through convolutional and max pooling layers to produce a feature map. For each proposal, a Region of Interest (RoI) pooling layer extracts a fixed-length feature vector using max pooling to convert features of any spatial extent into a fixed HxW grid (typically 7x7). These vectors feed into fully connected layers branching into two sibling outputs: softmax probabilities over K+1 classes (including background) and per-class bounding-box regression offsets.

The multi-task loss combines classification and localization: L(p,u,t,v) = L_cls(p,u) + λ[u≥1] L_loc(t,v). The classification loss L_cls is the standard log loss over K+1 classes. The localization loss uses smooth L1, which is more robust to outliers than L2 loss: it transitions from quadratic to linear at |x|=1, preventing exploding gradients from large errors. Training mini-batches sample N=2 images with R=128 RoIs per batch. Hierarchical sampling shares computation efficiently: 25% of RoIs come from proposals with IoU ≥ 0.5 (positives), and the rest from the IoU range [0.1, 0.5) (hard negatives).

On VOC 2007, Fast R-CNN achieves 66.9% mAP with VGG16; on VOC 2012, 65.7% mAP baseline and 68.4% with additional training data. Training is 8.8x faster than R-CNN (9.5 vs. 84 hours). Testing speed is 146x faster without SVD; 213x with SVD compression, achieving 0.32 seconds per image with VGG16. Freezing convolutional layers reduces mAP from 66.9% to 61.4%, validating end-to-end fine-tuning. Multi-task training improves classification accuracy by 0.8-1.1 mAP points over single-task baselines. Truncated SVD reduces FC layer computation by 30% with only 0.3 mAP loss.

Fast R-CNN presents an efficient, unified training approach enabling end-to-end training of object detection networks while achieving significant computational speedups and accuracy improvements over R-CNN and SPPnet. The RoI pooling layer, multi-task loss, and single-stage training pipeline collectively resolve the fragmented, multi-stage nature of prior detection systems. The method demonstrates that jointly training classification and bounding-box regression is not only simpler but also more effective than the disconnected pipelines that preceded it.