We propose CornerNet, a new approach to object detection where we detect an object bounding box as a pair of keypoints — the top-left corner and bottom-right corner. By detecting corners instead of anchor boxes, we eliminate the need for anchor box design, a critical but often heuristic component of modern one-stage detectors. We introduce corner pooling, a new type of pooling layer that helps the network better localize corners by encoding the information along the boundary of the object. CornerNet achieves 42.1% AP on MS COCO, outperforming all existing one-stage detectors at the time of publication.

Modern object detection methods are dominated by anchor-based approaches that densely tile predefined anchor boxes across the image and classify each anchor as foreground or background. One-stage methods like SSD, RetinaNet, and YOLO achieve real-time speed by directly predicting class labels and bounding box offsets from anchors. However, anchor-based detectors suffer from two significant drawbacks. First, the design of anchor boxes requires careful tuning of sizes, aspect ratios, and densities, which is typically done through heuristic rules or exhaustive search. Second, anchor boxes create a severe imbalance between positive and negative samples, as most anchors correspond to background, necessitating complex sampling strategies or focal loss.

We propose a fundamentally different detection paradigm: detecting objects as pairs of keypoints (top-left and bottom-right corners) rather than classifying anchor boxes. This approach offers several advantages. First, it completely eliminates the need for anchor box design, removing a significant source of hyperparameters. Second, the set of possible corners is much smaller than the set of possible anchors, leading to a more balanced distribution between positive and negative training examples. Third, corners encode the extent of objects more efficiently than centers, as a corner only needs to attend to one side of the object boundary rather than all four sides.

Two-stage detectors such as Faster R-CNN and R-FCN first generate region proposals and then classify each proposal. One-stage detectors like SSD and RetinaNet directly predict detections from a dense set of anchor boxes. Both families rely heavily on anchor box design. Keypoint-based methods have been successfully used for tasks like human pose estimation, where body joints are detected as heatmap peaks and connected into skeletons. CornerNet adapts this keypoint detection paradigm to object detection, treating each bounding box as two keypoints connected by their associative embeddings.

CornerNet uses a single hourglass network as the backbone to produce feature maps. The network predicts two sets of heatmaps — one for top-left corners and one for bottom-right corners — along with embedding vectors for grouping corners and offsets for precise localization. For each corner, the heatmap indicates the probability of a corner of a specific category existing at each location. The associative embedding assigns a vector to each detected corner such that corners belonging to the same object have similar embeddings, enabling us to group top-left and bottom-right corners into bounding boxes by matching embeddings with small distances.

We introduce corner pooling, a novel pooling operation designed to help the network locate corners more accurately. The intuition is that a corner is defined by two edges of the bounding box, and these edges may be far from the corner itself. For a top-left corner, corner pooling takes the maximum value looking rightward along the horizontal direction and the maximum value looking downward along the vertical direction, and adds them together. This allows the network to gather information from the entire top edge and left edge of the bounding box, even when the corner itself lacks distinctive visual features. Corner pooling is implemented efficiently using cumulative maximum operations.

The training of CornerNet uses a variant of focal loss for the corner heatmaps, with Gaussian penalty reduction around each ground-truth corner location to tolerate minor localization errors. The associative embedding loss uses a pull-push formulation: it pulls the embeddings of corners belonging to the same object close together while pushing embeddings of different objects apart. The offset loss uses smooth L1 loss to refine the corner locations from the quantized heatmap positions to sub-pixel accuracy. At inference, corners are extracted as heatmap peaks via non-maximum suppression, paired using embedding distances, and filtered by detection score.

CornerNet is evaluated on MS COCO test-dev. Using the Hourglass-104 backbone, CornerNet achieves 42.1% AP, outperforming all existing one-stage detectors including RetinaNet (40.8%), YOLOv3 (33.0%), and DSSD (33.2%). CornerNet also achieves competitive results compared to two-stage detectors, matching Cascade R-CNN (42.8%) closely. Particularly notable is CornerNet's strong performance on small objects (20.8% AP_small), which benefits from the absence of anchor-based priors that often underperform on small objects due to coarse feature resolution. Ablation studies confirm that corner pooling improves AP by 2.0 points, and associative embeddings outperform alternative grouping strategies.

Error analysis reveals that CornerNet's primary failure mode is incorrect corner grouping, where corners from different objects are mistakenly paired. This accounts for approximately 19.3% of false positive detections. The grouping errors are most common in crowded scenes where many objects of the same category overlap. Additionally, CornerNet shows relatively weaker performance on large objects compared to anchor-based methods, likely because the two corners of a large object are far apart and the embedding vectors must encode long-range spatial relationships. These observations motivate future work on improved grouping mechanisms and center-aware detection.

We have presented CornerNet, a novel anchor-free object detection approach that detects objects as pairs of corner keypoints. By eliminating anchor boxes, CornerNet removes a significant source of hyperparameters and achieves state-of-the-art performance among one-stage detectors on MS COCO. The introduction of corner pooling provides an effective mechanism for the network to leverage boundary information for accurate corner localization. Our work demonstrates that keypoint-based representations provide a viable and powerful alternative to anchor-based detection, opening new directions for object detection research including center-point detection and other geometric representations.