We present a novel denoising training method to accelerate DETR (DEtection TRansformer) training and offer a deepened understanding of the slow convergence issue of DETR-like methods. We show that the slow training convergence of DETR is largely attributed to the instability of bipartite graph matching, which causes inconsistent optimization goals in early training stages. To address this, we introduce query denoising as a new training paradigm: in addition to the Hungarian matching part, we feed ground-truth bounding boxes with noises into the Transformer decoder and train the model to reconstruct the original boxes. This approach bypasses the bipartite matching and provides a more stable and consistent training signal. Our method, DN-DETR, achieves 44.1 AP with a ResNet-50 backbone in 50 epochs, a +1.9 AP improvement over DAB-DETR with negligible extra computation. DN-DETR can be easily plugged into any DETR-like method to boost performance.

DETR (DEtection TRansformer) has revolutionized object detection by formulating it as a set prediction problem and eliminating the need for hand-crafted components such as anchor generation and non-maximum suppression (NMS). This elegant end-to-end framework uses a Transformer encoder-decoder architecture with learnable object queries and bipartite matching loss to directly predict detection results. Despite its conceptual simplicity, DETR suffers from prohibitively slow training convergence, requiring 500 training epochs compared to only 12 epochs for Faster R-CNN to achieve competitive performance on COCO.

Several approaches have been proposed to accelerate DETR training. Deformable DETR introduces multi-scale deformable attention to improve spatial resolution handling. Conditional DETR and Anchor DETR redesign the cross-attention mechanism with spatial priors. More recently, DAB-DETR formulates queries as dynamic anchor boxes that are iteratively updated. While these methods improve convergence to varying degrees, they primarily focus on architectural modifications and do not directly address the fundamental instability in the bipartite matching process. The matching inconsistency — where the same query is matched to different objects across epochs — remains a critical bottleneck.

In this paper, we propose DN-DETR, which introduces a denoising training approach that directly provides stable optimization targets to the decoder. Our key insight is straightforward: instead of relying solely on Hungarian matching to assign ground-truth targets, we additionally feed noised versions of ground-truth bounding boxes and labels as queries into the decoder, and train the model to reconstruct the original clean ground-truth. This denoising task bypasses the bipartite matching entirely for these queries, providing a consistent and unambiguous learning signal. An attention mask mechanism prevents information leakage between the denoising part and the matching part. Our approach is orthogonal to architectural improvements and can be combined with any DETR variant as a plug-in training strategy.

Object detection has long been dominated by CNN-based approaches. Two-stage detectors like Faster R-CNN and HTC generate region proposals followed by classification and refinement. One-stage methods like YOLO and FCOS directly predict bounding boxes from dense feature maps. These methods rely heavily on hand-designed components including anchor boxes, NMS, and feature pyramid networks. DETR offered a paradigm shift by treating detection as direct set prediction with bipartite matching, eliminating these hand-crafted modules. However, the slow convergence and high computational cost limited its practical adoption.

A rich line of work has emerged to improve DETR. Deformable DETR replaces global attention with deformable attention on multi-scale feature maps, significantly reducing computation and improving convergence. Conditional DETR decouples content and spatial queries to provide better spatial priors for cross-attention. Anchor DETR and DAB-DETR further refine query design by using anchor points or dynamic anchor boxes as positional queries that are iteratively refined across decoder layers. Efficient DETR initializes queries from encoder outputs. All these methods improve the decoder's ability to locate objects but do not fundamentally change the training paradigm based on bipartite matching.

Denoising has been widely used in representation learning and generative modeling. Denoising autoencoders (DAE) learn robust representations by reconstructing clean inputs from corrupted versions. More recently, diffusion models have demonstrated the power of iterative denoising for image generation. In the context of object detection, methods like DiffusionDet treat detection as a denoising process over bounding boxes. Our work draws inspiration from the denoising paradigm but applies it differently: we use denoising not as the detection mechanism itself, but as an auxiliary training task that provides additional supervision signals to accelerate convergence while being completely removed during inference.

The overall framework of DN-DETR builds upon DAB-DETR, which represents queries as 4D anchor boxes (x, y, w, h). The decoder receives two types of queries: learnable matching queries that undergo standard Hungarian matching with ground-truth, and denoising queries constructed by adding noise to ground-truth boxes and labels. These two groups of queries are processed together in the Transformer decoder but are separated by an attention mask to prevent information leakage. During inference, the denoising queries are simply removed, so DN-DETR introduces zero additional computation at test time.

The denoising queries are constructed by perturbing ground-truth bounding boxes and class labels. For box noise, the center coordinates (x, y) are shifted by random offsets satisfying |delta_x| < lambda_1 * w/2 and |delta_y| < lambda_1 * h/2, where w and h are the box width and height. The width and height are independently scaled by random factors sampled from [(1 - lambda_2), (1 + lambda_2)]. For label noise, class labels are randomly flipped to other categories with probability gamma. The default hyperparameters are lambda_1 = lambda_2 = 0.4 and gamma = 0.2. This noise design ensures that the noised boxes still overlap significantly with the original ground-truth, making the reconstruction task learnable.

Each denoising query consists of two components following the DAB-DETR formulation: a positional part (the noised anchor box) and a content part (an indicator embedding of the noised class label). The positional part encodes the noised box coordinates through sinusoidal positional encoding followed by an MLP, identical to how matching queries encode their anchor boxes. The content part uses a learnable class embedding that maps the (potentially flipped) label to a feature vector. This unified query representation means that the decoder treats denoising queries and matching queries identically in terms of architecture, enabling the denoising task to directly improve the decoder's localization and classification capabilities.

A critical design element is the attention mask applied in the self-attention layers of the decoder. Without proper masking, denoising queries could leak ground-truth information to matching queries, rendering the matching task trivial and the model unable to detect objects at inference (when denoising queries are absent). The attention mask enforces three rules: (1) matching queries cannot attend to any denoising query; (2) denoising queries within one denoising group can attend to each other; and (3) denoising queries from different groups cannot attend to each other. This ensures that each denoising group independently reconstructs its version of the ground-truth.

The ablation study reveals the critical importance of the attention mask. When the mask is removed entirely, the model's AP drops catastrophically from 43.4 to 24.0, confirming that information leakage completely undermines the matching-based detection. Interestingly, allowing denoising queries from different groups to attend to each other also degrades performance, since a query can "cheat" by directly copying the reconstruction from another group's less-noised version of the same object. The strict group isolation forces each denoising group to genuinely learn the reconstruction mapping independently.

To maximize the training signal from denoising, DN-DETR employs multiple denoising groups. Each group creates an independent noised copy of all ground-truth objects in the image. With G denoising groups and K ground-truth objects, the decoder receives G * K denoising queries plus N matching queries in total. The default number of groups is G = 5, and the matching queries remain at N = 300. Increasing the number of groups provides more diverse noised examples for training, effectively acting as a form of data augmentation at the query level. The reconstruction loss for denoising includes L1 loss and GIoU loss for boxes, and focal loss for classification, consistent with the matching part.

A notable property of DN-DETR is the clean separation between training and inference. During training, the total query set consists of both matching and denoising queries, processed jointly through the decoder. During inference, the denoising queries and associated attention mask are simply removed, leaving the decoder identical to the base DAB-DETR architecture. This means that DN-DETR adds zero latency and zero parameters at inference time. The entire benefit comes from better-optimized decoder weights learned through the auxiliary denoising supervision. This design philosophy — enriching the training signal without modifying the inference pipeline — makes DN-DETR exceptionally easy to adopt in practice.

Experiments are conducted on MS-COCO 2017 with 118K training images. Under the standard 50-epoch training setting with ResNet-50 backbone, DN-DETR achieves 44.1 AP, compared to 42.2 AP for DAB-DETR (a +1.9 AP improvement). With ResNet-101, DN-DETR reaches 45.2 AP versus 43.5 AP for DAB-DETR (+1.7 AP). In the more challenging 12-epoch (1x schedule) setting, the improvement is even more pronounced: DN-DETR-DC5-R50 achieves 41.7 AP while DAB-DETR reaches only 38.0 AP, a remarkable +3.7 AP gap. This confirms that the denoising training is most beneficial when training budget is limited.

Convergence analysis shows that DN-DETR achieves the baseline's 50-epoch performance in approximately 25 epochs, effectively halving the required training time. At 50 epochs, DN-DETR continues to outperform, indicating that the benefit is not merely earlier convergence but also a higher performance ceiling. Compared to other DETR variants, DN-DETR combined with Deformable DETR as backbone achieves DN-Deformable-DETR-R50++: 46.0 AP in only 12 epochs, setting a new state-of-the-art for DETR-family methods under the 1x training schedule. This demonstrates the universal applicability of the denoising training approach across different DETR architectures.

Detailed ablation studies dissect the contribution of each component. Starting from the DAB-DETR baseline at 42.2 AP, adding box denoising alone (without label noise) yields 42.2 AP — no improvement. Combining box denoising with label denoising raises performance to 43.0 AP, a +0.8 AP gain. Further adding the attention mask achieves 43.4 AP (+1.2 AP total). Scaling to 5 denoising groups reaches the full 44.1 AP. These results reveal that label denoising and the attention mask are both essential — box denoising alone is ineffective, confirming that the class label signal is crucial for the decoder to benefit from the denoising task.

The noise scale analysis reveals a clear sweet spot. When lambda_1 and lambda_2 are too small (e.g., 0.1), the denoising task becomes too easy and provides insufficient training signal. When too large (e.g., 0.8), the noised boxes deviate too far from the ground-truth, making reconstruction too difficult and the task loses its beneficial effect. The optimal range centers around lambda_1 = lambda_2 = 0.4. Similarly, the number of denoising groups shows diminishing returns: performance improves from 1 group (43.4 AP) to 5 groups (44.1 AP) but further increasing to 10 or more groups yields marginal gains. The label flip ratio gamma is optimal at 0.2, with higher ratios degrading classification performance.

To validate the plug-and-play nature of DN-DETR, the denoising training is applied to multiple DETR variants. When integrated with Deformable DETR, the denoising approach yields consistent improvements across all training schedules. DN-Deformable-DETR with ResNet-50 achieves 46.0 AP in the 12-epoch setting, surpassing the original Deformable DETR's 50-epoch result. When applied to the basic DETR architecture with DC5 (dilated C5) backbone, the gains are even more dramatic due to the baseline's severe convergence issues. These cross-architecture results provide strong evidence that the denoising training addresses a fundamental issue in DETR training rather than being architecture-specific.

This paper presents DN-DETR, which introduces query denoising as a new training paradigm for DETR-like object detectors. By identifying the instability of bipartite matching as a key factor in slow convergence, we propose to feed noised ground-truth boxes and labels into the decoder and train it to reconstruct the original targets. This denoising task provides stable and consistent optimization signals that complement the Hungarian matching. With an attention mask preventing information leakage and multiple denoising groups enriching training diversity, DN-DETR achieves significant improvements across multiple DETR variants and training schedules, particularly excelling in the 1x (12-epoch) setting.

The denoising training paradigm opens several future directions. Our current approach uses a simple uniform noise distribution; more sophisticated noise scheduling strategies — such as curriculum-based noise that starts easy and gradually increases difficulty — may further improve training efficiency. The denoising idea can potentially extend beyond detection to other set prediction tasks like instance segmentation and pose estimation. We believe that rethinking the training paradigm, rather than solely focusing on architectural innovations, is a promising and complementary direction for advancing DETR-based methods.