The "Roaring 20s" of visual recognition began with the introduction of Vision Transformers (ViTs), which quickly superseded ConvNets as the state-of-the-art image classification model. A vanilla ViT, on the other hand, faces difficulties when applied to general computer vision tasks such as object detection and semantic segmentation. It is the hierarchical Transformers (e.g., Swin Transformers) that reintroduced several ConvNet priors, making Transformers practically viable as a generic vision backbone. However, the effectiveness of such hybrid approaches is still largely credited to the intrinsic superiority of Transformers, rather than the inherent inductive biases of convolutions. In this work, the authors reexamine the design spaces and test the limits of what a pure ConvNet can achieve. They gradually "modernize" a standard ResNet toward the design of a Vision Transformer, and discover several key components that contribute to the performance difference along the way. The outcome of this exploration is a family of pure ConvNet models dubbed ConvNeXt. ConvNeXt models, constructed entirely from standard ConvNet modules, compete favorably with Transformers in terms of accuracy and scalability, achieving 87.8% ImageNet top-1 accuracy and outperforming Swin Transformers on COCO detection and ADE20K segmentation, while maintaining the simplicity and efficiency of standard ConvNets.

The "Roaring 20s" of visual recognition began with the introduction of Vision Transformers (ViTs). The 2010s were dominated by convolutional neural networks (ConvNets), beginning with AlexNet's breakthrough in 2012. The field evolved through representative models like VGGNet, ResNet, DenseNet, and EfficientNet, each focusing on different aspects of accuracy, efficiency, and scalability. ConvNets possessed several inherent advantages: translation equivariance (crucial for object detection), efficiency through shared computation in sliding-window approaches, and success across diverse applications from digit recognition to pedestrian detection.

However, around 2020, Vision Transformers (ViTs) fundamentally altered the architectural landscape. Unlike ConvNets, vanilla ViTs introduced minimal image-specific inductive bias. While ViTs demonstrated superior scaling behavior on image classification, they faced practical challenges for general computer vision tasks due to quadratic complexity in global attention mechanisms. Hierarchical Transformers like Swin addressed this by reintroducing sliding-window strategies and other ConvNet priors, achieving state-of-the-art results across multiple vision tasks. Yet the authors questioned: does the superiority stem from Transformers' inherent advantages, or simply from reincorporating convolution-based principles?

The paper's central investigation asks: "How do design decisions in Transformers impact ConvNets' performance?" The authors gradually modernize a ResNet toward the design of a Swin Transformer, without introducing any attention-based modules. This exploration yields ConvNeXt, a family of pure ConvNet models that achieves 87.8% ImageNet top-1 accuracy, outperforms Swin Transformers on COCO detection and ADE20K segmentation, while maintaining the simplicity and efficiency of standard ConvNets. The key finding is that many of the design choices borrowed from Transformers have been individually explored in the ConvNet literature over the past decade, but never collectively assembled.

Beyond architectural innovations, training procedures significantly affect performance. Vision Transformers introduced different optimization strategies and hyperparameters compared to traditional ConvNets. The authors trained baseline ResNet-50/200 models using a recipe similar to DeiT and Swin Transformer training approaches. The training is extended to 300 epochs from the original 90 epochs for ResNets. They use the AdamW optimizer, data augmentation techniques such as Mixup, Cutmix, RandAugment, Random Erasing, and regularization schemes including Stochastic Depth and Label Smoothing.

This enhanced training recipe alone increased ResNet-50 performance from 76.1% to 78.8% (+2.7%), suggesting that "a significant portion of the performance difference between traditional ConvNets and vision Transformers may be due to the training techniques." This 78.8% baseline uses fixed hyperparameters throughout the remainder of the modernization process, with reported ImageNet-1K accuracies averaged across three random seeds.

Changing Stage Compute Ratio. ResNet's empirical design distributed computation across stages, with the res4 stage being particularly heavy to support downstream object detection tasks operating on 14x14 feature planes. Swin Transformer employed a different stage compute ratio of 1:1:3:1 (1:1:9:1 for larger models). The authors adjusted ResNet-50's block distribution from (3, 4, 6, 3) to (3, 3, 9, 3), aligning FLOPs with Swin-T. This change improved accuracy from 78.8% to 79.4%.

Changing Stem to "Patchify". Standard ResNet stems aggressively downsample inputs through a 7x7 convolution with stride 2 plus max pooling, yielding 4x downsampling. Vision Transformers use more aggressive "patchify" strategies with large kernels (14 or 16) and non-overlapping convolutions. Swin Transformer employs a smaller patch size of 4 for its multi-stage design. The authors replaced the ResNet stem with a 4x4 stride-4 convolutional layer, changing accuracy marginally from 79.4% to 79.5%. "The stem cell in a ResNet may be substituted with a simpler 'patchify' layer a la ViT which will result in similar performance."

ResNeXt principles emphasize grouped convolution with the guideline "use more groups, expand width." The authors employed depthwise convolution, where the number of groups equals the number of channels, popularized by MobileNet and Xception. "Depthwise convolution is similar to the weighted sum operation in self-attention, which operates on a per-channel basis, i.e., only mixing information in the spatial dimension." This separation of spatial and channel mixing mirrors Vision Transformers' design. Using depthwise convolution initially reduced accuracy, while expanding network width from 64 to 96 channels (matching Swin-T's channel count) improved performance to 80.5% with 5.3G FLOPs.

Transformer blocks feature inverted bottlenecks where the MLP hidden dimension is four times wider than the input dimension. This design also appears in ConvNets, popularized by MobileNetV2 with expansion ratio 4. The authors repositioned the depthwise convolution layer in the block structure. Despite increased FLOPs for depthwise operations, overall network FLOPs decreased to 4.6G due to significant reduction in downsampling blocks' 1x1 convolution layers. Performance slightly improved from 80.5% to 80.6%. At the ResNet-200/Swin-B regime, the gains were more substantial: from 81.9% to 82.6% with reduced FLOPs.

Moving up depthwise conv layer. One of Vision Transformers' distinguishing features is non-local self-attention, providing a global receptive field. While large kernels appeared in early ConvNets, VGGNet established 3x3 stacking as the gold standard due to efficient GPU implementations. Swin Transformers employ window sizes of at least 7x7, significantly larger than ResNet's 3x3. To explore large kernels, the depthwise conv layer was repositioned to precede the dense 1x1 layers, paralleling Transformers where MSA blocks precede MLP layers. With inverted bottleneck blocks, complex/inefficient modules now operate on fewer channels while efficient 1x1 layers handle the heavy computation. This intermediate step reduced FLOPs to 4.1G but temporarily degraded performance to 79.9%.

Increasing kernel size. With the preparations complete, large kernel adoption showed significant benefits. Experiments tested kernel sizes of 3, 5, 7, 9, and 11. Performance improved from 79.9% (3x3) to 80.6% (7x7) while FLOPs remained roughly constant. "The benefit of larger kernel sizes reaches a saturation point at 7x7." The authors verified this behavior at larger scale: "a ResNet-200 regime model does not exhibit further gain when we increase the kernel size beyond 7x7." This finding is noteworthy because it matches Swin Transformer's default window size, suggesting a convergence in optimal receptive field size between ConvNets and Transformers.

Replacing ReLU with GELU. Rectified Linear Units (ReLU) remain extensively used in ConvNets for simplicity and efficiency. Gaussian Error Linear Units (GELU), a smoother variant, appear in advanced Transformers including BERT, GPT-2, and ViTs. Substituting GELU for ReLU in ConvNeXt showed "the accuracy stays unchanged (80.6%)", though the authors note it becomes useful in conjunction with later modifications.

Fewer activation functions. Transformer blocks contain fewer activation functions than ResNet blocks. Transformers include only one activation function in the MLP block, whereas ResNets typically append activations to every convolutional layer including 1x1 convolutions. Eliminating GELU layers from residual blocks except between two 1x1 layers (replicating Transformer style) improved performance by 0.7% to 81.3%, "practically matching the performance of Swin-T."

Fewer normalization layers. Transformer blocks typically include fewer normalization layers than ConvNet blocks. Removing two BatchNorm layers and retaining only one before the 1x1 convolution layers further boosted performance to 81.4%, "already surpassing Swin-T's result."

Substituting BN with LN. Batch Normalization (BN) improved convergence and reduced overfitting but contains complexities that may potentially harm performance. Layer Normalization (LN), simpler and widely used in Transformers, previously showed suboptimal results when directly substituting BN in original ResNets. However, with the accumulated architectural modifications and training techniques, "our ConvNet model does not have any difficulties training with LN; in fact, the performance is slightly better, obtaining an accuracy of 81.5%."

Separate downsampling layers. ResNet achieves spatial downsampling through residual blocks using 3x3 convolution with stride 2 at block beginnings. Swin Transformers add separate downsampling layers between stages. The authors explored 2x2 stride-2 convolution for downsampling but encountered training divergence. Adding normalization wherever spatial resolution changes stabilized training, including LayerNorm layers before each downsampling layer, after the stem, and after the final global average pooling. This improved accuracy to 82.0%, "significantly exceeding Swin-T's 81.3%." The authors remark: "These designs are not novel even in the ConvNet literature — they have all been researched separately, but not collectively, over the last decade."

ImageNet-1K Results. ConvNeXt variants demonstrate competitive performance across model sizes. ConvNeXt-T achieves 82.1% accuracy compared to Swin-T's 81.3%. At higher resolutions (384x384), ConvNeXt-B reaches 85.1% versus Swin-B's 84.5% with superior throughput (95.7 vs. 85.1 image/s). The model family includes five variants: ConvNeXt-T (29M parameters, 4.5G FLOPs), ConvNeXt-S (50M parameters, 8.7G FLOPs), ConvNeXt-B (89M parameters, 15.4G FLOPs), ConvNeXt-L (198M parameters, 34.4G FLOPs), and ConvNeXt-XL (350M parameters, 60.9G FLOPs).

ImageNet-22K Pre-training. With ImageNet-22K pre-training and fine-tuning, the scaling behavior of ConvNeXt becomes even more impressive. ConvNeXt-XL achieves 87.8% accuracy, outperforming EfficientNetV2-XL (87.3%) and matching ViT-L results. ConvNeXt-B reaches 86.8% compared to Swin-B's 86.4%. The results demonstrate that ConvNets scale effectively with larger datasets, challenging the widely held belief about Transformer superiority in the large-data regime.

Isotropic ConvNeXt vs. ViT. When applied to non-hierarchical (isotropic) architectures, ConvNeXt blocks perform competitively with Vision Transformers. Isotropic ConvNeXt-B achieves 82.0% accuracy, matching ViT-B's performance while using comparable parameters and FLOPs. This is a particularly strong result because it shows that the ConvNeXt block design is effective even without the hierarchical multi-stage structure that was borrowed from ConvNet traditions.

COCO Object Detection and Segmentation. Using Mask R-CNN and Cascade Mask R-CNN frameworks, ConvNeXt demonstrates strong performance on COCO. With Mask R-CNN, ConvNeXt-T achieves 46.2 AP^box and 41.7 AP^mask compared to Swin-T's 46.0 AP^box and 41.6 AP^mask, with better throughput (25.6 vs. 23.1 FPS). With Cascade Mask R-CNN and ImageNet-22K pre-training, ConvNeXt-B achieves 54.0 AP^box and 46.9 AP^mask, significantly outperforming Swin-B's 53.0 AP^box and 45.8 AP^mask (e.g., +1.0 AP^box).

ADE20K Semantic Segmentation. Using the UperNet framework, ConvNeXt consistently outperforms Swin across all scales. With ImageNet-1K pre-training, ConvNeXt-T achieves 46.7 mIoU vs. Swin-T's 45.8 mIoU; ConvNeXt-S reaches 49.6 mIoU vs. Swin-S's 49.5; and ConvNeXt-B attains 49.9 mIoU vs. Swin-B's 49.7. With ImageNet-22K pre-training, the gap widens: ConvNeXt-B achieves 53.1 mIoU vs. Swin-B's 51.7 mIoU, ConvNeXt-L reaches 53.7 mIoU vs. Swin-L's 53.5, and ConvNeXt-XL achieves the best result at 54.0 mIoU.

Model Efficiency. The authors address concerns about depthwise convolution efficiency. Despite theoretical efficiency concerns, "inference throughputs of ConvNeXts are comparable to or exceed that of Swin Transformers." Training memory consumption for Cascade Mask R-CNN with ConvNeXt-B requires 17.4GB versus 18.5GB for Swin-B. On A100 GPUs with TF32 support, ConvNeXt demonstrates "up to 49% higher throughput" compared to Swin Transformers, further highlighting that pure ConvNets can be both faster and more accurate than hierarchical Transformers.

The paper challenges the prevailing narrative that "vision Transformers are more accurate, efficient, and scalable than ConvNets." ConvNeXt demonstrates that pure convolutional architectures can compete favorably with state-of-the-art hierarchical vision Transformers across multiple computer vision benchmarks, while retaining the simplicity and efficiency of standard ConvNets. The authors note that "many design choices have all been examined separately over the last decade, but not collectively," suggesting their contribution lies in the systematic synthesis of existing techniques. They express hope that findings will "challenge several widely held views and prompt people to rethink the importance of convolution in computer vision."

Limitations. The authors acknowledge that ConvNeXt may be more suited for certain tasks, while Transformers may be more flexible for others. Multi-modal learning scenarios where cross-attention is preferable represent cases where Transformers maintain advantages. Tasks requiring "discretized, sparse, or structured outputs" may also favor Transformer approaches. The authors advocate choosing architectures based on task needs while striving for simplicity, rather than defaulting to the most hyped approach.