We propose a deep convolutional neural network architecture codenamed Inception, which was responsible for setting the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14). The main hallmark of this architecture is the improved utilization of the computing resources inside the network. This was achieved by a carefully crafted design that allows for increasing the depth and width of the network while keeping the computational budget constant. To optimize quality, the architectural decisions were based on the Hebbian principle and the intuition of multi-scale processing. One particular incarnation used in our submission for ILSVRC14 is called GoogLeNet, a 22 layers deep network, the quality of which is assessed in the context of classification and detection.

In the last three years, mainly due to the advances of deep learning, more concretely convolutional networks, the quality of image recognition and object detection has been progressing at a dramatic pace. One encouraging news is that most of this progress is not just the result of more powerful hardware, larger datasets and bigger models, but mainly a consequence of new ideas, algorithms and improved network architectures. The GoogLeNet network that won ILSVRC14 uses 12x fewer parameters than AlexNet while being significantly more accurate. The biggest gains in object detection have not come from the utilization of deep networks alone but from the synergy of deep architectures and classical computer vision.

In this paper, we focus on an efficient deep neural network architecture for computer vision, codenamed Inception, which derives its name from the "network in network" paper by Lin et al. as well as from the famous "we need to go deeper" internet meme. In our context, the word "deep" is used in two different senses: first, we introduce a new level of organization in the form of the "Inception module" and also in the more direct sense of increased network depth. In general, one can view the Inception model as a logical culmination of Network in Network while drawing inspiration from Arora et al. on the theoretical justification.

Starting with LeNet-5, convolutional neural networks have typically had a standard structure — stacked convolutional layers followed by one or more fully-connected layers. Variations of this basic design are prevalent in the image classification literature. For larger datasets such as ImageNet, the recent trend has been to increase the number of layers and layer size, while using dropout to address the problem of overfitting. Network in Network is an approach by Lin et al. that increases the representational power of neural networks by applying micro neural networks (1x1 convolutions) within each stage. This approach is adopted in our architecture for dimension reduction to remove computational bottlenecks.

The most straightforward way of improving the performance of deep neural networks is by increasing their size, including both the depth — the number of levels — of the network and its width — the number of units at each level. This is an easy and safe way of training higher quality models, especially given the availability of large labeled training sets. However, this approach has two major drawbacks. Bigger size typically means a larger number of parameters, which makes the enlarged network more prone to overfitting, especially if the number of labeled examples is limited. It also results in dramatically increased use of computational resources. The fundamental way of solving both issues would be by moving from fully connected to sparsely connected architectures, even inside the convolutions.

Arora et al. provided theoretical justification suggesting that if the probability distribution of the dataset is representable by a large, very sparse deep neural network, then the optimal network topology can be constructed layer by layer by analyzing the correlation statistics of the activations of the last layer and clustering neurons with highly correlated outputs. Although the mathematical proof requires very strong conditions, the fact that the Inception architecture was a close approximation to a sparse structure appears to validate the underlying assumption. The main idea of the Inception architecture is to consider how an optimal local sparse structure of a convolutional vision network can be approximated and covered by readily available dense components.

The Inception module performs parallel convolutions with multiple filter sizes (1x1, 3x3, 5x5) along with a parallel max pooling path. The outputs are concatenated along the channel dimension. A naive version of this module would suffer from computational explosion due to the large number of filters. To address this, 1x1 convolutions are used as dimensionality reduction modules before the expensive 3x3 and 5x5 convolutions. This reduces the computational cost by an order of magnitude while preserving representational power. The resulting architecture can increase both the depth and width of the network without uncontrolled increase in computational complexity.

The overall GoogLeNet architecture is 22 layers deep (27 layers if counting pooling) and uses approximately 5 million parameters, which is 12x fewer than AlexNet (60M) and significantly fewer than VGGNet (138M). The network employs 9 Inception modules stacked upon each other, with occasional max-pooling layers with stride 2 to halve the resolution. Notably, the network uses an average pooling layer before the final classifier instead of fully-connected layers, which further reduces parameters. Additionally, two auxiliary classifiers are attached to intermediate layers to combat the vanishing gradient problem and provide regularization.

On the ILSVRC 2014 classification challenge, GoogLeNet achieved a top-5 error rate of 6.67%, ranking first among all participants. This was significantly better than the previous year's best result and close to human-level performance. An ensemble of 7 GoogLeNet models was used for the competition submission. On the detection task, the GoogLeNet approach achieved 43.9% mAP, which was also the winning entry, surpassing the previous best by a large margin. Importantly, the strong detection results were achieved by combining the Inception architecture with the R-CNN approach by Girshick et al., demonstrating the generalizability of the Inception features beyond pure classification.

Our results yield a solid evidence that approximating the expected optimal sparse structure by readily available dense building blocks is a viable method for improving neural networks for computer vision. The main advantage of this method is a significant quality gain at a modest increase of computational requirements compared to shallower and less wide networks. Our Inception architecture demonstrates that moving to sparser architectures is feasible and useful. This suggests that there is hope for automatic creation of network topologies based on correlation statistics in the future, complementing or even replacing hand-crafted solutions.