Currently, the neural network architecture design is mostly guided by the indirect metric of computation complexity, i.e., FLOPs. However, the direct metric, e.g., speed, also depends on the other factors such as memory access cost and platform characteristics. Thus, using FLOPs as the only metric for computation complexity is insufficient and could lead to sub-optimal design.

In this work, we propose to evaluate the direct metric on the target platform, beyond only considering FLOPs. Based on a series of controlled experiments, we derive several practical guidelines for efficient network design. Accordingly, a new architecture is presented, called ShuffleNet V2. Comprehensive ablation experiments verify that our model is the state-of-the-art in terms of speed and accuracy tradeoff.

Deep neural networks have driven remarkable progress in many computer vision tasks. The growing demands for mobile and embedded applications require more efficient network architectures. Many works have been devoted to designing lightweight networks, such as MobileNet, ShuffleNet, and CondenseNet. A common practice is to use FLOPs as the metric for computation complexity. However, FLOPs is an indirect metric that does not directly correspond to the speed or latency on different hardware platforms.

There are mainly two reasons for the discrepancy between FLOPs and speed. First, memory access cost (MAC) is not taken into account by FLOPs but constitutes a significant portion of the runtime in many operations like group convolution. Second, the degree of parallelism also affects speed, and a model with the same FLOPs could run faster if it has higher parallelism. These factors motivate us to study the direct metrics for evaluating network efficiency.

Based on our analysis, we derive four practical guidelines for efficient network architecture design. G1: Equal channel width minimizes memory access cost. When the input and output channel widths of a 1x1 convolution are different, the MAC increases. Our experiments show that when input and output channels are equal, the MAC achieves the minimum given fixed FLOPs. G2: Excessive group convolution increases MAC. While group convolution reduces FLOPs, the MAC increases dramatically with the number of groups due to increased data reading and writing operations.

G3: Network fragmentation reduces degree of parallelism. Although multi-path structures like those in GoogLeNet and NASNet are beneficial for accuracy, they reduce the efficiency on GPUs due to the difficulty of parallelization. G4: Element-wise operations are non-negligible. Operations like ReLU, AddTensor, AddBias have small FLOPs but relatively high MAC and constitute a significant portion of the total runtime, especially in lightweight networks where the proportion of element-wise operations is larger.

Following the above guidelines, we design a new network architecture. The key building block is the channel split operation. At the beginning of each unit, the input of c feature channels is split into two branches with c-c' and c' channels. Following G3, one branch remains as identity. The other branch consists of three convolutions with the same input and output channels to satisfy G1. The 1x1 convolutions are no longer group-wise, following G2. After convolution, the two branches are concatenated instead of added, following G4. After concatenation, a channel shuffle operation is applied to enable information communication between the two branches.

We compare ShuffleNet V2 with several state-of-the-art lightweight networks on ImageNet 2012 classification dataset. All models are evaluated on GPU (NVIDIA GTX 1080Ti) and ARM (Qualcomm Snapdragon 810) platforms. Under similar computation complexity (~300 MFLOPs), ShuffleNet V2 achieves 72.6% top-1 accuracy, surpassing ShuffleNet V1 (71.5%), MobileNet V2 (72.0%), and Xception (69.5%). More importantly, ShuffleNet V2 is the fastest among all compared models on both GPU and ARM platforms.

We further evaluate ShuffleNet V2 on COCO object detection as backbone network. Using the light-head RCNN framework, ShuffleNet V2 achieves better detection accuracy than ShuffleNet V1 and MobileNet V2 while being faster. This demonstrates that the efficiency guidelines derived from classification tasks transfer well to other tasks such as object detection, validating the generality of our approach.

We propose that network architecture design should be guided by the direct metric of speed on the target platform, rather than the indirect metric of FLOPs. We present four practical guidelines derived from comprehensive experiments, and design ShuffleNet V2 following these guidelines. Our model achieves state-of-the-art speed-accuracy tradeoff on both GPU and ARM platforms. We believe that our guidelines will be useful for future network architecture design and neural architecture search.