The ability to perform pixel-wise semantic segmentation in real-time is of paramount importance in mobile applications. Recent deep neural networks have achieved impressive accuracy on segmentation benchmarks, but they require significant computational resources and are far from real-time on embedded devices. We propose ENet (Efficient Neural Network), a novel neural network architecture designed specifically for tasks requiring low latency operation. ENet is up to 18x faster, requires 75x less FLOPs, has 79x less parameters, and provides similar or better accuracy compared to existing models. We have tested it on CamVid, Cityscapes, and SUN datasets and report on both accuracy and speed.

Semantic segmentation is important for understanding visual scenes, with applications in autonomous driving, augmented reality, and robotics. State-of-the-art networks like SegNet and FCN have achieved good accuracy but remain too slow for real-time applications, especially on mobile and embedded platforms. The key challenge is to design a network that is both accurate and efficient. We observe that existing architectures are over-parameterized for the task of segmentation, and that careful architectural design can dramatically reduce computation without sacrificing accuracy. ENet is designed from the ground up with efficiency as the primary constraint.

ENet consists of an asymmetric encoder-decoder structure with a large encoder and a small decoder. The encoder follows the design of SegNet but with significantly fewer layers. The key building block is the bottleneck module, inspired by ResNet, consisting of a 1x1 projection that reduces dimensionality, a main convolutional layer (regular, dilated, or asymmetric), and a 1x1 expansion. We also use early downsampling — the initial block aggressively reduces resolution through a combination of max pooling and convolution. The decoder is intentionally lightweight: its role is only to upsample the encoder output and fine-tune details, not to learn complex features.

Several key design choices contribute to ENet's efficiency: (1) PReLU activation is used instead of ReLU, providing a small but consistent accuracy improvement; (2) Spatial dropout with varying rates in different stages helps regularize the network; (3) Asymmetric convolutions (5x1 followed by 1x5) replace standard 5x5 convolutions, reducing parameters; (4) Dilated convolutions are used in later encoder stages to increase the receptive field without increasing computational cost. These choices collectively enable ENet to achieve only 0.37M parameters, compared to SegNet's 29.5M.

We evaluate ENet on Cityscapes, CamVid, and SUN RGB-D. On Cityscapes, ENet achieves 58.3% class IoU, comparable to SegNet's 56.1% while being 18x faster. On CamVid, ENet achieves 51.3% class IoU at 76.9 FPS on an NVIDIA TX1 embedded platform. ENet processes a 640x360 image in 7ms on a Titan X GPU — true real-time performance. On the Jetson TX1 mobile platform, ENet runs at 20+ FPS for high-resolution inputs. While ENet does not achieve the absolute highest accuracy, its accuracy-to-speed ratio is unmatched, making it the ideal choice for embedded and mobile applications.

We have proposed ENet, an efficient neural network architecture designed for real-time semantic segmentation on resource-constrained platforms. Through careful architectural design — asymmetric encoder-decoder, early downsampling, bottleneck modules with dilated and asymmetric convolutions — ENet achieves competitive segmentation accuracy while being orders of magnitude more efficient than existing methods. ENet enables semantic segmentation on embedded devices and mobile platforms for the first time.