In this paper, we introduce a new channel pruning method to accelerate very deep convolutional neural networks. Given a trained CNN model, we propose an iterative two-step algorithm to effectively prune each layer, by a LASSO regression based channel selection and least square reconstruction. We reduce the feature map width — obtaining thinner networks that are both compact and directly deployable on conventional hardware without requiring sparse matrix libraries or any special implementations. Experiments demonstrate the effectiveness of our approach: we achieve 5x speedup on VGG-16 with only 0.3% increase in top-5 error, and 2x speedup on ResNet-56 with 1.0% accuracy loss.

CNN acceleration approaches can be categorized into three types: optimized implementation (e.g., FFT-based convolution), quantization (e.g., BinaryNet), and structured simplification. Channel pruning belongs to the third category and offers distinct advantages: unlike tensor factorization methods that decompose filters, channel pruning "directly reduces feature map width, which shrinks a network into thinner one". Unlike sparse connection methods that produce irregular layers requiring specialized libraries, channel pruning yields regular, dense convolutions that work efficiently on both CPU and GPU with standard libraries. Our key insight is to minimize reconstruction error on output feature maps rather than analyzing filter weights directly, which better preserves the network's functional behavior.

Weight pruning methods (Han et al.) achieve high compression ratios but produce sparse, irregular networks that require dedicated hardware or software for efficient execution. Filter-level pruning methods (Li et al.) remove entire filters based on magnitude but use simple heuristics (e.g., L1-norm of weights) that may not optimally preserve network accuracy. Tensor factorization approaches decompose weight tensors but encounter "performance decadence" on GPU when applied to 1x1 convolutions common in modern architectures. Our method differs in that it "fully exploits redundancy inter feature maps" by directly optimizing the reconstruction of output feature maps rather than analyzing individual filter properties.

Given a trained CNN, we formulate channel pruning as minimizing the reconstruction error on output feature maps. For each layer, we solve: minimize ||Y - sum(beta_i * X_i * W_i^T)||_F^2 subject to ||beta||_0 <= c', where beta is the channel selection vector, X_i are input channels, W_i are filter weights, and c' is the target number of channels. Since L0 minimization is NP-hard, we relax it to L1 regularization (LASSO). The algorithm alternates between two steps: Step (i) — Channel Selection via LASSO: solve for beta using LASSO regression, identifying which channels contribute most information (setting redundant channels' beta to zero). Step (ii) — Feature Map Reconstruction: with selected channels fixed, solve for optimal filter weights W via least squares, minimizing ||Y - X'(W')^T||_F^2. The penalty coefficient lambda is gradually increased until the desired sparsity is achieved.

For residual networks, two key modifications are needed. For the last layer of the residual branch: rather than approximating only the residual output Y_2, we "approximate Y_1 + Y_2" (shortcut + residual combined) to compensate for the parameter-free shortcut connection. For the first layer of the residual branch: since input channels cannot be pruned (shared with shortcut), we perform "feature map sampling before the first convolution to save computation", selecting representative input channels without altering the regular convolutional structure. This multi-branch enhancement provides 4.0% improvement in error reduction compared to naive application of the single-branch algorithm.

VGG-16 Results: At 2x speedup, the method achieves 0% accuracy loss after fine-tuning. At 4x speedup, top-5 error increases by only 1.0%. Combined with tensor factorization, 5x speedup with only 0.3% error increase is achieved, outperforming all previous state-of-the-art methods. GPU benchmarks show 2.5x actual GPU speedup, substantially better than tensor factorization approaches which encountered "performance decadence." Comparison with training from scratch shows that "the same model is difficult to be obtained from scratch" — fine-tuned pruned models outperform networks trained from scratch at equivalent computational budgets.

ResNet and Xception Results: On ResNet-50 at 2x speedup, the method achieves 1.4% top-5 error increase after fine-tuning (from 8.0% without enhancement and 4.0% with multi-branch enhancement). On Xception at 2x speedup, only 1.0% error increase is observed. The authors note that modern networks "tend to have less redundancy by design", explaining the smaller gains compared to VGG-16. For object detection transfer (Faster R-CNN on PASCAL VOC), 4x speedup results in only 1.8% mAP loss (actual runtime: 94ms vs. 220ms baseline), demonstrating the transferability of pruned features to downstream tasks.

We demonstrated that very deep CNNs possess substantial channel-level redundancy exploitable through structured pruning. Our combination of LASSO-based channel selection and least-squares reconstruction achieves competitive speedups while maintaining accuracy, and "only requires off-the-shelf libraries" for implementation. The method successfully accelerates modern architectures like ResNet and Xception where tensor factorization struggles with 1x1 convolutions. Channel pruning offers a practical path to deploying deep networks in resource-constrained environments without specialized hardware support.