We present Deep Neural Decision Forests — a novel approach that unifies classification trees with the representation learning functionality known from deep convolutional networks. These two architectures are seamlessly combined by introducing a stochastic and differentiable decision tree model, which steers the representation learning usually conducted in the initial layers of a (deep) convolutional network. Our principled, joint and global optimization of split and leaf node parameters is made possible through back-propagation. We achieve competitive results on benchmark datasets including MNIST and ImageNet, with Top-5 errors of only 7.84%/6.38% on ImageNet validation data when integrating our forests in a single-crop, single/seven model GoogLeNet architecture.

Deep convolutional neural networks (CNNs) have become the dominant paradigm in visual recognition, yet their monolithic end-to-end architecture makes them difficult to interpret. On the other hand, decision trees and random forests offer inherent interpretability through their hierarchical partitioning of feature space, but they rely on hand-crafted features that limit their representational power. Combining the representation learning capability of deep networks with the structured, interpretable decision-making of forests promises to capture the best of both worlds.

Previous attempts to combine neural networks and decision trees have been limited to shallow architectures or disjoint training procedures. Some approaches first train a CNN, then use its features to build a separate forest; others alternate between optimizing tree parameters and network weights without true joint optimization. Our key insight is that by formulating the decision tree with stochastic routing functions based on sigmoid activations, we can make the entire forest differentiable and thus amenable to end-to-end training via stochastic gradient descent.

Random forests have a long history in computer vision, from Breiman's original formulation to specialized variants for pose estimation, segmentation, and object detection. Their strength lies in efficient ensemble learning with built-in feature selection. However, they traditionally rely on pre-defined feature spaces (e.g., HOG, SIFT), limiting adaptation to task-specific representations. Deeply-supervised networks and conditional computation approaches share the spirit of hierarchical decision-making but lack the explicit tree structure that enables principled ensemble methods.

A conventional decision tree routes each input deterministically to a single leaf node. In contrast, we introduce stochastic routing: at each split node n, the input is sent to the left child with probability d_n(x; Θ) = σ(f_Θ(x)), where σ is the sigmoid function and f_Θ(x) is the output of the CNN at the corresponding split node. The probability of reaching a leaf l is the product of routing probabilities along the path from root to l. The tree's prediction is a weighted combination of all leaf distributions π_l, where each weight equals the probability of reaching that leaf.

The Deep Neural Decision Forest (dNDF) integrates multiple stochastic decision trees into a forest, where all trees share the same CNN feature extractor. The CNN maps input images to a high-dimensional feature representation, and different subsets of these features are routed to different split nodes across trees. This architecture allows the CNN's representation learning to be guided by the forest's decision-making structure — the forest provides gradient signals that encourage the CNN to learn features discriminative at each level of the hierarchical partition. The final prediction averages over all trees: P(y|x) = (1/T) Σ_t P_t(y|x).

Training proceeds by alternating between two steps. In the first step, the leaf node distributions π are updated by fixing the CNN parameters and solving a convex optimization problem per tree. In the second step, the CNN parameters Θ are updated via back-propagation with the leaf distributions held fixed. The loss function is the negative log-likelihood of the data under the forest model. Because the routing functions are differentiable sigmoid functions, gradients flow seamlessly from the tree structure back into the CNN, enabling true end-to-end learning. This alternating optimization converges reliably in practice.

Experiments are conducted on MNIST and ImageNet (ILSVRC 2012). On MNIST, the dNDF achieves state-of-the-art accuracy with a compact model. On ImageNet, integrating the neural decision forest with GoogLeNet as the feature backbone yields Top-5 error rates of 7.84% (single model) and 6.38% (seven-model ensemble) on the validation set, improving upon the GoogLeNet baseline of 6.67% in the ensemble setting. Notably, these results are obtained without dataset augmentation beyond standard cropping and flipping. Ablation studies show that increasing the number of trees and tree depth both contribute to performance gains, and that the learned representations differ meaningfully from those of a standard softmax classifier.

We have presented Deep Neural Decision Forests, a principled approach for unifying deep representation learning with stochastic decision forests. By making the decision tree differentiable through sigmoid routing functions, we enable end-to-end joint optimization of the CNN feature extractor and the forest's split and leaf parameters. The resulting model benefits from the complementary strengths of both paradigms: the CNN provides powerful learned features while the forest offers structured, ensemble-based prediction. Experiments on MNIST and ImageNet demonstrate competitive performance, validating the potential of this hybrid architecture for large-scale visual recognition tasks.