Neural networks for image recognition have evolved through extensive manual design from simple chain-like models to structures with multiple wiring paths. This paper investigates randomly wired neural networks using stochastic network generators based on three classical random graph models: Erdos-Renyi, Barabasi-Albert, and Watts-Strogatz. Rather than constraining architecture searches through hand-designed network generators, the authors explore broader connectivity patterns. Their key finding: "several variants of these random generators yield network instances that have competitive accuracy on the ImageNet benchmark," suggesting that network generator design may be more important than previously recognized in neural architecture search.

The authors frame neural network design as fundamentally about "how computational networks are wired." Recent advances moved from sequential architectures to designs like ResNets and DenseNets with sophisticated connectivity patterns. While neural architecture search (NAS) methods jointly optimize wiring and operations, their search spaces remain constrained by hand-designed network generators. The paper poses a critical question: "What happens if we loosen this constraint and design novel network generators?"

Rather than manually designing wiring patterns, the authors propose using classical random graph models from graph theory to generate diverse connectivity patterns. Notably, networks generated by the Watts-Strogatz model achieved approximately 79% ImageNet accuracy, comparable to or exceeding ResNet-50's 77.1% accuracy under similar computational budgets. The authors argue this represents a conceptual shift: "from designing an individual network to designing a network generator," paralleling the transition from hand-engineered features to learned representations.

The paper traces randomly wired systems to early AI pioneers: Turing's "unorganized machines," Minsky's randomly wired perceptron hardware, and Rosenblatt's Mark I Perceptron with randomly connected photocells. The work connects to neuroscience observations that biological neural networks exhibit small-world properties and substantial random connectivity patterns. In the NAS literature, the authors identify that NAS cells contain exactly 5 nodes with precisely 2 input edges and 1 output edge, highlighting the "hand-designed prior that fundamentally limits the search space despite being largely overlooked."

The authors formally define a network generator as a mapping function from parameter space to neural network architectures. Stochastic network generators accept a random seed s, creating families of networks where multiple calls with fixed parameters but varying s produce a probability distribution over network instances. Edge operations represent data flow (directed tensor transmission); node operations perform aggregation (weighted sum with learnable positive weights), transformation (ReLU-convolution-batch normalization triplet), and distribution. This design ensures "FLOPs and parameter count remain nearly independent of network wiring," with typical deviations of plus or minus 2%.

Three classical models are employed. Erdos-Renyi (ER): each node pair connects with independent probability P; graphs remain likely connected when P > ln(N)/N. Barabasi-Albert (BA): sequential node addition using preferential attachment (nodes connect with probability proportional to existing degree). Watts-Strogatz (WS): starts with ring topology where nodes connect to K/2 neighbors, then rewires edges with probability P, producing small-world properties with high clustering. Testing shows non-random WS variants (P=0) performed worse than randomized counterparts, suggesting "randomness benefits generator design."

In the small computation regime (~580M FLOPs), all generators achieved >73% ImageNet accuracy with remarkably low standard deviation (0.2-0.4%) across 5 instances. WS(4, 0.75) achieved 73.8% mean accuracy. In the regular computation regime, RandWire-WS achieved 79.0% accuracy versus ResNet-50's 77.1% (1.9% improvement) and 80.1% versus ResNet-101's 78.8%. On COCO object detection transfer, the WS variant achieved 39.9 AP versus ResNet-50's 37.1 AP. Node operation variants showed generator accuracy rankings remained stable with Pearson correlation of 0.91-0.98, suggesting "network wiring plays a role somewhat orthogonal" to operation choice.

The authors demonstrated that classical random graph models can generate competitive neural network architectures for image recognition. Mean accuracies achieved through randomly wired networks rival both hand-designed architectures and NAS-discovered models. These findings highlight that "network generator design" deserves greater research attention, as "the generator design determines a probabilistic distribution over networks." Rather than focusing solely on search algorithms with fixed generators, the community should explore novel generator designs producing new architecture families.