We present a method that achieves state-of-the-art results for synthesizing novel views of complex scenes by optimizing an underlying continuous volumetric scene function using a sparse set of input views. Our algorithm represents a scene using a fully-connected (non-convolutional) deep network, whose input is a single continuous 5D coordinate (spatial location and viewing direction) and whose output is the volume density and view-dependent emitted radiance at that spatial location. We synthesize views by querying 5D coordinates along camera rays and use classic volume rendering techniques to project the output colors and densities into an image.

In this work, we address the long-standing problem of view synthesis — generating photorealistic novel views of a scene from a set of input photographs. Existing approaches using discrete representations such as triangle meshes or voxel grids are fundamentally limited by their discrete nature, which requires prohibitively large storage for high-resolution complex scenes. We propose to represent a continuous scene as a 5D neural radiance field — a multilayer perceptron (MLP) that maps a 3D location and 2D viewing direction to color and density. This continuous representation naturally handles arbitrary resolution and avoids the storage limitations of discrete methods.

While neural networks have been used for implicit scene representations before, prior work has been limited to simple scenes with low geometric complexity. We overcome this by introducing two key improvements: positional encoding of input coordinates to enable the MLP to represent high-frequency functions, and a hierarchical sampling strategy that efficiently allocates samples to regions with visible scene content. Together, these allow us to represent complex real-world scenes with unprecedented photorealistic quality.

We represent a continuous scene as a 5D vector-valued function whose input is a 3D location x = (x, y, z) and 2D viewing direction (theta, phi), and whose output is an emitted color c = (r, g, b) and volume density sigma. In practice, we express direction as a 3D Cartesian unit vector d. We approximate this continuous function with an MLP network F_Theta: (x, d) -> (c, sigma) and optimize its weights Theta to map from each input 5D coordinate to its corresponding volume density and directional emitted color. We encourage the representation to be multiview consistent by restricting the network to predict volume density sigma as a function of only the location x, while allowing the color c to be predicted as a function of both location and viewing direction.

We render the color of any ray passing through the scene using classical volume rendering principles. The volume density sigma(x) can be interpreted as the differential probability of a ray terminating at an infinitesimal particle at location x. The expected color C(r) of camera ray r(t) = o + td with near and far bounds t_n and t_f is computed by integrating the color weighted by the accumulated transmittance along the ray. In practice, we estimate this integral using stratified sampling, dividing [t_n, t_f] into N evenly-spaced bins and drawing one sample uniformly at random from within each bin.

Despite containing universal approximation properties, we found that having the network F_Theta directly operate on (x, d) input coordinates results in poor performance at representing high-frequency variation in color and geometry. This is consistent with recent work showing that deep networks are biased towards learning lower frequency functions — a phenomenon known as spectral bias. We address this by mapping the inputs to a higher dimensional space using high frequency functions gamma(p) = (sin(2^0 pi p), cos(2^0 pi p), ..., sin(2^{L-1} pi p), cos(2^{L-1} pi p)) before passing them to the network. This positional encoding enables the MLP to more easily approximate higher frequency functions.

We quantitatively evaluate NeRF on both synthetic and real-world scenes. On the synthetic dataset of 8 complex scenes, NeRF achieves an average PSNR of 31.01 dB, compared to 26.50 dB for SRN, 27.81 dB for LLFF, and 24.15 dB for NV. On real forward-facing scenes, NeRF achieves 26.50 dB PSNR, outperforming all baselines. The visual quality shows sharp details, accurate specular reflections, and coherent geometry that prior methods fail to capture.

We perform an ablation study to analyze the contribution of each component. Removing positional encoding reduces PSNR by more than 4 dB, confirming its critical role. Removing view dependence reduces PSNR by about 1 dB and eliminates specular effects. Replacing the hierarchical sampling with uniform sampling reduces quality and increases training time. Each component is shown to contribute meaningfully to the final performance.

We have presented neural radiance fields (NeRF), a method that represents scenes as continuous 5D functions parameterized by neural networks for photorealistic view synthesis. Our approach achieves unprecedented visual quality by leveraging positional encoding and hierarchical volume sampling. While our method requires per-scene optimization and is computationally intensive, we believe that neural radiance fields open up a promising new direction for representing and rendering 3D scenes from captured images.