Neural Radiance Fields (NeRF) have shown impressive results for novel view synthesis from well-lit photographs. However, standard NeRF relies on processed low dynamic range (LDR) images, losing valuable high dynamic range (HDR) information and introducing bias from the camera's image signal processor. The authors propose RawNeRF, which modifies NeRF to train directly on linear raw images rather than tonemapped outputs. This enables high dynamic range view synthesis from noisy inputs captured in near-darkness, leveraging 3D multiview consistency across 25-200 input frames to effectively denoise while preserving full HDR information. The method enables novel applications including post-capture exposure adjustment, tonemapping manipulation, and synthetic defocus with physically accurate bokeh.

NeRF and its extensions have become the dominant paradigm for neural scene representation and novel view synthesis. Yet virtually all existing methods assume well-exposed, noise-free input images processed by the camera's image signal processor (ISP). This processing pipeline applies nonlinear transformations including demosaicing, white balancing, gamma correction, and tonemapping, which irreversibly compress the scene's dynamic range and discard information in highlights and shadows. In low-light conditions, the ISP further amplifies sensor noise through its nonlinear processing, producing images with severe artifacts. The authors argue that training NeRF directly on raw sensor data, bypassing the ISP entirely, can simultaneously solve the problems of noise, limited dynamic range, and inflexible post-processing.

The key insight is that NeRF's multiview aggregation provides a natural mechanism for denoising through 3D consistency. When multiple noisy observations of the same 3D point are available from different viewpoints, fitting a smooth radiance field to these observations implicitly averages out the noise. Unlike traditional single-image denoisers that must hallucinate missing details, RawNeRF leverages genuine multiview information to recover both geometry and appearance. Furthermore, by operating in linear raw space, the reconstructed radiance field preserves the full dynamic range of the scene, enabling post-capture manipulation such as exposure changes, tonemapping, and synthetic defocus.

This work synthesizes three research areas. In novel view synthesis, methods from NeRF to mip-NeRF have progressively improved quality but always assume clean, tonemapped inputs. In image denoising, learning-based approaches have shown impressive results on raw-domain denoising, including SID (See in the Dark) which demonstrated extreme low-light enhancement from raw sensor data. However, these methods process single images or video frames independently, without exploiting 3D scene structure. In computational photography, HDR imaging typically requires bracketed exposures or specialized hardware, while synthetic defocus requires depth estimation or multi-aperture systems. RawNeRF uniquely addresses all three capabilities — denoising, HDR recovery, and computational photography — through a single 3D-consistent framework.

RawNeRF is built upon mip-NeRF, which uses integrated positional encoding over conical frustums for anti-aliased rendering. The key modification is that the network outputs linear raw color values instead of sRGB colors. An exponential activation function is used for the color output to ensure positivity and to naturally represent the wide dynamic range of linear raw values, which can span several orders of magnitude. The network directly processes full-resolution mosaicked Bayer pattern data, treating each color channel (R, G, G, B) as a separate observation at each pixel location, which provides additional training signal.

Standard L2 loss on linear raw values is dominated by bright pixels, causing the network to ignore dark regions where noise is most problematic. The authors propose a tone-curve-weighted loss function that weights errors by the derivative of a logarithmic tone curve: (y_hat - y) / (sg(y_hat) + epsilon)^2, where sg() denotes stop-gradient. This weighting assigns equal perceptual importance to dark and bright regions, ensuring the network devotes capacity to reconstructing shadow details. Critically, the stop-gradient operator ensures unbiased gradient estimates even when training targets (raw pixels) contain noise, preventing the network from fitting to the noise.

To maximize dynamic range, input captures may include bracketed exposures with varying shutter speeds. RawNeRF handles this by scaling network outputs by the shutter speed ratio — the network learns a canonical radiance at a reference exposure, and different exposures are modeled by multiplying the output by the appropriate shutter time. Additionally, per-channel affine calibration factors are learned to correct for sensor miscalibration and vignetting that may vary across frames. This design enables training from heterogeneous exposure collections, and at inference time, the user can render at any desired exposure level and apply sophisticated tonemapping algorithms such as HDR+.

RawNeRF is evaluated on indoor and outdoor scenes captured in low-light conditions. For denoising performance, the method achieves results competitive with state-of-the-art denoisers including SID, Unprocess, RViDeNet, and UDVD, despite having no clean training data and using only camera pose information rather than the test image itself. For HDR applications, the method demonstrates exposure variation, sophisticated tonemapping (HDR+), and synthetic defocus with physically accurate bokeh effects on out-of-focus light sources. A key advantage is that RawNeRF produces 3D-consistent results across viewpoints, while single-image methods can produce temporally inconsistent outputs when applied to video sequences.

RawNeRF demonstrates that training neural radiance fields directly on raw sensor data enables high dynamic range, low-noise view synthesis from challenging captures. By operating in linear raw space, the method preserves the full dynamic range of the scene and enables post-capture manipulation of exposure, tonemapping, and focus. The 3D multiview consistency inherent in NeRF serves as a powerful implicit denoiser. The authors acknowledge computational demands, dependence on COLMAP pose estimation, and inability to handle dynamic scenes as current limitations, but view RawNeRF as progress toward robust, high-quality capture of real-world environments in any lighting condition.