Neural Radiance Fields (NeRF) can be used to reconstruct scenes using photos taken under controlled settings. However, "applying NeRF to casually captured images or Internet photo collections remains challenging" due to variable illumination and transient occluders (e.g., tourists, vehicles). The authors introduce NeRF-W, which extends NeRF by modeling per-image appearance variations and separately modeling transient phenomena. This enables accurate 3D reconstruction from unstructured, in-the-wild photo collections, such as Internet photos of famous landmarks.

NeRF achieves remarkable novel view synthesis quality by representing a scene as a continuous 5D function (3D position + 2D viewing direction) mapped to density and color via an MLP. However, the original formulation assumes that "the scene is static and captured under consistent lighting conditions." Real-world photo collections, such as those found on Flickr or Google Images for tourist landmarks, violate these assumptions: photos are taken at different times of day, seasons, and weather conditions, and commonly contain transient objects that are not part of the permanent scene structure.

Traditional structure-from-motion (SfM) and multi-view stereo (MVS) pipelines can handle some appearance variation through feature matching invariance, but their reconstructions are often "incomplete, noisy, and cannot render photorealistic novel views." Simply applying NeRF to such collections results in "blurry, artifact-laden renderings" as the network attempts to average over inconsistent observations. The authors argue that the solution requires explicitly modeling the factors that cause inter-image variation rather than treating them as noise.

Prior work on 3D reconstruction from Internet photos includes landmark-scale SfM systems that recover camera poses and sparse point clouds from thousands of photos. Appearance modeling in multi-view reconstruction has been explored through per-image affine color transforms and learned appearance embeddings. For transient object handling, some methods use semantic segmentation to mask out people, but this requires pre-trained detectors and cannot handle all types of transient phenomena. NeRF and its concurrent extensions focus on controlled capture settings, leaving the in-the-wild scenario largely unaddressed.

The original NeRF represents a static scene as a function F: (x, d) -> (c, sigma) mapping a 3D position x and viewing direction d to an RGB color c and volume density sigma. Novel views are synthesized via classical volume rendering: integrating color weighted by accumulated transmittance along camera rays. The model is trained by minimizing the photometric loss between rendered and observed pixel colors. This formulation implicitly assumes that each 3D point has a single, fixed appearance regardless of when or how it was photographed.

To handle variable illumination, the authors introduce a per-image appearance embedding vector l_i that is optimized jointly with the network parameters. The NeRF function becomes F: (x, d, l_i) -> (c_i, sigma), where the color output now depends on which image is being rendered. The density sigma remains shared across all images, ensuring that "the underlying geometry is consistent while allowing photometric variation." This models phenomena such as different times of day, seasonal changes, and varying camera white balance settings.

For transient occluders, the authors add a separate transient head that predicts per-image transient color and density: F_tau: (x, l_tau_i) -> (c_tau, sigma_tau, beta). The transient density sigma_tau is image-specific and not shared across views, reflecting that transient objects appear in specific images only. An uncertainty output beta is used to down-weight the loss for pixels containing transient objects. The total rendered color combines static and transient components, and the loss applies per-pixel heteroscedastic uncertainty weighting plus a regularizer that encourages zero transient density.

Experiments are conducted on the Phototourism dataset, which contains Internet photo collections of landmarks including the Brandenburg Gate, the Sacre Coeur, and the Trevi Fountain. NeRF-W is compared against original NeRF, NeRF with appearance embedding only, and traditional MVS methods. Results show that NeRF-W significantly outperforms baseline NeRF in PSNR, SSIM, and LPIPS metrics. The appearance embeddings successfully capture lighting variations — interpolating between embeddings produces smooth transitions between day and night appearances. The transient model correctly identifies and down-weights tourists, scaffolding, and other temporary objects. Ablation studies confirm that both appearance and transient modeling are necessary for optimal results.

NeRF-W extends neural radiance fields to handle unconstrained photo collections by introducing per-image appearance embeddings for photometric variation and a transient object model with uncertainty-based loss weighting. The method enables high-quality 3D reconstruction and novel view synthesis from Internet photo collections, a setting previously inaccessible to neural rendering methods. The learned appearance and transient decomposition also opens possibilities for appearance editing and scene cleaning applications.