Neural Radiance Fields (NeRF) have emerged as a powerful representation for synthesizing novel views of complex scenes from a sparse set of input views. However, NeRF is limited to static scenes. This paper introduces D-NeRF, a method that extends neural radiance fields to dynamic, time-varying scenes. The key idea is to decompose the scene into a canonical configuration and a time-conditioned deformation field. Given a monocular video as input, D-NeRF simultaneously learns a canonical radiance field and a deformation network that maps observation-space 3D points at each time step to their canonical positions, enabling novel view synthesis of dynamic objects from a single camera viewpoint.

Novel view synthesis aims to generate images from viewpoints not present in the input data. NeRF achieves photorealistic results by representing a scene as a continuous volumetric function mapping 3D coordinates and viewing directions to color and density, optimized through differentiable volume rendering. Despite its remarkable quality, the original NeRF formulation assumes a completely static scene captured from multiple viewpoints. This assumption severely limits applicability to real-world scenarios where objects move, deform, or change over time.

Extending NeRF to dynamic scenes poses fundamental challenges: the number of observations per time step is typically very limited (often just one view), making the problem severely under-constrained. Existing approaches either require multi-view synchronized capture systems or rely on pre-computed 3D reconstructions as input. D-NeRF addresses this by learning a shared canonical representation alongside a time-dependent deformation field, effectively amortizing observations across time to overcome the single-view-per-timestep limitation.

Dynamic scene reconstruction has been a long-standing challenge. Traditional methods such as non-rigid structure from motion (NRSfM) and scene flow estimation typically require multi-view input or strong priors about scene geometry. Recent learning-based approaches include Neural Volumes, which uses an encoder-decoder architecture to predict a voxel grid, but is limited by the voxel resolution. Nerfies and Neural Scene Flow Fields address dynamic scenes but require multi-view capture or scene flow supervision. In contrast, D-NeRF operates on monocular video input without any 3D supervision.

D-NeRF models a dynamic scene through two networks: a canonical network and a deformation network. The canonical network follows the standard NeRF formulation, mapping a 3D position x = (x, y, z) and viewing direction d = (theta, phi) to color c = (r, g, b) and volume density sigma. This canonical network represents the scene in a fixed reference configuration, analogous to a template mesh in traditional non-rigid registration. The canonical space is chosen as the configuration at time t=0.

The deformation network learns a mapping Delta_x = D(x, t) that transforms a 3D point x at time t into its corresponding position in canonical space: x_canonical = x + Delta_x. This network is implemented as an MLP conditioned on both spatial coordinates and time, with positional encoding applied to both inputs. The deformation field implicitly enforces temporal coherence by sharing the canonical representation across all time steps. At t=0, the deformation network should output zero displacement, which is achieved through a regularization loss penalizing non-zero deformations at the canonical time.

D-NeRF is evaluated on a set of synthetic dynamic scenes including bouncing balls, jumping jacks, a hook, a T-Rex, a standing up motion, a mutant, a hellwarrior, and a lego bulldozer. Each scene consists of monocular video frames rendered from a fixed viewpoint with ground truth novel views for evaluation. Quantitative results show that D-NeRF achieves PSNR values ranging from 29 to 35 dB across scenes, significantly outperforming baselines including NeRF applied per-frame (21-25 dB) and T-NeRF without canonical decomposition (26-30 dB). The method also demonstrates convincing novel view synthesis at unseen time steps through temporal interpolation.

Ablation studies confirm the importance of key design choices: removing positional encoding for time reduces PSNR by 2-4 dB, demonstrating that high-frequency temporal signals are necessary for capturing fine-grained motions. Without the canonical-time regularization, performance drops by 1-2 dB as the canonical space becomes ambiguous. The two-stage coarse-to-fine training strategy inherited from NeRF proves essential, with the fine network contributing 2-3 dB improvement over the coarse network alone.

This paper presents D-NeRF, the first approach that successfully extends Neural Radiance Fields to dynamic scenes using only monocular video input. By decomposing the problem into a canonical radiance field and a time-conditioned deformation field, D-NeRF effectively shares information across time steps to overcome the single-view-per-timestep constraint. Experiments on synthetic dynamic scenes demonstrate significant improvements over baselines in both novel view synthesis quality and temporal consistency. The work opens new directions for neural scene representations that handle the full complexity of the dynamic visual world.