Radiance field methods have recently revolutionized novel-view synthesis of scenes captured with multiple photos or videos. However, achieving high visual quality still requires neural networks that are costly to train and render, while recent faster methods inevitably trade off speed for quality. We introduce three key elements that allow us to achieve state-of-the-art visual quality while maintaining competitive training times and importantly enabling real-time (≥30 fps) novel-view synthesis at 1080p resolution: (1) anisotropic 3D Gaussians as a flexible scene representation, (2) interleaved optimization with adaptive density control of the 3D Gaussians, and (3) a fast, visibility-aware GPU rendering algorithm that supports anisotropic splatting.

Neural Radiance Fields (NeRF) and its variants have enabled stunning novel-view synthesis quality. However, they rely on costly volumetric ray-marching with neural network evaluations at each sample point. Fast NeRF variants like InstantNGP and Plenoxels use spatial data structures to accelerate rendering but cannot achieve real-time rendering at high resolutions. A key insight from point-based rendering is that "point-based alpha-blending and NeRF-style volumetric rendering share essentially the same image formation model". We build on this insight, demonstrating that a "continuous representation is not strictly necessary for fast, high-quality training".

Traditional reconstruction methods used Structure-from-Motion (SfM) and Multi-View Stereo (MVS) for view synthesis through image reprojection and blending. NeRF-based methods employ volumetric ray-marching with neural networks; recent fast variants (InstantNGP, Plenoxels) use spatial data structures but still require hundreds of network evaluations per pixel per frame. Point-based rendering has a long history, from surfels to differentiable point rasterization. Our work establishes that the discrete point-based representation is flexible enough to allow creation, destruction, and displacement of geometric primitives during optimization.

Each 3D Gaussian is defined by a mean mu (position) and full 3D covariance matrix Sigma in world space. Rather than directly optimizing Sigma (which must remain positive semi-definite), we decompose it as Sigma = R S S^T R^T, storing separately a scaling vector s and rotation quaternion q. Each Gaussian additionally stores an opacity alpha and spherical harmonics (SH) coefficients for view-dependent appearance. For rendering, 3D covariance projects to 2D screen-space covariance via: Sigma' = J W Sigma W^T J^T, where J is the Jacobian of the projective transformation and W is the viewing matrix.

The loss function combines L = (1 - lambda) * L1 + lambda * L_D-SSIM with lambda = 0.2. The key innovation is adaptive density control with two complementary strategies. Cloning: for under-reconstructed regions (small Gaussians with large positional gradients), duplicate the Gaussian and offset along the gradient direction. Splitting: for over-reconstructed regions (large Gaussians covering too much space), replace with two smaller Gaussians scaled by factor 1.6. Gaussians with opacity below threshold are periodically pruned, and every 3000 iterations, opacity is reset to near-zero to allow pruning of floaters accumulated during optimization.

Our GPU rasterizer uses a tile-based approach: the screen is divided into 16x16 tiles. Each Gaussian is instantiated for every overlapping tile, assigned a 64-bit key combining depth and tile ID, then globally sorted via GPU radix sort. The forward pass processes per-tile thread blocks that accumulate color and opacity front-to-back until alpha saturation. The backward pass traverses stored sorted lists back-to-front, recovering intermediate opacity values without storing explicit per-pixel lists. This enables "gradients on arbitrary numbers of blended Gaussians" while achieving 134 FPS on Mip-NeRF360 scenes at 1080p.

We evaluate on Mip-NeRF360, Tanks & Temples, Deep Blending, and synthetic Blender scenes. On Mip-NeRF360 (30K iterations): SSIM 0.815 (vs. Mip-NeRF360: 0.792), PSNR 27.21 dB (vs. 27.69 dB), training time ~42 minutes (vs. 48 hours), and rendering at 134 FPS (vs. 0.1 FPS). The method achieves comparable or superior visual quality to Mip-NeRF360 with over 1000x faster rendering and over 60x faster training. Ablation studies confirm that all components are critical — removing anisotropy, densification cloning/splitting, or unlimited gradient backpropagation each significantly degrades quality.

We have presented 3D Gaussian Splatting, demonstrating that a continuous neural representation is not strictly necessary for fast, high-quality radiance field rendering. Our approach of representing scenes with anisotropic 3D Gaussians, optimized with adaptive density control, and rendered via a tile-based GPU rasterizer achieves state-of-the-art quality with real-time rendering speeds. Limitations include artifacts in poorly-observed regions, challenges with view-dependent effects, and higher memory consumption (peak ~20 GB) compared to NeRF-based methods. We believe this work opens a new direction in the tradeoff between representation expressiveness, optimization efficiency, and rendering speed.