We present VGGT, a feed-forward neural network that directly infers all key 3D attributes of a scene, including camera parameters, point maps, depth maps, and 3D point tracks, from one, a few, or hundreds of its views. This approach is a step forward in 3D computer vision, where models have typically been constrained to and specialized for single tasks. It is also simple and efficient, reconstructing images in under one second, and still outperforming alternatives that require post-processing with visual geometry optimization techniques. The network achieves state-of-the-art results in multiple 3D tasks, including camera parameter estimation, multi-view depth estimation, dense point cloud reconstruction, and 3D point tracking.

We consider the problem of estimating the 3D attributes of a scene, captured in a set of images, utilizing a feed-forward neural network. Traditionally, 3D reconstruction has been approached with visual-geometry methods, utilizing iterative optimization techniques like Bundle Adjustment. Machine learning has often played an important complementary role, addressing tasks that cannot be solved by geometry alone, such as feature matching and monocular depth prediction. The integration has become increasingly tight, and now state-of-the-art Structure-from-Motion (SfM) methods like VGGSfM combine machine learning and visual geometry end-to-end.

As networks become ever more powerful, we ask if, finally, 3D tasks can be solved directly by a neural network, eschewing geometry post-processing almost entirely. Recent contributions like DUSt3R and its evolution MASt3R have shown promising results. However, these methods can only process two images at a time, and still require post-processing with visual geometry optimization techniques to handle multiple views. In this paper, we take a further step towards removing the need to optimize 3D geometry in post-processing. We do so by introducing Visual Geometry Grounded Transformer (VGGT), a feed-forward neural network that performs 3D reconstruction from one, a few, or even hundreds of input views of a scene in a single forward pass, often outperforming optimization-based alternatives without further processing.

VGGT is based on a fairly standard large transformer, with no particular 3D or other inductive biases, but trained on a large number of publicly available datasets with 3D annotations. Our key contributions include: (1) A large feed-forward transformer predicting camera parameters, point maps, depth maps, and 3D tracks in seconds; (2) Predictions that are directly usable and competitive with state-of-the-art methods using slow post-processing; (3) State-of-the-art results when combined with Bundle Adjustment. The model has approximately 1.2 billion parameters and operates across varying numbers of input views — from a single image to hundreds — within a unified framework.

Structure from Motion (SfM) involves estimating camera parameters and reconstructing sparse point clouds from collections of images. The traditional pipeline includes multiple stages, including image matching, triangulation, and bundle adjustment. These methods, exemplified by COLMAP, are highly effective but computationally expensive and fragile in challenging conditions. Recent methods explore end-to-end differentiable SfM, with VGGSfM combining machine learning and visual geometry in a unified differentiable framework, achieving improved robustness while maintaining geometric accuracy.

Multi-view Stereo (MVS) methods reconstruct scene geometry densely from overlapping images, typically with known cameras. Recent learning-based approaches like DUSt3R and MASt3R directly estimate aligned dense point clouds from image pairs without requiring camera parameters. However, they are restricted to two-view inputs and rely on global alignment optimization to fuse multi-view predictions. Tracking Any Point (TAP) methods track arbitrary surface points across video sequences, including through occlusions. Recent methods like CoTracker utilize correlations between points to handle occlusions. VGGT demonstrates that its features yield state-of-the-art tracking performance when coupled with existing point trackers.

The transformer maps a sequence of N RGB images to 3D annotations: f((I_i)^N_i=1) = (g_i, D_i, P_i, T_i)^N_i=1. Camera parameters use the parametrization g = [q, t, f] representing rotation quaternion, translation vector, and field of view, with the principal point assumed at image center. Depth maps D_i associate each pixel with depth values. Point maps P_i associate pixels with 3D scene points, defined in the coordinate system of the first camera as the world reference frame. The model is permutation equivariant except for the first frame serving as reference.

Following recent works in 3D deep learning, we design a simple architecture with minimal 3D inductive biases, letting the model learn from ample quantities of 3D-annotated data. We implement the model as a large transformer. To this end, each input image is initially patchified into a set of K tokens through DINO. We slightly adjust the standard transformer design by introducing Alternating-Attention (AA), making the transformer focus within each frame and globally in an alternate fashion. Frame-wise self-attention processes tokens within each frame separately, while global self-attention processes tokens across all frames. The model uses L=24 layers of global and frame-wise attention with no cross-attention layers.

The Alternating-Attention design addresses a fundamental challenge: when processing N images with K tokens each, standard global self-attention has O(N²K²) complexity, which becomes prohibitive for large numbers of views. By alternating between frame-wise attention (O(NK²)) and global attention across corresponding tokens (O(N²K)), the model achieves effective multi-view reasoning while maintaining tractable computational cost. This is crucial for scaling to hundreds of input views — a regime where prior methods like DUSt3R would require O(N²) pairwise forward passes followed by expensive global optimization.

Image tokens are augmented with camera tokens and register tokens. The camera head predicts intrinsics and extrinsics from camera tokens using four self-attention layers and a linear layer. Dense predictions — depth, point maps, and tracking features — use image tokens converted to dense feature maps via a DPT layer, then mapped to outputs using 3x3 convolutions. Aleatoric uncertainty is predicted for both depth and point maps, providing a measure of prediction confidence at each pixel. For tracking, the CoTracker2 architecture processes dense tracking features. Given a query point in a query image, the tracking head predicts the set of 2D points in all images that correspond to the same 3D point.

The multi-task loss combines: L = L_camera + L_depth + L_pmap + λL_track. Camera loss uses Huber loss comparing predicted and ground-truth cameras. Depth loss implements aleatoric-uncertainty loss with gradient-based terms. Point map loss follows analogously. Training employs 160K iterations on 64 A100 GPUs over nine days with diverse datasets: Co3Dv2, BlendMVS, DL3DV, MegaDepth, Kubric, WildRGB, ScanNet, HyperSim, Mapillary, Habitat, Replica, MVS-Synth, PointOdyssey, Virtual KITTI, Aria Synthetic Environments, Aria Digital Twin, and a synthetic dataset of artist-created assets. Images are resized to 518 pixels maximum dimension. Aggressive color augmentation is applied independently across frames.

Ground truth tracks are built by unprojecting depth maps to 3D and reprojecting to target frames, providing dense correspondence supervision without requiring explicit track annotations. The peak learning rate is 0.0002 with 8K warmup iterations. The total model has approximately 1.2 billion parameters. This large-scale training strategy is central to VGGT's philosophy: rather than encoding geometric priors in the architecture, the model learns geometry directly from diverse, large-scale 3D data. The approach demonstrates that sufficient data diversity and scale can substitute for hand-crafted inductive biases in 3D vision.

Camera Pose Estimation. On CO3Dv2 and RealEstate10K datasets, VGGT achieves 85.3 AUC@30 on unseen RealEstate10K and 88.2 on CO3Dv2 in feed-forward mode, outperforming competing methods in approximately 0.2 seconds. When combined with Bundle Adjustment, performance reaches 93.5 and 91.8 AUC@30 respectively in approximately 1.8 seconds. These results significantly surpass prior methods including DUSt3R, MASt3R, and VGGSfM, which require substantially longer processing times and iterative geometric optimization.

Multi-view Depth Estimation. On the DTU dataset, VGGT substantially outperforms DUSt3R, reducing the Overall score from 1.741 to 0.382. It achieves comparable results to methods with known ground-truth cameras, attributed to its multi-image training scheme that teaches it to reason about multi-view triangulation natively. Point Map Estimation. On the ETH3D dataset, VGGT's feed-forward performance (0.677 Overall score) surpasses DUSt3R and MASt3R despite their expensive optimization. Interestingly, combining predicted depth maps and predicted camera parameters produces more accurate 3D points compared to directly employing a specialized point map branch.

Image Matching. On ScanNet-1500, VGGT achieves the highest accuracy: 33.9 AUC@5, 55.2 AUC@10, and 73.4 AUC@20, despite not being specialized for two-view matching, demonstrating the generalization capability of its features. Dynamic Point Tracking. Fine-tuning CoTracker2 with pretrained VGGT weights significantly enhances TAP-Vid performance, achieving δ_avg^vis of 84.0 versus 78.9 baseline. Feed-forward Novel View Synthesis. Finetuned VGGT achieves competitive performance of 30.41 PSNR on the GSO dataset without requiring known camera parameters, using only 20% of LVSM's training data.

Feature Backbone. Alternating-Attention outperforms both global self-attention only (0.827 Overall) and cross-attention variants (1.061 Overall), validating the design choice. Multi-task Learning. Simultaneous training of camera, depth, and tracking estimation yields 0.709 Overall score, versus 0.790 without tracking and 0.727 without depth, demonstrating that multi-task learning provides synergistic benefits. DINOv2 tokenization provides better performance and training stability than alternatives. Differentiable Bundle Adjustment was explored but made training approximately 4x slower, leading to its exclusion from the final model.

Limitations. The current model does not support fisheye or panoramic images. Additionally, reconstruction performance drops under conditions involving extreme input rotations. The model handles minor non-rigid motions but fails with substantial deformation. However, the approach offers flexibility: addressing these limitations can be straightforwardly achieved by fine-tuning the model on targeted datasets with minimal architectural modifications. Processing varies with input frames: 1 frame (0.04s, 1.88GB) to 200 frames (8.75s, 40.63GB). Camera head adds approximately 5% runtime and 2% memory. Users can process frame-by-frame when GPU-constrained.

VGGT establishes that a feed-forward neural network can directly estimate all key 3D scene properties for hundreds of input views, achieving competitive results across multiple benchmarks without post-processing. This represents a paradigm shift from optimization-dependent approaches toward neural-first 3D reconstruction. The model's simple architecture with minimal inductive biases, trained on diverse large-scale data, demonstrates that scale and data diversity are powerful substitutes for hand-crafted geometric priors. Single-view reconstruction works surprisingly well despite not being explicitly trained for it, further evidencing the depth of geometric understanding learned by the network.