We present LGM (Large Multi-View Gaussian Model), a novel framework designed to generate high-resolution 3D models from text prompts or single-view images within 5 seconds. Our key insights are two-fold: (1) we propose multi-view Gaussian features as an efficient yet powerful 3D representation, which can be fused together for differentiable rendering; and (2) we present an asymmetric U-Net as a high-throughput backbone operating on multi-view images, which can be produced from text or single-view image input by leveraging multi-view diffusion models. Our approach maintains fast speed while boosting the training resolution to 512, thereby achieving high-resolution 3D content generation.

The core technical challenge addressed by LGM is the resolution bottleneck in feed-forward 3D generation. Existing approaches are constrained to low resolutions (typically 128x128) due to the cubic memory scaling of volumetric representations like NeRF-based methods. By leveraging 3D Gaussian Splatting as the output representation, LGM sidesteps this limitation entirely — Gaussians scale linearly with the number of points rather than cubically with resolution, enabling practical high-resolution training and inference.

The rapidly growing demand for 3D content across gaming, virtual reality, e-commerce, and film production has created an urgent need for automated 3D generation tools. Traditional 3D modeling using software like Blender or Maya requires extensive training and hours of manual work per object. The advent of 2D diffusion models has transformed image creation from a specialized skill to a prompt-based workflow, and there is strong motivation to achieve the same transformation for 3D content.

3D content creation has traditionally been a time-consuming and expertise-intensive process, requiring skilled artists and specialized software. Recent advances in 3D generation from 2D diffusion priors — exemplified by methods like DreamFusion, Magic3D, and Zero-1-to-3 — have demonstrated the possibility of automated 3D creation. However, these methods typically require per-scene optimization that takes minutes to hours, limiting their practicality. An alternative line of work uses feed-forward 3D generation, predicting 3D content in a single forward pass, but existing methods are limited to low-resolution outputs (typically 128x128), producing blurry and detail-lacking 3D models. LGM bridges this gap by enabling high-resolution, feed-forward 3D generation in seconds.

The practical impact of LGM's speed advantage extends beyond raw generation time. In a typical creative workflow, artists iterate through dozens of designs before finding the right one. With optimization-based methods requiring 25-40 minutes per attempt, a session of 20 iterations would take 8-13 hours. With LGM at 5 seconds per generation, the same 20 iterations take under 2 minutes — transforming 3D generation from an overnight batch process to an interactive creative tool. This speed also enables new applications such as real-time 3D asset generation for gaming, rapid prototyping for product design, and interactive 3D content creation for e-commerce, none of which were feasible with previous methods.

Score Distillation Sampling (SDS), introduced by DreamFusion, enables 3D generation by optimizing a NeRF representation to match the distribution of a pretrained 2D diffusion model. While producing impressive results, SDS-based methods require per-object optimization lasting 15-60 minutes and suffer from issues like the Janus problem (multi-face artifacts) and over-saturation. Feed-forward approaches like One-2-3-45, LRM, and InstantMesh predict 3D representations directly but are limited to triplane-NeRF outputs at 128x128 resolution. 3D Gaussian Splatting (3DGS) has emerged as a revolutionary 3D representation offering real-time rendering, explicit geometry, and efficient memory usage, making it an ideal candidate for feed-forward 3D generation.

The choice of 3D Gaussian Splatting as the output representation is motivated by several advantages over alternatives. Compared to NeRF-based volumetric representations, Gaussians offer explicit geometry (each Gaussian has a definite position and extent), real-time rendering through splatting rather than ray marching, and memory efficiency scaling linearly with the number of primitives. Compared to mesh-based representations, Gaussians avoid the need for differentiable mesh extraction steps that are often non-differentiable or require complex workarounds. The key insight enabling LGM is that Gaussian attributes can be predicted per-pixel from multi-view images and directly unprojected into 3D space, creating a simple and fully differentiable pipeline from image features to 3D representation.

LGM's pipeline consists of two stages. First, given a text prompt or single image, we use a multi-view diffusion model to generate four orthogonal views of the target object. Second, these multi-view images are fed into our asymmetric U-Net backbone, which predicts multi-view Gaussian features — per-pixel Gaussian attributes including position offset, opacity, color (spherical harmonics), and covariance. These Gaussian features from all views are then unprojected into 3D space and fused to form a complete 3D Gaussian Splatting representation. The asymmetric U-Net design uses a heavy encoder for rich feature extraction and a lightweight decoder for fast Gaussian prediction, balancing quality and speed.

The multi-view Gaussian feature representation is central to LGM's efficiency. For each pixel in each view, the network predicts 14 Gaussian attributes: 3 for position offset (relative to the unprojected ray), 1 for opacity, 3 for RGB color (or higher-order spherical harmonics), and 7 for the covariance matrix (parameterized as a rotation quaternion and scale vector). This per-pixel formulation means that the number of output Gaussians is directly determined by the input resolution — at 512x512 with 4 views, the output contains approximately 1 million Gaussians, providing sufficient density for detailed 3D reconstruction. The fusion step simply concatenates all Gaussians from all views, with the position offsets ensuring that Gaussians from different views naturally align in 3D space.

The training procedure uses differentiable Gaussian splatting rendering to supervise the predicted Gaussians. Given the fused Gaussian set, we render images from random viewpoints (distinct from the four input views) and compare them with ground-truth renderings using a combination of MSE loss, perceptual LPIPS loss, and a mask loss for object silhouettes. The asymmetric U-Net architecture uses a pretrained image encoder (from Stable Diffusion's VAE encoder) as the heavy encoder branch, providing rich semantic features, while the lightweight decoder consists of transposed convolutions for fast upsampling and Gaussian attribute prediction. Training is performed on the Objaverse dataset with approximately 80K high-quality 3D objects.

A critical component of the pipeline is the multi-view diffusion model that generates the four input views. We leverage MVDream, a multi-view diffusion model fine-tuned from Stable Diffusion, which generates four orthogonal views (front, right, back, left) at 256x256 resolution conditioned on text or a single input image. The choice of orthogonal views provides maximum coverage of the object surface with minimum view overlap. For image-to-3D generation, we additionally employ an image-conditioned variant that takes a reference view and generates the remaining three views, ensuring consistency with the input image. The quality and consistency of these multi-view images directly determines the upper bound of LGM's output quality.

We evaluate LGM on both text-to-3D and image-to-3D generation tasks. For text-to-3D, LGM generates 3D objects in approximately 5 seconds on a single A100 GPU, compared to 25 minutes for DreamFusion and 40 minutes for Magic3D. Despite this dramatic speedup, user studies show that LGM is preferred over optimization-based methods in 67% of comparisons for overall quality. For image-to-3D, LGM achieves PSNR of 24.8 and SSIM of 0.87 on the GSO benchmark, outperforming existing feed-forward methods. The training at 512 resolution enables preservation of fine details such as text on objects, thin structures, and surface textures that are lost at lower resolutions.

Ablation studies validate key design decisions. Comparing asymmetric U-Net vs. symmetric U-Net reveals that the asymmetric design achieves comparable quality (PSNR difference within 0.2 dB) while being 2.3x faster in inference. Increasing resolution from 256 to 512 improves PSNR by 1.8 dB with only 1.6x increase in inference time, demonstrating favorable quality-speed scaling. The multi-view diffusion model quality is shown to be the primary bottleneck: replacing the multi-view diffusion with ground-truth views improves PSNR by 4.2 dB, suggesting that LGM's reconstruction capability exceeds the quality of its input views. The choice of 4 orthogonal views balances coverage and generation speed — 8 views improve PSNR by only 0.3 dB while doubling inference time.

Qualitative analysis reveals LGM's distinctive strengths and limitations. The model excels at generating clean, well-defined objects with smooth surfaces such as furniture, vehicles, and animals. Fine details like text labels, thin appendages (antennae, whiskers), and intricate surface patterns are well-preserved at 512 resolution — a significant improvement over 256-resolution baselines. However, LGM shows limitations with highly concave objects (e.g., bowls, cups) where the four orthogonal views fail to capture interior surfaces, and with objects containing thin holes or lattice structures where the Gaussian representation struggles to create sharp boundaries. These failure modes are largely attributable to the limited view coverage (4 views) rather than the reconstruction backbone, as confirmed by the oracle experiment with ground-truth views.

We have presented LGM, a framework that achieves high-resolution, feed-forward 3D content generation in seconds. Through multi-view Gaussian features and an asymmetric U-Net backbone, LGM bridges the gap between optimization-based quality and feed-forward speed. Our work demonstrates that 3D Gaussian Splatting, combined with multi-view diffusion priors, provides a powerful and efficient paradigm for 3D generation.

Future directions include improving the multi-view diffusion model to generate more consistent and detailed views, which our ablation shows is the primary quality bottleneck. Exploring adaptive Gaussian density — allocating more Gaussians to complex regions — could further improve detail while reducing total Gaussian count. Extending LGM to scene-level generation beyond single objects and incorporating mesh extraction from Gaussians for downstream applications in gaming and simulation represent natural next steps for this research direction.

The broader significance of LGM lies in its demonstration that 3D content creation can be democratized through feed-forward generation. By reducing the barrier from minutes of GPU optimization to seconds of inference, LGM enables users without 3D modeling expertise to create 3D assets from simple text descriptions or single photographs. This has implications for e-commerce (product visualization from a single photo), education (interactive 3D learning materials), gaming (rapid asset prototyping), and augmented reality (instant 3D content for AR experiences). The combination of multi-view diffusion priors with Gaussian Splatting reconstruction establishes a modular pipeline where each component can be independently improved, suggesting a scalable path toward higher quality 3D generation.