We introduce Scalable Interpolant Transformers (SiT), a family of generative models built on the backbone of Diffusion Transformers (DiT). The interpolant framework allows for connecting two distributions in a more flexible way than standard diffusion models, making possible a modular study of various design choices impacting generative models built on dynamical transport: learning in discrete or continuous time, the objective function, the interpolant that connects the distributions, and deterministic or stochastic sampling. SiT surpasses DiT uniformly across model sizes on the conditional ImageNet 256x256 and 512x512 benchmark, achieving an FID-50K score of 2.06 and 2.62 respectively.

A distinctive feature of our work is its emphasis on understanding rather than solely pursuing state-of-the-art numbers. By building on the exact same DiT architecture, we isolate the contributions of the interpolant framework from architectural effects. Our systematic ablation reveals actionable guidelines: velocity prediction outperforms score prediction, linear interpolation outperforms trigonometric schedules, and tunable stochastic sampling outperforms both pure ODE and pure SDE sampling. These findings provide a practical recipe for improving any DiT-based model with zero additional computational cost.

The recent surge in generative AI applications has been driven primarily by diffusion models, yet the design space of these models remains insufficiently explored. Most practitioners adopt the standard DDPM formulation or its latent diffusion variant without questioning whether the default noise schedule, prediction target, or sampling strategy are optimal. This is partly because changing one aspect typically requires changes to others, making controlled experimentation difficult. SiT's interpolant framework resolves this issue by providing a principled decomposition of the design space into independently variable dimensions.

Diffusion models and flow-based models are two prominent frameworks for generative modeling. Standard diffusion models define a fixed forward process that progressively adds Gaussian noise, and learn a reverse denoising process. Flow-based models instead learn a deterministic mapping between a simple base distribution and the data distribution. The stochastic interpolant framework provides a unified perspective that encompasses both diffusion and flow-based models as special cases. By defining a general interpolation between noise and data, we can independently vary the interpolation schedule, the objective function, and the sampling strategy, enabling a systematic exploration of the design space.

Despite the success of DiT in establishing Transformers as the backbone for diffusion models, the design choices beyond architecture remain underexplored. DiT inherits the standard DDPM formulation with fixed noise schedules and epsilon prediction, but there is no principled reason why these specific choices should be optimal. The interpolant framework enables us to ask: what is the best way to connect the noise and data distributions? Should we predict the velocity, the score, or the noise? Should sampling be deterministic or stochastic? By decoupling these choices from the architecture, we can provide clear, controlled answers to each question independently.

The practical significance of SiT lies in its demonstration that substantial improvements are achievable through better training and sampling recipes alone, without any architectural changes. This is particularly valuable because architectural improvements typically require retraining from scratch, often with modified codebases and different hyperparameter searches. In contrast, SiT's improvements can be applied to any existing DiT training pipeline by simply changing the loss function and sampling procedure. For the rapidly growing community of practitioners training custom diffusion models, this represents an immediate, zero-cost upgrade path — change a few lines of training code and sampling configuration, and receive a 9.3% FID improvement.

The theoretical foundation for SiT draws from stochastic interpolants and flow matching. Stochastic interpolants, introduced by Albergo and Vanden-Eijnden, provide a general framework for learning transport maps between arbitrary distributions by defining interpolation paths parameterized by time-dependent coefficients. Flow matching, developed concurrently by Lipman et al., offers a simulation-free training objective for continuous normalizing flows. Both frameworks generalize the DDPM formulation while providing greater flexibility in choosing noise schedules and objectives. Our contribution is to bring these theoretical advances into the practical setting of large-scale class-conditional image generation with Transformer architectures.

A key distinction between diffusion models and flow matching lies in the presence or absence of stochasticity in the forward process. Standard diffusion (DDPM) defines a stochastic forward process that adds Gaussian noise according to a fixed schedule, while flow matching defines a deterministic linear interpolation between data and noise. The interpolant framework subsumes both by parameterizing the interpolation with general time-dependent coefficients: x_t = alpha(t)*x_0 + beta(t)*epsilon. Setting alpha(t) = sqrt(1 - sigma(t)^2) and beta(t) = sigma(t) recovers DDPM, while alpha(t) = 1-t and beta(t) = t recovers linear flow matching. This unified parameterization enables fair comparison of design choices that were previously confounded with framework-specific assumptions.

SiT is built on the exact same architecture as DiT — a Transformer operating on latent patches with adaptive layer normalization conditioning. The only changes are in the training objective and sampling procedure, which are governed by the interpolant framework. We systematically explore: (1) linear vs. trigonometric interpolants; (2) velocity prediction vs. score prediction objectives; (3) continuous-time vs. discrete-time training; and (4) deterministic (ODE) vs. stochastic (SDE) sampling with varying diffusion coefficients. Our key finding is that by decoupling the diffusion coefficient from the learning process, we can tune it at inference time to find the optimal balance between sample quality and diversity.

The interpolant defines how noise and data are mixed at intermediate times. A linear interpolant creates x_t = (1-t)x_0 + t*epsilon, while a trigonometric interpolant uses x_t = cos(pi*t/2)*x_0 + sin(pi*t/2)*epsilon. These choices affect the signal-to-noise ratio schedule and consequently the difficulty of prediction at each timestep. The velocity prediction objective trains the network to predict dx_t/dt (the time derivative of the interpolant), while score prediction targets the score function of the noisy distribution. Velocity prediction has a natural connection to flow matching and produces more uniform loss magnitudes across timesteps, potentially leading to more balanced training.

The most consequential finding concerns sampling with tunable stochasticity. Standard diffusion models use a fixed SDE for sampling, coupling the diffusion coefficient to the learned score. SiT decouples these by learning a velocity field that can be used with any diffusion coefficient at inference time. Setting the coefficient to zero yields deterministic ODE sampling (similar to DDIM); increasing it adds stochasticity that can improve sample quality by correcting accumulated errors along the trajectory. The optimal coefficient is typically between 0.5 and 1.5, and can be tuned on a small validation set. This single hyperparameter provides a principled way to trade off sample quality against diversity without retraining the model.

On conditional ImageNet 256x256, SiT-XL/2 achieves an FID-50K of 2.06, compared to DiT-XL/2's FID of 2.27 — a 9.3% improvement using the exact same model architecture, parameters, and GFLOPs. On ImageNet 512x512, SiT achieves FID 2.62 vs DiT's 3.04. Critically, SiT's advantages are consistent across all model sizes (S, B, L, XL), confirming that the improvements stem from the interpolant framework rather than scale-dependent factors. Our ablation studies reveal that velocity prediction with linear interpolation outperforms score prediction with standard diffusion noise schedules, and that SDE sampling with tuned diffusion coefficients outperforms deterministic ODE sampling.

A detailed breakdown of the design space reveals clear winners in each dimension. For interpolation: linear interpolation achieves FID 2.06 vs. trigonometric's 2.18, a gap that is consistent across model sizes. For objectives: velocity prediction achieves FID 2.06 vs. score prediction's 2.31, confirming that the more uniform loss landscape of velocity prediction leads to better optimization. For sampling: SDE with diffusion coefficient 1.0 achieves FID 2.06 vs. ODE's 2.25, demonstrating that controlled stochasticity helps correct trajectory errors. The continuous-time training and discrete-time training perform comparably (FID 2.06 vs. 2.09), suggesting that this choice is less critical than the others.

We further analyze the relationship between the tunable diffusion coefficient and generation quality. Sweeping the coefficient from 0 (pure ODE) to 2.0 (heavy SDE) reveals a clear optimal region between 0.8 and 1.2 where FID is minimized. Below 0.8, the deterministic trajectory accumulates discretization errors that manifest as reduced diversity. Above 1.2, excessive stochasticity introduces noise that degrades sharpness. Interestingly, the optimal coefficient is relatively stable across model sizes (varying by less than 0.1 between SiT-S and SiT-XL), suggesting that this finding transfers across scales. We also observe that the optimal coefficient for FID differs from the optimal for Inception Score (which favors slightly higher stochasticity at 1.3-1.5), providing users with a principled knob for controlling the quality-diversity tradeoff.

We have introduced SiT, demonstrating that the interpolant framework provides a principled and effective approach to improving diffusion-based generative models. By enabling independent control over interpolation schedules, objectives, and sampling strategies, SiT achieves consistent improvements over DiT across all model scales with zero additional computational cost. Our systematic study provides practical guidelines for the design of future generative models based on dynamical transport.

The implications of SiT extend beyond the specific ImageNet benchmarks. The finding that velocity prediction with linear interpolation and tunable SDE sampling constitutes a superior training-sampling recipe is directly applicable to any model using the DiT architecture, including text-to-image models, video generation models, and audio synthesis systems. Future work should explore whether these findings transfer to latent diffusion settings with different VAE architectures and to conditional generation tasks beyond class conditioning. The interpolant framework also opens the door to learning the optimal interpolation schedule jointly with the model, potentially discovering schedules that outperform hand-designed options.

The methodological contribution of SiT extends beyond its specific numerical results. It establishes a template for systematic empirical investigation in generative modeling: fix the architecture, vary one design dimension at a time, and measure the impact. This controlled experimental methodology is surprisingly rare in the generative modeling literature, where new methods typically change multiple aspects simultaneously, making it difficult to attribute improvements to specific design choices. SiT's approach of building on the exact same codebase and training setup as DiT ensures that every observed difference is attributable to the interpolant framework. This level of experimental rigor provides the community with trustworthy, actionable guidance rather than results that may not reproduce under different conditions.