We introduce AnyDoor, a diffusion-based image generator that teleports target objects to new scenes at user-specified locations in a harmonious way. Unlike existing methods that require parameter tuning for each new object, our model trains once and generalizes to diverse object-scene combinations during inference. We design the framework by combining identity features extracted with a self-supervised backbone and detail features obtained through high-frequency maps, which together preserve object texture while allowing local variations in lighting and orientation. We further leverage video datasets to observe multiple forms of the same objects, enhancing generalizability and robustness. Applications include virtual try-on, object swapping, and multi-object composition.

While diffusion models have enabled remarkable image generation capabilities through text, scribbles, and other conditions, the specific task of object teleportation — accurately placing target objects into desired scene locations — remains underexplored despite its practical importance for image composition, rendering, poster design, and virtual try-on. Previous methods present significant limitations: Paint-by-Example and ObjectStitch cannot generate identity-consistent content for untrained categories; customization methods like DreamBooth require extensive fine-tuning (approximately one hour with multiple images) and cannot specify object locations within given scenes.

AnyDoor addresses these constraints through zero-shot customization by representing objects using both identity and detail-related features, then compositing them within background scenes. The approach integrates ID tokens extracted by DINO-V2 and high-frequency detail maps into a pre-trained Stable Diffusion model. Training incorporates both video and image datasets, with an adaptive timestep sampler to leverage different data sources effectively. The framework supports single and multiple object placement, object moving, and object swapping — all without per-object parameter tuning.

For identity feature extraction, the method employs DINO-V2 as the ID extractor backbone rather than CLIP encoders used in prior work. Background is first removed via segmentation models, then features are extracted as a concatenation of global tokens (1x1536 dimensions) and patch tokens (256x1536 dimensions) through a linear projector, producing ID tokens (257x1024 dimensions) aligned with the text-to-image UNet embedding space. The choice of DINO-V2 over CLIP is motivated by its ability to encode more discriminative patch-level information that captures fine-grained object identity beyond semantic-level similarity.

To address the spatial resolution limitations of ID tokens, the authors introduce complementary detail features. A collage representation stitches background-removed objects to target scene locations, but direct stitching lacks diversity. Instead, high-frequency maps are extracted using Sobel kernels — horizontal and vertical filters combined with the original image through element-wise operations, with an eroded mask removing outer contour information. The resulting high-frequency map feeds into a ControlNet-style UNet encoder producing hierarchical-resolution detail maps. This design ensures ID tokens capture overall structure while high-frequency maps preserve fine details like logos and textures.

For feature injection, ID tokens replace text embeddings in cross-attention mechanisms within the UNet, while detail maps concatenate with UNet decoder features at each resolution. The UNet encoder remains frozen to preserve learned priors, while the decoder adapts through mean squared error loss. The training strategy leverages video datasets (YouTubeVOS, YouTubeVIS, UVO, MOSE, VIPSeg, BURST) where the same object appears across frames, along with multi-view datasets (MVImgNet, VitonHD, FashionTryon) and single-image datasets — totaling 270,821 training samples. An adaptive timestep sampler assigns video data to early denoising steps (coarse structure) and image data to late steps (fine details).

The base generator is Stable Diffusion V2.1, processing images at 512x512 resolution. Evaluation uses a constructed benchmark with 30 new concepts from DreamBooth and 80 manually-selected scenes from COCO-Val, generating 2,400 combinations. Metrics include CLIP-Score and DINO-Score measuring similarity between generated and target object features. A user study with 15 annotators rates fidelity, quality, and diversity on 1-4 scales. Compared against reference-based methods (Paint-by-Example, Graphit, Stable Diffusion inpainting) and tuning-based methods (DreamBooth, Custom Diffusion, Cones).

User study results show AnyDoor achieves quality score 3.04, fidelity 3.06, and diversity 2.88, outperforming Paint-by-Example (2.71, 2.10, 3.04) and Graphit (2.65, 2.11, 2.84) on quality and fidelity while being competitive on diversity. Ablation studies reveal the critical role of each component: replacing DINO-V2 with CLIP causes results to lose identity features, retaining only semantic consistency. The full model achieves CLIP Score 82.1 and DINO Score 67.8, versus the CLIP-encoder baseline at 73.8 and 31.5. Background removal before DINO-V2 processing further improves DINO Score from 64.1 to 67.8, confirming that disentangling foreground from background is essential for identity preservation.

For virtual try-on applications, AnyDoor preserves the color, texture, and patterns of target clothes and performs well for large human gestures. Unlike traditional GAN-based methods that require human parsing maps as additional input, AnyDoor needs only a bounding box indicating the upper-body position. The framework also supports flexible interactions through integration with inpainting and interactive segmentation models: users can click and drag objects, swap locations between objects, or adjust object shapes via bounding box manipulation. The pipeline uses inpainting to fill original positions and AnyDoor to regenerate objects at new locations.

AnyDoor provides a diffusion-based solution for object teleportation through a discriminative ID extractor and frequency-aware detail extractor that together characterize target objects comprehensively. By training on combined video and image data with an adaptive timestep strategy, the model achieves high-fidelity compositions at user-specified scene locations without parameter tuning. The approach offers a universal solution for general region-to-region mapping tasks, with demonstrated applications spanning image composition, virtual try-on, and interactive object manipulation. The key insight is that combining self-supervised identity features with frequency-domain detail features bridges the gap between semantic understanding and fine-grained texture preservation.