This work presents AnyDoor, a diffusion-based image generator with the power to teleport target objects to new scenes at user-specified locations in a harmonious way. The model trains once and generalizes to diverse object-scene combinations without tuning parameters. It uses identity features complemented by detail features that maintain texture while allowing local variations. The approach leverages video datasets to observe object variations, enhancing generalizability. Experiments demonstrate superiority over existing methods with applications in virtual try-on and object moving.

Image generation has advanced significantly through diffusion models. While existing work explores editing posture, style, and content through instructions, AnyDoor specifically addresses "object teleportation" — accurately placing target objects into desired scene locations by re-generating box-marked regions. Paint-by-Example and ObjectStitch take a target image as template to edit scene regions, but could not generate ID-consistent contents, especially for untrained categories. AnyDoor overcomes this by enabling zero-shot, high-quality ID-consistent compositions without fine-tuning.

The core contribution involves representing objects through identity and detail-related features, composite interaction with background scenes, and injecting these features into pre-trained diffusion models. Training leverages both video and image data with an adaptive timestep sampler. The identity features are extracted using DINO-V2 backbone which encodes images as global tokens (1x1536) and patch tokens (256x1536), concatenated and projected into the embedding space of the pre-trained UNet via a single linear layer.

Local image editing methods focused on text-guided local region editing. Blended Diffusion conducts multi-step blending; InpaintAnything combines SAM and Stable Diffusion; Paint-by-Example uses CLIP image encoders; ObjectStitch proposes content adaptors. However, "those methods could only give coarse guidance for generations and often fail to synthesize ID-consistent results." Customized image generation methods like Textual Inversion and DreamBooth fine-tune models for specific objects, achieving high fidelity but lacking scenario/location specification and requiring extensive per-object tuning.

The method employs pre-trained visual encoders for identity extraction. "Before feeding the target image into the ID extractor, we remove the background with a segmentor and align the object to the image center." The DINO-V2 backbone encodes images as global tokens (1x1536) and patch tokens (256x1536), concatenated for preservation. "A single linear layer as a projector aligns these tokens to the embedding space of the pre-trained text-to-image UNet," producing ID tokens of 257x1024 dimensions. Unlike CLIP's semantic-level information, DINO-V2 captures discriminative visual features crucial for identity preservation.

Recognizing that ID tokens lose spatial resolution for fine details, the method adds detail guidance through a high-frequency map. "Using collage as controls could provide strong priors," however collages risk over-constraining results. An information bottleneck addresses this through high-frequency mapping using Sobel kernels (horizontal and vertical), applies Hadamard products for RGB color extraction, and filters contour information with eroded masks. This "maintains the fine details yet allows versatile local variants like gesture, lighting, orientation." A ControlNet-style UNet encoder processes the stitched collage to produce hierarchical resolution detail maps.

"ID tokens replace text embedding and inject into each UNet layer via cross-attention. Detail maps concatenate with UNet decoder features at each resolution." Training freezes pre-trained UNet encoder parameters while tuning the decoder for task adaptation. The training strategy collects image pairs from video datasets (YouTubeVOS, YouTubeVIS, UVO, MOSE, VIPSeg, BURST) and image sources (MVImgNet, VitonHD, FashionTryon, MSRA-10K, DUT, HFlickr, LVIS, SAM subset). An adaptive timestep sampler adjusts denoising focus: "early denoising steps focus on overall structure, pose, and view; later steps cover fine details like texture and colors."

The method uses Stable Diffusion V2.1 with 512x512 image processing. Evaluation uses 30 new concepts from DreamBooth with 80 COCO-Val scene images, generating 2,400 combinations. Metrics include CLIP-Score, DINO-Score, and user studies with 15 annotators. Compared against reference-based methods (Paint-by-Example, Graphit, SD inpainting), AnyDoor maintains "highly-faithful details" where competitors maintain only semantic consistency. Against tuning-based methods (DreamBooth, Custom Diffusion), AnyDoor achieves "high-fidelity results for multi-subject composition without parameter tuning" while tuning-based methods require approximately one hour of fine-tuning per object. User study scores: AnyDoor — quality 3.04, fidelity 3.06, diversity 2.88, outperforming Paint-by-Example (2.71, 2.10, 3.04) and Graphit (2.65, 2.11, 2.84).

AnyDoor provides "a universal solution for general region-to-region mapping tasks" through discriminative ID extraction and frequency-aware detail extraction. Training on combined video and image data enables zero-shot object teleportation with applications across image editing, composition, and synthesis tasks. The approach demonstrates that a single trained model can handle diverse object-scene combinations without per-object fine-tuning, opening possibilities for virtual try-on, object moving, and flexible scene composition.