This paper investigates the Mixture of Experts (MoE) architecture in real-time open-vocabulary object detection. The authors observe that in the shallow layers, experts tend to cooperate with diverse peers to expand the search space, while deeper layers develop stable expert collaboration, with specialized combinations for specific token patterns. Based on these findings, they propose Dynamic-DINO, which converts Grounding DINO 1.5 Edge into a dynamic inference framework through efficient MoE-Tuning. A granularity decomposition mechanism splits the Feed-Forward Network into finer expert networks, and a pre-trained weight allocation strategy ensures stable initialization. Trained on only 1.56M open-source data, Dynamic-DINO surpasses the baseline pretrained on a private 20M dataset with comparable inference speed.

Open-vocabulary object detection enables flexible localization of arbitrary objects by integrating language modality, advancing beyond fixed-category detection systems. Real-time detectors have gained emphasis due to practical applications in anomaly detection, robotics, and autonomous driving. Current approaches employ dense models with fixed inference architectures, where a single FFN in each layer is required to process all tokens, encompassing extensive patterns in open scenarios, including visual patterns such as category and attribute, and contextual patterns like relative position and relationship. This causes gradient conflicts and long-tail issues.

The search space for MoE with N experts and top-k activation across L layers is (C_N^k)^L. Two straightforward expansion strategies exist: increasing k raises computational costs; increasing N increases memory and training overhead. The authors propose Dynamic-DINO, implementing efficient MoE-Tuning based on Grounding DINO 1.5 Edge. They employ a granularity decomposition strategy, which splits a single FFN into multiple expert networks. Crucially, activated parameters remain equivalent to a single FFN, maintaining inference efficiency.

Representative works in open-vocabulary object detection include GLIP, OpenSeeD, OWL-ViT, Grounding DINO, and DetCLIP variants. Real-time detectors like YOLO-World inherit efficiency from the YOLO series. Grounding DINO 1.5 Edge focuses on computational efficiency, with subsequent versions expanding pre-training datasets. Dynamic-DINO uniquely incorporates MoE-driven dynamic inference to achieve significant improvements in accuracy without compromising efficiency.

Mixture of Experts (MoE) enables conditional computation, scaling model capacity while ensuring efficient computation through selective expert activation. Early works adopted hard routing; recent LLM and LVLM work employs soft routers enabling dynamic token allocation. DeepSeekMoE and QwenMoE segment experts by splitting FFN intermediate dimensions. Dynamic-DINO differs by segmenting pre-trained FFN parameters for incremental fine-tuning rather than random initialization for full pre-training.

Dynamic-DINO extends Grounding DINO 1.5 Edge into a dynamic framework via MoE-Tuning. MoE application targets the decoder because: (1) Language-guided Query Selection retains only 900 tokens, minimizing router computational costs; (2) decoder output directly influences bounding box regression, improving fine-tuning efficiency. Training allows Cross-Attention in Feature Enhancer, MoE Layer in Cross-Modality MoE Decoder, and Detection Head to participate while freezing other parameters.

The FFN in each decoder layer expands into N identical FFNs via Supernet Expansion. Each FFN's intermediate hidden dimension divides evenly into k partitions, constructing k x N fine-grained experts. This creates a larger subnet search space through finer granularity. During Subnet Inference, a router R(x) selects experts using softmax normalization. Top-k experts activate via gating mechanism, ensuring activated parameters equal a single FFN: h(x) = sum of g_i multiplied by E_i(x), where g_i indicates whether expert i is selected.

Expert Initialization segments parameters from the pre-trained FFN. The first layer weight W1 divides horizontally into k blocks, and the second layer W2 divides vertically. Bias is adjusted as b2* = b2/k. This ensures "the sum of the outputs from the k fine-grained experts matches the output of the original FFN". For Router Initialization, random weights replicate each centroid k times, ensuring the router invariably selects the k experts derived from the same FFN at the start of fine-tuning. The total loss combines detection loss L_det (L1, GIOU, and Focal losses) with an auxiliary load balancing loss L_aux to prevent expert collapse.

Experiments use Objects365 (V1), GoldG, and V3Det datasets totaling approximately 1.56M images, excluding COCO images from GoldG. Zero-shot evaluation is conducted on COCO, LVIS, and ODinW using standard Average Precision metrics. The model uses EfficientViT-L1 image backbone and BERT-base text backbone with 900 queries and 6 decoder layers. Dynamic-DINO achieves comparable performance with official Grounding DINO 1.5 Edge across resolutions. Notably, performance on rare classes significantly improves, indicating MoE-Tuning effectively alleviates the long-tail problem.

Statistical analysis of routing distributions reveals that expert loading varies notably across layers, showing experts learned task-specific division mechanisms. In shallow layers, experts cooperate with a diverse range of peers to explore a wider search space. Deeper layers focus on consistent collaborations with 2-3 specific partners to process distinct patterns. Token routing examples confirm that distinct expert combinations are specialized in processing specific patterns: refrigerator tokens select experts 0,3 while clothing tokens select 1,7.

Dynamic-DINO explores MoE integration in real-time open-vocabulary detection. The framework demonstrates that diverse expert combinations can adaptively process specific patterns, activating only relevant experts during inference. Building on a reproduced Grounding DINO 1.5 Edge, Dynamic-DINO extends it into a dynamic framework via MoE-Tuning with granularity decomposition, expanding the subnet search space while maintaining activated parameters. A novel weight allocation and router initialization strategy prevents performance degradation at the start of fine-tuning. Extensive validation on multiple benchmarks confirms the effectiveness of the approach.