We introduce the Segment Anything (SA) project: a new task, model, and dataset for image segmentation. Using our efficient model in a data collection loop, we built the largest segmentation dataset to date, with over 1 billion masks on 11 million licensed and privacy-respecting images. The model is designed and trained to be promptable, so it can transfer zero-shot to new image distributions and tasks. We evaluate its capabilities on numerous tasks and find that its zero-shot performance is impressive — often competitive with or even superior to prior fully supervised results. We release the Segment Anything Model (SAM) and corresponding dataset (SA-1B) to foster research in foundation models for computer vision.

In natural language processing, foundation models pretrained on broad web-scale data can be adapted to a wide range of downstream tasks via prompting. Can we build a similar foundation model for image segmentation? Such a model would need to solve a promptable segmentation task: given any prompt — a point, a box, a mask, or text — it must return a valid segmentation mask. This task is general enough to serve as a pretraining objective and enables zero-shot transfer to downstream segmentation problems through appropriate prompt engineering.

We define a promptable segmentation task where the goal is to return a valid segmentation mask given any segmentation prompt. A prompt can be spatial (points, boxes, masks) or semantic (free-form text). The requirement of a "valid" mask means that even when a prompt is ambiguous — for instance, a single point on a shirt could refer to the shirt itself or the person wearing it — the output should be a reasonable mask for at least one of the possible objects. This task generalizes naturally to several existing segmentation tasks (interactive segmentation, edge detection, object proposal generation) and can serve as a unifying pretraining objective.

SAM consists of three components. The image encoder employs an MAE-pretrained Vision Transformer (ViT) adapted for high-resolution input processing, generating 64x64 embeddings from 1024x1024 images. The prompt encoder represents sparse prompts (points, boxes) through positional encodings summed with learned embeddings, while dense prompts (masks) use convolutional processing with element-wise addition to image embeddings. The mask decoder features a lightweight two-layer transformer decoder utilizing cross-attention mechanisms, enabling real-time mask generation in approximately 50ms in a web browser.

To handle prompt ambiguity, SAM predicts multiple output masks (three by default) for each prompt, each with an associated confidence score (IoU prediction). This design directly addresses the one-to-many nature of the task: a single point on an object may correspond to the part, the whole object, or a larger group. During training, the loss is computed only against the minimum-loss mask, encouraging each output slot to specialize in a different granularity level. This multi-mask output with automatic ambiguity resolution is essential for the promptable interface.

The SA-1B dataset was built through a three-stage data engine. In the assisted-manual stage, professional annotators used SAM interactively, producing 4.3 million masks. In the semi-automatic stage, SAM generated candidate masks that annotators refined, yielding 5.9 million masks and increasing mask diversity. In the fully automatic stage, SAM generated masks using a 32x32 grid of point prompts per image, followed by non-maximum suppression and confidence filtering, producing the vast majority of the final 1.1 billion masks across 11 million images. Human quality ratings consistently scored mask quality between 7 and 9 on a 10-point scale.

We evaluate SAM across 23 diverse segmentation datasets in a zero-shot setting. On single-point segmentation, SAM outperforms the baseline RITM model on 16 of 23 datasets. Additional evaluations demonstrate zero-shot capability on edge detection (comparable to learned approaches on BSDS500), object proposals (competitive on LVIS), and instance segmentation. Preliminary experiments with text-to-mask functionality show promising but less refined results. Critically, the model achieves these results without any task-specific finetuning, purely through prompt engineering — supporting our thesis that SAM functions as a foundation model for segmentation.

The Segment Anything project demonstrates that a promptable foundation model for image segmentation is both feasible and effective. Through the synergy of the SA task, SAM model, and SA-1B dataset, we show that large-scale self-training with human-in-the-loop data curation can produce a model with strong zero-shot generalization. SAM's ability to segment any object with a simple prompt opens new possibilities for interactive annotation, downstream task adaptation, and compositional AI systems. We believe this work represents a step toward building foundation models for computer vision analogous to those that have transformed NLP.