Executive Overview
Meta Fundamental AI Research (FAIR) team has open-sourced SAM 2 (Segment Anything Model 2), a unified foundation model for promptable visual segmentation in images and video. The architecture processes video frames in real time, tracking and segmenting complex occluded objects across dynamic camera movements.
Streaming Memory Architecture
SAM 2 incorporates a memory attention module that conditions current frame predictions on past object masks and user prompt interactions. When an object is temporarily occluded behind an obstacle, SAM 2 remembers its visual signature and instantly recovers tracking when it re-emerges.
Performance Benchmarks
- Inference Throughput: Processes 44 frames per second at 1080p resolution on single consumer GPU nodes.
- Interactive Refinement: Users can refine object boundaries by clicking a single point in any video frame.
- Dataset Scale: Trained on SA-V dataset containing over 50,000 diverse videos and 640,000 spatial masklet annotations.