SAM-Audio

SAM-Audio is Meta’s Segment Anything Model for audio: a foundation model that isolates any sound within a recording using text, visual or temporal prompts. Describe the sound you want, point to it in a video, or mark a time span, and the model separates that source from a complex mixture.

Overview

SAM-Audio is released by Meta’s research team on GitHub, where it has about 3,670 stars, with the repository updated in 2026. The model builds on the Perception-Encoder Audio-Visual (PE-AV) backbone and a companion Judge model to understand which part of a mixture matches a prompt. The repository provides inference code, example notebooks and links to download trained checkpoints from Hugging Face, with access granted on request. It runs in Python 3.11 and works best with a CUDA-capable GPU, supporting prompts as simple as a phrase like a dog barking or a person speaking. The release includes both large and base model sizes, so users can trade accuracy against compute, and the separation quality is judged with the help of the accompanying Judge model, which scores how well an isolated segment matches the prompt that produced it.

Key Features

• Isolates individual sounds from complex audio mixtures with a single prompt
• Three prompting modes: natural-language text, visual cues from video, and time spans
• Foundation-model design based on the Perception-Encoder Audio-Visual backbone
• Inference code, example notebooks and downloadable pretrained checkpoints
• Handles speech, music and environmental sounds in one unified model
• Released by Meta AI research alongside a paper, blog post and demo

💳 Pricing

SAM-Audio is free to use for research purposes under Meta’s model licensing terms. You download the checkpoints from Hugging Face after an access request and run inference on your own hardware.

Who Is It For

Audio researchers, media-production engineers and ML developers who need flexible, prompt-driven sound separation for editing, analysis or dataset creation.