WhisperX builds on OpenAI’s Whisper to deliver fast, accurate speech recognition with word-level timestamps and speaker diarization. Using batched inference through faster-whisper, it transcribes the large-v2 model at up to around 70 times real-time speed, then aligns the transcript with a phoneme-based model to pinpoint exactly when each word was spoken, and labels who said what with pyannote diarization. It has close to 25,000 stars on GitHub and is a standard tool for serious transcription work.
Whisper’s own timestamps are segment-level and often drift, which is not good enough for subtitles, interviews or conversation analytics. WhisperX fixes this with a three-stage pipeline: voice activity detection trims silence and improves batching, the faster-whisper backend (CTranslate2) transcribes with a fraction of the memory, and forced alignment using wav2vec2 models produces accurate word-level timing for many languages. Optional speaker diarization assigns speaker labels, so a two-person interview comes back as a clean, attributed transcript. The tool runs from the command line or as a Python library, supports output formats suitable for subtitling and downstream processing, and can cut GPU memory usage dramatically compared with the original Whisper implementation.
WhisperX is free and open source under the BSD-2-Clause license. You run it on your own hardware; note that speaker diarization uses pyannote models hosted on Hugging Face, which require a free account and accepting their terms.
WhisperX is for journalists, podcasters, researchers, subtitle creators and developers who need precise word timings and speaker labels from audio, and for teams building transcription pipelines where vanilla Whisper output is not accurate enough.