Whisper JAX

Whisper JAX is an open-source reimplementation of OpenAI’s Whisper speech recognition model in JAX, built by Sanchit Gandhi during his time at Hugging Face. It achieves up to 70x faster transcription than the original PyTorch implementation on TPUs, and the GitHub repository has earned roughly 4,700 stars as a reference for high-throughput speech-to-text.

Overview

Whisper is accurate but slow when transcribing at scale. Whisper JAX rewrites the model with JAX and Flax, applies just-in-time compilation, data parallelism across devices and half-precision execution, so long-form audio can be transcribed in a fraction of the time on TPU pods and GPUs. The repo includes a full training and fine-tuning pipeline as well, letting you adapt Whisper to new languages or domains with the same speed-oriented stack. The project is also a popular teaching artifact: its code and accompanying analysis show concretely how compilation, sharding and precision choices translate into order-of-magnitude inference wins on modern accelerators.

Key Features

• Up to 70x faster Whisper inference on TPUs versus the reference implementation
• Faithful JAX/Flax port of all Whisper model sizes, weights convertible from Hugging Face
• Batched, sharded inference pipeline for transcribing large audio corpora
• Complete fine-tuning scripts for adapting Whisper to new languages and domains
• Long-form audio support with chunked decoding
• Detailed benchmarks and a companion write-up explaining the optimizations
• Pretrained checkpoints for every Whisper size, loadable directly from the Hugging Face Hub

💳 Pricing

Free and open source under the Apache 2.0 license. Costs are only the TPU or GPU time you run it on.

Who Is It For

Speech researchers and engineering teams transcribing audio at serious scale (media archives, call centers, dataset builders) who need Whisper’s accuracy at a fraction of the compute bill.