NATSpeech is a non-autoregressive text-to-speech framework in PyTorch that collects the official implementations of two well-known neural speech systems: PortaSpeech (NeurIPS 2021) and DiffSpeech, also known as DiffSinger (AAAI 2022). Maintained by the NATSpeech team, the repository is MIT licensed and has earned over 1,000 stars on GitHub.
Non-autoregressive models generate speech in parallel rather than token by token, which makes them much faster than classic autoregressive TTS while keeping quality high. NATSpeech provides the full pipeline around those models: data processing built on the Montreal Forced Aligner, a convenient and scalable framework for training and inference, and the model code for PortaSpeech’s portable, high-quality synthesis and DiffSpeech’s shallow-diffusion approach to singing voice synthesis. Demo pages and Hugging Face demos accompany the release, so you can hear the results before training anything yourself, and the framework’s structure makes it practical to extend the same pipeline to new non-autoregressive speech models.
The two headline systems target different ends of the synthesis problem. PortaSpeech focuses on portability and fidelity for spoken content, using a lightweight architecture that still produces natural, high-quality voices, while DiffSpeech introduced a shallow diffusion mechanism that brought diffusion-quality audio to both speech and singing, where pitch accuracy and expressiveness are harder to achieve. Housing both in one framework lets researchers compare approaches on shared data tooling and swap components โ aligners, acoustic models, and vocoders โ without rebuilding their pipeline for each new paper they want to try.
NATSpeech is free and open source under the MIT license.
It is aimed at speech researchers and audio engineers working on TTS or singing synthesis who want paper-faithful implementations with a complete training pipeline.