Audio Diffusion PyTorch is an open-source library with 2,000+ stars on GitHub that brings diffusion models — the technique behind modern image generation — to audio, generating sound with denoising diffusion in pure PyTorch.
Diffusion models generate data by learning to reverse a gradual noising process, and this library applies that recipe to waveforms and spectrograms. It provides the building blocks a researcher needs to train audio diffusion models: the diffusion process itself, model components suited to audio, and training scaffolding, all expressed in idiomatic PyTorch so it can be read, modified and extended. Projects of this type underpin text-to-audio and music-generation research, unconditional sound synthesis and audio inpainting experiments. The library was most active around 2023, when audio diffusion was a fast-moving research front, and it should be approached today as a compact, readable reference implementation and experimentation base rather than a maintained product — expect to bring your own data pipeline and to adapt pieces to current PyTorch versions. For engineers learning how diffusion works beyond images, the audio domain is a particularly instructive variation because the data is one-dimensional, long and highly structured.
Audio Diffusion PyTorch is free and open source under the MIT license. Training and sampling run on your own hardware; the only costs are your compute and any datasets you bring.
Machine-learning researchers and audio engineers who want a hands-on PyTorch codebase for understanding and experimenting with diffusion-based audio generation, rather than a hosted text-to-audio product.