🤖 The free, Open Source OpenAI alternative. Self-hosted, community-driven and local-first. Drop-in replacement for OpenAI running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. It allows to generate Text, Audio, Video, Images. Also with voice cloning capabilities.

Amphion (/æmˈfaɪən/) is a toolkit for Audio, Music, and Speech Generation. Its purpose is to support reproducible research and help junior researchers and engineers get started in the field of audio, music, and speech generation research and development.

AudioLDM: Generate speech, sound effects, music and beyond, with text.

Text-to-Audio/Music Generation

A timeline of the latest AI models for audio generation, starting in 2023!

Audio generation using diffusion models, in PyTorch.

TTS Generation Web UI (Bark, MusicGen + AudioGen, Tortoise, RVC, Vocos, Demucs, SeamlessM4T, MAGNet, StyleTTS2, MMS)

Implementation of SoundStorm, Efficient Parallel Audio Generation from Google Deepmind, in Pytorch

A family of diffusion models for text-to-audio generation.

Official PyTorch implementation of BigVGAN (ICLR 2023)

[CVPR'23] MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video Generation

Source code for "Taming Visually Guided Sound Generation" (Oral at the BMVC 2021)

AI Audio Datasets 🎵. A list of datasets consisting of speech, music, and sound effects, which can provide training data for Generative AI, AIGC, AI model training, intelligent audio tool development, and audio applications.

FunCodec is a research-oriented toolkit for audio quantization and downstream applications, such as text-to-speech synthesis, music generation et.al.

Python library for designing and training your own Diffusion Models with PyTorch.

This is a list of sound, audio and music development tools which contains machine learning, audio generation, audio signal processing, sound synthesis, spatial audio, music information retrieval, music generation, speech recognition, speech synthesis, singing voice synthesis and more.

Daily tracking of awesome audio papers, including music generation, zero-shot tts, asr, audio generation

Pytorch implementation of BigVSAN

Official pytorch implementation of the paper: "Catch-A-Waveform: Learning to Generate Audio from a Single Short Example" (NeurIPS 2021)

Reading list for research topics in Sound AI