Initial commit featuring OpenVINO-module src's compatible with Audaci…

…ty 3.4.2

Signed-off-by: Metcalfe, Ryan <ryan.metcalfe@intel.com>

CONTRIBUTING.md

@@ -2,7 +2,7 @@
 
 ### License
 
-<PROJECT NAME> is licensed under the terms in [LICENSE]<link to license file in repo>. By contributing to the project, you agree to the license and copyright terms therein and release your contribution under these terms.
+OpenVINO™ AI Plugins for Audacity* is licensed under the terms in [LICENSE](LICENSE.txt). By contributing to the project, you agree to the license and copyright terms therein and release your contribution under these terms.
 
 ### Sign your work
 

README.md

@@ -1,5 +1,40 @@
-# OpenVINO™ AI Plugins for Audacity*
+# OpenVINO™ AI Plugins for Audacity* :metal:
+![](doc/assets/openvino_ai_plugins.png)
+[![License: GPL v3](https://img.shields.io/badge/License-GPL%20v3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0)  
 
-Coming soon! 
+A set of AI-enabled effects, generators, and analyzers for [Audacity®](https://www.audacityteam.org/). These AI features run 100% locally on your PC :computer: -- no internet connection necessary! [OpenVINO™](https://github.com/openvinotoolkit/openvino) is used to run AI models on supported accelerators found on the user's system such as CPU, GPU, and NPU.
 
-Click watch to be notified when the plugins are released!
+- [**Music Separation**](doc/feature_doc/music_separation/README.md):musical_note: -- Separate a mono or stereo track into individual stems -- Drums, Bass, Vocals, & Other Instruments. 
+- [**Music Style Remix**](doc/feature_doc/music_style_remix/README.md):cd: -- Uses Stable Diffusion to alter a mono or stereo track using a text prompt.
+- [**Noise Suppression**](doc/feature_doc/noise_suppression/README.md):broom: -- Removes background noise from an audio sample.
+- [**Music Generation**](doc/feature_doc/music_generation/README.md):notes: -- Uses Stable Diffusion to generate snippets of music from a text prompt.
+- [**Whisper Transcription**](doc/feature_doc/whisper_transcription/README.md):microphone: -- Uses [whisper.cpp](https://github.com/ggerganov/whisper.cpp) to generate a label track containing the transcription or translation for a given selection of spoken audio or vocals.
+
+## Installation :floppy_disk: 
+  Go [here](https://github.com/intel/openvino-plugins-ai-audacity/releases) to find installation packages & instructions for the latest Windows release. 
+
+## Build Instructions :hammer:
+  - [Windows Build Instructions](doc/build_doc/windows/README.md)  
+  - [Linux Build Instructions](doc/build_doc/linux/README.md)
+
+## Help, Feedback, & Bug Reports 🙋‍♂️
+  We welcome you to submit an issue [here](https://github.com/intel/openvino-plugins-ai-audacity/issues) for
+  * Questions
+  * Bug Reports
+  * Feature Requests
+  * Feedback of any kind -- how can we improve this project?
+
+## Contribution :handshake:
+  Your contributions are welcome and valued, no matter how big or small. Feel free to submit a pull-request!
+
+## Acknowledgements :pray:
+* Audacity® development team & Muse Group-- Thank you for your support!  
+* Audacity® GitHub -- https://github.com/audacity/audacity  
+* Whisper transcription & translation analyzer uses whisper.cpp (with OpenVINO™ backend): https://github.com/ggerganov/whisper.cpp  
+* Music Generation & Music Style Remix use Riffusion's UNet model, Riffusion pipelines that were ported to C++ from this project: https://github.com/riffusion/riffusion    
+* Music Separation effect uses Meta's Demucs v4 model (https://github.com/facebookresearch/demucs), which has been ported to work with OpenVINO™  
+* Noise Suppression uses noise-suppression-denseunet-ll model from OpenVINO™'s Open Model Zoo: https://github.com/openvinotoolkit/open_model_zoo  
+
+## Disclaimer :warning:
+Stable Diffusion & Riffusion's data model is governed by the Creative ML Open Rail M license, which is not an open source license.
+https://github.com/CompVis/stable-diffusion. Users are responsible for their own assessment whether their proposed use of the project code and model would be governed by and permissible under this license.

doc/build_doc/linux/README.md

@@ -0,0 +1,228 @@
+# Audacity OpenVINO module build for Linux (Ubuntu 22.04)
+
+Hi! The following is the process that we use when building the Audacity modules for Linux. These instructions are for Ubuntu 22.04, so you may need to adapt them slightly for other Linux distros.
+
+## High-Level Overview
+Before I get into the specifics, at a high-level we will be doing the following:
+* Cloning & building whisper.cpp with OpenVINO support (For transcription audacity module)
+* Cloning & building openvino-stable-diffusion-cpp (This is to support Music Generation & Remix features)
+* Cloning & building Audacity 3.4.2 without modifications (just to make sure 'vanilla' build works fine)
+* Adding our OpenVINO module src's to the Audacity source tree, and re-building it.
+
+## Dependencies
+Here are some of the dependencies that you need to grab. If applicable, I'll also give the cmd's to set up your environment here.
+* Build Essentials (GCC & CMake)
+```
+sudo apt install build-essential
+```
+* OpenVINO - You can use public version from [here](https://storage.openvinotoolkit.org/repositories/openvino/packages/2023.2/linux/l_openvino_toolkit_ubuntu22_2023.2.0.13089.cfd42bd2cb0_x86_64.tgz)
+```
+# Extract it
+tar xvf l_openvino_toolkit_ubuntu22_2023.2.0.13089.cfd42bd2cb0_x86_64.tgz 
+
+#install dependencies
+cd l_openvino_toolkit_ubuntu22_2023.2.0.13089.cfd42bd2cb0_x86_64/install_dependencies/
+sudo -E ./install_openvino_dependencies.sh
+cd ..
+
+# setup env
+source setupvars.sh
+```
+* OpenCV - Only a dependency for the  OpenVINO Stable-Diffusion CPP samples (to read/write images from disk, display images, etc.). You can install like this:
+```
+sudo apt install libopencv-dev
+```
+* Libtorch (C++ distribution of pytorch)- This is a dependency for the audio utilities in openvino-stable-diffusion-cpp (like spectrogram-to-wav, wav-to-spectrogram), as well as some of our htdemucs v4 routines (supporting music separation). We are currently using this version: [libtorch-cxx11-abi-shared-with-deps-2.1.1+cpu.zip ](https://download.pytorch.org/libtorch/cpu/libtorch-cxx11-abi-shared-with-deps-2.1.1%2Bcpu.zip). Setup environment like this:
+```
+unzip libtorch-cxx11-abi-shared-with-deps-2.1.1+cpu.zip
+export LIBTORCH_ROOTDIR=/path/to/libtorch
+```
+
+## Sub-Component builds
+We're now going to build whisper.cpp and stablediffusion-pipelines-cpp.  
+```
+# OpenVINO
+source /path/to/l_openvino_toolkit_ubuntu22_2023.2.0.13089.cfd42bd2cb0_x86_64/setupvars.sh
+
+# Libtorch
+export LIBTORCH_ROOTDIR=/path/to/libtorch
+```
+
+### Whisper.cpp 
+```
+# Clone it & check out specific commit
+git clone https://github.com/ggerganov/whisper.cpp
+cd whisper.cpp
+git checkout ec7a6f04f9c32adec2e6b0995b8c728c5bf56f35
+cd ..
+
+# Create build folder
+mkdir whisper-build
+cd whisper-build
+
+# Run CMake, specifying that you want to enable OpenVINO support.
+cmake ../whisper.cpp/ -DWHISPER_OPENVINO=ON
+
+# Build it:
+make -j 4
+
+# Install built whisper collateral into a local 'installed' directory:
+cmake --install . --config Release --prefix ./installed
+```
+With the build / install complete, the Audacity build will find the built collateral via the WHISPERCPP_ROOTDIR. So you can set it like this:
+```
+export WHISPERCPP_ROOTDIR=/path/to/whisper-build/installed
+export LD_LIBRARY_PATH=${WHISPERCPP_ROOTDIR}/lib:$LD_LIBRARY_PATH
+```
+(I'll remind you later about this though)
+
+### OpenVINO Stable-Diffusion CPP
+```
+# clone it
+git clone https://github.com/intel/stablediffusion-pipelines-cpp.git
+
+#create build folder
+mkdir stablediffusion-pipelines-cpp-build
+cd stablediffusion-pipelines-cpp-build
+
+# Run cmake
+cmake ../stablediffusion-pipelines-cpp
+
+# Build it
+make -j 8
+
+# Install it
+cmake --install . --config Release --prefix ./installed
+
+# Set environment variable that Audacity module will use to find this component.
+export CPP_STABLE_DIFFUSION_OV_ROOTDIR=/path/to/stablediffusion-pipelines-cpp-build/installed
+export LD_LIBRARY_PATH=${CPP_STABLE_DIFFUSION_OV_ROOTDIR}/lib:$LD_LIBRARY_PATH
+
+```
+
+## Audacity 
+
+Okay, moving on to actually building Audacity. Just a reminder, we're first going to just build Audacity without any modifications. Once that is done, we'll copy our openvino-module into the Audacity src tree, and built that.
+
+### Audacity initial (vanilla) build
+```
+#Install some build dependencies
+sudo apt-get install -y build-essential cmake git python3-pip
+sudo pip3 install conan
+sudo apt-get install libgtk2.0-dev libasound2-dev libjack-jackd2-dev uuid-dev
+
+# clone Audacity
+git clone https://github.com/audacity/audacity.git
+
+# Check out Audacity-3.4.2 tag, 
+cd audacity
+git checkout Audacity-3.4.2
+cd ..
+
+# Create build directory
+mkdir audacity-build
+cd audacity-build
+
+# Run cmake (grab a coffee & a snack... this takes a while)
+cmake -G "Unix Makefiles" ../audacity -DCMAKE_BUILD_TYPE=Release
+
+# build it 
+make -j`nproc`
+```
+
+When this is done, you can run Audacity like this (from audacity-build directory):
+```
+Release/bin/audacity
+```
+
+### Audacity OpenVINO module build
+
+Now we'll run through the steps to actually build the OpenVINO-based Audacity module.
+
+First, clone the following repo. This is where the actual Audacity module code lives today.
+```
+:: clone it
+git clone https://github.com/intel/openvino-plugins-ai-audacity.git
+
+# Check out the release tag that matches the Audacity version you're using
+cd openvino-plugins-ai-audacity
+git checkout v3.4.2-R1
+```
+
+We need to copy the ```mod-openvino``` folder into the Audacity source tree.
+i.e. Copy ```openvino_audio_workloads\mod-openvino``` folder to ```audacity\modules```.
+
+
+We now need to edit ```audacity\modules\CMakeLists.txt``` to add mod-openvino as a build target. You just need to add a ```add_subdirectory(mod-openvino)``` someplace in the file. For example:
+
+```
+...
+foreach( MODULE ${MODULES} )
+   add_subdirectory("${MODULE}")
+endforeach()
+
+#YOU CAN ADD IT HERE
+add_subdirectory(mod-openvino)
+
+if( NOT CMAKE_SYSTEM_NAME MATCHES "Darwin" )
+   if( NOT "${CMAKE_GENERATOR}" MATCHES "Visual Studio*")
+      install( DIRECTORY "${_DEST}/modules"
+               DESTINATION "${_PKGLIB}" )
+   endif()
+endif()
+...
+```
+
+Okay, now we're going to (finally) build the module. Here's a recap of the environment variables that you should have set:
+
+```
+# OpenVINO
+source /path/to/l_openvino_toolkit_ubuntu22_2023.2.0.13089.cfd42bd2cb0_x86_64/setupvars.sh
+
+# Libtorch
+export LIBTORCH_ROOTDIR=/path/to/libtorch
+
+# Whisper.cpp
+export WHISPERCPP_ROOTDIR=/path/to/whisper-build/installed
+export LD_LIBRARY_PATH=${WHISPERCPP_ROOTDIR}/lib:$LD_LIBRARY_PATH
+
+# CPP Stable Diffusion
+export CPP_STABLE_DIFFUSION_OV_ROOTDIR=/path/to/stablediffusion-pipelines-cpp-build/installed
+export LD_LIBRARY_PATH=${CPP_STABLE_DIFFUSION_OV_ROOTDIR}/lib:$LD_LIBRARY_PATH
+```
+
+Okay, on to the build:  
+```
+# cd back to the same Audacity folder you used to build Audacity before
+cd audacity-build
+
+# and build the new target, mod-openvino.
+# (Note: CMake will automatically re-run since you modified CMakeLists.txt)
+cmake --build . --config Release --target mod-openvino
+```
+
+If it all builds correctly, you should see mod-openvino.so sitting in Release/lib/audacity/modules/
+
+You can go ahead and run audacity-build/Release/bin/audacity
+
+Once Audacity is open, you need to go to Edit->Preferences. And on the left side you'll see a 'Modules' tab, click that. And here you (hopefully) see mod-openvino entry set to 'New'. You need to change it to 'Enabled', as shown in the following picture.  
+
+![](preferences_enabled.png)
+
+Once you change to 'Enabled', close Audacity and re-open it. When it comes back up, you should now see the OpenVINO modules listed.
+
+## OpenVINO Models Installation  
+And we're done, at least with the module build. To actually use these modules, we need to generate / populate ```/usr/local/lib/``` with the models for noise-separation, music separation (htdemucs), whisper, and riffusion. Start by downloading the model package zip file from here:  
+[openvino-models.zip](https://github.com/intel/openvino-plugins-ai-audacity/releases/download/v3.4.2-R1/openvino-models.zip)
+
+And extract / install them into /usr/local/lib like this:
+```
+# unzip the packages. 
+unzip openvino-models.zip
+
+# After above command you should have a single ```openvino-models``` folder, which you can copy to /usr/local/lib:
+sudo cp -R openvino-models /usr/local/lib/
+```
+
+# Need Help? :raising_hand_man:
+For any questions about this build procedure, feel free to submit an issue [here](https://github.com/intel/openvino-plugins-ai-audacity/issues)