HARP

HARP is a sample editor that allows for hosted, asynchronous, remote processing of audio with machine learning. HARP works by routing audio through Gradio endpoints. Since Gradio applications can be hosted locally or in the cloud (e.g. with HuggingFace Spaces), HARP lets users of Digital Audio Workstations (DAWs) capable of connecting with external sample editors (e.g. Reaper, Logic Pro X, or Ableton Live) access large state-of-the-art models using cloud-based services, without breaking the within-DAW workflow.

Check out our paper: https://neuripscreativityworkshop.github.io/2023/papers/ml4cd2023_paper23.pdf.

TODO - update main figure with up-to-date screenshots

OS & DAW compatibility

HARP has been tested on the following:

• MacOS (ARM) versions 13.0 and 13.4
• MacOS (x86) version 10.15
• Windows 10
• Ubuntu 22.04
• TODO - Windows 11

with Reaper and Logic Pro X and as a standalone application.

Installing HARP

MacOS

• Download the macOS DMG file for HARP from the releases page.

• Double click on the DMG file. This will open the window shown below.

• Drag HARP.app to the Applications/ folder to install HARP.

Windows

• Download the Windows ZIP file for HARP from the releases page.

• Extract the contents of the ZIP file (HARP.exe) to a folder of your choice, e.g. C:\Program Files.

Linux

• Download the Linux ZIP file for HARP from the releases page.

• Extract the contents of the ZIP file (HARP) to a folder of your choice, e.g. /usr/local/bin/.

HARP Guide

WARNING! HARP is a destructive file editor.

After recording or loading audio into a track within your preferred DAW, it is recommended to bounce the track in order to avoid overwriting the original audio. If you would only like to process an excerpt of the track, trim the audio before performing the bounce.

When using HARP, it is recommended that you "bounce-in-place" any audio regions you'd like to process with HARP before processing them. This gives you the chance to undo changes and revert to a backup of your original file.

Standalone

Opening HARP

MacOS

• Run HARP.app to start the application.

Windows

• Run HARP.exe to start the application.

Linux

• Run HARP to start the application.

REAPER

Setting Up HARP

• Choose REAPER > Preferences on the file menu.

• Scroll down to External Editors and click Add.

• Click Browse to the right of the Primary Editor field.

• Navigate to your HARP.app installation and select "OK".

Opening HARP

• Right click the audio for the track you want to process and select Render items as new take to bounce the track.

• Right click the bounced audio and select Open items in editor > Open items in 'HARP.app'.

Logic Pro X

Setting Up HARP

• Set HARP.app as an external sample editor following this guide.

Opening HARP

• Select any audio region and press Shift+W to open the corresponding audio file in HARP.

Ableton Live

Setting Up HARP

TODO

Opening HARP

TODO

HARP Usage

• After opening HARP as an external sample editor, the following window will appear.

• Type the Gradio endpoint of an available HARP-compatible model (e.g. "hugggof/harmonic_percussive") in the field with the text path to a gradio endpoint.

  • This will populate the window with controls for the model.

  • Loading may take some time if the HuggingFace Space is asleep.

• Adjust the model controls to your liking and click process.

• The resulting audio can be played by pressing the space bar or by clicking Play/Stop.

• Any changes made in HARP will be automatically reflected in your DAW.

Available HARP-Compatible Models

While any algorithm or model can be made HARP-compatible with the PyHARP API, at present, the following are available for use within HARP:

• Pitch Shifting: hugggof/pitch_shifter

• Harmonic/Percussive Source Separation: hugggof/harmonic_percussive

• Music Audio Generation: descript/vampnet

• Convert Instrumental Music into 8-bit Chiptune: hugggof/nesquik

• Music Audio Generation: hugggof/MusicGen

• Pitch-Preserving Timbre-Removal: cwitkowitz/timbre-trap

Making a Model HARP-Compatible

We also provide PyHARP, a lightweight API to build HARP-compatible Gradio apps with optional interactive controls. PyHARP allows machine learning researchers to create DAW-friendly user interfaces for virtually any audio processing code using a minimal Python wrapper.

Building HARP

HARP can be built from scratch with the following steps:

1. Clone the repository

git clone --recurse-submodules https://github.com/audacitorch/HARP

2. Enter the project

cd HARP/

3. Configure

mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Debug

(ARM vs. x86 MacOS) The OSX architecture for the build can be specified explicitly by setting CMAKE_OSX_ARCHITECTURES to either arm64 or x86_64:

cmake .. -DCMAKE_OSX_ARCHITECTURES=x86_64

4. Build

MacOS/Linux

make -j <NUM_PROCESSORS>

Windows

cmake --build . --config Debug -j <NUM_PROCESSORS>

Codesigning & Distribution

MacOS

Codesigning and packaging for distribution is done through the script located at packaging/package.sh. You'll need to set up a developer account with Apple and create a certificate in order to sign the plugin. For more information on codesigning and notarization for macOS, please refer to the pamplejuce template.

The script requires the following variables to be passed:

# Retrieve values from either environment variables or command-line arguments
DEV_ID_APPLICATION # Developer ID Application certificate
ARTIFACTS_PATH # should be packaging/dmg/HARP.app
PROJECT_NAME # "HARP"
PRODUCT_NAME # "HARP"
NOTARIZATION_USERNAME # Apple ID
NOTARIZATION_PASSWORD # App-specific password for notarization
TEAM_ID # Team ID for notarization

Usage:

bash packaging/package.sh <DEV_ID_APPLICATION> <ARTIFACTS_PATH> <PROJECT_NAME> <PRODUCT_NAME> <NOTARIZATION_USERNAME> <NOTARIZATION_PASSWORD> <TEAM_ID>

After running package.sh, you should have a signed and notarized dmg file in the packaging/ directory.

Windows

TODO

Debugging a HARP Build

Visual Studio Code

1. Download Visual Studio Code.
2. Install the C/C++ extension from Microsoft.
3. Open the Run and Debug tab in VS Code and click create a launch.json file using CMake Debugger.
4. Create a configuration to attach to the process (see the following example code to be placed in launch.json).

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "(lldb) Standalone",
            "type": "cppdbg",
            "request": "launch",
            "program": "${workspaceFolder}/build/HARP_artefacts/Debug/HARP.app", // for macOS
            // "program": "${workspaceFolder}/build/HARP_artefacts/Debug/HARP.exe", // for Windows
            // "program": "${workspaceFolder}/build/HARP_artefacts/Debug/HARP", // for Linux
            "args": ["../test.wav"], // TODO - remove?
            "cwd": "${fileDirname}",
            "MIMode": "lldb" // for macOS
        }
    ]
}

5. Build the plugin using the flag -DCMAKE_BUILD_TYPE=Debug.
6. Add break points and run the debugger.