Add filter for rnnoise noise reduction

[bmatherly]

There is also a filter based on the link, but it is hidden

Depends on mltframework/mlt#1249

[Copilot]

Pull request overview

Adds a new RNNoise-based audio noise reduction filter to Shotcut’s QML filter catalog, along with build-script support to fetch/build the rnnoise library as part of the Shotcut dependency build workflow (noting the dependency on the referenced MLT change).

Changes:

• Added audio_rnnoise filter UI + metadata for an RNNoise noise reduction audio filter.
• Added a hidden audio_rnnoise_link metadata entry (and UI) for an RNNoise-based link filter.
• Updated build-shotcut.sh and build-shotcut-msys2.sh to optionally clone/build/install xiph/rnnoise (v0.2).

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
src/qml/filters/audio_rnnoise/ui.qml	New QML UI for RNNoise suppression (“mix”) control + preset integration.
src/qml/filters/audio_rnnoise/meta.qml	New filter metadata entry for the RNNoise audio filter.
src/qml/filters/audio_rnnoise_link/ui.qml	New QML UI (duplicated) for the hidden RNNoise link entry.
src/qml/filters/audio_rnnoise_link/meta.qml	New hidden link metadata entry for RNNoise.
scripts/build-shotcut.sh	Adds rnnoise repo/config/build/install steps to the dependency build script.
scripts/build-shotcut-msys2.sh	Adds rnnoise repo/config/build/install steps for the MSYS2 build script.

[ddennedy]

Why is the link hidden? Is there some magical way that links are automatically added that I forgot about?

[bmatherly]

Why is the link hidden? Is there some magical way that links are automatically added that I forgot about?

I implemented both filter and link to make it easy to test both services in MLT. But I only exposed the filter. Here is the complication:

• The filter adds a delay because the rnnoise library adds a delay and the filter can not read samples ahead to flush out the delay. This results in 30ms of silence at the start of the filter. If a user adds the filter to multiple clips with splits, it will sound like a discontinuity.
• the link does not add a delay because it can read samples ahead - and therefore does not add the silence at the start of the clip.
• The filter is more natual for users because they are probably going to look under Audio and not Time for this filter (we already have this problem with the declick filter)
• The link can not be applied to tracks - only clips

Ideally, the filter would be applied to tracks and the link would be applied to clips. But I do not think that is user friendly.

It is easy to hide/unhide either or both. I wonder what you think about this.

[ddennedy]

I was wondering if there is some way to make this link and "Declick Audio" appear in the Audio filters. I was making some changes to make a new type AudioLink, but then I realized it looks like I simply add isAudio: true to its meta.qml and it works. Adding it

[Error  ] <MLT> [link avfilter.adeclick] Init: f32le	2c	48000Hz
[adeclick @ 0x7ff7110c6d00] Detected clicks in 0 of 3600 samples (0%).
[in @ 0x7ff7108f6240] tb:1/48000 samplefmt:flt samplerate:48000 chlayout:stereo
[out @ 0x7ff7108f8ac0] auto-inserting filter 'auto_aresample_0' between the filter 'adeclick' and the filter 'out'
[adeclick @ 0x7ff71112b280] auto-inserting filter 'auto_aresample_1' between the filter 'in' and the filter 'adeclick'
[auto_aresample_1 @ 0x7ff710d74000] ch:2 chl:stereo fmt:flt r:48000Hz -> ch:2 chl:stereo fmt:dblp r:48000Hz
[auto_aresample_0 @ 0x7ff7110c6d00] ch:2 chl:stereo fmt:dblp r:48000Hz -> ch:2 chl:stereo fmt:flt r:48000Hz

I was wondering about that "Error" but looking at the MLT code, that looks like the wrong log level.
It shows up in Time as well, but that is an easy fix. Searching "declick" only shows once.:

diff --git a/src/models/metadatamodel.cpp b/src/models/metadatamodel.cpp
index ae15db9c..fdb28577 100644
--- a/src/models/metadatamodel.cpp
+++ b/src/models/metadatamodel.cpp
@@ -232,7 +232,7 @@ bool MetadataModel::filterAcceptsRow(int sourceRow, const QModelIndex &sourcePar
                 return false;
             break;
         case LinkFilter:
-            if (meta->type() != QmlMetadata::Link)
+            if (meta->type() != QmlMetadata::Link || meta->isAudio())
                 return false;
             break;
         case FilterSetFilter:
diff --git a/src/qml/filters/audio_adeclick/meta.qml b/src/qml/filters/audio_adeclick/meta.qml
index 22c24773..69793950 100644
--- a/src/qml/filters/audio_adeclick/meta.qml
+++ b/src/qml/filters/audio_adeclick/meta.qml
@@ -3,6 +3,7 @@ import org.shotcut.qml
 
 Metadata {
     type: Metadata.Link
+    isAudio: true
     name: qsTr("Declick Audio")
     mlt_service: 'avfilter.adeclick'
     keywords: qsTr('declick crackle pop', 'search keywords for the Declick audio filter') + ' declick audio'

One small quirk is that it still must appear at the top of attached filters for correct link-based behavior:

What do you think? OK?

[bmatherly]

What do you think? OK?

I tried it myself and I am surprised that it works that well. I'm sure users will find it to be unexpected and report it as a bug. Or at least ask why they can't move it around. Maybe, we could name audio links something else so users can understand the order. Like we could call that section heading "Audio Preprocess". Or, since we only have noise reduction and declick, the section heading could be "Noise & Repair".

To be clear, I am only suggesting to change the text in the section header of the applied filters list. When selecting filters, they would both still be in the "Audio" list. I am not proposing to add a whole new category of filters.

[bmatherly]

Here is a patch that demonstrates my suggestion. I do not feel strongly about the name of the section as long as we don't duplicate "Audio" twice.

diff --git a/src/models/attachedfiltersmodel.cpp b/src/models/attachedfiltersmodel.cpp
index c12e3ce0c..605e259f9 100644
--- a/src/models/attachedfiltersmodel.cpp
+++ b/src/models/attachedfiltersmodel.cpp
@@ -232,7 +232,9 @@ QVariant AttachedFiltersModel::data(const QModelIndex &index, int role) const
     case TypeDisplayRole: {
         QVariant result;
         const QmlMetadata *meta = m_metaList[index.row()];
-        if (meta && meta->isAudio()) {
+        if (meta && meta->isAudio() && meta->type() == QmlMetadata::Link) {
+            result = tr("Audio Preprocess");
+        } else if (meta && meta->isAudio()) {
             result = tr("Audio");
         } else if (meta && meta->type() == QmlMetadata::Link) {
             result = tr("Time");
diff --git a/src/models/metadatamodel.cpp b/src/models/metadatamodel.cpp
index ae15db9c7..fdb285772 100644
--- a/src/models/metadatamodel.cpp
+++ b/src/models/metadatamodel.cpp
@@ -232,7 +232,7 @@ bool MetadataModel::filterAcceptsRow(int sourceRow, const QModelIndex &sourcePar
                 return false;
             break;
         case LinkFilter:
-            if (meta->type() != QmlMetadata::Link)
+            if (meta->type() != QmlMetadata::Link || meta->isAudio())
                 return false;
             break;
         case FilterSetFilter:
diff --git a/src/qml/filters/audio_adeclick/meta.qml b/src/qml/filters/audio_adeclick/meta.qml
index 22c24773f..cd8da8d0c 100644
--- a/src/qml/filters/audio_adeclick/meta.qml
+++ b/src/qml/filters/audio_adeclick/meta.qml
@@ -8,5 +8,6 @@ Metadata {
     keywords: qsTr('declick crackle pop', 'search keywords for the Declick audio filter') + ' declick audio'
     objectName: 'audioDeclick'
     qml: 'ui.qml'
+    isAudio: true
     help: 'https://forum.shotcut.org/t/declick-audio-filter/42273/1'
 }
diff --git a/src/qml/filters/audio_rnnoise_link/meta.qml b/src/qml/filters/audio_rnnoise_link/meta.qml
index 3172e0502..b586b7a0c 100644
--- a/src/qml/filters/audio_rnnoise_link/meta.qml
+++ b/src/qml/filters/audio_rnnoise_link/meta.qml
@@ -8,4 +8,5 @@ Metadata {
     keywords: qsTr('reduce noise denoise background wind speech clean', 'search keywords for the Reduce Noise: Audio filter') + ' reduce noise: audio rnnoise'
     objectName: 'audioRnnoiseLink'
     qml: 'ui.qml'
+    isAudio: true
 }

[ddennedy]

Um, but then if you add a Time filter there can be up 4 categories in the attached list, which is not ideal! And neither is the next idea, but maybe these should be listed in the "Audio" menu but the "Time" category:

Or possibly change Time to Preprocess:

[bmatherly]

Or possibly change Time to Preprocess

That's my favorite version so far.

I asked AI for more name ideas for "Preprocess":

• Input Filters
• Source Filters
• Clip Filters
• Base Filters
• Before Effects
• Pre Effects

I kind of like Input/Source/Clip to imply that the processing occurs on the source clip rather than on individual frames. But I do not know if it is good to expose that implementation detail to the user. And "Preprocess" is a pretty common word that people can understand conceptually.