Adaptive Volume Threshold and speed like NewPipe does, or even better.

[1nd1g0]

Hello there, Bennett. Right to the point.

Would you be so kind to add an adaptive (dynamic) threshold option?
It's a part of software adaptive noise gates and dynamic range compressors in real-time digital signal processing for decades. Our task is also "real-time" processing. Presumably, no large preload or buffering needed yet.

The basic idea is to start with zero (silence) threshold and x1 speed gradually integrate the volume level of incoming samples , calculating the whole volume scale of the already played part: the highest integral level, the lowest integral level (noise/music) etc. The threshold gradually moves up to the calculated noise/music level during the whole playback. Moving back towards safe (0%) threshold has to be accelerated so no audio is lost when music/noise level dramatically decreased. Also the next trick with quiet audio speed slowdown comes handy.

Adaptive speed option can be very useful. Generally - the lower highest integral volume or (option) the higher the threshold over time is, the less FF speed is used for the user to hear better what was said.

Thanks in advance for your consideration.

[vantezzen]

Hello there, sorry for only now coming back to you on this. Do you have any further resources, maybe even code samples, for such an algorithm or would be willing to try to implement this?

I have created a new branch "dynamic-threshold" that holds the code needed around the dynamic threshold calculating code (adding a new setting, saving previous sample volumes etc.) but I don't really understand how your described algorithm should work.
As a placeholder, I added a script that simply that uses 30% of the average volume of the last 500 samples (at ca. 60Hz) as the threshold:

skip-silence/src/pages/content/lib/DynamicThresholdCalculator.ts

Lines 34 to 38 in 88d7098

	// Calculate the dynamic silence level
	// TODO: Implement a real algorithm
	const averageVolume = this.previousSamples.reduce((acc, val) => acc + val) / this.previousSamples.length;
	this.threshold = averageVolume * 0.3;

This would need to get replaced with a real implementation of your described algorithm.

[vantezzen]

I've updated the implementation to be closer to what you've described. It will now slowly get to the volume of the lowest 10th percentile. This will be released as "beta" in Skip Silence 4

skip-silence/src/pages/content/lib/DynamicThresholdCalculator.ts

Lines 39 to 49 in cc5b155

	const currentTreshold = this.threshold;
	const sortedSamples = this.previousSamples.sort((a, b) => a - b);
	const lowest10Percentile = sortedSamples[Math.floor(this.previousSamples.length * 0.1)];
	const delta = Math.abs(this.threshold - lowest10Percentile);
	if (lowest10Percentile > this.threshold) {
	this.threshold += delta * 0.1;
	} else if (lowest10Percentile < this.threshold) {
	// Threshold should decrease faster so we can better adapt to fast volume changes
	this.threshold -= delta * 0.4;
	}

[1nd1g0]

Wow, thanks a lot, man. I've also lost your message within a sea of other notifications. I'll test the implementation in the wild.
Meanwhile lurking through NewPipe source I've learned, that android's ExoPlayer(2) with built-in silence skipping is used.
Here is the last commit that introduces silence skipping parameterization.

google/ExoPlayer@ad1dffc

[vantezzen]

It looks like ExoPlayer implements its Skip Silence functionality in https://github.com/google/ExoPlayer/blob/bd54394391/library/core/src/main/java/com/google/android/exoplayer2/audio/SilenceSkippingAudioProcessor.java, but I can't really find adaptive threshold code there.

The code uses silenceThresholdLevel to determine if audio is SILENCE, NOISY or MAYBE_SILENT (the transition state between the two) but I can't find the code where this threshold is changed - in fact, the threshold is set as final so it can't be changed unless the complete class is re-initialized.

Maybe I'm overseeing something, NewPipe is implementing the dynamic threshold itself or its simply using a static threshold instead of dynamically updating the threshold based on the current volume.

[AlexLev59]

I have an error when installing in the Google Chrome browser:
Unrecognized manifest key 'browser_specific_settings'.
{"description":"Skip silent parts in videos and audio files","version":"4.0.0","name":"Skip Silence","background":{"page":"background.html"},"page_action":{"default_icon":{"32":"assets/img/disabled.png"},"default_popup":"popup.html"},"icons":{"128":"assets/img/icon-128.png"},"permissions":["activeTab","storage"],"browser_specific_settings":{"gecko":{"strict_min_version":"91.0"}},"content_scripts":[{"all_frames":true,"matches":["http:///","https:///","<all_urls>"],"js":["contentScript.bundle.js"],"css":["content.styles.css"]}],"web_accessible_resources":["content.styles.css","assets/img/icon-128.png","assets/img/icon-34.png","assets/img/close.svg"],"commands":{"toggle-enable":{"suggested_key":{"default":"Ctrl+Shift+S"},"description":"Enable/disable Skip Silence for the current tab"},"toggle-command-bar":{"suggested_key":{"default":"Alt+Shift+S"},"description":"Show/hide the command bar"}},"manifest_version":2,"content_security_policy":"script-src 'self' 'unsafe-eval' https://scripts.simpleanalyticscdn.com https://a.vantezzen.io; object-src 'self'"}

[WofWca]

@AlexLev59 Wrong topic.
And I believe this is not an error but just a warning that doesn't prevent you from using the extension. Right?