[BUG] Monitor of all speakers is very delayed

[matanui159]

Desktop (please complete the following information):

• Version of Plasma Framework: git latest (515f49d)

Describe the bug
When switching to "Mixing all features" for PulseAudio the visualiser is delayed by about three seconds from the audio playing (I can test this by pausing and playing a song). This does not happen when using the other options.

Any error message shown in the console

The log:

QML debugging is enabled. Only use this in a safe environment.
qml: View QML loaded
org.kde.plasmaquick: Applet preload policy set to 1
file:///usr/share/plasma/wallpapers/org.kde.image/contents/ui/main.qml:76:9: Unable to assign [undefined] to QStringList
file:///usr/share/plasma/wallpapers/org.kde.image/contents/ui/main.qml:75:9: Unable to assign [undefined] to int
file:///usr/share/plasma/wallpapers/org.kde.image/contents/ui/main.qml:75:9: Unable to assign [undefined] to int
file:///usr/share/plasma/wallpapers/org.kde.image/contents/ui/main.qml:76:9: Unable to assign [undefined] to QStringList
Toolbox not loading, toolbox package is either invalid or disabled.
qml: New Containment: ContainmentInterface(0x5641098252d0)
QQuickOpenGLShaderEffect: 'source' does not have a matching property!
QQuickOpenGLShaderEffect: 'source' does not have a matching property!
kf5.kio.core: Invalid URL: QUrl("")
trying to show an empty dialog
file:///home/matanui159/.extra/panon/plasmoid/contents/ui/Spectrum.qml:121:45: QML Image: Unknown error
QProcess: Destroyed while process ("/bin/sh") is still running.
trying to show an empty dialog
file:///usr/share/plasma/shells/org.kde.plasma.plasmoidviewershell/contents/configuration/AppletConfiguration.qml:153:9: QML RowLayout: Detected anchors on an item that is managed by a layout. This is undefined behavior; use Layout.alignment instead.
QCoreApplication::postEvent: Unexpected null receiver
file:///usr/lib/qt/qml/org/kde/newstuff/qml/QuestionAsker.qml:106:5: QML Connections: Cannot assign to non-existent property "onClosing"
QProcess: Destroyed while process ("/bin/sh") is still running.
QProcess: Destroyed while process ("/bin/sh") is still running.
QProcess: Destroyed while process ("/bin/sh") is still running.

[rbn42]

Sorry, I can't reproduce your issue in my system.

A possible solution is to replace line 68

panon/panon/backend/client.py

Lines 67 to 70 in 515f49d

	if spectrum_data is None:
	# Set fps to 2 to lower CPU usage, when audio is unavailable.
	latest_wave_data = spectrum_source.read(fps=2)

with

   if False:

[rbn42]

And if the solution above failed for you, please try this python script:

import soundcard as sc
import numpy as np
sample_rate=44100
channel_count=2
fps=60

blocksize = sample_rate // fps
mics = sc.all_microphones(exclude_monitors=False)

streams = []

for mic in mics:
    stream = mic.recorder(
        sample_rate,
        channel_count,
        blocksize,
    )
    stream.__enter__()
    streams.append(stream)

for i,mic in enumerate(mics):
    print('Stream %d id:%s'%(i,mic.id))
    print('Stream %d name:%s'%(i,mic.name))

while True:
    msg=''
    for i,stream in enumerate(streams):
        data = stream.record(blocksize)
        if np.sum(data)==0:
            msg+='Stream %d:paused'%i
        else:
            msg+='Stream %d:playing'%i
    print(msg,end='\r')

The script requires NumPy and SoundCard. You can test this script by pausing and playing a song, to see if it is also delayed.

[matanui159]

Can confirm that the provided script also has about a 3 second delay. Might be because I'm using a bluetooth device? Will test with other devices.

[matanui159]

Other devices have a very low delay

[matanui159]

Also this does not happen when specifically selecting my bluetooth headset in the Panon configuration, only when mixing all the streams.

[rbn42]

I guess I can't solve it. Maybe you can use the test script to open an issue in SoundCard's repo?

[Nano-Ocelot]

I have this issue as well on openSUSE Tumbleweed and I'm not using a bluetooth device at all. It's a wired headset and monitor only.

[rbn42]

I have this issue as well on openSUSE Tumbleweed and I'm not using a bluetooth device at all. It's a wired headset and monitor only.

Only if someone can fix the python script mentioned above (by fixing an upstream issue, or by fixing the pulseaudio configurations), will this issue be solved.

[yuannan]

Try out the new panon :) rbn42 and I have added a new feature to work around this.

I'm still curious to see why this was happening.

Try "pacmd list-sinks | grep latency" to show the latencies. Some devices will by default will use timed scheduling. aka dynamic buffer size.

You can set it to a constant size if that is the indeed the issue. It was for me on my USB DAC. When my DAC was not in use it would increase the buffer up to 300ms. I assume it's probably higher for bluetooth devices.
https://wiki.ubuntu.com/PulseAudio/performance

If this this cannot be fixed via mitigations I plan to make panon multi-process and fix this once and for all.

[ohPaco]

If i use my line-out analog stereo speakers i get about a 4 second delay from when i hear the music to when panon picks it up. this delay is significantly less using my usb headset as audio device.

[ohPaco]

Update - the delay isnt present when having my line-out analog stereo speakers.. it delays when "Mixing all speakers".

[yuannan]

Update - the delay isnt present when having my line-out analog stereo speakers.. it delays when "Mixing all speakers".

Howdy ohPaco. This is a known issue and requires quite a bit to fix. Try using "Monitor of Current Device". But this brings with it it's own caveats.

To fix this it will require a fix/feature to the underlying python library that addresses the audio elements. This can be seen here:

bastibe/SoundCard#125

In short. To fix this there is 3 parts.

1. Rig up a CFFI interface from the native Linux sound system via subscribe interface. An example can be found at "pactl subscribe". This however is a command line interface but thanks to the magic of FOSS you can view it here (https://gitlab.freedesktop.org/pulseaudio/pulseaudio/-/blob/master/src/pulse/subscribe.c)

2. The underlying library for panon is SoundCard. This interface will have to be rigged up to CFFI and merged into SoundCard which is written in Python.

3. ONLY now can we start to update the code in panon to be smarter in it's audio detection.

This unfortunately is a whole lot of work made by volunteers like me. Read as "unpaid dudes".

This is as far as I'm aware the "best" solution as it allows for better performance than right now. Which if you read the code is pretty much a hack parsing commands in a subprocess. Slow and prone to breaking.

As much as I would love to just "fix" this. There is a whole bucket load of problems that it brings. This fix is only for pulse audio. So people who use JACK or anything else are left high and dry. Not to mention that Linux as a whole is slowly moving towards Pipewire.

TL;DR:

It's simple in concept but actually a massive job that no one is paid for. By all mean I'm happy to collab if someone wants to help me, but when I say "in short". I meant it. It's quite a big job.