Pulseaudio "cracking" underruns after several hours of playback (no equalizer) or after minutes (with equalizer)

[Lizoffa]

Qubes OS version (e.g., R3.2):

R 3.2

Affected TemplateVMs (e.g., fedora-23, if applicable):

Dom0

(involvement of others possible)

Expected behavior:

Sound plays normally even if more than one source (that is, more than one AppVM) tries to play, irrespective of time played

pulseaudio-equalizer not expected to cause additional glitches or exacerbations

Actual behavior:

If more than one VM plays for appreciable time, sound starts getting lots of hiccups / crackles

Verbose pulseaudio log shows numerous
"requesting rewind due to end of underrun" messages followed by many (tens) "have to rewind" and "processing rewind" messages whenever a hiccup/crackle occurs.

Usually it may take a long time to arise

However, installing and activating
pulseaudio-equalizer.noarch module

triggers this issue within minutes of starting playback.

After the issue is triggered it does not subside without reboot, and restarting pulseadio in dom0 doesn't help.

Steps to reproduce the behavior:

1. have more than one VM play stuff for appreciable time (hours)

OR

1. have one VM playing AND have pulseaudio-equalizer installed and active

General notes:

Audiocard is a Realtek HD, built in.

---

Related issues:

[tonsimple]

Reproduced it easily in under 1 minute when using pulseaudio-equalizer and optical output of my motherboard's built in audiocard (motherboard is Asus Hero VIII)

Reproducing with analog output takes a few moments longer.

Even with equalizer thing uninstalled it eventually starts happening for both optical out and normal out, just takes a wee longer (way less than an hour on my config... maybe ~35 minutes)

P.S.: I've been having these clicks for a long time, but kind of assumed it was headphones approaching their end of life... so I guess this being a qubes+pulseaudio bug is kinda good news for me.

EDITED TO ADD:
Another neat way to trigger this behavior is to open SYSTEM TOOLS -> Pulse audio manager while playing back

Most of the times, clicking (and log spam about underruns and rewinds) follows as soon as the manager is closed and repeats every 1-2 minutes of playback.

It seems to me that it's particularly easy to trigger while watching a youtube video, but I have also "succeeded" in inducing it when playing back mp3s and video files.

[tonsimple]

Reproduced it easily in under 1 minute when using pulseaudio-equalizer and optical output of my motherboard's built in audiocard (motherboard is Asus Hero VIII)

Reproducing with analog output takes a few moments longer.

Even with equalizer thing uninstalled it eventually starts happening for both optical out and normal out, just takes a wee longer (way less than an hour on my config... maybe ~35 minutes)

P.S.: I've been having these clicks for a long time, but kind of assumed it was headphones approaching their end of life... so I guess this being a qubes+pulseaudio bug is kinda good news for me.

EDITED TO ADD:
Another neat way to trigger this behavior is to open SYSTEM TOOLS -> Pulse audio manager while playing back

Most of the times, clicking (and log spam about underruns and rewinds) follows as soon as the manager is closed and repeats every 1-2 minutes of playback.

It seems to me that it's particularly easy to trigger while watching a youtube video, but I have also "succeeded" in inducing it when playing back mp3s and video files.

[marmarek]

Does it by a chance happen at specific times: every 6 minutes (so, 0th, 6th, etc minute each hour)? If so, try commenting out line in /etc/cron.d/qubes-sync-clock.cron in dom0.

[marmarek]

Does it by a chance happen at specific times: every 6 minutes (so, 0th, 6th, etc minute each hour)? If so, try commenting out line in /etc/cron.d/qubes-sync-clock.cron in dom0.

[tonsimple]

@marmarek

Kind of float-y for me (+/- 2 minutes) but I'll try doing that to /etc/cron.d/qubes-sync-clock.cron and see what happens

[tonsimple]

@marmarek

Kind of float-y for me (+/- 2 minutes) but I'll try doing that to /etc/cron.d/qubes-sync-clock.cron and see what happens

[tonsimple]

@marmarek no cigar, still happening at approximately same rate

[tonsimple]

@marmarek no cigar, still happening at approximately same rate

[tonsimple]

Also, there is a ton of "memblock.c: Pool Full" messages in my pulseaudio log output. Like, thousands upon thousands. Not sure if it's related

EDIT

Tried the "tsched=0" thing many people in the forums suggest in different threads, no cigar, "hiccups and crackles" still occasionally happen, so it's not just timer-based-scheduler thing

EDIT 2

Uninstalling pulseaudio-equalizer and editing /etc/pulse/daemon.conf to have a slightly bigger buffer (default-fragments=4 and default-fragment-size-msec=60) has reduced the clicking very appreciably for me, about 2-3 clicks per hour (every time a click happens, log fills with "underrun" and "rewind" messages, so the issue persists, just at a less obnoxious rate)

[tonsimple]

Also, there is a ton of "memblock.c: Pool Full" messages in my pulseaudio log output. Like, thousands upon thousands. Not sure if it's related

EDIT

Tried the "tsched=0" thing many people in the forums suggest in different threads, no cigar, "hiccups and crackles" still occasionally happen, so it's not just timer-based-scheduler thing

EDIT 2

Uninstalling pulseaudio-equalizer and editing /etc/pulse/daemon.conf to have a slightly bigger buffer (default-fragments=4 and default-fragment-size-msec=60) has reduced the clicking very appreciably for me, about 2-3 clicks per hour (every time a click happens, log fills with "underrun" and "rewind" messages, so the issue persists, just at a less obnoxious rate)

[tonsimple]

It appears to me I might have a "hotfix/workaround" for this, particularly as regarding to equalizer, but I need some advice on qubes's pulseaudio-with-vchan module https://groups.google.com/forum/#!topic/qubes-devel/IWu6HLCyjPs

[tonsimple]

It appears to me I might have a "hotfix/workaround" for this, particularly as regarding to equalizer, but I need some advice on qubes's pulseaudio-with-vchan module https://groups.google.com/forum/#!topic/qubes-devel/IWu6HLCyjPs

[tonsimple]

Okay, I think I have found a workaround (valid, at least, for IEC958 optical out)

(for the record, I did not succeed in getting audio out of the VMs as anything other than s16le format, and still get an unsolicited click-clicky underrun every 45-120 minutes of continuous playback, but only if equalizer is present and active. Click-clicks are gone completely for non-equalized playback)

Some steps one has to do irrespective of whether they have equalizer:

Fist, one has to up the buffers in daemon.con in Dom0
nano /etc/pulse/daemon.conf

find lines

; default-fragments = 4
; default-fragment-size-msec = 25_

uncomment and set to

default-fragments = 7
default-fragment-size-msec = 25_

Next we need to kill all "smartypants" modules in templates that will be playing audio

Following lines need to be commented out in /etc/pulse/qubes-default.pa inside the template vm which is used for the "playback AppVM"

load-module module-intended-roles

load-module module-suspend-on-idle

load-module module-position-event-sounds

.ifexists module-cork-music-on-phone.so
load-module module-cork-music-on-phone
.endif
.ifexists module-role-cork.so
load-module module-role-cork
.endif

load-module module-filter-heuristics
load-module module-filter-apply

Now we need to kill those same modules in our Dom0 pulseaudio

nano /etc/pulse/default.pa

Find those very same lines as above and comment them out (I think that the cork line is slightly different in Dom0 , specifically, the "load-module module-cork-music-on-phone" and its attendant ifexists things are missing, but the general idea is the same - disable those modules)

Now reboot.

Done!

Now, the following step will depend on whether you intend to use the equalizer.

if you do intend to use it, edit Dom0 daemon.conf again
nano /etc/pulse/daemon.conf

find the line
; default-sample-format = s16le

uncomment it and change it to

default-sample-format = float32le

If you DO NOT intend to use the equalizer
edit Dom0 daemon.conf again
nano /etc/pulse/daemon.conf

find the line
; default-sample-format = s16le

and JUST uncomment it
default-sample-format = s16le

Don't forget to restart pulseaudio and all affected VMs (or better yet, reboot just to be sure)

P.S.
Also, when you instruct pulse equalizer to "save settings" it will create a new default pa file in user's directory in Dom0

~/.pulse/default.pa

And dom0 pulse will start using that instead.

Make sure all the "smartypants modules" are commented out there, too

Relax and enjoy click-free (or in case EQ is used, almost click-free) audio playback

P.P.S.:
pretty sure that if it was possible to get the VMs to send audio to Dom0 in float32le sample format, EQ-adjusted playback would become absolutely click-free too (but seriously, about one clicky-click per hour isn't too bad).

[tonsimple]

Okay, I think I have found a workaround (valid, at least, for IEC958 optical out)

(for the record, I did not succeed in getting audio out of the VMs as anything other than s16le format, and still get an unsolicited click-clicky underrun every 45-120 minutes of continuous playback, but only if equalizer is present and active. Click-clicks are gone completely for non-equalized playback)

Some steps one has to do irrespective of whether they have equalizer:

Fist, one has to up the buffers in daemon.con in Dom0
nano /etc/pulse/daemon.conf

find lines

; default-fragments = 4
; default-fragment-size-msec = 25_

uncomment and set to

default-fragments = 7
default-fragment-size-msec = 25_

Next we need to kill all "smartypants" modules in templates that will be playing audio

Following lines need to be commented out in /etc/pulse/qubes-default.pa inside the template vm which is used for the "playback AppVM"

load-module module-intended-roles

load-module module-suspend-on-idle

load-module module-position-event-sounds

.ifexists module-cork-music-on-phone.so
load-module module-cork-music-on-phone
.endif
.ifexists module-role-cork.so
load-module module-role-cork
.endif

load-module module-filter-heuristics
load-module module-filter-apply

Now we need to kill those same modules in our Dom0 pulseaudio

nano /etc/pulse/default.pa

Find those very same lines as above and comment them out (I think that the cork line is slightly different in Dom0 , specifically, the "load-module module-cork-music-on-phone" and its attendant ifexists things are missing, but the general idea is the same - disable those modules)

Now reboot.

Done!

Now, the following step will depend on whether you intend to use the equalizer.

if you do intend to use it, edit Dom0 daemon.conf again
nano /etc/pulse/daemon.conf

find the line
; default-sample-format = s16le

uncomment it and change it to

default-sample-format = float32le

If you DO NOT intend to use the equalizer
edit Dom0 daemon.conf again
nano /etc/pulse/daemon.conf

find the line
; default-sample-format = s16le

and JUST uncomment it
default-sample-format = s16le

Don't forget to restart pulseaudio and all affected VMs (or better yet, reboot just to be sure)

P.S.
Also, when you instruct pulse equalizer to "save settings" it will create a new default pa file in user's directory in Dom0

~/.pulse/default.pa

And dom0 pulse will start using that instead.

Make sure all the "smartypants modules" are commented out there, too

Relax and enjoy click-free (or in case EQ is used, almost click-free) audio playback

P.P.S.:
pretty sure that if it was possible to get the VMs to send audio to Dom0 in float32le sample format, EQ-adjusted playback would become absolutely click-free too (but seriously, about one clicky-click per hour isn't too bad).

[zamaudio]

Has anyone tried using the superior JACK server instead of pulseaudio for audio in Qubes OS?

[zamaudio]

Has anyone tried using the superior JACK server instead of pulseaudio for audio in Qubes OS?

[tonsimple]

@zamaudio
While at this point I hate pulseaudio with a dangerous passion, I gotta admit trying to cut it out of Qubes would likely bring more issues than it solves, at least inside appVMs, because TBH a lot of "end user" software heavily relies on pulseaudio.

Whether "fully grown pulse" is something worth having in Dom0 is an open question, but something tells me that having Pulseaudio in appvms and Jack as Dom0 backend is also liable to break in funny ways.

P.S.:
I have experimented with equalizer further and found that adding a loopback from equalizer's monitor to a null sink changes the character and rate of incidence of underrun clicky-clicks in a very interesting way. Experiments ongoing.

[tonsimple]

@zamaudio
While at this point I hate pulseaudio with a dangerous passion, I gotta admit trying to cut it out of Qubes would likely bring more issues than it solves, at least inside appVMs, because TBH a lot of "end user" software heavily relies on pulseaudio.

Whether "fully grown pulse" is something worth having in Dom0 is an open question, but something tells me that having Pulseaudio in appvms and Jack as Dom0 backend is also liable to break in funny ways.

P.S.:
I have experimented with equalizer further and found that adding a loopback from equalizer's monitor to a null sink changes the character and rate of incidence of underrun clicky-clicks in a very interesting way. Experiments ongoing.

[zamaudio]

I installed Qubes 3.2 and disabled pulseaudio, then used usb passthrough to an appvm running JACK on my usb sound card.. it kind of works, but when the machine is under heavy load, it crackles and pops.

[zamaudio]

I installed Qubes 3.2 and disabled pulseaudio, then used usb passthrough to an appvm running JACK on my usb sound card.. it kind of works, but when the machine is under heavy load, it crackles and pops.

[zamaudio]

@tonsimple i am glad you are not a fan of pulseaudio, but I disagree that a lot of "end user" software relies heavily on it. Take, for example, the leading free software DAW, Ardour. It doesn't even have an option to use pulse, because you simply can't do realtime audio properly with pulse. This isn't really the right channel to discuss which sound server we should use or not, but I do want to point out that I am creating an alternative backend for Qubes audio that uses JACK. See https://github.com/zamaudio/qubes-jack 👍 I could use some help nutting out how best to design it, but I think I have a working server side.

[zamaudio]

@tonsimple i am glad you are not a fan of pulseaudio, but I disagree that a lot of "end user" software relies heavily on it. Take, for example, the leading free software DAW, Ardour. It doesn't even have an option to use pulse, because you simply can't do realtime audio properly with pulse. This isn't really the right channel to discuss which sound server we should use or not, but I do want to point out that I am creating an alternative backend for Qubes audio that uses JACK. See https://github.com/zamaudio/qubes-jack 👍 I could use some help nutting out how best to design it, but I think I have a working server side.