-
Notifications
You must be signed in to change notification settings - Fork 69
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Latency increasing over time #51
Comments
You are "appending" which means you are reallocating the memory at every iteration. After a while, the memory blocks becomes larger and larger and larger and it takes more time to reallocate them. This is not a good practice at all. The memory allocation should be managed by a separate process. Read this article of Ross Bencina, it should help you how to and how not to deal with audio and syscalls. |
@Chum4k3r That's not actually the case in the code above: The code is using lists, which can be appended to without penalty. Numpy arrays would be a different matter. This might be a buffering issue. Try It might also be a performance issue. If pulseaudio repeatedly notices that your code doesn't provide/consume audio data on time, it can decide to raise the internal block size. You could also time the execution of |
Any update on this issue @tetelestia ? |
@mpariente Not really. I never found any reliable audio streaming on Linux and am focusing all on mobile development now. I tried the fix bastibe suggested, about removing the specification I'd be very interested if you do find a solution though. The closest I got was with JACK Audio Connection Kit, but that had catastrophic failures when processing time on my frame was greater than the buffer size. |
I read your code too fast at first and I didn't notice that the frame size you want and the block size you ask from pulseaudio ( Also, I'm not sure you understood the role of Finally, I did manage to get reasonably stable audio streaming with |
Did you find a solution to your problem? |
First of all, many thanks for this great library. It's a real relieve after using other audio libraries in Python. Anyways, I have a similar issue with recording and playing at the same time (like a loopback). I'm using a null-sink as a test speaker to play the audio that I record in real time with a microphone. When I run pulseaudio with -vvvv flags, I can see messages like the following in pulseaudio's debug log:
As you can see my block sizes are really low to achieve real time. When I initialize the player like
If I buffer too many frames upfront, then there's too much latency for the player because it never catches up. I've seen in the C library, there's a function to find out how much data can be sent to a speaker. I guess I'd need this exposed to not get buffer underruns. Apart from all that, the suggestion from @mpariente does work for me, so just dropping frames when the player is too slow. Unfortunately it's slower than it would be to be without this rewinding and hanging for 2 seconds. Anyways, with dropping frames, after those initial 2 seconds, I can catch up and everything runs smoothly. Just not in the first 2 seconds. PS: Using Debian 11 Linux. |
Thank you for your kind words! It might be that pulse is not honoring your requested block size. There's a Also, you can query pulse at any time as to how many bytes are available to read or write with |
I stumbled upon similar problems, when I used SoundCard in a real-time application. Whenever the process encounters a buffer underrun, Pulseaudio increases the internal buffer, till the latency exceeds 2 seconds or more. I used a quick hack to set a hard limit to the Pulseaudio backend, since in my case occasional buffer underruns were acceptable but the ever increasing latency was not. If not for personal reasons, I would have followed up with a proper solution and a merge request. However, I hope the linked commit points you in the right direction. |
I also experimented with a try-record function, which instead of waiting returns None, if there are not enough frames in the Pulseaudio buffer: I'm not sure though, if this helps with yout problem. |
Thank you for your input, @szlop! I wonder how the try-record function is functionally different from If I understand this problem correctly, it is that recording returns just a tiny bit too little data for playing. Is that correct? Or is it just a mismatch of block sizes (which could be solved with caching). The former is really unsolvable in soundcard itself, but quietly appending a frame or two of plausible data to the recording would probably work around the issue without too much trouble. The latter would need some more engineering as a solution. |
The only difference is, that |
@szlop Thanks for your hint. My problem is actually not with the record function but with playing. Also I also cannot observe that pulseaudio would automatically increase the latency. It doesn't do that in my installation but maybe it's also related to the type of device. I'm playing to a virtual null-sink device. @bastibe Thanks for your help. I had found the play function and tested around a bit. In https://github.com/bastibe/SoundCard/blob/master/soundcard/pulseaudio.py#L751 it tries to find out using how many bytes are writable, just as you've said. It does I assume nwrite should be the number of frames that are writable, thus dividing by 4 bytes of a float32 but it doesn't consider the channels. It tried to write too many bytes to pulseaudio, more than it says that are writeable. Then again, this has no impact at all because according to the pulseaudio docs
SoundCard sets maxlength to a really high value, so maybe the while loop in the play() function is not necessary at all? I've corrected the nwrite calculation in a local test but it unfortunately doesn't improve the situation with the initial hick-ups and rewinding for me. PS: I've also monitored the latency for the player and recorder using the SoundCard feature and they give really low values until the player hangs. The recorder is constantly < 1 ms. The player < 10 ms. Only in the beginning, after the first one or two writes, the playing hangs (due to rewing) and it gives 2000 ms and afterwards goes down to < 10ms again as I'm dropping frames. |
Thank you for your analysis. Good idea that pa_stream_writable_size might need to be divided by the number of configured channels! That might be true, and might even explain the latency hikes. The maxlength used by SoundCard is (if I remember correctly) pulse's "default" value. It just means that pulse may freely choose. Does the situation improve if this is set to a fixed value? |
Here's a link to the PulseAudio docs: https://freedesktop.org/software/pulseaudio/doxygen/structpa__buffer__attr.html It says to get the default you should set the value of maxlength to I've tested around a bit more and I've solved my problem by removing the loop from the play function like
Now I can record and play (so loopback) in realtime and there are no buffer underflows or overflows, also not in the beginning. I now also suspect the
in the original version with the loop is causing the issue for the original poster. This 1 ms will slowly add up given the calculation of nwrites. I think under no circumstances it is a good idea to add 1 ms there from nowhere. It will automatically cause latency issues if you are playing real time because that 1 ms of audio cannot appear from nowhere, so in the best case with all the protection in PulseAudio (the rewinding stuff) it will cause some cracks in the output. What I don't understand yet is why it didn't work for me when I had corrected the nwrites calculation but it only works now after I've removed the loop completely and write the whole data at once, no matter how much is writeable. I rely on the really huge maxlength and my blocks are really small anyways. |
Hmm, ok, I was wrong about removing the loop solving my problem. I have no 2 s delay but therefore the overall latency goes up in the interface. I'm afraid I'll need some more testing. |
It's really mysterious. This morning I come back and I run that same test script from yesterday evening and everything works smoothly without any changes. I kept the computer running, it's all the same code, at least as far as I am aware, but still the results are much better. |
That's the same. In C you'd write Regarding your point about the millisecond sleep: I think the common use case is to either play or record. If you just want to play audio, waiting is the right thing to do. But in your case it is not. Actually the same thing is implemented for recording as well, where the default case waits until the requested number of samples is available. But for recording, we also have the the non-blocking record (with |
I'm having difficulty keeping consistent low latency while streaming audio (input and output) with SoundCard. It appears as if something occurs periodically causing a small bump in latency which is never recovered. Flushing the microphone has some effect, but in the long run, the latency continues climbing upward. Has anyone encountered this problem, or know the cause of it?
Below I've posted a scatter plot showing the increase in latency over time.
I'm using Ubuntu 18.04, kernel 4.15.0-46-lowlatency, SoundCard version 0.3.2. Code to reproduce this plot is posted below. I've managed to replicate it on 2 different computers, although both Ubuntu 18.04, with the lowlatency kernel.
The text was updated successfully, but these errors were encountered: