-
-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Audio crackles even with high latency, worse on some cores #8235
Comments
mGBA has known audio issues so I would not bother playing with that core when you are investigating these problems. What are your hardware specs? What is your monitor resolution and refresh rate? Always 60 or have you tried others? Have you tried 1.7.5? What audio driver(s) have you tried? Have you tried different sound cards? Some of them just have terrible latency that is unavoidable. Do you experience these issues with the default latency of 64ms? You might need the wasapi audio driver on Windows to go lower than that. I would also try with fresh configs (move/rename retroarch.cfg and retroarch-core-options.cfg). |
My apologies, it seems that I was a little hasty to pin this issue on Retroarch. Would you believe me if I told you that PulseAudio was the problem all along? When you mentioned sound cards and drivers, it reminded me about the PA options for fragment sizes. After changing Changing the fragment sizes in So, this issue is probably related the timer-based scheduling used by default in Pulse, and also in Windows since Vista (and also MacOS, apparently). RA should work correctly regardless of the audio scheduling method, but for now, reverting to IRQ-based scheduling is a good workaround. I can test some of these other variations later, but I'm going to be busy for a few days and won't have time to play with audio drivers. I'm running an FX-6350 @ 4.5 GHz (hardcore stuff I know). I have a 60 Hz LCD display as well as a CRT that can go up to 120 Hz (I still run it at 60 Hz for RetroArch, but I can test at other framerates). I'm using a USB-connected desktop DAC for audio (Behringer UCA202) so there shouldn't be any HDMI audio driver issues. |
If you still experience problems I would make sure to try with only a single monitor. Having multiple connected (especially with different refresh rates) has been known to cause audio issues as well for some people. |
For what it's worth - and I am not sure it is related specifically to the issue that you described here - I have had quite a few audio issues with the libretro core version of mGBA ever since version 0.5. For example, you cannot set the audio latency below the default rate of 64ms without triggering a lot of hitches, pops, crackles and overall stuttering, while with version 0.4.1 (I have a separate core I compiled from the 0.4.x branch) I can drop audio latency all the way to 16ms on certain titles and around 20-24ms with most content. The audio is probably not as clear or accurate, but audio latency performance is far, far better. @vanfanel pointed out the same issue here, indicating that it could be due to a bigger than normal audio buffer: libretro/mgba#109 Others are encountering similar problems with Audio Sync on the latest mGBA core: libretro/mgba#134 |
I had PA set up slightly wrong earlier—the fragment number/size options do work, and can be used to change the latency globally. I tested a few cores (not rigorously, just one game each) on my system to see how low I can push the latency. Using fragments of 1 ms each:
On top of this, mGBA's behavior is different from the others when there isn't enough buffer. The pops sound louder and have more body (I suspect this is because they last longer), and the entire core slows down if the buffer is small (4-8 ms). Both of these indicate that mGBA's audio implementation is a little screwy. However, if I run the same tests with timer-based scheduling back on (
Cores need 2-3 times their normal latency, except for genesis plus GX, which somehow manages with 0. So definitely some core-to-core variation, but I would expect these numbers to match the first set. My intuition tells me that these numbers should all be a little lower, but that might be wrong. So there seems to be two issues: (1) mGBA audio stuff, and (2) RetroArch doesn't play nice with timer-based audio scheduling. My pet theory is that, because timer-based scheduling will dynamically change the latency/buffer size, the way that audio samples are generated makes a difference. If they come at a steady cadence (genplus?), tsched should hit one underrun, back off by 10 ms, and then hold steady. But if a large number of samples are dumped at once—say, once per frame or so—then maybe tsched keeps tightening latencies while the samples are available, underruns and backs off, then gets greedy again when another batch is available. As a result, you have to tell PulseAudio to start with an overly-large buffer to "bias" it away from underruns. These results also give me the feeling that RetroArch's audio task has a lower priority than the video-rendering task. It explains why the latencies for bsnes, genplus, and beetle are about half to 3/4 of a frame—it fills the buffer, almost depletes the buffer while rendering a frame, then refills it at the last second. This prioritization makes sense for large latencies (> 1 frame), but once the deadline for audio becomes shorter than video's, we would ideally want these priorities to be swapped. If a core buffers some amount of audio internally before flushing it out to RA in chunks (mGBA?), this would increase the worst-case latency. Comparing the audio code in genplus vs. mGBA could probably give some insight into both issues. Also, if anybody knows more than I do about RA's task-scheduling paradigm, I would love to hear about it. It would be interesting to measure the actual audio latency—is 64 ms actually 64 ms?—but I don't have the equipment to do so. Not actually sure how to do that. @bparker06 I only output to one monitor while I'm using RA, so I don't think that's a factor. I don't physically unplug my LCD, but I use xrandr to disable it ( |
This issue can be relevant: |
reverted, was causing problems for GB, we just changed the SAMPLES to 512 instead. |
I would like to confirm that this is still an issue. While setting up RA on a new system (much more powerful than my old one), with a freshly-installed operating system (Kubuntu, no longer Mint), I encountered these same crackles, but much worse than I remember. They occurred in all cores I tried (mGBA of course, also bsnes and Genesis Plus GX). Raising the audio latency setting in RA upwards of 100 ms did not resolve it completely, or even nearly. I now have an adaptive-sync monitor, so I am using the "exact content framerate" option, but otherwise my settings are the same. I was able to attain a working setup using a similar method as before, but my settings from earlier were insufficient; more buffer/latency was required. This backs my perception that the issue has worsened. If anybody has this issue as well, my revised settings are:
|
It turned out that when cores submit audio in multiple steps, audio sync results in crackling with low audio latency settings. Some cores have been fixed now and they result in clean audio even at very low latency settings. The ones I know of that are now fixed are snes9x, mgba, puae, vice, and swanstation. |
Description
Highly audible "pops" and "crackles" occur while a core is running, and can only be remedied by raising the audio latency to excessive levels. This is most prominent by far in mGBA out of all cores tested, but can be reproduced in other cores as well (albeit at lower audio latencies).
Expected behavior
The audio output from Retroarch sounds the same as on the original console, unless the core is unable to run at full speed. The audio is imperceptibly time-stretched to sync with the video and resampled to the desired output sample frequency without distortion. Audio output from the core in use must be buffered for some length of time to accomplish this, inducing latency; however, a buffer of only a few milliseconds should be sufficient for cores which are not very demanding on the hardware in use.
Actual behavior
The audio output from Retroarch is heavily distorted unless the audio latency is set strangely high. The threshold varies by core and by content, and is not always consistent across repeated runs. Mother 3, running on mGBA, usually requires 48 ms minimum but on occasion has required latencies in excess of 80 ms. 20 ms is usually sufficient for most cores, but all mGBA games tested exhibit this audio crackle to some extent at the same latency.
This issue does not affect recorded "video dumps", which contain perfect audio at the emulated system's native sample rate, indicating that the core's output is not at issue here. The distorted audio, however, can be captured with Audacity by monitoring an audio output (e.g. my desktop DAC).
This particularly unlucky run of audio samples shows the full breadth of known failure modes. It affects the left and right channels at the same time. Most of the time, the audio simply "drops out" for a short, but varying length of time. Sometimes the signal appears to have been momentarily "paused", and resumes at the same level it stopped at, but other times it skips either ahead or behind to a completely different level, as if the second chunk of audio has been "time shifted". Additionally, two sections of audio seem to be "spliced" at 27.780---it seems that the audio was abruptly time-shifted without dropping to zero at all. Most of the audio isn't this bad, but these same patterns persist.
I have reproduced this behavior on Linux Mint 19 and Windows 10 on the same machine. The crackle is less severe in Windows, and latency can be lowered a bit compared to Linux, but remains in excess of two frames (32 ms minimum from light testing).
The issue seems to have nothing to do with the core's CPU utilization. The English-language patch of Mother 3 crackles heavily on the main menu screen, making it easy to test with, even though the core can run at approximately 700 FPS in fast-forward mode. Megaman X also exhibited crackling on bsnes-mercury accuracy, but not in any level---only the level select screen, which can also easily run at or above double speed in fast-forward mode.
I have tried changing the output sample rate in RetroArch (48kHz -> 44.1kHz), and I have tried increasing the dynamic audio rate control and the max timing skew (to 3x default values), but without any results. Enabling 0-frame hard GPU sync (ironically) requires additional latency, but I found the worst results (crackling at 64 ms) by reverting to a default config file. This might have been a fluke. My monitor's measured refresh rate converges to 60.000 ± .002 Hz, so my vertical refresh rate is set to 60.000. Changing this value causes more crackles, as expected.
I'm no expert in scheduling theory, but I do know a little bit of it. If memory serves: even with optimal task priorities, the CPU might still miss deadlines, even if it is often idle. I wouldn't be surprised if the audio began to stutter when the system is near full load, or if there was a rare, intermittent gap under moderate load. However, when running such a lightweight core that the system is approximately 90% idle, I would expect there to be enough time to top up the audio buffer at least once per frame, let alone just once every second or third frame.
Also, on the off chance that this is the intended functionality of the audio system, and I'm just setting the latency too low...isn't audio latency just as important as video latency? RetroArch has made pretty amazing progress in the second category recently, and I'd be happy to see audio get some of the same love.
Steps to reproduce the bug
Log file (default config, just let Mother 3 run for a minute; crackles abound)
Bisect Results
I have rough memories of this issue since years back, but never dug into it until now. I suspect it's a long-running feature of the audio system, but if there were any major updates to it historically, I would be happy to test out an earlier version. I can give the latest nightly a shot too...maybe tomorrow.
Version/Commit
Environment information
The text was updated successfully, but these errors were encountered: