Random short audio dropouts when recording or streaming

[floele]

Operating System Info

Windows 10

Other OS

No response

OBS Studio Version

27.0.0-rc2

OBS Studio Version (Other)

No response

OBS Studio Log URL

https://pastebin.com/WfdNU2QN

OBS Studio Crash Log URL

No response

Expected Behavior

Audio recording without dropouts.

Current Behavior

When recording or streaming, audio data contains multiple random short audio dropouts.

Looks like this:

Distribution of the dropouts seems to be random but happen at a certain rate depending on which computer it is being run on. On my computer it’s approx. every 5 minutes, I have seen it on another computer where it happens roughly twice per hour.
For a 40 min. test recording on my computer, dropouts have occurred at these time stamps:
00:03:19
00:09:40
00:15:38
00:19:51
00:23:41
00:29:55
00:36:07
The length of the dropouts are not necessarily all the same length.

Steps to Reproduce

1. Launch OBS.
2. Start test tone (I use https://www.szynalski.com/tone-generator/).
3. Record desktop audio using OBS (all default settings) and wait.
4. After stopping the recording, audio dropouts can be found in the resulting file.

As indicated above, the frequency of this issue seems to depend on hardware and may either not occur at all or very rarely on some systems, I'm not sure what it depends on.

The issue can also be reproduced without using a test tone of course, but a continuous tone makes it easier to check the resulting file for dropouts.

Anything else we should know?

Conditions

I have been testing on Windows 10 20H2 with OBS 26.1.2-276 and 27.0.rc2, 64 bit, without any video source configured (issue also happens with video configured though).

The issue seems to be independent of OBS settings (like audio sample rate or output format). I have not yet found any setting in the OS (like energy settings) that fix or reduce the issue.

The frequency of the issue seems to depend on the hardware. There are systems where I was not able to reproduce the issue, though I did not make excessively long recordings (> 4 h) to eliminate the possibility of the issue occurring on these systems.

The issue is not limited to recording from desktop audio but also applies when recording from external USB audio interfaces.
The issue is not limited to recording but also applies to streaming.

When an audio dropout occurs, the message “can't discard, data still pending” is seen in the debug output (if compiled with DEBUG_AUDIO).

The attached log file includes a debug message (Capture size: 480) I added, it outputs the captureSize variable in ProcessCaptureData() to check for any changes in amount of data being returned there.

Debug attempt

I’m not familiar with the OBS code base and the audio processing happening, but to me it seems like the problem is the fact that in audio_callback(), it always takes a fixed amount of data from the audio buffer (audio_size = AUDIO_OUTPUT_FRAMES * sizeof(float)) even if the buffer is not sufficiently filled due to potential delays in the capture thread.

What I have tried to solve this issue is basically waiting for the buffer to be filled, like this in audio_callback.

		if (!source->audio_pending && source->audio_ts > 0) {
			int iterations = 0;
			while (source->audio_input_buf[0].size < 4096) {
				blog(LOG_DEBUG,
				     "waiting 2 ms for audio buffer to fill (%s)",
				     obs_source_get_name(source));
				os_sleep_ms(2);
				iterations++;
			}

			if (iterations > 0) {
				blog(LOG_DEBUG,
				     "waited %i iterations for the audio buffer to fill",
				     iterations);
			}
		}

This significantly reduces the number of dropouts (on my system to like 1 in half an hour), but it’s not a good solution because additional to not entirely fixing the issue it doesn’t recover from the situation and adds continuous wait cycles after the first dropout that far exceed the number of dropouts that would have actually occurred. Also it seems kind of wrong to me doing that in the first place.

I would appreciate some guidance or a fix for this issue.

[WizardCM]

To confirm, this issue is not new in 27, as you've verified it in OBS 26.1.1?

[floele]

Yep, I have been investigating this in 26 already and only confirmed that it's still there before reporting.

[DJ-3ncoder]

I found a simillar or maybe the same issue in my latest stream recordings.

My software and hardware:
OBS 27.0.1
Windows 10 64bit
Focusrite Clarett 4Pre USB

My signal chain: Pioneer DJ Mixer puts out 48kHz signal via S/PDIF into the Focusrite Clarett.
The soundcard gets the S/PDIF clock from the mixer and is locked to it.
Every soft- and hardware is set to 48kHz.

On the top there is the recording done with Audacity and on the bottom there is the audio-track from the recorded video.
Recording is set to "stream encoder" (nvenc), mkv and remux to mp4 afterwards.
The drop-out occurs in the mkv and mp4.
Both audio-tracks are recorded using the same interface on the same computer. Audacity looks fine but OBS is struggling.

After scrolling through the waveforms I found this:

To get the first picture I aligned the waveforms after the drop-out. The second picture is before the audio-drop out.
You can see that the waveforms are not aligned. The bottom waveform (=OBS) is earlier in time.
After seeing that I was thinking that the drop-out is more like a "break" in the signal because when you delete it and move the bottom waveform to the right, both are aligned all the time.

These drop-outs or "breaks" happen randomly. I do not click anything because in the video everyone can see that I am not in front of my PC.

Due to the fact that the Audacity recording is fine on the same PC I think this issue is caused by OBS.
I do not know if it occurs in the stream too, nobody is listening that carefully.

Log-file: https://cloud.3ncoder.com/s/nsS7n98QYED9Tnr

[konsolenritter]

Unfortunately i must confirm. Still using obs 26.1.1 under win10. All sound devices disabled but a steinberg ur-22. External playback of 1khz tone. i7 10th generation nearby idling.
With recording or streaming the same experience: By chance (random) happen such short dropouts. Sometimes once within in a couple of minutes, sometimes two dropouts within 20 seconds or even less. Randomly distributed.
The try/workaround as of user floele i find very interesting.

Due to the nature of live events in the case of streaming people tend to be forgiving alot, especially in the case of ultra-short outages (its almost anticipated these days). But even then a short cut on audio is more distracting then a few lost frames. So ever stable audio wins (and is more important) over stable video for normal broadcast.

Would be unbelievable happy if that nasty thing could be fixed.

[floele]

Unfortunately I did not have time too look into the issue futher, just wanted to share the last thing I found out.
It looks like (at least on my PC) I'm dealing with two different kind of dropouts. A frequent one (occurs roughly every 5 minutes) and an infrequent one (maybe each 30 min.).

I was able to fix the frequent one by changing the sleep method to a more precise alternative (example commit floele@a34fda1), but the issue I actually want to fix is the infrequent one because it reproduces on the machine I'm concerned about (and many other machines as far as I know). This one does not seem to trigger a “can't discard, data still pending” debug message so at this point I'm not sure when it is triggered or what is causing it.

[konsolenritter]

Is anybody else working on that issue here?
My C or C++ is fifteen years ago..., so there is a little rust around. ;)
I could imagine to invest a couple of hours into the obs-audio world over the next few weeks. My problem is, that i don't have a working environment to test and compile for windows here by myself. I'm totally fan of free software, but when it comes to ms studio or something alike... shiver

[DJ-3ncoder]

I found out that the problem that I am facing occurs at around 1h 7min of stream/recording time. It occurs only once but everytime at the same time!

Audacity recordings are fine. Changing the buffer size in the Focusrite panel makes no difference.

Maybe I can find out how to change log level to debug or so to find out what happens at 1h 7min.

[bnacar]

It's not just on Windows. This has been happening to me during livestreams and recordings on both MacOS and Linux, on average once every ten minutes. Just now I was able to replicate this after compiling from the latest source on Linux (Debian Stable). With OBS and Audacity recording simultaneously for fifteen minutes, I got two dropouts in the OBS recording and none from Audacity.

@floele were you ever able to pinpoint the cause of the less-frequent dropouts?

[floele]

Unfortunately I have not yet looked into it any further...it's still bothering me though.

[DJ-3ncoder]

Since the beginning of 2022 I tried the ASIO plugin for OBS.
I set my input accordingly and the drop outs are still there.

I also tried on a different computer (but the same soundcard). Same result, 1 drop out at timestamp 01:xx:xx of recording.

I am still shure that this bug is caused by OBS itself and not by my soundcard. It annoys me very much.
Is there a way to change log level in OBS? I want to find out what is happening at that time.
Normal loglevel does not log that drop out.

[floele]

If you combile OBS yourself you can enable the DEBUG_AUDIO flag for more logs.

After fixing one of the issues (the frequent dropouts) there weren't any relevant logs left corresponding to the occurrences of the issue on my PC though. So it's likely that these logs won't help you at all.

I also tried ASIO without success.

[konsolenritter]

Hi floele, what is the toolchain needed to compile obs myself for/under win10? How much effort? Is there a clear documentation for this? My IMHO feeling is that the problem is bound by the amount of samples obs reads each cycle in the audio thread from the samplequeue, and how this thread is triggered. Is it a fixed timing, a kernel timer based, or something else? Do you know something about? Greez, #timo Am 10.02.2022 um 23:07 schrieb floele:

…

If you combile OBS yourself you can enable the DEBUG_AUDIO flag for more logs. After fixing one of the issues (the frequent dropouts) there weren't any relevant logs left corresponding to the occurrences of the issue on my PC though. So it's likely that these logs won't help you at all. I also tried ASIO without success. — Reply to this email directly, view it on GitHub <#4600 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ANLNV47JOTGCLW2Y2UFQXKTU2QZINANCNFSM43TMLWEQ>. Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you commented.Message ID: ***@***.***>

[floele]

Yes, there are build instructions available. I would link to it, but apparently there is currently some reorganisation happening and the page asks not to link to it for the moment. So just look for it on Google ("Build Instructions (Windows) - OBS Studio").
You basically only need VS 2019 with C++ and install some additional tools (like QT). It's doable.

As for the issue, I've been focusing on https://github.com/obsproject/obs-studio/blob/master/libobs/media-io/audio-io.c and https://github.com/obsproject/obs-studio/blob/master/libobs/obs-audio.c but I'm starting to believe that maybe the issue might be at a later point in execution, maybe the muxing process.

The reading is basically sleep() controlled as far as I can see. As per my previous comment, this seems to be unreliable on my hardware so I replaced it with a different method, but that doesn't solve the problem everyone is having.

[Fenrirthviti]

apparently there is currently some reorganisation happening and the page asks not to link to it for the moment.

Where exactly are you seeing this comment? Our official build directions are on our Wiki pages here: https://github.com/obsproject/obs-studio/wiki/install-instructions

There is a pending cmake rewrite that will change how some things work at a high level, but the overall process should be largely unchanged.

[bnacar]

I tried adapting @floele's solution - of making the sleep time in the audio thread more precise - for Linux, and for whatever reason, it didn't affect the frequency of the dropouts (on my system anyway). I did several other experiments and this what I observed on my system:

The vast majority of audio dropouts that I've encountered are indeed correlated with the "data still pending" message, and these dropouts have a duration equivalent to whatever I set AUDIO_OUTPUT_FRAMES to. Using Audacity to visually inspect the recorded audio, the waveforms at the beginning and end of these dropouts line up, i.e. audio data is getting delayed, not deleted, and delayed by just about one sleep cycle.

In addition to making the sleep time in audio_thread more accurate, I tried setting AUDIO_OUTPUT_FRAMES to 768, which corresponds to a nice round number of 16 milliseconds when recording at 48kHz. Unfortunately this didn't make any difference either.

I'm not familiar with what sequence of events will deterministically result in a call to add_audio_buffering, but I tried modifying this function to add an absurd amount of buffering when it does get called:

- ticks = (int)((frames + AUDIO_OUTPUT_FRAMES - 1) / AUDIO_OUTPUT_FRAMES);
+ ticks = (int)((frames + 3 * AUDIO_OUTPUT_FRAMES - 1) / AUDIO_OUTPUT_FRAMES);

On the occasions that add_audio_buffering does get called, the additional extra buffer results in an initial long stretch of recording (on the order of 20 to 30 minutes) completely free of dropouts, but over that time the buffer gradually shrinks until there isn't enough data available, and then the dropouts resume.

In short, audio data is being output slightly faster than it gets added to the buffer. If this discrepancy is unavoidable, the permanent solution might be closer to the original proposed fix of adjusting the sleep time on the fly with extra wait cycles (I haven't tried this myself yet).

[floele]

@Fenrirthviti Ah, thanks. That is the page I was looking for. I got to this page instead: https://obsproject.com/kb/build-instructions-windows

[konsolenritter]

@floele Thanks! Thats alot of stuff. And i hoped to get around VS... sigh

@bnacar Thanks for your significant investigations regarding the buffer! Regarding your observation of "delayed" audio in audacity: I think its a case of post- and pre-echo due to the typical side effects of windowing functions in lossy encoding like aac.

[UPDATE: The written below is still true. But i've re-read bnacar's post and realized that a different observation was meant: Yes. The audio data lines up around the gap, not being discarded.]

In the image above the upper screenshot is from a short dropout (simulated by myself by cutting a -12dB sine) in uncompressed, and below after re-reading in an exported aac version (160kbps) of the above. Lossy compression like in movie audio always shows up such ghosts at sharp signal changes, especially dropouts.

So i'm sure our disaster is not bound to the mux'ing stage, because all encoding happens before. The muxer doesn't break or decode the single streams again (AFAIK).
[UPDATE: end of objection]

In short, audio data is being output slightly faster than it gets added to the buffer. If this discrepancy is unavoidable...

Personally i think this is exactly our problem here: rarely buffer-underruns. And the bad news is: such discrepancy can't be avoided. Sleep- or timer-interrupt based threading is bound to the crystals of the board and cpu, while incoming audio (despite observed timing delays at usb stage or so...) is always bound to the crystal of the sound device or even external clocking. A clear sign of this would be that everyone of us experiences such dropouts after differnt runtime with different probability and interspacing of such dropouts. Because everyone of us has different combination of pc/mac and sound-card/-device.

I want definitely to get more into this issue (because i hate it that much) but i've had no time yet to get the whole process of setting up the compilation workflow these days (too much job-based trouble)... sniff

[konsolenritter]

@DJ-3ncoder

I found out that the problem that I am facing occurs at around 1h 7min of stream/recording time. It occurs only once but everytime at the same time!

According to my theory it should happen again around 2h 14min...? ;)

[bnacar]

@konsolenritter sorry for the confusion, I should have included a screenshot:

Those ghost artifacts that you were talking about do show up at either end of the dropout, so the glitch in the signal does occur before any compression. The alignment of the waveform before and after the dropout is thus approximate, but consistent.

[floele]

@konsolenritter Thanks for the hint, I have done some further debugging and can confirm that the issue occurs before encoding and muxing.

I have further noticed that the issue seems to occur whenever audio data is written to the puffer using circlebuf_place() instead of the regular push_back(). I'll try to investigate further when and why this happens, but this call seems to be reponsible for putting a NULL-gap into the buffer, which is then subsequently read and causing the issue.

[konsolenritter]

Where have you seen such ...place() calls?

[bnacar]

Also just to clarify wrt the circlebuf_place() calls, are you referring to the "data still pending" dropouts or the more-elusive less-frequent ones?

[konsolenritter]

"Both" of them (for me. Or did you mentioned floele?). I'm still not into the code yet. I proofed (like for a blackbox) what's happening if i tweak the technical (external) sampling rate a bit. Lowering the external 48k (in case of 48k) by 0.1% or 0.4% already brought up the dropouts on a regularly basis, every couple of seconds. It even doesn't need streaming or recording. I can hear it in the monitoring device, already. That points to the generic parts of audio sources ticking and mixing in the general audio thread, too (IMHO). I'm about to upload a vid showing this (possibly today already) to YT.

Sometimes (not in every case) a "data still pending" shows up in between. So i can't distinguish between the two cases yet. Sorry for that. I'm still in the phase of "what needed to produce such dropouts", or, "what-adjusting-screw-belongs-to-the-occurrence/frequentness-of-dropouts"... (leaving the pc/sound-device combo unchanged).

[floele]

In source_output_audio_data() there is a call to source_output_audio_place() which is called under certain circumstances. I'm still not entirely sure what those are. Possibly some timing issue with being more data on the buffer for a moment than expected, so it tries to put the next audio data further to - or rather beyond - the end of the input buffer. Maybe there is someone else who can explain the logic in source_output_audio_data() properly. It compares some timestamps and then possibly doesn't set the variable push_back to true.

So what I have observed is that data is added to the buffer so that there is some empty space in the middle. Like [11111111111111111100000000000000002222222222222] with 1 being the current data on the buffer, 0 the empty space in between and 2 the new data being added to the buffer.

So we have this state in source_output_audio_data() (notice the huge diff being > TS_SMOOTHING_THRESHOLD and thus not making a push_back happening):

And this buffer in source_output_audio_place(), buf_placement being 21692 in this case, so quite a bit beyond the buffer end (while resizing the buffer, the empty space is filled with zeros).

My proposed fix (though without understanding the intentions it might only be a workaround) for that would be to not put any data in the buffer beyond its end, I don't see why the gap would have to be created. Maybe the code in source_output_audio_data() was only created with the intention of overwriting existing data in the buffer and not accidentially create a gap? Or maybe it is to keep in sync to the video? Which wouldn't make sense for me because I don't even record any video for my tests.

The occurrence of the issue is not accompanied by a "data still pending" debug message. However, if you add a #define DEBUG_AUDIO 1 to libobs\obs-source.c you'll get a frames: %lu, size: %lu, placement: %lu, base_ts: %llu, ts: %llu" debug message each time the source_output_audio_place() method is being called, which should correlate to the issue.

I made a commit here: Would be great if someone could verify if that helps.
floele@6cde82c

[bnacar]

@floele I ran a handful of tests ranging between 30 and 70 minutes long.

The good news is that all the dropouts I saw that did not correlate to the "data still pending" message did correlate to the "frames ... size ... placement ..." message, and this second type of dropout reliably occurred for me between 40 and 45 minutes into recording, which is similar to what you were observing, so you definitely have a lead on the two distinct dropout sources.

The bad news is that the effect of the altered source_output_audio_place() was inconclusive. Keep in mind of course multiple confounding factors, different hardware, Windows vs Linux etc...

Out of four occurrences of the "frames ... size ... placement ..." message, two of them were associated with no dropout. I looked very carefully - the waveform in these two cases appeared continuous. Of the remaining two cases, one of them was associated with a dropout, interestingly right after a "data still pending" dropout:

(The second dropout pictured here had a duration _less_than AUDIO_OUTPUT_FRAMES, unlike the "data still pending" ones.)

And the other appearance of the "frames ... size ... placement ..." message was associated with this oddity:

Then, just for comparison, I recompiled with the original version of source_output_audio_place() and started recording, and after 45 minutes I hit a "frames ... size ... placement ..." message associated with a waveform similar to the second screenshot.

So yeah... can anyone else give this a try? I hate being the party pooper.

[bnacar]

@konsolenritter re: your observation about timing discrepancies between audio or video sources and the processor itself. If this is in fact the source of some or all of these dropouts, then practically speaking there is no perfect workaround - how do you decide what "true" 48kHz is? Or "true" 30fps for that matter? Is OBS feeding data to YouTube at exactly the rate YouTube is expecting it, or does YouTube just have such a massive buffer that it doesn't matter?

If all we care about is audio continuity, a simple workaround for such timing differences might be to manually specify extra buffer (like as much as a second) at startup, enough to cover the time drift over the course of the entire session.

It gets more complicated when you have multiple audio and video sources that might drift out of sync. For those users to whom strict continuity matters, in addition to a manually specified buffer they would need to be able to define a particular audio or video source as ground truth and lock all other sources to that one, accepting some disruptions in those other sources. For example, gamers are probably going to care more about video continuity and musicians are going to care more about audio continuity.

Ouch.

[floele]

@bnacar Thanks for the tests so far!

Since we might be dealing with different types of/reasons for dropouts, we probably need to carefully distinguish between them for verifying whether a change fixes at least one type of them or not.

The dropout I'm looking for and have been debugging a little more today at seems to depend on the TS_SMOOTHING_THRESHOLD constant, which decides on whether or not a "push back" is being done. Whenever the timestamp difference between source->next_audio_ts_min and in.timestamp is greater than that constant, it was only a very close miss on my system, I got these values with in.timestamp being greater:

70004300
70005600
70004700
70015100
70006700

I'm not sure why it's set to 70000000, seems a bit arbitrary and increasing the value by a bit could also be a solution. You could try that for comparison. It would be interesting to know if the timestamp differences are in the same range on your hardware.

However, the difference being calculated here is an absolute one, so there might also be situations where in.timestamp is smaller and not greater, which could potentially lead to the result of your second screenshot. This would not be fixed by my solution, but if you increase TS_SMOOTHING_THRESHOLD instead, it could also fix that situation (assuming that source_output_audio_place() is being used here to overwrite existing audio data).

In any case, it would be interesting to know how your timestamps are set in source_output_audio_data(), particularly if in.timestamp is way too large or too small.

The "data still pending" related dropout might be fixed by my other fix (not sure if you applied that as well?).

[floele]

I just realized my solution might be a fit flawed here...since I'm only switching to push_back when the buffer is still small, the fix might only work for the first dropout and revert back to the "place" behavior after that. Will verify that in another debug session. So it might actually be smarter increasing the TS_SMOOTHING_THRESHOLD, even if the value is a bit arbitrary.

[floele]

Got more insights today. Actually, even increasing TS_SMOOTHING_THRESHOLD will not help.

The actual root problem is something else. Still in source_output_audio_data(), if everything is "normal", there's a line that sets
in.timestamp = source->next_audio_ts_min.
In each previous call to that method, next_audio_ts_min is set like this source->next_audio_ts_min = in.timestamp + conv_frames_to_time(sample_rate, in.frames);

So this means that source->next_audio_ts_min will always be the "ideal" expected timestamp, since it's increased by a fixed value in each iteration. However, the incoming audio time stamps will always be a little bit later (higher) than the ideal theoretical values (it doesn't matter whether or not you have enabled "device timestamps" for the audio sources) since it also includes the execution time of the application and doesn't provide continuous values.

This means that the timestamp associated with the incoming audio data will deviate further and further away from the expected timestamp, eventually exceeding TS_SMOOTHING_THRESHOLD, causing an audio issue and then invoking a reset of the timer. This explains the behavior we have seen very well.

To give some actual data, it would look like this (only an extract of a few thousand iterations):

Perfect timestamp: 366688637418000, actual timestamp: 366688637507100, diff: 89100
Perfect timestamp: 366690227418000, actual timestamp: 366690227453000, diff: 350000
Perfect timestamp: 366697247418000, actual timestamp: 366697247892200, diff: 474200
Perfect timestamp: 366702747418000, actual timestamp: 366702748130500, diff: 712500
Perfect timestamp: 366705197418000, actual timestamp: 366705198281800, diff: 863800
Perfect timestamp: 366708257418000, actual timestamp: 366708258445200, diff: 1027200
...

New proposed solution: Increase next_audio_ts_min like this: source->next_audio_ts_min = data->timestamp + conv_frames_to_time(sample_rate, in.frames);
So we'd be using the unmodified timestamp to calculate the next expected timestamp, avoiding the buildup of "delay".

I'll be testing that a little now...

[bnacar]

OMG I just tested this - changed in.timestamp + conv_frames_to_time(sample_rate, in.frames); to data->timestamp + conv_frames_to_time(sample_rate, in.frames); and ran two separate hour-long tests.

Result: the "frames ... size ... placement ..." message did not occur, and the only dropouts I got were the "data still pending" dropouts!

(Regarding the latter, I had previously tried your other fix with no luck, see my comment upthread)

I'm not sure I completely understood what you did to get those timestamp comparisons - this is what I added right before setting source->last_audio_ts and source->next_audio_ts_min:

#ifdef DEBUG_AUDIO
    blog(LOG_DEBUG, "Perfect timestamp: %" PRIu64 ", "
                    "actual timestamp: %" PRIu64 ", "
                    " diff: %" PRIu64 "",
                    in.timestamp,
                    data->timestamp,
                    data->timestamp - in.timestamp);
#endif

Before altering the source->next_audio_ts_min line, this is what I would see:

debug: Perfect timestamp: 199851195957235, actual timestamp: 199851205733117,  diff: 9775882
debug: Perfect timestamp: 199851196207235, actual timestamp: 199851196961653,  diff: 754418
debug: Perfect timestamp: 199851205269735, actual timestamp: 199851204709421,  diff: 18446744073708991302
debug: ts 199851207628747-199851218295414
debug: audio discarded, new ts: 199851218295414
debug: Perfect timestamp: 199851206644735, actual timestamp: 199851212878274,  diff: 6233539
debug: Perfect timestamp: 199851208957235, actual timestamp: 199851209284523,  diff: 327288
debug: ts 199851218295414-199851228962081
deug: audio discarded, new ts: 199851228962081
debug: Perfect timestamp: 199851214957235, actual timestamp: 199851223846996,  diff: 8889761
debug: Perfect timestamp: 199851215248901, actual timestamp: 199851215568386,  diff: 319485
debug: ts 199851228962081-199851239628747
debug: audio discarded, new ts: 199851239628747
debug: Perfect timestamp: 199851223915567, actual timestamp: 199851233620234,  diff: 9704667
debug: Perfect timestamp: 199851224332233, actual timestamp: 199851225060634,  diff: 728401
debug: Perfect timestamp: 199851233394733, actual timestamp: 199851232872051,  diff: 18446744073709028934
debug: ts 199851239628747-199851250295414
debug: audio discarded, new ts: 199851250295414
debug: Perfect timestamp: 199851234686399, actual timestamp: 199851240897241,  diff: 6210842
debug: Perfect timestamp: 199851237769732, actual timestamp: 199851238503086,  diff: 733354
debug: Perfect timestamp: 199851243373898, actual timestamp: 199851242843501,  diff: 18446744073709021219
debug: ts 199851250295414-199851260962081
debug: audio discarded, new ts: 199851260962081
debug: Perfect timestamp: 199851244665564, actual timestamp: 199851251744927,  diff: 7079363
debug: Perfect timestamp: 199851246748897, actual timestamp: 199851247313393,  diff: 564496
debug: Perfect timestamp: 199851253353063, actual timestamp: 199851252757044,  diff: 18446744073708955597
debug: ts 199851260962081-199851271628747
debug: audio discarded, new ts: 199851271628747
debug: Perfect timestamp: 199851254540563, actual timestamp: 199851262549689,  diff: 8009126
debug: Perfect timestamp: 199851255623896, actual timestamp: 199851256099999,  diff: 476103
debug: Perfect timestamp: 199851263332229, actual timestamp: 199851263814794,  diff: 482565

So sometimes in.timestamp was actually greater than data->timestamp (hence the overflows when I would print the difference) but I didn't see anything that clearly indicated that the difference was monotonically increasing... had you put the debug statement somewhere else?

But the actual results were promising.

[cattywampus04]

Hello.

So I have been doing more testing and unfortunately, as others have discovered, the audio drifts out of sync over time. I tried with and without device timestamps checked. Both the same. I am doing a test in the main release build just to double check its not drifting on that too, and I will do another test using the motherboards internal audio.

Is this likely to be a hurdle that cannot be gotten over, being able to sync audio and video proper?

Thanks.

[norihiro]

Hi @Stuwot,
If you are interested, could you try another possible-solution that I made? It's implemented as a plugin instead of implemented into OBS Studio itself.

You can download from this page.
https://github.com/norihiro/obs-async-audio-filter/releases
It should work with released version of OBS Studio but need to apply a filter Asynchronous Audio Filter to your audio source. You might experience small distortion especially on high frequency.

It's behavior is that it will detect the drift of the coming audio, then it will gradually remove or add samples to compensate the drift. The idea is similar to this article but my implementation is much more simplified than the article.

[cattywampus04]

Hey @norihiro

Damn that sounds awesome! I am just testing the most recently PR build that @floele has put out. I'm streaming to a burner Twitch account while recording so I can watch/listen to the stream and do some sync tests live while also recording to check later...and at the moment everything seems in sync. @floele suggested this sync issue could happen sometimes and not others though, and he also mentioned about there being 2 PR builds and one of them might not cause the audio drift. I wasn't sure whether he meant his latest build was one of the 2 and that's why the drift might not be happening this recording.

Anyway...that's a long way of me saying I most certainly will test your plugin if the drifting keeps being an issue. I've done a lot of tests lately and not done much proper production so will need to break away from testing at some point but will surely check it out soon!

[floele]

Hi @Stuwot , I updated the PR #5993 today with some modifications that address the concerns norihiro raised. Since I am still unable to reproduce the out of sync issue, it would be great if you could test the changes. This PR is still based on an older version of OBS and should work fine for testing.

@norihiro Can you explain how you see the issue we're trying to address? You mention a "difference of clocks" as source for the problems in your filter plugin description, however, I don't see any such issues for the cases I am looking into. Especially since the problem also occurs when you don't use any external audio devices. Are you looking into a different kind of problem?

Also, if some kind of "catching up" is implemented by adding for removing frames, shouldn't this be done in the video stream since adding or removing frames there would basically not be noticable?

[norihiro]

Looking at the log on this issue and Stuwot's log, I saw a kind of USB audio interface is connected. I assume they have separated audio input and video input.

My setup is (a) M-200i, which output AES/EBU digital audio, (b) PCM2906B Audio CODEC, a hand-made USB-connected AES/EBU receiver, and a PC running OBS Studio on CentOS 8. I'm seeing ~20ppm difference in between (a) and (c).
I'm checking this article describing 'Asynchronous Mode', which I assume the root cause of the issue. Another journal on IEEE and ADI's Ee note describe the solution used in commercial products.

Also, if some kind of "catching up" is implemented by adding for removing frames, shouldn't this be done in the video stream since adding or removing frames there would basically not be noticable?

I agree that. Currently video capture sources provide an option Use Buffering. By disabling the buffering, the drift on the video should not happen as well. (This might degrade frames if frames does not arrive smoothly but working fine on my environment.)

[cattywampus04]

Hi @Stuwot , I updated the PR #5993 today with some modifications that address the concerns norihiro raised. Since I am still unable to reproduce the out of sync issue, it would be great if you could test the changes. This PR is still based on an older version of OBS and should work fine for testing.

@norihiro Can you explain how you see the issue we're trying to address? You mention a "difference of clocks" as source for the problems in your filter plugin description, however, I don't see any such issues for the cases I am looking into. Especially since the problem also occurs when you don't use any external audio devices. Are you looking into a different kind of problem?

Also, if some kind of "catching up" is implemented by adding for removing frames, shouldn't this be done in the video stream since adding or removing frames there would basically not be noticeable?

Testing right now. I have been recording and streaming to a burner Twitch account so I can check the sync live as it's happening, but with this build, it would stream for ~60 seconds and then die. Is there something that is making Twitch consider the build not suitable? I tried streaming with the proper OBS build and it worked fine? Not a massive problem. Just thought I'd ask.

Would it be possible for you to briefly explain the 2 different issues that you are trying to fix? I have read through the threads but I'm not fully understanding what 2 issues are and which build fixes what? I don't need paragraphs and paragraphs. I would be helpful for me though to know and understand which issue is being fixed by which builds?

[cattywampus04]

Looking at the log on this issue and Stuwot's log, I saw a kind of USB audio interface is connected. I assume they have separated audio input and video input.

My setup is (a) M-200i, which output AES/EBU digital audio, (b) PCM2906B Audio CODEC, a hand-made USB-connected AES/EBU receiver, and a PC running OBS Studio on CentOS 8. I'm seeing ~20ppm difference in between (a) and (c). I'm checking this article describing 'Asynchronous Mode', which I assume the root cause of the issue. Another journal on IEEE and ADI's Ee note describe the solution used in commercial products.

Also, if some kind of "catching up" is implemented by adding for removing frames, shouldn't this be done in the video stream since adding or removing frames there would basically not be noticable?

I agree that. Currently video capture sources provide an option Use Buffering. By disabling the buffering, the drift on the video should not happen as well. (This might degrade frames if frames does not arrive smoothly but working fine on my environment.)

Yes I am using a GOXLR Mini. When I use the Realtek audio line-in on the motherboard, there are no stutters. This is true on both of my computers. I get no syncing issues on the proper release of OBS either, just FYI 🙂

Thanks!

[cattywampus04]

Hi @Stuwot , I updated the PR #5993 today with some modifications that address the concerns norihiro raised. Since I am still unable to reproduce the out of sync issue, it would be great if you could test the changes. This PR is still based on an older version of OBS and should work fine for testing.

@norihiro Can you explain how you see the issue we're trying to address? You mention a "difference of clocks" as source for the problems in your filter plugin description, however, I don't see any such issues for the cases I am looking into. Especially since the problem also occurs when you don't use any external audio devices. Are you looking into a different kind of problem?

Also, if some kind of "catching up" is implemented by adding for removing frames, shouldn't this be done in the video stream since adding or removing frames there would basically not be noticable?

Hi, this build is not working for me. I thought it was just streaming that wouldn't work and recording would be fine, but I just did a 90minute recording, only to find that once I pressed the stop recording button, the file size is tiny. It basically records 60-120 seconds of video, then just records audio, but really sped up. A 90 minute recording is only ~2 minutes long.

Tried multiple times and same result.

[floele]

@Stuwot Which build did you use? I tried OBS-Studio-27.2.0-f3b94c752-Win64 today and it works fine for me. There shouldn't be any major difference to the previous build.

Would it be possible for you to briefly explain the 2 different issues that you are trying to fix?

OK let's make it TLDR:

1. Audio dropout or distortion, occurs roughly every 40 min. or so. Fixed in Fix random audio dropouts and corruption of audio data (1/2) #6133 or Fix random audio dropouts and corruption of audio data (2/2) #5993 and the corresponding builds.
2. Audio dropout, may occur every couple of minutes. Fixed only in Fix random audio dropouts and corruption of audio data (2/2) #5993 and the corresponding builds.

I'm not sure if either of these issues is reliably preventable by using audio interfaces that behave in a certain way. So for now I assume that both issues occur in a wide variety of configurations.

[cattywampus04]

Thank you so much for that TLDR. I cannot stress how much that helps (and thank you for bearing with me on being slow on the uptake on things lol).

I have just had another bash with build 27.2.0-f3b94c752. I was running it with no plugins except the ASIO plugin so I could grab the line-in audio from my GOXLR. I've removed that plugin and am grabbing the audio a different way and it seems to be working ok now. I'm streaming and recording at the same time so am able to watch live and check the syncing. Does that having the ASIO plugin installed (which has worked fine on all other builds so far) sound like a plausible explanation for the problems I was reporting?

Any-who. I am doing a recording right now. Will report back.

Thanks again!

[cattywampus04]

Hi, again!

So there is definitely something wrong with this build (27.2.0-f3b94c752). I'm running it with no plugins and I got 1hr 7m into a stream/recording and then it just gives up.

The first sign of trouble is the bitrate marker in the bottom right of the screen goes to 0kb/s...then after a minute or so I get a notif that the stream could not connect. If i then try and start the stream again, with the recording still supposedly going, it sits and thinks for a bit before failing to start again. Then, when I press the stop streaming button, that too sits there saying "stopping recording" - it sits like that indefinitely and I have to force close OBS if I want to be able to record and/or stream again using that build.

THEN...when I go and check the recording...even though OBS said it was still recording after it threw the error, the recording stops at that point. So OBS tells me the recording is 1hr 20mins at the point where I stop, but the file is 1hr 6m and it takes ages to seek around the file itself - as if something is corrupt in the file.

I've attached the log for the session. I got the stream error (where it drops to 0kb/s bitrate before throwing an error) at 01:57am. OBS then continued supposedly recording for another 20ish minutes...which it didn't actually record anything.

Thanks

2022-03-23 00-50-40.txt

[noodohs]

I am just chiming in to say that I am having the same issue as originally described, short stutters that occur seemingly randomly throughout streams/recordings. The only thing I have to add that might be of some value is that I seem to only get them with devices that have ASIO drivers available. So, for example, I get them when I use my Audient EVO 8 or when I previously used the PreSonus Revelator io44, but I don't seem to get them when using something like my THX Onyx or Shure MV7 which only have "standard" Windows drivers. I have only tested this for half an hour at a time, so I can't be 100% certain of that, but that seems to be consistent. The only common factor I can think of is that these sorts of ASIO audio interfaces generally have internal clocks. Not sure if any of that is helpful, but just wanted to chime in to try and help get this resolved.

[DonKatsu]

I'm not sure if what I'm experiencing is this issue or something similar. CPU is 5900X, audio device is ALC1220, using pipewire.

What I'm experiencing extends to output as well. While recording with OBS, I will occasionally hear a skip that ends up in the recording. These skips coincide with an increase in ERR counts for all processes listed in pw-top, including the output and input themselves. OBS gets more counts than the other process, and the output/input get the least.
It seems to happen more frequently the more output audio sources I have set and mixed in OBS. Multiple obs-pipewire-audio-capture sources especially. However, pipewire services aren't reporting xruns in any case.

Verbose log of a 21 minute recording. The only audio sources were my output and USB mic selected in Settings>Audio>Global Audio Devices. Desktop Audio on track 1 and 6, mic on track 2 and 6.
At some point in this recording, I got Audio timestamp for 'Mic/Aux' exceeded TS_SMOOTHING_THRESHOLD, diff=70042268 ns, expected 69524630824508, input 69524560782240. That should be around 16:13 but the closest dropout I can see at a glance is at 17:54. Screenshot. Track 1>Track 2>Track 6.
You can also see track 1 lagging behind track 6. It does this from the very start of the recording. Any track with more than one source mixed seem to start faster than tracks with a single source?

Audacity unsurprisingly picks up these drops if ran alongside OBS.
Audacity recording on its own doesn't see any drops, and pw-top doesn't gain any ERR counts.
Game used for everything is Deep Rock Galactic, which is fairly CPU intensive.

EDIT: Tried changing some pipewire/wireplumber settings.
Increasing clock.min-quantum stops the errors entirely, but needs to be set to 1024. Errors still happen at 512. But from what I can tell, changing this value isn't recommended.
Increasing api.alsa.headroom seems to only help prevent the output from gaining errors. The OBS process still generates them, with the other processes in the output gaining a lower amount. Tried up to a value of 4096.

Still not sure what to make of this. Pipewire issue? Seems like OBS doesn't have enough quantum since the output gets set to 256 with Proton.

[moonlightes]

Hello. I'm using the motu m2 audio interface and I have a similar problem. I understand that no solution was found?

[dojima]

The solution I found was to use norihiro's asynchronous audio filter plugin. Works perfectly in my testing.

[moonlightes]

I have tried using this. But with the plugin, the sound stutters.

[dojima]

What is your use case exactly? Which sound stutters; what you hear or what is recorded or both? Did you make sure to use the plugin on all audio sources in your scene?

[norihiro]

I have tried using this. But with the plugin, the sound stutters.

I'd like to suggest creating an issue on https://github.com/norihiro/obs-async-audio-filter/issues so that the topic on this thread is focused on obs-studio itself.

[moonlightes]

What is your use case exactly? Which sound stutters; what you hear or what is recorded or both? Did you make sure to use the plugin on all audio sources in your scene?

Quite possibly I did not understand how to use the plugin. I installed it and added a filter for the microphone and windows sound, then I launched the tone generator. Clicks are heard on the recording, and the more samples the plugin adds, the more often they.

[AlexGxtr]

The logs for this run have expired and are no longer available.
Trying to dl this for windows

[AlexGxtr]

Could you pls upload again the updated files for obs. I'm having the same issue.

[moonlightes]

Obs v30 in release and the problem is still present.
Hardware: motu m2.
--verbose when stuttering shows: "Audio timestamp for 'Playback Device' exceeded TS_SMOOTHING_THRESHOLD" and the same for the microphone
obs-async-audio-filter helps in my case.
Is it such a rare problem that it remains unresolved after all this time?

[0x01011970]

I can confirm the problem is still present on windows.

I could confirm the issue is happening in OBS by recording audio at the same time in OBS and in Bitwig (for ASIO) and Audacity (for WASAPI).

Both on a Focusrite USB audio card (both ASIO & WASAPI) and on an onboard realtek audio card (WASAPI).

I tried both "Use Device Timestamps" enabled and disabled.

In every single tests, no sample drops can be seen in the raw PCM audio file produced by Audacity or Bitwig.

In every single tests, several sample drops can be seen in the recorded file by OBS (.mkv, .mp4, AAC or PCM 24 sle or 32 float).

I make sure all software and windows settings use same sampling frequency and depth.

CPU and GPU both bellow 50% usage, plenty of RAM & VRAM

There is also a secondary issue. By aligning the start of both sound waves, you can see the OBS soundwave is stretched over time. The OBS audio ends at a later timestamp than the one in Audacity or Bitwig. Logic would dictate since samples are missing in the OBS audio file that it would be shorter, but it’s the reverse.

I did not try https://github.com/norihiro/obs-async-audio-filter/, it is not a viable workaround.

[AlexGxtr]

I can confirm the problem is still present on windows.

I could confirm the issue is happening in OBS by recording audio at the same time in OBS and in Bitwig (for ASIO) and Audacity (for WASAPI).

Both on a Focusrite USB audio card (both ASIO & WASAPI) and on an onboard realtek audio card (WASAPI).

I tried both "Use Device Timestamps" enabled and disabled.

In every single tests, no sample drops can be seen in the raw PCM audio file produced by Audacity or Bitwig.

In every single tests, several sample drops can be seen in the recorded file by OBS (.mkv, .mp4, AAC or PCM 24 sle or 32 float).

I make sure all software and windows settings use same sampling frequency and depth.

CPU and GPU both bellow 50% usage, plenty of RAM & VRAM

There is also a secondary issue. By aligning the start of both sound waves, you can see the OBS soundwave is stretched over time. The OBS audio ends at a later timestamp than the one in Audacity or Bitwig. Logic would dictate since samples are missing in the OBS audio file that it would be shorter, but it’s the reverse.

I did not try https://github.com/norihiro/obs-async-audio-filter/, it is not a viable workaround.

Lower your master by 10-20-30db!!! Your signal is way too powerful so the audio is causing such issues.

[HWesterhof]

I can confirm the problem is still present on windows.
I did not try https://github.com/norihiro/obs-async-audio-filter/, it is not a viable workaround.

I can confirm as well (see my duplicate report in #11407)

But for me, the mentioned plugin solves my issue totally.