Audio Sync Causes Video Stutter/Jitter

[Awakened0]

I've noticed that the mGBA core has a small amount of stutter/jitter when scrolling at 60fps. It's easiest to test with the 240p test suit "hill zone scroll test". I get small little hitches after a few seconds of scrolling, indicating dropped frames. If I turn off Audio Sync in RetroArch's audio settings, the hitches are gone and the scrolling looks perfect to me. It happens in windowed or true full screen, and with audio sync on it's usually more pronounced in windowed mode.

[Tatsuya79]

Happens to me too on win7 x64.
240p suite + Hill zone test, fast forwarding and going back to normal speed is a fast way to trigger the desync.

Audio sync off + "exact sync" in throttle menu is smooth.
So probably an audio problem?

[vanfanel]

I can confirm this happening here, too. Probably an audio buffer problem, IMHO.

[endrift]

Can you try these two things to see if either of them fix it:

1. Reducing the SAMPLES definition to 512
2. Applying this patch:

diff --git a/src/platform/libretro/libretro.c b/src/platform/libretro/libretro.c
index 5c59e598f..0ec3fdbba 100644
--- a/src/platform/libretro/libretro.c
+++ b/src/platform/libretro/libretro.c
@@ -39,7 +39,7 @@ FS_Archive sdmcArchive;
 #include <switch.h>
 #endif
 
-#define SAMPLES 1024
+#define SAMPLES 550
 #define RUMBLE_PWM 35
 
 static retro_environment_t environCallback;
@@ -52,7 +52,6 @@ static retro_set_rumble_state_t rumbleCallback;
 
 static void GBARetroLog(struct mLogger* logger, int category, enum mLogLevel level, const char* format, va_list args);
 
-static void _postAudioBuffer(struct mAVStream*, blip_t* left, blip_t* right);
 static void _setRumble(struct mRumble* rumble, int enable);
 static uint8_t _readLux(struct GBALuminanceSource* lux);
 static void _updateLux(struct GBALuminanceSource* lux);
@@ -69,7 +68,6 @@ static void* outputBuffer;
 static void* data;
 static size_t dataSize;
 static void* savedata;
-static struct mAVStream stream;
 static int rumbleUp;
 static int rumbleDown;
 static struct mRumble rumble;
@@ -323,11 +321,6 @@ void retro_init(void) {
 	logger.log = GBARetroLog;
 	mLogSetDefaultLogger(&logger);
 
-	stream.videoDimensionsChanged = 0;
-	stream.postAudioFrame = 0;
-	stream.postAudioBuffer = _postAudioBuffer;
-	stream.postVideoFrame = 0;
-
 	imageSource.startRequestImage = _startImage;
 	imageSource.stopRequestImage = _stopImage;
 	imageSource.requestImage = _requestImage;
@@ -450,13 +443,10 @@ void retro_run(void) {
 	core->desiredVideoDimensions(core, &width, &height);
 	videoCallback(outputBuffer, width, height, BYTES_PER_PIXEL * 256);
 
-	// This was from aliaspider patch (4539a0e), game boy audio is buggy with it (adapted for this refactored core)
-/*
 	int16_t samples[SAMPLES * 2];
 	int produced = blip_read_samples(core->getAudioChannel(core, 0), samples, SAMPLES, true);
 	blip_read_samples(core->getAudioChannel(core, 1), samples + 1, SAMPLES, true);
 	audioCallback(samples, produced);
-*/
 
 	if (rumbleCallback) {
 		if (rumbleUp) {
@@ -633,7 +623,6 @@ bool retro_load_game(const struct retro_game_info* game) {
 	}
 	mCoreInitConfig(core, NULL);
 	core->init(core);
-	core->setAVStream(core, &stream);
 
 	size_t size = 256 * 224 * BYTES_PER_PIXEL;
 #ifdef _3DS
@@ -936,14 +925,6 @@ void GBARetroLog(struct mLogger* logger, int category, enum mLogLevel level, con
 	logCallback(retroLevel, "%s: %s\n", mLogCategoryName(category), message);
 }
 
-static void _postAudioBuffer(struct mAVStream* stream, blip_t* left, blip_t* right) {
-	UNUSED(stream);
-	int16_t samples[SAMPLES * 2];
-	blip_read_samples(left, samples, SAMPLES, true);
-	blip_read_samples(right, samples + 1, SAMPLES, true);
-	audioCallback(samples, SAMPLES);
-}
-
 static void _setRumble(struct mRumble* rumble, int enable) {
 	UNUSED(rumble);
 	if (!rumbleCallback) {

I don't have the ability to test right now.

[Tatsuya79]

I tried the diff first and it worked to get rid of the stutters but then I experienced some crashes after doing several FF/standard speed tests.

Then I tried just Reducing the SAMPLES definition to 512 and it stutters a lot less as well.
I just see some stutters when going back to standard speed after a FF (which is quite common) but then it stabilizes and seems to stay smooth again.

All that is about fullscreen, I still see stutters in windowed mode.

[Awakened0]

With the samples definition set to 512, fullscreen with exact content sync and gsync is smooth on the hill zone scroll test. In windowed mode with both gsync and exact content sync off and swap interval set to 2 (since my desktop refresh is 120hz) there is periodic stutter, but seemingly less than with the old definition.

The diff patch seems just as smooth in fullscreen as just changing the samples definition, but also makes windowed mode smooth. At least with the hill zone test. Aria of Sorrow still seemed to have slight jitters here and there that aren't there in fullscreen.

I couldn't reproduce any crashes when spamming fast forward with the patch (I'm on the latest non-insider Win10 and latest Nvidia drivers. And using the Vulkan video driver in RA). I guess there is a little bit of stutter when coming out of FF, but it's hard for me to notice since it clears up fast.

[Tatsuya79]

I tried the diff again.
It works fine in fullscreen, I didn't have a crash this time.
I got some huge stops when I hammer FF on/off a lot after a while, that's where I had some crashes before, but it could be just some gsync thing.
Windowed still stutters though.

Without gsync: fine in fullscreen (no more stutters after FF or sound popping), still stutters in windowed mode.

[Tatsuya79]

But it's messing up GB sound...
So the simple change to 512 seems better.

[Awakened0]

The latest buildbot build has the samples set to 512, so the hillzone test is nice and smooth fullscreen. There's still some tiny jitters in windowed mode, but I'd be willing to close this since I don't really play windowed, or expect it to be perfectly smooth since DWM can mess with it. Unless anyone has any objections?