Implement audio syncing for "stream" sounds

[Herschel]

Playing a "Stream" sound on the timeline triggers frame-skipping behavior in the Flash Player to keep the animation in sync with the sound. Ruffle does not fully implement this.

Currently, Ruffle does frame-skipping at all times, which more-or-less keeps audio in sync for most cases. However, if the audio still manages to get out of sync (because the player or the audio stalls for some amount of time), there is no mechanism to re-sync the animation. This also occurs immediately in some exotic situations, such as when a stream sound is played in a loaded movie that has a different framerate from the root movie.

This should hopefully handle all of these cases:

• Keep animation and audio synced
• Fix audio sync in loaded movies where the loaded movie framerate differs from the root movie's framerate (Wrong frame rate for swf loaded via GetUrl2 #3020)
• Fix issues with audio cutting out when SoundStreamBlock tags are intermittent (Choppy sound when SoundStreamBlock isn't in every frame #3817)
• Properly handle silent gaps in sound streams (no SoundStreamBlock tag for many frames) (Don't stop audio streams in run_frame_internal to prevent audio stutter. Close #3817 #3819)

I'm thinking something like this:

• On the audio backend, stream sounds keep track of the frame they are on in the movieclip's timeline.
• Each frame while a stream sound is playing, the player can compare the movieclip's frame to the audio's frame. If these are different, adjust the player's frame timer to skip frames/stall the movie appropriately.
• Some investigation is necessary to figure out the Flash Player's behavior:
  • How does Flash behave when multiple stream sounds are playing? Does it choose one sound to sync with? (Particularly test when the sounds are from movies with different framerates.)
  • If there is a large discrepancy, say, the audio is suddenly a second+ ahead of the animation, how should we behave? Stop and restart the stream?

[ondaeduca]

Hi Mike.
I have several activities on the platform that have this problem of synchronization between audio and animation. They all have the audio on the timeline in "stream". I wanted to ask you if it is currently in development and open. I have read that you have rewritten the audio section and you have already worked on these problems, but is it finished or is there still implementation work?

I am attaching an example file, it is somewhat long, but it also happens in other very short 2-3 seconds.
prag_003.zip

thanks!
Javier YH

[Herschel]

Implemented in #5486.