Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sound export audio samples synchronization issues #1157

Closed
Jose-Moreno opened this Issue Jan 13, 2019 · 4 comments

Comments

Projects
None yet
3 participants
@Jose-Moreno
Copy link
Member

Jose-Moreno commented Jan 13, 2019

Issue Summary

After performing troubleshooting over the forum on this thread the problem encountered was the following:

Upon exporting the file shared by the user, the resulting video prompts synchronization issues where the sound starts playing BEFORE the first frame changes to the next one.

In my tests, at first there's an approximate difference of 10 frames between video and audio, however after inspecting the resulting soundtrack, each sound sample seems to be shifted progressively so simply shifting the track for x frames is impossible as each sample is shifted individually in time.

It almost seems as if the sound has not been mapped to the frame measurement during export, however when performing playback with the Pencil2D editor it works as expected.

Note: The overlapping sounds in the sound layer track are not causing the synchronization issues. I already tested removing them and the same problem persists. I actually think it's because it's a really long soundtrack however we still can't rule out it's because of the resulting track length in FFMPEG or that it's because each sound should have it's own separate track (which would be very difficult to do even in a dedicated video editor due to the amount of sound samples)

Expected Results

Exported video sound should have synchronized video and audio

Steps to reproduce

  1. Download the file provided by the user
  2. Playback the file a few times (for cache purposes as the sound lags a bit if it's not cached and if your cache number is too low)
  3. Then try a full playback to notice sound cues for reference.
  4. Export the video (In test MP4 was used)
  5. Notice there is a mismatch between audio and video which was not shown in the editor.

System Information

  • Pencil2D Version:
    Original User: Wind 64 - October 12th Nightly Build

Tests:
Version: 0.6.2
commit: da62598
date: 2018-12-20_12:07:36
Development build
Operating System: Windows 7 SP 1 (6.1)
CPU Architecture: x86_64

scribblemaniac added a commit to scribblemaniac/pencil that referenced this issue Jan 16, 2019

Fix audio synchroization issue
The sound track delay was based on the assumption of a sampling
rate of 44100 Hz, but the sound file was not being converted to
that rate until after the delay had already been applied.

Fixes pencil2d#1157
@scribblemaniac

This comment has been minimized.

Copy link
Member

scribblemaniac commented Jan 16, 2019

The issue should be fixed in the latest nightly build now. The source of the issue was with the sampling rate of the audio files being different from what was expected.

@scribblemaniac scribblemaniac added bug and removed investigate labels Jan 16, 2019

@Jose-Moreno

This comment has been minimized.

Copy link
Member Author

Jose-Moreno commented Jan 17, 2019

@scribblemaniac I just tested this by exporting from the latest build. Indeed it's much better. Awesome job!

Related to this issue I'm a bit conflicted with the following problem: as an example you can notice in the linked user file in frame 281 where there's a scene change to a title card and you see the sound starts there, however in the exported video the sound starts playing at frame 275, 6 frames before it actually should start. Now there seems that a slight general delay still persists between the exported movie and how it plays in the editor when using Quicktime Player (and inconsistently Windows Media Player).

However viewing the same video in VLC or using KDEnlive editor the sound is correctly aligned with the video and no delay is perceived. I don't know If I should note that down for future release notes, since it seems a problem with Quicktime player in general but I'd like to hear your opinion on that as well before calling this solved.

@Jose-Moreno

This comment has been minimized.

Copy link
Member Author

Jose-Moreno commented Jan 17, 2019

Actually now that I'm looking at the video now in KDE using a project at the same FPS rate (12fps) and there is indeed a 2 frame delay between where the sounds are placed in the Pencil2D editor and where the sounds are in the final video as you can see in the following images:

image
image

Of course this is not nearly as bad as it was before since 2 frames are not as noticeable (unless you are quicktime player apparently), but it still is a problem that needs to be solved.

Edit: Alright, now it's down to a 1 frame difference mainly because I noticed Qt and KDenlive start at 0 whereas Pencil2D exports from frame 1, so I moved the movie in the timeline 1 frame forward and now it's one less delay frame to worry about 😝

@chchwy chchwy added this to the 0.6.3 milestone Jan 18, 2019

@scribblemaniac

This comment has been minimized.

Copy link
Member

scribblemaniac commented Jan 18, 2019

@Jose-Moreno Yeah there's probably an off-by-one error somewhere. In fact I think I know where it is. I'll look into it later. It's not like Pencil2D is well suited for audio syncing even after this is fixed. As far as I know there are still multiple issues with the preview audio synchronization.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.