Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Playback Lag Affects Sound Perception #1567

Open
Jose-Moreno opened this issue Jan 26, 2021 · 0 comments
Open

Playback Lag Affects Sound Perception #1567

Jose-Moreno opened this issue Jan 26, 2021 · 0 comments

Comments

@Jose-Moreno
Copy link
Member

Issue Summary

A few weeks ago I and scribblemaniac tested an issue with a project I was using at my work, where there was a significant delay occurring on my work. After a lot of grinding we identified that this delay is perceptual and is influenced by the amount of layers and frames you have.

Basically the more layers / frames, the laggier the editor playback becomes and sound is audibly offset, however in reality what's offset is the playback of the frames trying to compensate for the lag, but the sound is played at real time speed instead so the discrepancy occurs.

This can be further verified by simply scrubbing and exporting the video since in both instances the sound is correctly mapped out to the animation.

The problem is that this is a pervasive problem that has been affecting users for years and at that moment I realized that every time someone complained about sound delays, this could be one of the potential problems (aside from those that were already fixed)

Expected Results

Animation playback is a tricky endeavor without a doubt. none of the commercial software I use ever tries to sell you real time playback speed with the editor, they usually have terms like pre-render, pre-caching, ram-preview, etc where they forcefully cache a range of images to be played or render them in some sort of intermediate or low latency file type to preview either in a separate panel widget or in the main viewer widget. In general there's always a preview feature regardless, sometimes there's even both or a hybrid of these ideas.

I know Pencil2D caches the images for playback but after years of believing this also cached the entire layer stack the current implementation doesn't seem to be enough to replicate a real time preview feature. This idea of caching is hinted in #965

Once of the first enhancements I'm requesting is improving the playback caching in a way that it really helps reduce the latency when there's a lot of elements. Perhaps we could pair it with the frame range feature so whatever is on this range will be guaranteed to play at real time speed. Perfect for shot playback and looping.

It's also been proposed before in #12 to have a separate preview option, which I would also agree to have as well if we consider that this can and should be used for the entire animation. pre-caching, software rendering or otherwise lowering the latency to have real time playback in a different widget (or a canvas mode so it doesn't look alien) is a technique widely used in various animation applications as well.

Video or Image Reference

This is a step by step explanation of how the issue occurs. It is quite involved so apologies if I don't provide the steps in written form.
https://youtu.be/u7PBZetRpN4

Steps to reproduce

Please refer to the video above but for a brief summary:

  1. Open Pencil2D
  2. Import a movie video to have a base line to test with plenty frames
  3. Import the appropriate movie audio
  4. Start adding layers on top without any frame. Test the playback latency per every layer you add and record your findings.
  5. Now add a few frames on every layer you added and draw something over. Test playback latency again and record the behavior
  6. Now hide all the layers except the main bitmap layer for the video. Test the playback latency and notice the difference.

System Information

  • Pencil2D Version:

Nightly Build Jan 4 2021

commit: 528aacb
date: 2021-01-04_15:21:03

Operating System: Windows 10 (10.0)
CPU Architecture: x86_64
Qt Version: 5.12.10

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Discussion
Enhancement Priority
  
Discussion
Development

No branches or pull requests

1 participant