Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Attempt to fix av desync in segmented video #8447

Open
wants to merge 3 commits into
base: master
from

Conversation

@vadosnaprimer
Copy link
Contributor

vadosnaprimer commented Nov 3, 2019

CRC32: 6C83A5FF
MD5: 2381ACD2199D6E7566932DF86901903D
SHA-1: C75F7936814636FFE03277F363FC3427C98602EE
  • Stop the movie after the level starts
  • Rename or move framedump0.avi
  • Enable Dump at Internal Resolution
  • Dump frames again.

If you splice the 2 dump segments that were done from internal resolution, you'll see that the level starts 68 frames earlier than in the single-segment dump that used the window resolution. It's because s_last_frame_is_valid gets reset, resulting in wrong frame duration timings and missing frames. This results in heavy audio-video desync if you dump the entire movie's audio and video and splice/mux everything together.

I tried to fix that by accounting for whether it's a new segment caused by video split, and now I consistently get 1 extra frame on every split. I've been staring at the code for the entire day, and all the frame timings are now identical to the correct dump scenario (single-segment), but the output still doesn't match. I'm unable to figure out what else should be tweaked to get rid of the extra frame. Help!

@vadosnaprimer

This comment has been minimized.

Copy link
Contributor Author

vadosnaprimer commented Nov 3, 2019

This should be better.

@vadosnaprimer

This comment has been minimized.

Copy link
Contributor Author

vadosnaprimer commented Nov 4, 2019

I found the problem, every last frame of a dump only has duration of 1 (since next frame's PTS is non-existent), even if it's supposed to repeat. The current approach uses just PTSs for every new frame, and the decoder then calculates how long each frame should last by looking at next frame's PTS. But when a video ends, there's no next frame, hence no duration info.

For a single-segment dump this problem is hard to notice, it's also non-apparent for games that render at 60fps. But if you have a game that renders at 30fps and changes resolution, every segment's last frame will last for 1 video frame instead of 2.

Now the solution would be to set pkt.duration explicitly I think, but we can't know it until the next frame has been emulated! I'm not sure how to resolve this.

@vadosnaprimer vadosnaprimer marked this pull request as ready for review Nov 4, 2019
@vadosnaprimer

This comment has been minimized.

Copy link
Contributor Author

vadosnaprimer commented Nov 4, 2019

Accidentally marked as ready, please ignore for now. It's still a WIP.

@vadosnaprimer vadosnaprimer changed the title Attempt to fix av desync in segmented video [WIP] Attempt to fix av desync in segmented video Nov 4, 2019
@altimumdelta

This comment has been minimized.

Copy link
Contributor

altimumdelta commented Nov 6, 2019

Now the solution would be to set pkt.duration explicitly I think, but we can't know it until the next frame has been emulated! I'm not sure how to resolve this.

I'll start looking at this Framedumping thing again then; also while it's not required, it doesn't stop anyone from developing locally, it's still open what kind of ffmpeg integration the community feels would like to have along with the update to the ffmpeg external for windows, delayed dll or static.

@vadosnaprimer

This comment has been minimized.

Copy link
Contributor Author

vadosnaprimer commented Nov 6, 2019

I'm working on making it output separate frames, actually CFR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
3 participants
You can’t perform that action at this time.