Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Video Recording #5057

Closed
jroweboy opened this issue Jan 15, 2020 · 0 comments · Fixed by #5083
Closed

Improve Video Recording #5057

jroweboy opened this issue Jan 15, 2020 · 0 comments · Fixed by #5083

Comments

@jroweboy
Copy link
Contributor

jroweboy commented Jan 15, 2020

The suggestions here are low priority, as we aren't in the business of recording video, we are an emulator project. BUT that said, this could be a nice task for someone who wants to start contributing to the project, so I'm going to write some suggestions here.

I recently did some investigation into what makes our video recording so slow, and found the following performance bottlenecks:

  • VP9 is a CPU encoder and is very slow
  • We flip the image on the CPU on the emu thread.
  • We use loseless encoding in VP9 with no way to configure it for real time streaming

As such, I want to make the following recommendations (and will put this in a new issue so we can track it if someone wants to improve this further)

VP9 is a CPU encoder and is very slow

Allow users to configure their choice of codec and encoder. Originally we stuck just with VP9 because its completely patent free and so we can ship the encoder, but in practice, we can use the operating system's encoders without issue. Theres no reason we can't use FFMPEG wrappers for NVENC or AMD/Intel or OS specific encoders since we aren't shipping the encoder, just a wrapper for the encoder which the user must have installed separately. Additionally, we can use compute shaders when available to skip downloading and reuploading the frame if needed, but this is probably not worth the effort if move the download to its own thread. See next section

We flip the image on the CPU on the emu thread.

GL coord space and the encoder coord space is different, so we need to flip the image. This should be simple enough to flip as part of the draw step in the renderer. Now that we support shared contexts, video encoding should go on its own graphics thread entirely, and use the mailbox system to get the latest image to render so that the backend isn't blocked on the download or flipping. The should be a somewhat easy change to make without having to dive into making more UI settings or make a new encoder, and it would speed up all future encoders as well.

We use loseless encoding in VP9 with no way to configure it for real time streaming

This could be done by someone with very limited encoder experience or graphics experience. Instead of using the hardcoded settings in the encoder, add a new settings window to allow the user to change the parameters of the encoder. The current settings are for loseless encoding, but some people just want to record while they play, so lossy encoding is acceptable. As a quick ascii example of what i'm thinking about for the settings menu, see the following diagram:

File to write to [____________]

Encoder          [_Software__V]
                 [_others...__]

Codec            [_VP9_______V]
                 [_others...__]

Quality          [_Loseless__V]
                 [_Good_______]
                 [_Realtime___]

Average Bitrate  |----[]------]

More settings could be added as we add support for more codecs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Features
  
Proposed
Development

Successfully merging a pull request may close this issue.

2 participants