-
-
Notifications
You must be signed in to change notification settings - Fork 8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Windows Graphics Capture support #2208
Conversation
b31cdf3
to
b7d37a8
Compare
Need to switch Windows SDK over to 10.0.18362.0 for this PR to build. |
8911843
to
3304271
Compare
It would appear that the old image Azure CI is currently using does not contain the latest Windows 10 SDK. Could you try changing https://github.com/obsproject/obs-studio/blob/master/azure-pipelines.yml#L64 and https://github.com/obsproject/obs-studio/blob/master/azure-pipelines.yml#L64 to |
3304271
to
6587d3f
Compare
The windows-2019 image does not appear to have VS 2017 installed. |
026a8a4
to
2de958b
Compare
I noticed that the new capture method stops working when changing OBS' frame rate in the video settings. The yellow border is still shown but the frame within OBS stops updating. |
2de958b
to
9ee8bf0
Compare
I hadn't realized OBS could recreate the graphics thread behind the sources backs until now. Added shenanigans handling, so changing video settings should work now. |
c30c15a
to
5aa675b
Compare
Obviously, testing can have a never-ending list of permutations, but I am satisfied that I more or less have the basics covered. The only thing holding me back from switching this out of draft status is the build situation. At a minimum, we need VS 2017 + Windows 10 SDK 10.0.18362.0 to compile this PR, but maybe we should move up to VS 2019 if that makes things easier for Azure. |
Getting the SDK available and being used by VS2017 is probably more pain than just switching to VS2019 which includes the newer SDK version. |
84c086d
to
e7d9a0e
Compare
Not sure if we're going to retire AppVeyor as well but for now we'd have to update it to the vs2019 image as well. |
Also, can we bump up to CMake 3.16? They recently added support for precompiled headers, and it reduces compile time for this PR from 6 seconds to less than a second. |
111a4c2
to
036aadb
Compare
An issue I noticed is that this capture method appears to disable the hardware cursor, meaning cusor movement will feel somewhat unresponsive (input lag). While not necessarily a big deal - on my 144 Hz monitor this is barely noticeable, on my 60 Hz one it is very apparent - I think that might be something for others to test and confirm. |
036aadb
to
c5b5e72
Compare
If the CI works then the SDK issue is fine for the record. For the hardware cursor thing, that's very likely Microsoft's doing, not ours, we probably aren't in control of that. |
There's a commit in this PR to switch CI over to VS 2019. Is that fine? |
Yea, that's fine. |
K, I'll see in a few hours if there's something I can do about hardware cursor before I remove draft status. |
c5b5e72
to
b10b4a7
Compare
I couldn't repro the hardware cursor issue; it's always active for me. |
Although I don't have any lag either, I really doubt that it's anything in the source implementation anyway, I would expect it to be due to something happening internally as part of the capture in microsoft code. Unless you have some cursor stuff in your code that you think would be the cause, but I doubt it. It doesn't really matter too much though, because this probably shouldn't be treated like a complete replacement for normal window capture. In my view, this is more important to us to capture windows that have been evading normal window capture, such as browser windows, microsoft store apps, office, or programs that run on chromium/cef/electon. |
This is just about perfect. I'm probably going to add an "auto" option and make it the default for the capture method dropdown though, so it can automatically capture windows we'd previously been having problems with. I feel like it should only be a few hours to make. I'm considering modifying the text for the items in the drop-down as well so the function is a bit more apparent. |
Don't make someone else define IUnknown.
Complex external systems using the D3D11 device may need to perform their own device loss handling, the upcoming Windows Graphics Capture support for example.
Necessary for upcoming Windows Graphics Capture support.
This provides a chance to clean up resources on the graphics thread in the hide callback, which Windows Graphics Capture requires.
The top-level window appears to be preferred for upcoming Windows Graphics Capture support, at least for UWP apps.
Currently wraps Windows Graphics Capture API.
Users will now have the option of legacy window capture via BitBlt, or Windows Graphics Capture, which is new to Windows 10. There are two annoyances with the new capture method though. One is that there is a bright, yellow border added to the original window (but not the OBS view of it). The other is that the mouse cursor is always captured, and we won't be able to capture without cursor until a later version of Windows 10 is released. It should also be noted that DPI scaling is now applied, which may result in blurrier images caused by Windows rescaling.
Reduces compile time from six seconds to less than a second.
The recent graphics capture changes require CMake 3.16 or above to function on Windows, so add a requirement check for 3.16 if on Windows.
The graphics capture changes require Windows SDK version 10.0.18362.0 or above to function, so check for that in CMake.
Adds an "Automatic" method to the capture method property of window capture. This allows the ability for window capture to automatically determine the best capture method to use. Primarily, it prefers BitBlt in most cases, but will use WGC when it detects the window is a browser, microsoft office, or a UWP program. This was implemented because the new capture method has a number of undesired effects associated with it -- the issue of the capture border that we can do nothing about, the fact that we can't control the capture of the cursor, and the fact that Microsoft designed it to switch the cursor away from hardware cursor mode when the capture is active (there was absolutely no reason to do this because even OBS can capture the hardware cursor with no issue). Until we get a new version of this API that doesn't absolutely blow, we're stuck preferring BitBlt instead. But hey, at least people will be able to capture browser windows now.
69521e1
to
572ce73
Compare
Description
Users will now have the option of legacy window capture via BitBlt, or
Windows Graphics Capture, which is new to Windows 10.
There are two annoyances with the new capture method though. One is that
there is a bright, yellow border added to the original window (but not
the OBS view of it). The other is that the mouse cursor is always
captured, and we won't be able to capture without cursor until a later
version of Windows 10 is released.
It should also be noted that DPI scaling is now applied, which may
result in blurrier images caused by Windows rescaling.
Motivation and Context
Works with hardware-accelerated Chrome. UWP apps also work with the new method.
How Has This Been Tested?
Scenarios tested:
Using QPF markers for CPU measurement, WGC tends to be a bit more expensive than BitBlt and wider variance, maybe 200-800 us vs 300-500 us. The new PeekMessage loop is very cheap when empty, about 0.2 microseconds.
WGC appears to be a bit cheaper than BitBlt on GPU just looking at Task Manager percentage, but it's hard to tell with the noise.
Since this feature is optional, and there's not much I can do to optimize these OS functions, I'm not too concerned about performance.
Types of changes
Checklist: