Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Video playback in Age of Empires II HD (2013) breaks d3d9 rendering #1726

Closed
aeikum opened this issue Aug 14, 2020 · 13 comments
Closed

Video playback in Age of Empires II HD (2013) breaks d3d9 rendering #1726

aeikum opened this issue Aug 14, 2020 · 13 comments
Assignees
Labels

Comments

@aeikum
Copy link
Contributor

aeikum commented Aug 14, 2020

In AOE2:HD, if you have functional video playback, then DXVK's d3d9 rendering is broken. In v1.7, the video blinks rapidly. In v1.7.1, the video appears stretched and corrupted. In both cases, the main menu of the game doesn't render (i.e. the DXVK HUD FPS counter is blank; no rendering). With wined3d, the video and game render just fine. Wine's quartz implementation is using d3d9 to render, see dlls/quartz/vmr9.c.

Since I know getting video playback set up is a pain, I'm including as much info here as I can. I'm happy to provide more info or test patches, etc.

Software information

Age of Empires II HD (2013) (steam game id 221380)

System information

  • GPU: Nvidia GTX 970
  • Driver: 450.57
  • Wine version: 5.13
  • DXVK version: v1.7.1

Apitrace file(s)

wined3d (correct) apitrace: https://smokingonabike.com/AoK%20HD%20wined3d.trace.xz
dxvk (incorrect) apitrace: https://smokingonabike.com/AoK%20HD%20dxvk.trace.xz

Log files

@aeikum
Copy link
Contributor Author

aeikum commented Aug 14, 2020

Here's what the video looks like in DXVK v1.7.1.

Screenshot_2020-08-14_12-52-44

@Joshua-Ashton Joshua-Ashton self-assigned this Aug 15, 2020
Joshua-Ashton added a commit that referenced this issue Aug 16, 2020
Needed for #1726 otherwise it will upload dump that upload garbage in a YUV2 texture.
@Joshua-Ashton
Copy link
Collaborator

Joshua-Ashton commented Aug 16, 2020

So it uses the format YV12 in the WineD3D trace, however we don't support that so it tries to use YUY2 in the DXVK trace.

It seems the YUY2 data that gets uploaded is garbage, same with how it is when replaying the trace on native for me.
Maybe this is a problem in Wine's quartz impl that needs to be looked into, or the game doing something wrong?

Anyway, I've implemented YV12. Please let me know if this works on your end.

Joshua-Ashton added a commit that referenced this issue Aug 16, 2020
Needed for #1726 otherwise it will upload dump that upload garbage in a YUV2 texture.
@aeikum
Copy link
Contributor Author

aeikum commented Aug 17, 2020

Thanks. I built that branch (v1.7.1-2-g7e0bb971). The ConvertFormat warning is gone and the video no longer flickers, however the colors are incorrect and the game's main menu still does not render.

AoK-dxvk-branch

@Joshua-Ashton
Copy link
Collaborator

@aeikum Is there anything special I need to do to get the video to play?

I tried in proton_5.0-next and it just skipped over the video and went straight to the main menu (which worked) for me.

@Joshua-Ashton
Copy link
Collaborator

So, the main menu worked before the video thing worked.

The problem is you're creating multiple swapchains/surfaces for a window, this is a big no-no in Vulkan, but it should be possible to fix.

@Joshua-Ashton
Copy link
Collaborator

Hm, perhaps not. This is happening on another d3d9 instance and another d3d9 device.

The primary vulkan swapchain from the game still thinks everything is okay and it can present and even returns VK_SUCCESS.

I'm not super sure what to do about that given I have 0 information... :-/


Anyway, I fixed the YV12 rendering at least. I'll need to have a think about the swapchain dilemma, I'm not sure if this is something I can work around, or if I can work around it in Wine or maybe the driver is wrong for thinking everything is okay.

Joshua-Ashton added a commit that referenced this issue Aug 17, 2020
Needed for #1726 otherwise it will upload dump that upload garbage in a YUV2 texture.
Joshua-Ashton added a commit that referenced this issue Aug 17, 2020
Needed for #1726 otherwise it will upload dump that upload garbage in a YUV2 texture.
@aeikum
Copy link
Contributor Author

aeikum commented Sep 28, 2020

Just curious, any further thoughts on this? Do we know where the problem is?

@Plagman
Copy link

Plagman commented Dec 18, 2020

@Joshua-Ashton

@Joshua-Ashton
Copy link
Collaborator

Joshua-Ashton commented Jan 8, 2021

I have tested this on Windows with DXVK and it works fine there.

On Windows in D3D9, you can have multiple swapchains on the same window.

The problem is that technically this is illegal in the Vulkan spec -- but it seems to just work on Windows too?

__
In WineX11, I would recommend wrapping the swapchain object, tracking ownership of swapchain->window, and make sure whoeever wants to present at a given time has the ownership. If they don't currently own the swapchain, return VK_OUT_OF_DATE_KHR on present and make them re-make the swapchain (and therefore update the ownership) when that happens.

Whoever creates a swapchain for a window last should be the "owner".


We should eventually write a VALVE extension that we enable all the time to make things spec-positive here. I can take that part once we know this works.

@nsivov
Copy link
Contributor

nsivov commented Jul 16, 2021

Another game that needs YUY2 in d3d9 path is Mary Skelter: Nightmares. It's using mediafoundation for intro video, with EVR renderer, that goes through dxva2. YUY2 is a main planar format there as I understand, at least for software devices. Winegstreamer is already able to output in YUY2, but that needs a hack to trigger + some smaller patches that are waiting to be included for mainline wine. Let me know if there is interest to test this, I can provide more info then.

@aeikum
Copy link
Contributor Author

aeikum commented Oct 20, 2021

As we're getting more videos online, we're running into this problem more often. Any hope of a workaround?

@aeikum
Copy link
Contributor Author

aeikum commented Oct 26, 2021

Rémi actually had some fixes for this in one of his trees. We've backported his work to 6.3 and it fixes AOE2 and some other affected games. It works by allowing multiple surfaces & swapchains per window, and rendering offscreen if multiple surfaces are active.

I've pushed this to my repo and it'll be in Experimental soon: aeikum/wine@08db7c2

ivyl pushed a commit to ValveSoftware/wine that referenced this issue Oct 27, 2021
Fixes games failing to render after displaying a video, e.g. Age of
Empires II (2013).

    doitsujin/dxvk#1726
ivyl pushed a commit to ValveSoftware/wine that referenced this issue Nov 24, 2021
Fixes games failing to render after displaying a video, e.g. Age of
Empires II (2013).

    doitsujin/dxvk#1726
ivyl pushed a commit to ValveSoftware/wine that referenced this issue Dec 9, 2021
Fixes games failing to render after displaying a video, e.g. Age of
Empires II (2013).

    doitsujin/dxvk#1726
rbernon added a commit to rbernon/wine that referenced this issue Dec 17, 2021
Fixes games failing to render after displaying a video, e.g. Age of
Empires II (2013).

    doitsujin/dxvk#1726
imaami pushed a commit to imaami/wine that referenced this issue Feb 16, 2022
Fixes games failing to render after displaying a video, e.g. Age of
Empires II (2013).

    doitsujin/dxvk#1726
imaami pushed a commit to imaami/wine that referenced this issue Jun 5, 2022
Fixes games failing to render after displaying a video, e.g. Age of
Empires II (2013).

    doitsujin/dxvk#1726
imaami pushed a commit to imaami/wine that referenced this issue Jun 5, 2022
Fixes games failing to render after displaying a video, e.g. Age of
Empires II (2013).

    doitsujin/dxvk#1726
@K0bin
Copy link
Collaborator

K0bin commented Apr 8, 2023

This has been working for some time now.

@K0bin K0bin closed this as completed Apr 8, 2023
GloriousEggroll pushed a commit to GloriousEggroll/proton-wine that referenced this issue May 5, 2023
Fixes games failing to render after displaying a video, e.g. Age of
Empires II (2013).

    doitsujin/dxvk#1726
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

5 participants