Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Native Port #1264

Open
wants to merge 29 commits into
base: master
from
Open

WIP: Native Port #1264

wants to merge 29 commits into from

Conversation

@Joshua-Ashton
Copy link
Collaborator

Joshua-Ashton commented Nov 30, 2019

This PR allows DXVK to be built natively on Linux with an option to specify the WSI you want to use.

This allows titles to port to Linux and potentially other platforms further down the line much easier by letting them keep their existing rendering code for D3D11.

The exact same code path can also be used on Windows if the title wishes to ship with DXVK there too, and is compatible with the regular Windows headers (ie. you can use SDL2 WSI intergration on Windows builds as well.)

@Joshua-Ashton Joshua-Ashton marked this pull request as ready for review Nov 30, 2019
@Joshua-Ashton

This comment has been minimized.

Copy link
Collaborator Author

Joshua-Ashton commented Nov 30, 2019

[Turns out Github drafts are not the same as Gitlab ones 🐸]

@doitsujin

This comment has been minimized.

Copy link
Owner

doitsujin commented Dec 4, 2019

I'm going to leave this open for now until a final decision is made, but as discussed on Discord, with DXVK entering maintenance mode, I'd like to avoid any significant changes or additions to the code base that are not strictly necessary.

@Guy1524 Guy1524 mentioned this pull request Dec 7, 2019
@eszlari

This comment has been minimized.

Copy link
Contributor

eszlari commented Dec 10, 2019

with DXVK entering maintenance mode

Is this, because you consider DXVK to be feature complete?

@doitsujin

This comment has been minimized.

Copy link
Owner

doitsujin commented Dec 10, 2019

Is this, because you consider DXVK to be feature complete?

It's because DXVK has become a fragile, unreliable and frustrating maintenance nightmare. Most of the 1.4.x releases introduced major regressions which I cannot reproduce, and therefore cannot debug and fix.

This includes GPU hangs in Overwatch on specific maps with Nvidia GPUs (some users claim it's fixed in 1.4.6 while others still have them), rendering issues in Dishonored 2 which I can't reproduce (see ValveSoftware/Proton#823 (comment)), vertex explosions in some games which I also can't reproduce, an ongoing Star Citizen issue which I still need to debug (see #1262), and tons of weird issues that don't make any sense whatsoever (like #1266 which only seems to affect RADV).

Most of these problems are still unresolved and I have no idea how to even track them down, let alone fix them, and the ones that got "fixed" got fixed by reverting otherwise useful changes because I simply do not understand any of the issues at all.

Doing any sort of active development with this broken mess of a code base would only make this worse, and I wish I had drawn the line sooner. The only thing I still plan to do is wire up some useful Vulkan extensions and eventually merge D9VK, the rest will be bug fixing only.

@jediafr

This comment has been minimized.

Copy link

jediafr commented Dec 11, 2019

It does sound like a burnout. I would advise 2 weeks iatus to stop working in crunch mode. A rethinking of the framework might benefit the project long term. Valve can understand this.

EDIT : my advice to doitsujin comes from the heart for the difference he made by writing dxvk AND his commitment to it. After two decades of IT, i've experienced some burnout situations.
It's avoidable and nothing justifies being pushed to crunch , least of all gaming. Dxvk is a labour of love, and it shows, to doitsujin : take it easy and relax.

@Pyrathar

This comment has been minimized.

Copy link

Pyrathar commented Dec 11, 2019

It does sound like burnout, but sir i would like to say that im pretty sure everybody in gaming linux community is thankful for your work, this made such a huge impact in my life and im sure it did on a lot of other people aswell, take a vacation mate! you more than earned it!

@veryprofessionaldodo

This comment has been minimized.

Copy link

veryprofessionaldodo commented Dec 11, 2019

I agree with what was stated above, you did/are doing some amazing work, and I'm incredibly thankful for your achievements! @doitsujin

@devland

This comment has been minimized.

Copy link

devland commented Dec 11, 2019

Hey, Philip. We really appreciate your work. It's been inspiring to watch this project evolve over time and to see real world applications at a small (me playing Windows games in Lutris under Linux) and large scale (Valve's Proton).

I'm a developer myself and what you're experiencing sounds a lot like you need a vacation. Working on the same piece of code for long periods of time will get you tunnel vision and the only way out is to just take a break and relax. :)

@RealXor

This comment has been minimized.

Copy link

RealXor commented Dec 11, 2019

Hey Phil, same here, I can probably speak for the community for this matter, take a break. Do it for yourself, and instead just focus on your personal live man, you need to catch up!

Have a nice day bro!

@PMunkes

This comment has been minimized.

Copy link

PMunkes commented Dec 11, 2019

Just wanted to say thank you for all your hard work. Take a few weeks off, at least till the new year.

@wdbm

This comment has been minimized.

Copy link

wdbm commented Dec 11, 2019

DXVK is awesome. I simply will not use malware like Windows, so DXVK means I can enjoy the likes of Alien: Isolation or Gwent. Thank you. :)

@brenthuisman

This comment has been minimized.

Copy link

brenthuisman commented Dec 11, 2019

Hang in there @doitsujin, you've been doing the unimaginable with unimaginable success. Take care of yourself and take your time!

@julienbenjamin

This comment has been minimized.

Copy link

julienbenjamin commented Dec 11, 2019

Is this, because you consider DXVK to be feature complete?

It's because DXVK has become a fragile, unreliable and frustrating maintenance nightmare. Most of the 1.4.x releases introduced major regressions which I cannot reproduce, and therefore cannot debug and fix.

This includes GPU hangs in Overwatch on specific maps with Nvidia GPUs (some users claim it's fixed in 1.4.6 while others still have them), rendering issues in Dishonored 2 which I can't reproduce (see ValveSoftware/Proton#823 (comment)), vertex explosions in some games which I also can't reproduce, an ongoing Star Citizen issue which I still need to debug (see #1262), and tons of weird issues that don't make any sense whatsoever (like #1266 which only seems to affect RADV).

Most of these problems are still unresolved and I have no idea how to even track them down, let alone fix them, and the ones that got "fixed" got fixed by reverting otherwise useful changes because I simply do not understand any of the issues at all.

Doing any sort of active development with this broken mess of a code base would only make this worse, and I wish I had drawn the line sooner. The only thing I still plan to do is wire up some useful Vulkan extensions and eventually merge D9VK, the rest will be bug fixing only.

Is there any way to help you out, without knowing how DXVK has been designed?

@Fordi

This comment has been minimized.

Copy link

Fordi commented Dec 11, 2019

I don't know about the current structure, but in my experience, compatibility layers tend to start as a straightforward API surface, which descends into a mess of heuristics and hacks as edge cases are found and addressed, until a breaking point where Things Must Change. It sounds like you're nearing that crest (which sucks).

If you had all the time in the world, and didn't have to worry about risk, is there a way you would restructure DXVK to be easier to maintain? If you can define it readily, it might be that you can petition Valve to get you more resources for a re-arch to simplify maintenance without having to promise performance improvements for a while. Get them to commit to a bug moratorium and give you people to help you clean house.

But like others said, you should maybe take some vacation for the holiday season, do something nice for yourself. Graphics drivers and compatibility layers and other things users shouldn't notice are thankless, grueling work, rife with heisenbugs and opaque stacks - but you still manage to fix some crazy shit.

Let yourself have a real reward, and I think I speak for the whole community when I say you seriously deserve it.

Then, please, come back once again and be amazing.

@telmotrooper

This comment has been minimized.

Copy link

telmotrooper commented Dec 11, 2019

@doitsujin I just can't thank you enough for how much of a difference you've made in the status quo of computer gaming. Wish you best of luck!

@Joshua-Ashton

This comment has been minimized.

Copy link
Collaborator Author

Joshua-Ashton commented Dec 11, 2019

Come play TF2 with me and @liam-middlebrook 🐸

@doitsujin

This comment has been minimized.

Copy link
Owner

doitsujin commented Dec 11, 2019

If you had all the time in the world, and didn't have to worry about risk, is there a way you would restructure DXVK to be easier to maintain?

My main issue is that I simply don't understand most of the issues that are popping up everywhere lately, not that the code base itself is poorly structured. There are some things that could use some cleanups, but the last attempts at doing that have caused exactly the kind of regressions I was complaining about, and it really seems like touching anything causes some arbitrary breakage for no particular reason.

I don't know if we're running into driver issues (could be given that these things tend to be limited to a single vendor) or if there's some insane case of undefined behaviour going on in my code (of course dxvk being a windows library makes it a lot harder to debug such things since none of the standard tools like valgrind work properly), but it doesn't really matter either way, breakage is bad and should be avoided.

@Fordi

This comment has been minimized.

Copy link

Fordi commented Dec 11, 2019

If opacity is the problem, would the next big step be instrumentation?

@doitsujin

This comment has been minimized.

Copy link
Owner

doitsujin commented Dec 11, 2019

No since that would either cause a major performance hit even if it's not needed, or make everything incredibly messy due to #ifdef magic (or both).

@Fordi

This comment has been minimized.

Copy link

Fordi commented Dec 11, 2019

Could try some sugar that encapsulates basic patterns to avoid ifdef cruft, like:

#ifdef TRACING
#define INSTRUMENT(stmt) beginInstrument();\
 stmt \
endInstrument();
#else
#define INSTRUMENT(stmt) stmt
#endif

Then use the macro around code that you want to keep track of, e.g.,

int thingDoer() { INSTRUMENT(
  doTheThing();
) }

Dunno how generic it could be, or what insights you could gain, and I'm probably wasting your time, though. Don't know what the patterns would even look like. I just know I've used that kinda thing to make conditional compilation look a lot neater in my own code (which, admittedly, has been dumb little arduino stuff).

@doitsujin

This comment has been minimized.

Copy link
Owner

doitsujin commented Dec 11, 2019

I mean it's not like it couldn't be done, usually I do that manually when tracking down a crash or bug for the functions where it matters, but there are multiple issues with that approach.

  • We can't debug GPU hangs this way, at all. This needs GPU-side work, and obviously extra code on the CPU side to actually do that.
  • Even if purely CPU-side debugging is useful (e.g. when something segfaults), demanding games easily do >10 million API calls per second. Logging all of those, plus all the backend work, would be insane.
  • There's just so much going on on multiple threads that just spamming messages everywhere won't necessarily make it easier to figure out what the actual problem was. Wine's debug channels kind of have the same problem; if things don't work due to an unimplemented feature it's easy to figure out; if the problem sits deeper than that, it often isn't.
@barotto

This comment has been minimized.

Copy link

barotto commented Dec 11, 2019

The fact that many PC gamers like to overclock their systems is not helping either. I think one requirement for bug reporting should be "please, if your system is overclocked in any way (CPU, GPU, and/or RAM) retest at stock before reporting bugs."

@brianinthered

This comment has been minimized.

Copy link

brianinthered commented Dec 11, 2019

I also thank you Phillip for all of your dedication and hard work!
I test windows games in Wine on a daily basis and post step-by-step guides in PlayOnLinux weekly. Without DXVK most of my tests will fail because of DirectX 10/11
Thanks to you we have so many games and guides that allow us to play indie "windows only" games from itch.io and AAA games from GOG!

Definitely take a break and clear your head and give it a second pair of eyes later on. Refresh yourself and enjoy some time off. It won't affect us at all... we can wait! I've waited years to see Windows DirectX 10/11 games to run in Linux so I have no problem waiting a few weeks or even months!

By the way, what bugs have you noticed? Almost every Unity3D and Unreal Engine game I've tested runs great with every version of DXVK I test. I rarely notice any issues at all.

@dhollinger

This comment has been minimized.

Copy link

dhollinger commented Dec 11, 2019

@doitsujin Is there any situation where you'd feel comfortable handing the project maintainership off to another person or a community to maintain?

@o-bri

This comment has been minimized.

Copy link

o-bri commented Dec 11, 2019

Hi Philip,

thanks for your work so far!

I think it would be important, that you don not do all this work alone. It is amazing, that you came this far with this complex piece of work.

Perhaps you can take a break an Valve should organize, that you have 1-3 devs to help you and discuss problems an solutions with them.
I was working as a developer and I can remember thinking about bugs/problems for hours. And then simply explaining them to another dev made my brain understand it. Sometimes it really didn't even need an answer from the other one - just someone listening.

I hope this is not the end of the amazing duo DXVK/Proton.

Kind regards
Oliver

@gardotd426

This comment has been minimized.

Copy link

gardotd426 commented Dec 11, 2019

DXVK is awesome. I simply will not use malware like Windows, so DXVK means I can enjoy the likes of Alien: Isolation or Gwent. Thank you. :)

Alien: Isolation has a native version though?

Anyway, @doitsujin I just want to pile on with what everyone else said. Your project here has made a gigantic impact on my life, and I hope that isn't lost on you. A lot of us REALLY love games, but the thought of having to even dual boot and use Windows to play them makes us legitimately upset/sick. Sure, there are some huge games I'd love to play but can't because I won't make that sacrifice of using Windows at all, but because of you, and Valve, and @GloriousEggroll, and the people at @lutris, I can play MORE than enough amazing games. Without you, I honestly would have had to swallow my disgust and use Windows, but because of you, I get to keep thoroughly enjoying my computer while also retaining control of it and use the operating system I truly love.

You're honestly one of the Mount Rushmore faces of Linux Gaming along with GE and @Tk-Glitch, and even if you never develop DXVK again, you've done more than enough. Take a break.

@mphuZ

This comment has been minimized.

Copy link

mphuZ commented Dec 12, 2019

doitsujin, I hope you will put your efforts into the VKD3D project.

@brunonymous

This comment has been minimized.

Copy link

brunonymous commented Dec 12, 2019

Hello Philip,

I would like to thank you for all the work you have done on DXVK. Thanks to your work my son can play a lot of Windows games under Ubuntu Linux 18.04 (Overwatch, Monster Hunter: World, ...).

Thank you again and don't give up.

Best regards
Bruno

@Joshua-Ashton Joshua-Ashton force-pushed the Joshua-Ashton:dxvk-native branch 2 times, most recently from f358a97 to 7e78809 Dec 30, 2019
Joshua-Ashton added 23 commits Nov 16, 2019
Includes some basic defs/hresults etc

A setup for implementing __uuidof is also included using constexpr + templates
Use native std::threads, ids using a counter
Disable OpenVR for this WSI also.
…native

Currently unimplemented as there is no real nice way to do so without getting pthread involved.
@Joshua-Ashton Joshua-Ashton force-pushed the Joshua-Ashton:dxvk-native branch from 7e78809 to bd989ed Jan 2, 2020
@Diliz

This comment has been minimized.

Copy link

Diliz commented Jan 22, 2020

Hello!

Thanks for all the work, I wanted to post a big hug message to the community here, but most of the ppls would not have read it xD

So this is just a big thanks, much love and stay cool ppls, sometime take a break is the best way to handle things.

PS: If you don't know what's the problem with some buggy stuff, I recommend trying to list the broken stuff, and make a study with your mates, let some mates try to identify what's broken, more brains, more possibilities :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.