Elite Dangerous logs and trace

[agates]

The game freezes at "preparing planet generation system" after shader generation completes. This is before the main menu loads.

Software information

Elite Dangerous: Horizons

System information

• GPU: Radeon RX 580 Series (POLARIS10 / DRM 3.19.0 / 4.14.14-1-default, LLVM 5.0.1)
• Driver: RADV with Mesa 17.3.3
• Wine version: wine-2.20 (Staging)
• DXVK version: commit 59a4e80

Apitrace file(s)

• https://mega.nz/#!G891RZBL!wh-xGw0mdEmKxFRFCGJAKpTrthArzoPXAuNUiPwIsQM

Log files

• d3d11.log: EliteDangerous64_d3d11.log
• dxgi.log: EliteDangerous64_dxgi.log

[roothorick]

I wonder... if I'm remembering this correctly... does the non-Horizons version behave differently? IIRC you can choose between them in the launcher.

[agates]

@roothorick Yep, the non-Horizons client just gets a login error which is of course unrelated.

I think there's a demo of the game to try -- might do that next.

[doitsujin]

According to this video, the game seems to (sort of) work with latest master, but there are still validaton errors when replaying the apitrace.

[agates]

The video just covers the demo which doesn't include all of the functionality as far as I'm aware. I'll give this another shot sometime this week or weekend and report back :).

[varris1]

The game is freezing because most terrain shaders need 40+ seconds to compile. If you have DXVK_LOG_LEVEL=debug set, you can see it for yourself.

Just be patient and it will continue, despite looking like it's frozen. After the first start it will be quick.

[winiciuscota]

for me the demo version is able to launch but gets stuck within 1 minute.

The non horizon version only shows a black screen on launch, sometimes it loads the background music(still with the blackscreen).

Logs:

Demo version
EliteDangerous32_d3d11.log
EliteDangerous32_dxgi.log

Non horizon version
EliteDangerous64_d3d11.log
EliteDangerous64_dxgi.log

[doitsujin]

@varris1 sent me some of the shaders that take up to two minutes each to compile on the driver. We're talking about DXBC shaders with over 20'000 instructions, which translate to anything up to 170'000 SPIR-V instructions.

This is a game bug. Frontier probably thought it was a good idea to force loop unrolling in the HLSL compiler when it clearly wasn't.

[doitsujin]

Not sure how that qualifies as a game bug.

The offending shader has 170k instructions. Just to put this into perspective, the biggest regular shaders I've seen across a large number of games have 3-4k instructions at most (those work fine, but even that takes a while to compile). You cannot expect any Vulkan driver to make sense of the mess that this game generates.

[steffenWi]

Hah, yes I searched a bit and got numbers indicating a few hundred instructions is 'normal'.

That's why I removed my comment. Sorry.

[winiciuscota]

So there is no hope on seeing this game working on Linux unless Frontier fixes it on their side(which is never going to happen)?

[jp7677]

Would the shader cache cover these extreme shaders? Or ask differently, with a current DXVK version, are the "freezes" gone once the shader cache is filled?

[steffenWi]

Completely Offtopic but to answer you guys: I got to try E:D on an account of a friend a few days ago. Used Lutris with DXVK 0.9 and current wine. E:D Horizons would freeze my entire system shortly after hitting 18% in the planet generation thing.

Normal E:D and E:D Battle Arena started but threw an error that I should start the game from the launcher (I did) as there was a version mismatch. So this only seems to affect Horizons while the other modes suffer from other problems.

[doitsujin]

@jp7677 the problem is that the shader takes ridiculously long to compile (might even be several hours, not sure if anyone tried letting it run). The state cache isn't going to help here since the shader still needs to be compiled by the driver, and while the driver's shader cache is nice, it'll only solve that problem until the next update.

[jp7677]

@doitsujin ok, thanks for your insights.

[winiciuscota]

@doitsujin but how is dx11 able to handle that?

[doitsujin]

@winiciuscota That's something you need to ask driver developers. My guess is that since DXBC is already optimized by Microsoft's HLSL compiler and has its own register model, drivers just skip some optimization passes and may run much faster register allocation algorithms, which can speed up compilation quite a bit.

[fls2018]

@doitsujin

Game is now running in Wine at least with DXVK, turns out the CRC error was due to a Microsoft Cryptography machineid keys being mismatched in wine:

https://youtu.be/JcDY4WFENug

Proton still has a freezing issue which may or may not be due to the fact it doesn't have mono.

[steffenWi]

@fls2018 this bug/issue doesn't affect E:D. It only occurs in the E:D Horizon DLC

[fls2018]

@fls2018 this bug/issue doesn't affect E:D. It only occurs in the E:D Horizon DLC

I am playing horizons.

Edit: I'm on nvidia with the latest Vulkan 396.54.09 driver, maybe that's the difference.

[steffenWi]

Ah, sorry. That is interesting/weird.

[winiciuscota]

@fls2018 Have you tried the game without the Horizon DLC?

[Pecisk]

Let me explain this a bit. There are two game clients for Elite Dangerous. Core version is DX10 compatible and have regular shaders compilation at the beginning of the game. However, Horizons version, which allows you to land on planets, uses compute shaders, and requires DX11.3. So when planetary generation happens as mentioned in OP, it is where compute shaders are getting compiled.

As for whole story - CRC error got fixed, along with other smaller issues for ED, that's why there's now working ED Horizons version under Wine with dxvk.

[Pecisk]

Also it seems Nvidia works, and AMD doesn't, judging by this bug report.

[agates]

I just got Elite Dangerous (core, not Horizions) working. Haven't tested it extensively. Horizons still hangs for me on "preparing planet generation system". I think that's the AMD-specific issue. Otherwise shader generation is very quick on my system.

• GPU: Radeon RX 580 Series (POLARIS10, DRM 3.27.0, 4.19.0-999-lowlatency, LLVM 8.0.0)
• Driver: RADV with Mesa 18.3.0-devel - padoka PPA
• Wine version: wine-3.19 (Staging)
• DXVK version: Release 0.90

[hhtns]

Horizons still hangs for me on "preparing planet generation system". I think that's the AMD-specific issue.

Horizons works with AMD if you replace TerrainComputeShaders.csa with TerrainComputeShadersNvidia.csa in the game files.

It kind of hints the issue with the non-nvidia shaders is not the scalability of shader compilation, but then the nvidia file is a bit smaller so I guess it could be. It could be interesting if someone with an nvidia gpu tried switching the files the other way.

[agates]

Horizons still hangs for me on "preparing planet generation system". I think that's the AMD-specific issue.

Horizons work with AMD if you replace TerrainComputeShaders.csa with TerrainComputeShadersNvidia.csa in the game files.

Wow, that worked! At least it loaded correctly. Going to test planet landing now.

agates@tempest ~/EDLaunch $ find . -type f -name "TerrainComputeShaders*"
./Products/elite-dangerous-64/PlanetShaders/TerrainComputeShadersNvidia.csa
./Products/elite-dangerous-64/PlanetShaders/TerrainComputeShadersDP.csa
./Products/elite-dangerous-64/PlanetShaders/TerrainComputeShaders.csa
agates@tempest ~/EDLaunch $ cd ./Products/elite-dangerous-64/PlanetShaders/
agates@tempest ~/EDLaunch/Products/elite-dangerous-64/PlanetShaders $ ls -alh
total 63M
drwxr-xr-x 2 agates agates 4.0K Nov  4 11:51 .
drwxr-xr-x 9 agates agates 4.0K Nov  4 11:56 ..
-rw-r--r-- 1 agates agates  24M Nov  4 11:51 TerrainComputeShaders.csa
-rw-r--r-- 1 agates agates 4.4M Nov  4 11:51 TerrainComputeShadersDP.csa
-rw-r--r-- 1 agates agates  17M Nov  4 11:51 TerrainComputeShadersNvidia.csa
-rw-r--r-- 1 agates agates  19M Nov  4 11:51 TerrainPointsComputeShaders.csa
agates@tempest ~/EDLaunch/Products/elite-dangerous-64/PlanetShaders $ cp TerrainComputeShaders.csa TerrainComputeShaders.csa.old
agates@tempest ~/EDLaunch/Products/elite-dangerous-64/PlanetShaders $ cp TerrainComputeShadersNvidia.csa TerrainComputeShaders.csa

[agates]

Well, landing on a planet worked without a problem!

[varris1]

It could be interesting if someone with an nvidia gpu tried switching the files the other way.
Still seems to work fine. It just takes way longer to generate the planet shaders at the beginning. (GTX 970, 396.54.09)

[doitsujin]

@agates can you test if putting the following into a dxvk.conf fixes the problem when using the original TerrainComputeShaders.csa file:

dxgi.customVendorId = 10de
dxgi.customDeviceId = 1b83

You might need to set DXVK_CONFIG_FILE to the absolute path to your dxvk.conf.

This should make the game think it's running on a GTX 1060.

[agates]

@doitsujin That appears to have worked.

[doitsujin]

@agates alright, guess I'm going to enable that workaround by default for this game. Can you please post the full executable name of the game? Capitalization matters.

[fls2018]

The games executable is: EliteDangerous64.exe

But you may also need the launcher: EDLaunch.exe

[doitsujin]

The launcher should be irrelevant. Workaround already implemented in 3d2e5a7.

[Pecisk]

Thanks but I almost forgot, there is another executable. Not the main game but the combat demo, which has been quite helpful for those testing this:

EliteDangerous32.exe

Adding it would be pointless as 32-bit version of the game is discontinued. Also all testing is now done on 64-bit prefix anyway.

[dmarcuse]

The workaround is only necessary for the full game with the horizons expansion anyways, right?

[Pecisk]

The workaround is only necessary for the full game with the horizons expansion anyways, right?

Also true. Base game doesn't have compute shaders and regular shaders already worked fine.

[fls2018]

Never mind I didn't think of that as I don't have AMD myself, I deleted my post.

The demo is kind of useful though particularly in troubleshooting, a lot of people on the frontier forums are trying this out and a lot of the steps up until now have been challenging for some to follow. Getting the demo running is usually the first goal for most.

[agates]

Never mind I didn't think of that as I don't have AMD myself, I deleted my post.

The demo is kind of useful though particularly in troubleshooting, a lot of people on the frontier forums are trying this out and a lot of the steps up until now have been challenging for some to follow. Getting the demo running is usually the first goal for most.

That's a good point, but I don't think the demo has planetary landing anyway :). Unless it's been updated to include it.

[Hoimar]

Hi guys. My ED: Horizons exposes the exact same behavior that the OP had in his first post. Although this issue should already be solved, for me it isn't.
Here are my system specs:
OS: Xubuntu 18.04 with Linux 4.19.9-041909-generic
Wine: "wine-4.0-rc3 (Staging)" from official PPA
Graphics stack: Mesa 19.0.0-devel, LLVM 7.0.1 (oibaf PPA, so also latest RADV etc.), latest DXVK (currently 0.94).
GPU: AMD Radeon RX 580
The normal ED works, but Horizons hangs in the "preparing planet generation system" screen; after a while the Frontier crash reporter pops up, but the loading continues a few more seconds (farthest I got was 49% so far), then the whole ED client crashes. Also saw that it ate up all of my system memory (8GB) over time.
The DXVK logs didn't show me anything special, no errors, just normal things like "compiling compute shader…" etc. The only big difference between my setup and @agates' is that he has LLVM 8.0.0 .
What could be the issue? Where should I look? Is this even DXVK related?

Help is much appreciated.
Cheers

[agates]

@Hoimar I can think of a few things to try to narrow it down.

First and easiest, try a different DXVK version. I would try the version I tested with using the DXVK config file. If it happens to work, bisect the DXVK versions until it breaks so you can find it change. I don't think this is the cause, but if it happens to be then it would be useful to catch.

You can also try copying the affected compute shader file mentioned above.

Then, I would test with the Padoka PPA, instead of oibaf, which includes the newer LLVM you mentioned. This can definitely affect shader compilation.

https://launchpad.net/~paulo-miguel-dias/+archive/ubuntu/mesa/

To adequately switch PPAs, you might want to purge first oibaf with this utility: https://askubuntu.com/questions/307/how-can-ppas-be-removed

[Hoimar]

UPDATE: So after giving my system a rather huge swap file, ED had enough space to load etc., so it got through the shader compilation stage and I could start Horizons. BUT: It takes ages everytime I start Horizons to get past the "preparing planet generation system" stage (about 5-10min). Also, the Frontier crash reporter still pops up every time. But once ingame, it works at least.

Thanks for your answer, @agates , I'll try your recommendations next.

[agates]

Aha, I do have 32GB of ram in my system. It would not surprise me if that's the main differentiator.

EDIT: When you said it ate your system memory I saw 8GB and for some reason assumed it was the graphic memory

[Hoimar]

As we suspected @agates, the shader compilation was the problematic part of the whole. So switching the PPA from oibaf to padoka (LLVM 7.0.1 -> 8.0.0, Mesa stays the same) did the trick. It launches within 1min (also no crash reporter, I suspect it pops up when the ED Client becomes unresponsive) and works like a charm.
Thank @agates for his work here, @doitsujin for DXVK as a whole, and all other people involved for making running this game possible with Linux.

Cheers

[fls2018]

There appears to be two issues since the 3.3 update, one is the terrain issue but that's related to the latest Nvidia driver (also affects Windows on older hardware than Maxwell).

Second is I'm finding significant performance degradation with the new Volumetric Effects that were added in 3.3 on Linux vs Windows. I'm seeing up to a third of the performance I would be normally in some specific locations that probably leverage the new effects (Iota Hydri station, some asteroid rings). Graphics appears to be fully utilized in these scenarios but lower GPU power draw? Clocks also seem to be normal.

[TRPB]

This has returned for Odyssey, I'm trying to confirm that it's still correctly pretending to be an nvidia card but I'm not sure where to check.

edit: Swapping TerrainComputeShaders.csa for the nvidia version doesn't seem to help so it may not be the same problem. I'll try a couple of other things and create a new issue.

edit again: It does work eventually if you leave it long enough.