WIP: Mesa cache fix #93946

corngood · 2020-07-27T02:53:46Z

Motivation for this change

Fixes #92807

Things done

enable build-id for llvm and radv
remove old disk-cache-key patch from mesa

corngood · 2020-07-27T02:54:54Z

The mesa patch should probably go upstream. I'll look into that when I get a chance.

corngood · 2020-07-27T03:15:57Z

Upstream MR: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6081

primeos

Is this really related to #92807? Your case seems to differ since that issue was fixed by the update to Mesa 20.1.3 which didn't fix it for you. So you seem to be hitting another issue.

pkgs/development/compilers/llvm/10/llvm.nix

primeos · 2020-07-27T11:14:04Z

pkgs/development/libraries/mesa/default.nix

@@ -57,7 +57,7 @@ stdenv.mkDerivation {
  patches = [
    ./missing-includes.patch # dev_t needs sys/stat.h, time_t needs time.h, etc.-- fixes build w/musl
    ./opencl-install-dir.patch
-    ./disk_cache-include-dri-driver-path-in-cache-key.patch


We should check carefully if this patch is indeed not required anymore. From a quick look at the source-code it looks ok, but I haven't checked all drivers yet. Some still refer to the parameter as *timestamp (inconsistently even src/util/disk_cache.h) while using the build-id.

I guess I'm mainly concerned since #91145 apparently didn't affect all drivers.

I did a somewhat thorough check. All the dri and vulkan drivers have build-id.

lib/dri/radeonsi_drv_video.so does not have build-id, and does contain si_disk_cache_create. I'm not sure if that's a problem. None of the _drv_video.so libs or vdpau libs have build-id.

The problem with the cache key patch is that it only affects the actual disk cache keys, not other uses of cache_uuid, such as the one in radv.

I'd be happy just to merge a23c6888d399b291ba691e43028e1b295ad10efc while this gets investigated more.

This seems to be the only driver that still uses timestamps directly:

https://gitlab.freedesktop.org/mesa/mesa/-/blob/master/src/freedreno/vulkan/tu_device.c#L61

corngood · 2020-07-27T11:55:25Z

Is this really related to #92807? Your case seems to differ since that issue was fixed by the update to Mesa 20.1.3 which didn't fix it for you. So you seem to be hitting another issue.

I'm not sure if it's the same problem everyone in that thread was having, but this comment mentions ac_rtld error: !part->elf, which I believe is caused by the cache collision:

#92807 (comment)

Now that mesa uses build-id for cache keys, this shouldn't be necessary.

This is used by mesa drivers as a cache key. llvmPackages_9 is currently used by mesa in nixpkgs, so we just enable it there and in llvmPackages_10 so it doesn't get lost in future versions.

primeos · 2020-07-28T14:10:53Z

I'm not sure if it's the same problem everyone in that thread was having, but this comment mentions ac_rtld error: !part->elf, which I believe is caused by the cache collision:

Ok, yeah it seems like multiple things got mixed up in that issue.

I've gone ahead and pushed the important fix into staging 535a3e8 and rebased your branch.

Regarding the rest of the commits:

ff4481b: I'll try to double check / test this to avoid regressions but it LGTM (@corngood thanks for your great work btw!)
48d248e: I haven't yet checked if this works with patchelf 0.11 (Some Mesa drivers break with patchelf 0.11 #91145). Other than that I don't see any problems with it but I think someone else should review it (maybe @lovek323 @7c6f434c @dtzWill?).

vcunat · 2020-07-28T14:31:18Z

How is this related to patchelf 0.11? ATM we're still using 0.9 for mesa drivers because of that bug (and I can't see this PR changing that).

corngood · 2020-07-28T14:56:27Z

Well I guess the concern was patchelf 0.11 being using on libLLVM? I did build llvm 9 and 10, and the notes are intact. I also stepped through the uuid code to verify it gets used.

$ readelf -n result-lib/lib/libLLVM-10.so                                                                                                                                                                                                                                                ~/src/nixpkgs

Displaying notes found in: .note.gnu.build-id
  Owner                 Data size       Description
  GNU                  0x00000014       NT_GNU_BUILD_ID (unique build ID bitstring)
    Build ID: bf824074735a0a4a13681e577d65adc101281359

primeos · 2020-07-28T17:08:29Z

Well I guess the concern was patchelf 0.11 being using on libLLVM?

Yes, exactly.

I did build llvm 9 and 10, and the notes are intact. I also stepped through the uuid code to verify it gets used.

Awesome, thanks :)

@vcunat do you have an opinion / any comments regarding ff4481b? I think it's the way to go but I cannot really estimate the potential regressions. The main drivers should all be fine but as @corngood noted here not everything is using a build-id yet. If we encounter regressions we could then fix them properly (mesa shouldn't rely on any timestamps for reproducible builds and they seem to agree) but it might be difficult for users to track these regressions down (which is my main concern as I know how annoying graphics bugs can unfortunately be). But I don't have any objections if anyone would like to merge this right away, I just don't want to make that decision alone with my limited knowledge. (And I guess Mesa updates regularly come with some regressions/bugs anyway, so I might be just overthinking this.)

vcunat · 2020-07-28T17:15:31Z

I've never really looked into details around that patch.

corngood · 2020-07-28T17:20:44Z

@primeos I marked this WIP because I'm also concerned about regressions in ff4481b.

$ find `nix-build -A mesa_drivers` -name \*.so\* -type f | xargs readelf -n
File: /nix/store/c7sg1y7z9s13bnc117blfxqsjf6qmnv5-mesa-custom-20.1.4-drivers/lib/dri/i915_dri.so

Displaying notes found in: .note.gnu.build-id
  Owner                 Data size	Description
  GNU                  0x00000014	NT_GNU_BUILD_ID (unique build ID bitstring)
    Build ID: 2368960098c4ebc32b23d97d516d5d4abefb4e2f

File: /nix/store/c7sg1y7z9s13bnc117blfxqsjf6qmnv5-mesa-custom-20.1.4-drivers/lib/dri/i965_dri.so

Displaying notes found in: .note.gnu.build-id
  Owner                 Data size	Description
  GNU                  0x00000014	NT_GNU_BUILD_ID (unique build ID bitstring)
    Build ID: 2368960098c4ebc32b23d97d516d5d4abefb4e2f

File: /nix/store/c7sg1y7z9s13bnc117blfxqsjf6qmnv5-mesa-custom-20.1.4-drivers/lib/dri/radeon_dri.so

Displaying notes found in: .note.gnu.build-id
  Owner                 Data size	Description
  GNU                  0x00000014	NT_GNU_BUILD_ID (unique build ID bitstring)
    Build ID: 2368960098c4ebc32b23d97d516d5d4abefb4e2f

File: /nix/store/c7sg1y7z9s13bnc117blfxqsjf6qmnv5-mesa-custom-20.1.4-drivers/lib/dri/r200_dri.so

Displaying notes found in: .note.gnu.build-id
  Owner                 Data size	Description
  GNU                  0x00000014	NT_GNU_BUILD_ID (unique build ID bitstring)
    Build ID: 2368960098c4ebc32b23d97d516d5d4abefb4e2f

File: /nix/store/c7sg1y7z9s13bnc117blfxqsjf6qmnv5-mesa-custom-20.1.4-drivers/lib/dri/nouveau_vieux_dri.so

Displaying notes found in: .note.gnu.build-id
  Owner                 Data size	Description
  GNU                  0x00000014	NT_GNU_BUILD_ID (unique build ID bitstring)
    Build ID: 2368960098c4ebc32b23d97d516d5d4abefb4e2f

File: /nix/store/c7sg1y7z9s13bnc117blfxqsjf6qmnv5-mesa-custom-20.1.4-drivers/lib/dri/radeonsi_dri.so

Displaying notes found in: .note.gnu.build-id
  Owner                 Data size	Description
  GNU                  0x00000014	NT_GNU_BUILD_ID (unique build ID bitstring)
    Build ID: 75fba4914990966831d7cf85e525770d868c1ab9

File: /nix/store/c7sg1y7z9s13bnc117blfxqsjf6qmnv5-mesa-custom-20.1.4-drivers/lib/dri/nouveau_dri.so

Displaying notes found in: .note.gnu.build-id
  Owner                 Data size	Description
  GNU                  0x00000014	NT_GNU_BUILD_ID (unique build ID bitstring)
    Build ID: 75fba4914990966831d7cf85e525770d868c1ab9

File: /nix/store/c7sg1y7z9s13bnc117blfxqsjf6qmnv5-mesa-custom-20.1.4-drivers/lib/dri/swrast_dri.so

Displaying notes found in: .note.gnu.build-id
  Owner                 Data size	Description
  GNU                  0x00000014	NT_GNU_BUILD_ID (unique build ID bitstring)
    Build ID: 75fba4914990966831d7cf85e525770d868c1ab9

File: /nix/store/c7sg1y7z9s13bnc117blfxqsjf6qmnv5-mesa-custom-20.1.4-drivers/lib/dri/kms_swrast_dri.so

Displaying notes found in: .note.gnu.build-id
  Owner                 Data size	Description
  GNU                  0x00000014	NT_GNU_BUILD_ID (unique build ID bitstring)
    Build ID: 75fba4914990966831d7cf85e525770d868c1ab9

File: /nix/store/c7sg1y7z9s13bnc117blfxqsjf6qmnv5-mesa-custom-20.1.4-drivers/lib/dri/iris_dri.so

Displaying notes found in: .note.gnu.build-id
  Owner                 Data size	Description
  GNU                  0x00000014	NT_GNU_BUILD_ID (unique build ID bitstring)
    Build ID: 75fba4914990966831d7cf85e525770d868c1ab9

File: /nix/store/c7sg1y7z9s13bnc117blfxqsjf6qmnv5-mesa-custom-20.1.4-drivers/lib/dri/r300_dri.so

Displaying notes found in: .note.gnu.build-id
  Owner                 Data size	Description
  GNU                  0x00000014	NT_GNU_BUILD_ID (unique build ID bitstring)
    Build ID: 75fba4914990966831d7cf85e525770d868c1ab9

File: /nix/store/c7sg1y7z9s13bnc117blfxqsjf6qmnv5-mesa-custom-20.1.4-drivers/lib/dri/r600_dri.so

Displaying notes found in: .note.gnu.build-id
  Owner                 Data size	Description
  GNU                  0x00000014	NT_GNU_BUILD_ID (unique build ID bitstring)
    Build ID: 75fba4914990966831d7cf85e525770d868c1ab9

File: /nix/store/c7sg1y7z9s13bnc117blfxqsjf6qmnv5-mesa-custom-20.1.4-drivers/lib/dri/vmwgfx_dri.so

Displaying notes found in: .note.gnu.build-id
  Owner                 Data size	Description
  GNU                  0x00000014	NT_GNU_BUILD_ID (unique build ID bitstring)
    Build ID: 75fba4914990966831d7cf85e525770d868c1ab9

File: /nix/store/c7sg1y7z9s13bnc117blfxqsjf6qmnv5-mesa-custom-20.1.4-drivers/lib/dri/virtio_gpu_dri.so

Displaying notes found in: .note.gnu.build-id
  Owner                 Data size	Description
  GNU                  0x00000014	NT_GNU_BUILD_ID (unique build ID bitstring)
    Build ID: 75fba4914990966831d7cf85e525770d868c1ab9

File: /nix/store/c7sg1y7z9s13bnc117blfxqsjf6qmnv5-mesa-custom-20.1.4-drivers/lib/dri/r600_drv_video.so

File: /nix/store/c7sg1y7z9s13bnc117blfxqsjf6qmnv5-mesa-custom-20.1.4-drivers/lib/dri/radeonsi_drv_video.so

File: /nix/store/c7sg1y7z9s13bnc117blfxqsjf6qmnv5-mesa-custom-20.1.4-drivers/lib/dri/nouveau_drv_video.so

File: /nix/store/c7sg1y7z9s13bnc117blfxqsjf6qmnv5-mesa-custom-20.1.4-drivers/lib/vdpau/libvdpau_r300.so.1.0.0

File: /nix/store/c7sg1y7z9s13bnc117blfxqsjf6qmnv5-mesa-custom-20.1.4-drivers/lib/vdpau/libvdpau_r600.so.1.0.0

File: /nix/store/c7sg1y7z9s13bnc117blfxqsjf6qmnv5-mesa-custom-20.1.4-drivers/lib/vdpau/libvdpau_radeonsi.so.1.0.0

File: /nix/store/c7sg1y7z9s13bnc117blfxqsjf6qmnv5-mesa-custom-20.1.4-drivers/lib/vdpau/libvdpau_nouveau.so.1.0.0

File: /nix/store/c7sg1y7z9s13bnc117blfxqsjf6qmnv5-mesa-custom-20.1.4-drivers/lib/d3d/d3dadapter9.so.1.0.0

Displaying notes found in: .note.gnu.build-id
  Owner                 Data size	Description
  GNU                  0x00000014	NT_GNU_BUILD_ID (unique build ID bitstring)
    Build ID: 5630f668630388d1f46b11c54117d16b0c75b69e

File: /nix/store/c7sg1y7z9s13bnc117blfxqsjf6qmnv5-mesa-custom-20.1.4-drivers/lib/libXvMCr600.so.1.0.0

File: /nix/store/c7sg1y7z9s13bnc117blfxqsjf6qmnv5-mesa-custom-20.1.4-drivers/lib/libXvMCnouveau.so.1.0.0

File: /nix/store/c7sg1y7z9s13bnc117blfxqsjf6qmnv5-mesa-custom-20.1.4-drivers/lib/libxatracker.so.2.5.0

File: /nix/store/c7sg1y7z9s13bnc117blfxqsjf6qmnv5-mesa-custom-20.1.4-drivers/lib/libvulkan_intel.so

Displaying notes found in: .note.gnu.build-id
  Owner                 Data size	Description
  GNU                  0x00000014	NT_GNU_BUILD_ID (unique build ID bitstring)
    Build ID: 9d2f81e6a959ecf55acc1f5e1c34e14d980221c4

File: /nix/store/c7sg1y7z9s13bnc117blfxqsjf6qmnv5-mesa-custom-20.1.4-drivers/lib/libvulkan_radeon.so

Displaying notes found in: .note.gnu.build-id
  Owner                 Data size	Description
  GNU                  0x00000014	NT_GNU_BUILD_ID (unique build ID bitstring)
    Build ID: 955bd5fbe91cb8c722cf750b483c2376ba0f3faa

File: /nix/store/c7sg1y7z9s13bnc117blfxqsjf6qmnv5-mesa-custom-20.1.4-drivers/lib/libEGL_mesa.so.0.0.0

File: /nix/store/c7sg1y7z9s13bnc117blfxqsjf6qmnv5-mesa-custom-20.1.4-drivers/lib/libGLX_mesa.so.0.0.0

si_disk_cache_create is included in a bunch of these libraries, including the non-radeon ones, which is odd.

This is used by mesa.drivers (still on LLVM 9) as a cache key. I've ported that change to LLVM 11 to test it and so that it doesn't get lost in future versions. Credit for the change goes to David McFarland. See #93946 for details. Co-Authored-By: David McFarland <corngood@gmail.com>

jansol · 2020-10-21T15:55:54Z

It's been a while, this PR can probably be closed as outdated since upstream has updated after merging the fix from here already.

@corngood unless I missed something in the discussion? Admittedly I didn't follow it too closely.

corngood · 2020-10-21T16:05:28Z

@jansol yeah, we don't need this PR. I'll close it. Thanks.

This is used by mesa.drivers (still on LLVM 9) as a cache key. I've ported that change to LLVM 11 to test it and so that it doesn't get lost in future versions. Credit for the change goes to David McFarland. See NixOS#93946 for details. Co-Authored-By: David McFarland <corngood@gmail.com>

corngood requested review from FRidh, jonringer and matthewbauer as code owners July 27, 2020 02:53

corngood changed the base branch from master to staging July 27, 2020 02:54

corngood mentioned this pull request Jul 27, 2020

steam crashes amdgpu with mesa 20.0.8 #92807

Closed

ofborg bot added 6.topic: python and removed 6.topic: python labels Jul 27, 2020

ofborg bot requested review from ttuegel, 7c6f434c, dtzWill, lovek323, vcunat, jtojnar and primeos July 27, 2020 03:02

ofborg bot added 10.rebuild-darwin: 501+ 10.rebuild-darwin: 5001+ 10.rebuild-linux: 501+ 10.rebuild-linux: 5001+ labels Jul 27, 2020

ofborg bot added 10.rebuild-darwin: 1001-2500 and removed 10.rebuild-darwin: 5001+ labels Jul 27, 2020

corngood force-pushed the mesa-cache-fix branch from 1568607 to 9f47644 Compare July 27, 2020 03:54

primeos reviewed Jul 27, 2020

View reviewed changes

jtojnar removed request for lovek323, ttuegel, jtojnar, dtzWill and vcunat July 27, 2020 12:48

jtojnar removed the request for review from 7c6f434c July 27, 2020 12:48

corngood added 2 commits July 28, 2020 15:59

mesa: remove disk-cache-key patch

ff4481b

Now that mesa uses build-id for cache keys, this shouldn't be necessary.

llvm: enable build-id when linking libLLVM shared libs

48d248e

This is used by mesa drivers as a cache key. llvmPackages_9 is currently used by mesa in nixpkgs, so we just enable it there and in llvmPackages_10 so it doesn't get lost in future versions.

primeos force-pushed the mesa-cache-fix branch from 9f47644 to 48d248e Compare July 28, 2020 14:02

ofborg bot requested review from 7c6f434c, primeos, lovek323, dtzWill and vcunat July 28, 2020 14:11

corngood changed the title ~~Mesa cache fix~~ WIP: Mesa cache fix Jul 28, 2020

primeos mentioned this pull request Jul 31, 2020

MESA-LOADER: failed to open $driver #94315

Closed

FRidh added the 2.status: work-in-progress This PR isn't done label Aug 15, 2020

primeos mentioned this pull request Aug 20, 2020

helm: fails to start with MESA errors #95806

Closed

corngood closed this Oct 21, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP: Mesa cache fix #93946

WIP: Mesa cache fix #93946

corngood commented Jul 27, 2020 •

edited

Loading

corngood commented Jul 27, 2020

corngood commented Jul 27, 2020

primeos left a comment

primeos Jul 27, 2020

corngood Jul 27, 2020

corngood Jul 27, 2020

corngood commented Jul 27, 2020

primeos commented Jul 28, 2020 •

edited

Loading

vcunat commented Jul 28, 2020 •

edited

Loading

corngood commented Jul 28, 2020

primeos commented Jul 28, 2020

vcunat commented Jul 28, 2020

corngood commented Jul 28, 2020

jansol commented Oct 21, 2020

corngood commented Oct 21, 2020

WIP: Mesa cache fix #93946

WIP: Mesa cache fix #93946

Conversation

corngood commented Jul 27, 2020 • edited Loading

Motivation for this change

Things done

corngood commented Jul 27, 2020

corngood commented Jul 27, 2020

primeos left a comment

Choose a reason for hiding this comment

primeos Jul 27, 2020

Choose a reason for hiding this comment

corngood Jul 27, 2020

Choose a reason for hiding this comment

corngood Jul 27, 2020

Choose a reason for hiding this comment

corngood commented Jul 27, 2020

primeos commented Jul 28, 2020 • edited Loading

vcunat commented Jul 28, 2020 • edited Loading

corngood commented Jul 28, 2020

primeos commented Jul 28, 2020

vcunat commented Jul 28, 2020

corngood commented Jul 28, 2020

jansol commented Oct 21, 2020

corngood commented Oct 21, 2020

corngood commented Jul 27, 2020 •

edited

Loading

primeos commented Jul 28, 2020 •

edited

Loading

vcunat commented Jul 28, 2020 •

edited

Loading