tsm-client package: broken on 20.09 due to "patchelf: unsupported overlap of SHT_NOTE and PT_NOTE"

[florianjacob]

Describe the bug
Upgrading to NixOS 20.09 failed for me because the tsm-client package is broken on 20.09.
patchelf: unsupported overlap of SHT_NOTE and PT_NOTE seems to be the main error message from the rebuild log output pasted below. While this sounds similar to NixOS/patchelf#255, tsm-client is written in Java and not in Rust.
The tsm-client package built fine on NixOS 20.03.

To Reproduce

programs.tsmClient.enable = true;

Expected behavior
The tsm-client package should compile on 20.09.

Screenshots

[…]
searching for dependencies of /nix/store/m0nw6ravvg8q6gzgh8mzghyvbcf977g2-tsm-client-8.1.8.0-unwrapped/opt/tivoli/tsm/client/ba/bin/dsmutillnx
  libstdc++.so.6 -> found: /nix/store/vran8acwir59772hj4vscr7zribvp7l5-gcc-9.3.0-lib/lib/libstdc++.so.6
setting RPATH to: /nix/store/bf82z1xgvpsidj26c12mm4r7fhijcpzz-lvm2-2.03.10/lib:/nix/store/vran8acwir59772hj4vscr7zribvp7l5-gcc-9.3.0-lib/lib
patchelf: unsupported overlap of SHT_NOTE and PT_NOTE
builder for '/nix/store/43w826xr9646yvhkz234wfl3hmiplyd8-tsm-client-8.1.8.0-unwrapped.drv' failed with exit code 1
cannot build derivation '/nix/store/151jk73200n362cy8bs2x3f7smijnnyy-tsm-client-8.1.8.0.drv': 1 dependencies couldn't be built
[…]

Notify maintainers
Maintainer: @Yarny0 can you reproduce this?

Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute: pkgs.tsm-client
# a list of nixos modules affected by the problem
module: programs.tsmClient services.tsmBackup

[Yarny0]

Hi @florianjacob , thanks for your report and research!

The problem is a bug in patchelf. There is already a patch in NixOS/patchelf#230, however, it didn't reach nixpkgs releases yet. Apparently this bug is rarely triggered.

I prepared a pull request for master here: #106340 . It just adds the patch to nixpkgs build recipe for patchelf. If the pull requests gets accepted, I will prepare a backporting pull request for release 20.09.

In the meantime, you can use the test recipe I attached to my pull request. This recipe builds a patched patchelf, then uses it to build tsm-client. I tested this successfully on unstable and 20.09 Maybe you can even assemble a package overlay to include this workaround script in your NixOS system configuration.

[florianjacob]

@Yarny0 I am so happy that you happen to know what you're doing here, and how to deal with this. 😅
Thanks for the information / PRs / workarounds, will go and create a package overlay later & report back.

[florianjacob]

Tried to get it to work with this overlay, but failed:

nixpkgs.config.overlays = [ (self: super: {
  patchelf = super.patchelf.overrideAttrs (oldAttrs: {
    patches = [(fetchpatch {
      name = "PR230-fix-false-alarm-for-non-overlapping-sections.patch";
      url = "https://raw.githubusercontent.com/Yarny0/nixpkgs/f9d5e1c8278cbd05148859b05016c01889351c59/pkgs/development/tools/misc/pa    tchelf/PR230-fix-false-alarm-for-non-overlapping-sections.patch";
      sha256 = "1l69grlij1ycqacf0lgka8dq14wdi49wn173hrw2vw18n8rz4md1";
    })];
  });
})];

The error message from tsm building stays the same, but probably the patch is just not really applied / a vanilla patchelf version is actually used, due to the bootstrapping role it plays in the build process. :/

[Yarny0]

Tried to get it to work with this overlay, but failed:

nixpkgs.config.overlays = [ (self: super: {
  patchelf = super.patchelf.overrideAttrs (oldAttrs: {
    patches = [(fetchpatch {
      name = "PR230-fix-false-alarm-for-non-overlapping-sections.patch";
      url = "https://raw.githubusercontent.com/Yarny0/nixpkgs/f9d5e1c8278cbd05148859b05016c01889351c59/pkgs/development/tools/misc/pa    tchelf/PR230-fix-false-alarm-for-non-overlapping-sections.patch";
      sha256 = "1l69grlij1ycqacf0lgka8dq14wdi49wn173hrw2vw18n8rz4md1";
    })];
  });
})];

The error message from tsm building stays the same, but probably the patch is just not really applied / a vanilla patchelf version is actually used, due to the bootstrapping role it plays in the build process. :/

I'm not sure if nixpkgs.config.overlays does anything at all. If you use nixpkgs.overlays instead, it does hav an effect: It complains about infinite recursion. The reason is that patchelf is a dependency of fetchpatch (see the discussion in #106340 for details. Maybe this can be work-arounded by using super.fetchpatch or self.fetchpatch -- I didn't test that.

You can also work around that by adding the patch file directly into your nixos configuration, but since patchelf is a dependency of the stdenv, nixos-rebuild will rebuild almost all packages locally then.

I guess there is no way around: You need to add an overlay that essentially replaces the build script of tsm-client with the one in my test recipe. This ensures the modified patchelf is only used in the tsm-client build.

[stale]

I marked this as stale due to inactivity. → More info

[florianjacob]

I really have to test this again with NixOS 21.05

[Yarny0]

With #135477 the branch staging-next got merged into master. This includes an update to patchelf version 0.13 in commit 4639832 . That update should fix the issue at hand. I successfully tested tsm-client 8.1.8.0 with recent unstable commit ff748a8 . However, I don't expect patchelf updates to be backported to NixOS 21.05 (let alone 20.09).

Given the opportunity, I filed a pull request to update tsm-client to the latest version: #138386 . I intend to file another pull request for NixOS 21.05 which backports the update and also includes the patchelf workaround from #106340 . However, to avoid duplicate work, I will wait until #138386 gets accepted.

Dear @florianjacob, sorry it took so long until it got fixed. Can you verify that tsm-client now works for you in nixos-unstable channel? If you can spare the time, maybe you can also test the changes in #138386 in your environment and post a review there.

[florianjacob]

No worries, thanks for investigating this intricate issue!
I'm sorry to say that I am unable to verify, as that server where I used tsm never got the upgrade to 21.05 and is decomissioned now. 😅

👍 for updating tsm-client to the latest version, that'll be helpful for the next time. :)

As the patchelf issues also appeared in other contexts and were reproduced by other people, and are presumably fixed now, I'll proceed and close this issue.