Cannot run x86 binaries on x86_64 (Exec format error)

[cmhe]

Issue description

I currently cannot execute any 32 bit Linux binaries on my 64 bit system and this breaks updates, since I have 32 bit support enabled in the nix options:

while setting up the build environment: executing '/nix/store/4qj9vzggfzsc34ci79ckaim5cjrwc6ns-bash-4.4-p23/bin/bash': Exec format error
builder for '/nix/store/hnr8gaipmzkrmbfzjzm21r46dw861m3y-libtxc_dxtn-1.0.1.drv' failed with exit code 1
building '/nix/store/bffa5kw5q1y7rsmh5yr7h04vyihhbsfv-main.conf.drv'...
cannot build derivation '/nix/store/3a7vkl1izryr1ak6cm6yydxrmqvhqqh5-mesa-drivers+txc-18.2.4.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/s0f52h6icvd6ccf7xvw6zxd7yjwikhng-opengl-drivers-32bit.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/3pamd9rd5pwv6l3wjz9ki42pss6fz5h1-etc-nixos.conf.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/1z8a8gcd01hm3spf9940rlpc0bj5nvmf-etc.drv': 1 dependencies couldn't be built

I remember that this worked about a week ago, so something has changed.

I am new to NixOS, so its possible that I have done something wrong. I used ArchLinux for about 9 years before that and now NixOS for just 3 months and its concepts are still foreign for me.

As a long term linux user (about 15 years) I do feel a bit out of place in NixOS. So I am not sure how to fix that myself.

Steps to reproduce

Not sure. I have wine installed and support for 32 bit games enabled.

Then I normally do just 'nixos-rebuild switch --upgrade' to upgrade, and sometimes 'nix-channel --update'. I also regularly do 'nix-collect-garbage -d'.

No idea if that is reproducible or not.

Technical details

 - system: `"x86_64-linux"`
 - host os: `Linux 4.20.0-rc3, NixOS, 19.03pre159739.3ae0407d3c9 (Koi)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.1.3`
 - channels(root): `"nixos-19.03pre159472.80738ed9dc0, nixpkgs-19.03pre160446.206a1c00bae"`
 - channels(ch): `""`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

[matthewbauer]

This might be from 4a8fc5b. Could you see if it works without that commit?

Also verify that /nix/store/4qj9vzggfzsc34ci79ckaim5cjrwc6ns-bash-4.4-p23/bin/bash executes. I can run it on my system. You can run this to verify that your store is not corrupted:

nix-store --verify --check-contents

[cmhe]

I solved it now for me.

Instead of using the 'linuxPackages_testing_hardened' 4.20 linux kernel I switched back to 'linuxPackages_4_19' 4.19.2 kernel and now executing 32 binaries work again.

So I guess the hardened kernel does not support executing 32 bit binaries.

Does it make sense to have some kind of visible warning here?

I normally would not have used the hardened kernel, if that wouldn't have been the only 4.20 version is I get. Its not clear to me when and how those changes in master appear in my nixpkgs/nixos channel (I am on unstable). I would like to use the master branch directly, but I haven't figured out how to best do that yet.

[vicgc]

The same here, hardened kernel 4.18.17 and build failed.

building '/nix/store/6h9asn7rc93dbwfcs6xaj3k0qgxmg0vh-libtxc_dxtn-1.0.1.drv'...
while setting up the build environment: executing '/nix/store/xgf4vra77qrww0n228hd80jsdccdp54w-bash-4.4-p23/bin/bash': Exec format error
builder for '/nix/store/6h9asn7rc93dbwfcs6xaj3k0qgxmg0vh-libtxc_dxtn-1.0.1.drv' failed with exit code 1
building '/nix/store/cfwj5yjby2xvagyj69zd1yx8hdhi27ay-lightdm.pam.drv'...
cannot build derivation '/nix/store/maynq4fxxjc217figygdrgg5q9xnw6yy-mesa-drivers+txc-18.1.7.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/wv9b4ayvgnpkvwz81zkk0lm9w4v5pd1y-opengl-drivers-32bit.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/q61vc27rdbcsrhxifiw89w1qdjax1amh-nixos-system-brix-18.09.1534.d45a0d7a4f5.drv': 1 dependencies couldn't be built
error: build of '/nix/store/q61vc27rdbcsrhxifiw89w1qdjax1amh-nixos-system-brix-18.09.1534.d45a0d7a4f5.drv' failed

Thanks

[ivan]

Maybe the hardened kernel shouldn't set IA32_EMULATION n when NixOS' bootstrap-tools/bin/bash is an ELF 32-bit LSB executable?

[matthewbauer]

It should be 64 bit for the x86_64-linux stdenv. But some packages use the i686 stdenv.

[NeQuissimus]

I am not too familiar with stdenv. Is this expected behaviour that a 64-bit system uses 32-bit binaries? This issue is reproducible building glibc as well.

[stale]

Thank you for your contributions.

This has been automatically marked as stale because it has had no activity for 180 days.

If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.

Here are suggestions that might help resolve this more quickly:

1. Search for maintainers and people that previously touched the related code and @ mention them in a comment.
2. Ask on the NixOS Discourse.
3. Ask on the #nixos channel on irc.freenode.net.

[gloaming]

This should be resolved by #82006 - please reopen if there's still a problem.