Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stdenv: Add hack to fix cmake canExecute cross-compilation #251853

Merged
merged 1 commit into from
Sep 2, 2023

Conversation

Artturin
Copy link
Member

Fixes pkgsCross.musl64.llvmPackages_16.clang.cc on x86_64-linux, which used to fail with /bin/sh: clang-tblgen: not found.

Same hack is used in other projects:
https://github.com/search?q=%2FCMAKE_CROSSCOMPILING_EMULATOR.%2B%5C%2Fusr%5C%2Fbin%5C%2Fenv%2F+NOT+is%3Afork&type=code

Comment from https://github.com/vsrinivas/fuchsia/blob/30435a9d0f0b67c94e3c70760b522c9f7fbbd6be/build/cmake/HostLinuxToolchain.cmake#L64

Required to run host Linux executables during the build itself.
An example would be https://gitub.com/KhronosGroup/Vulkan-Loader and
its "asm_offset" program.

NOTE: Alternatives have been tried unsuccessfully, i.e.:

With $(set CMAKE_CROSSCOMPILING_EMULATOR), the build fails because
the CMake ninja/Make script tries to find the executable in the current
path, as in:

[3/16] Generating gen_defines.asm
FAILED: loader/gen_defines.asm
cd /tmp/cc/build-Vulkan-Loader/loader && asm_offset GAS
/bin/sh: asm_offset: command not found
ninja: build stopped: subcommand failed.

With $(set CMAKE_CROSSCOMPILING_EMULATOR ""), the build fails because
the shell cannot find the "" program as in:

[3/16] Generating gen_defines.asm
FAILED: loader/gen_defines.asm
cd /tmp/cc/build-Vulkan-Loader/loader && "" /tmp/cc/build-Vulkan-Loader/loader/asm_offset GAS
/bin/sh: : command not found
ninja: build stopped: subcommand failed.

It seems that the root of the problem comes from how the CMake function
cmCustomCommandGenerator::GetArgc0Location() computes the target
executable's location. At this point it's unclear whether this is a CMake
bug or a feature.

Risicle discovered this hack.

Description of changes

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 23.11 Release Notes (or backporting 23.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

@Artturin Artturin added the 6.topic: cross-compilation Building packages on a different sort platform than than they will be run on label Aug 27, 2023
@Artturin Artturin requested a review from a user August 27, 2023 22:36
@github-actions github-actions bot added the 6.topic: stdenv Standard environment label Aug 27, 2023
Fixes `pkgsCross.musl64.llvmPackages_16.clang.cc` on `x86_64-linux`,
which used to fail with `/bin/sh: clang-tblgen: not found`.

Same hack is used in other projects:
https://github.com/search?q=%2FCMAKE_CROSSCOMPILING_EMULATOR.%2B%5C%2Fusr%5C%2Fbin%5C%2Fenv%2F+NOT+is%3Afork&type=code

Comment from https://github.com/vsrinivas/fuchsia/blob/30435a9d0f0b67c94e3c70760b522c9f7fbbd6be/build/cmake/HostLinuxToolchain.cmake#L64

> Required to run host Linux executables during the build itself.
> An example would be https://gitub.com/KhronosGroup/Vulkan-Loader and
> its "asm_offset" program.
>
> NOTE: Alternatives have been tried unsuccessfully, i.e.:
>
>  With $(set CMAKE_CROSSCOMPILING_EMULATOR), the build fails because
>  the CMake ninja/Make script tries to find the executable in the current
>  path, as in:
>
>    [3/16] Generating gen_defines.asm
>    FAILED: loader/gen_defines.asm
>    cd /tmp/cc/build-Vulkan-Loader/loader && asm_offset GAS
>    /bin/sh: asm_offset: command not found
>    ninja: build stopped: subcommand failed.
>
> With $(set CMAKE_CROSSCOMPILING_EMULATOR ""), the build fails because
> the shell cannot find the "" program as in:
>
>    [3/16] Generating gen_defines.asm
>    FAILED: loader/gen_defines.asm
>    cd /tmp/cc/build-Vulkan-Loader/loader && "" /tmp/cc/build-Vulkan-Loader/loader/asm_offset GAS
>    /bin/sh: : command not found
>    ninja: build stopped: subcommand failed.
>
> It seems that the root of the problem comes from how the CMake function
> cmCustomCommandGenerator::GetArgc0Location() computes the target
> executable's location. At this point it's unclear whether this is a CMake
> bug or a feature.

Risicle discovered this hack.

Co-authored-by: Robert Scott <code@humanleg.org.uk>
@Artturin Artturin changed the base branch from master to staging August 29, 2023 19:12
Copy link
Contributor

@Mindavi Mindavi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's try it out and see

@Artturin Artturin merged commit 0e7c244 into NixOS:staging Sep 2, 2023
22 checks passed
@Artturin Artturin deleted the cmakefix1 branch September 2, 2023 00:53
@hmenke
Copy link
Member

hmenke commented Oct 10, 2023

Tests are failing now: nix-build -A pkgsStatic.libjpeg_turbo

build.log

@alyssais
Copy link
Member

I got a fix for libjpeg-turbo accepted upstream, but we'll need to update libjpeg_turbo in Nixpkgs before we can apply it. #285136

alyssais added a commit to alyssais/nixpkgs that referenced this pull request Feb 1, 2024
alyssais added a commit to alyssais/nixpkgs that referenced this pull request Feb 1, 2024
@alyssais
Copy link
Member

alyssais commented Feb 1, 2024

#285624

alyssais added a commit to alyssais/nixpkgs that referenced this pull request Feb 2, 2024
alyssais added a commit to alyssais/nixpkgs that referenced this pull request Feb 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.topic: cross-compilation Building packages on a different sort platform than than they will be run on 6.topic: stdenv Standard environment 10.rebuild-darwin: 1-10 10.rebuild-darwin: 1 10.rebuild-linux: 101-500
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants