New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Always cross compile #21471

Open
Ericson2314 opened this Issue Dec 28, 2016 · 24 comments

Comments

Projects
None yet
5 participants
@Ericson2314
Member

Ericson2314 commented Dec 28, 2016

Exherbo, another linux distribution that has done good work with cross-compiling, always builds cross compilers---i.e. --host --build and --target are always passed to gcc's build system even if some of those platforms are the same: http://git.exherbo.org/arbor.git/tree/packages/sys-devel/gcc/gcc.exlib#n187. The big advantage here is by using the cross-compiling code path in all cases, there's less to maintain, and actual cross compilation is less likely to rot assuming native compilation will always be better tested.

In #21268 (specifically 633feb4 but I'll probably rebase at some point breaking that link), I introduce always-defined buildPlatform hostPlatform and targetPlatform for the same reasons (I keep around nullable crossSystem and not-always present stdenv.cross for compatibility). So if/when that PR is merged would be a good time to tackle this. that PR will probably be closed as a bunch else happened separately, including always defining buildPlatform hostPlatform and targetPlatform and removing crossSystem and stdenv.cross altogether.

Naturally this would be a gcc-caused mass rebuild on Linux. Darwin shouldn't be as affected as LLVM, by default, includes all the targets we care about.

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Jan 26, 2017

Member

@DavidEGrayson's work in https://github.com/DavidEGrayson/nixcrpkgs/ should greatly accelerate this!

Member

Ericson2314 commented Jan 26, 2017

@DavidEGrayson's work in https://github.com/DavidEGrayson/nixcrpkgs/ should greatly accelerate this!

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Jan 26, 2017

Member

@cleverca22 points out https://github.com/taktoa/arcane-chat/blob/master/default.nix has a bunch of hacks needed to get windows to work. Would be good to keep these things in mind.

Member

Ericson2314 commented Jan 26, 2017

@cleverca22 points out https://github.com/taktoa/arcane-chat/blob/master/default.nix has a bunch of hacks needed to get windows to work. Would be good to keep these things in mind.

@Ericson2314 Ericson2314 added this to the 18.03 milestone Aug 20, 2017

@Ericson2314 Ericson2314 referenced this issue Aug 20, 2017

Merged

binutils: Pass --build --host on non-arm #28374

0 of 8 tasks complete

@Ericson2314 Ericson2314 modified the milestones: 18.03, 17.09, 18.09 Mar 15, 2018

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Mar 15, 2018

Member

OK, hope this actually happens this cycle 🤞.

Member

Ericson2314 commented Mar 15, 2018

OK, hope this actually happens this cycle 🤞.

@wmertens

This comment has been minimized.

Show comment
Hide comment
@wmertens

wmertens Mar 15, 2018

Contributor

This will be seriously amazing!

So after a quick look, this change requires changing all occurrences of crossSystem to hostPlatform or targetPlatform? Is there no way to guess the correct value at evaluation time?

In other words, it's a huge single-shot change and after that it will just work?

If so, I propose we declare a week "cross-compile week" and only merge changes that make this work. Once everything builds on staging we merge to master and we're done?

Or is there more to it? Will there be things that break?

Contributor

wmertens commented Mar 15, 2018

This will be seriously amazing!

So after a quick look, this change requires changing all occurrences of crossSystem to hostPlatform or targetPlatform? Is there no way to guess the correct value at evaluation time?

In other words, it's a huge single-shot change and after that it will just work?

If so, I propose we declare a week "cross-compile week" and only merge changes that make this work. Once everything builds on staging we merge to master and we're done?

Or is there more to it? Will there be things that break?

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Mar 15, 2018

Member

The crossSystem vs hostPlatform things are actually all taken care of, now! What's left is:

  • #36867 LLVM-based cross compilation
  • Change configureFlags on binutils and gcc to be [ "host" "build" "target" ]. Watch out for this arm (armv5 I think it is?) case where then things fail to build. Also do for ghc and any other compiler where we have configureFlags set today. Also make the various cases of targetPrefix unconditionally hostPlatform.config "-". (Must be done with the previous.
Member

Ericson2314 commented Mar 15, 2018

The crossSystem vs hostPlatform things are actually all taken care of, now! What's left is:

  • #36867 LLVM-based cross compilation
  • Change configureFlags on binutils and gcc to be [ "host" "build" "target" ]. Watch out for this arm (armv5 I think it is?) case where then things fail to build. Also do for ghc and any other compiler where we have configureFlags set today. Also make the various cases of targetPrefix unconditionally hostPlatform.config "-". (Must be done with the previous.
@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Mar 27, 2018

Member

stdenv.cross and `crossSystem are already sufficiently purged!

The first thing to do is just make all targetPrefixes unconditional, and also --target for compilers that might use it. Then start passing --build and --host to everything else. @lheckemann found a safer alternative for that second step.

Member

Ericson2314 commented Mar 27, 2018

stdenv.cross and `crossSystem are already sufficiently purged!

The first thing to do is just make all targetPrefixes unconditional, and also --target for compilers that might use it. Then start passing --build and --host to everything else. @lheckemann found a safer alternative for that second step.

@lheckemann

This comment has been minimized.

Show comment
Hide comment
@lheckemann

lheckemann Mar 27, 2018

Member

Said alternative is setting the build_alias, host_alias and target_alias environment variables. However, I'm not sure if this is really a desirable solution, both because it'll only work for autotools-based configure scripts and because I'm not sure this way of setting the platforms is actually considered a public/documented/stable interface by autotools.

Member

lheckemann commented Mar 27, 2018

Said alternative is setting the build_alias, host_alias and target_alias environment variables. However, I'm not sure if this is really a desirable solution, both because it'll only work for autotools-based configure scripts and because I'm not sure this way of setting the platforms is actually considered a public/documented/stable interface by autotools.

@wmertens

This comment has been minimized.

Show comment
Hide comment
@wmertens

wmertens Mar 27, 2018

Contributor
Contributor

wmertens commented Mar 27, 2018

@lheckemann

This comment has been minimized.

Show comment
Hide comment
@lheckemann

lheckemann Mar 27, 2018

Member

No. The derivation hashes will be different, and as a result the output hashes will also be different. In an ideal scenario the only differences between cross-compiled and natively-compiled outputs will be the store paths, but I highly doubt that we'll achieve that in practice.

Member

lheckemann commented Mar 27, 2018

No. The derivation hashes will be different, and as a result the output hashes will also be different. In an ideal scenario the only differences between cross-compiled and natively-compiled outputs will be the store paths, but I highly doubt that we'll achieve that in practice.

@wmertens

This comment has been minimized.

Show comment
Hide comment
@wmertens

wmertens Mar 27, 2018

Contributor
Contributor

wmertens commented Mar 27, 2018

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Mar 27, 2018

Member

@wmertens Yes that is a goal, but it will be very hard. CAS / intentional store is necessary, but not sufficient, as the input binaries for the build tools would not be anywhere near bit-for-bit the same.

[Also, you mean build platform not host platform, BTW. The names are indeed awful.]

Member

Ericson2314 commented Mar 27, 2018

@wmertens Yes that is a goal, but it will be very hard. CAS / intentional store is necessary, but not sufficient, as the input binaries for the build tools would not be anywhere near bit-for-bit the same.

[Also, you mean build platform not host platform, BTW. The names are indeed awful.]

@lheckemann

This comment has been minimized.

Show comment
Hide comment
@lheckemann

lheckemann May 9, 2018

Member

@dezgeg raised the concern of tests not being run when cross-compiling. Does exherbo maybe have a solution that we can steal?

Member

lheckemann commented May 9, 2018

@dezgeg raised the concern of tests not being run when cross-compiling. Does exherbo maybe have a solution that we can steal?

@eternaleye

This comment has been minimized.

Show comment
Hide comment
@eternaleye

eternaleye May 9, 2018

@lheckemann: My experience on Exherbo has been that tests run just fine when build == host (build == run), and would likely run just fine any time host (run) is runnable (such as i686 or x32 on an x86_64 builder). That could be further expanded using binfmt_misc and qemu-user.

eternaleye commented May 9, 2018

@lheckemann: My experience on Exherbo has been that tests run just fine when build == host (build == run), and would likely run just fine any time host (run) is runnable (such as i686 or x32 on an x86_64 builder). That could be further expanded using binfmt_misc and qemu-user.

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 May 9, 2018

Member

I can't find it well on my phone, but the answer must be in https://git.exherbo.org/paludis/paludis.git/

Member

Ericson2314 commented May 9, 2018

I can't find it well on my phone, but the answer must be in https://git.exherbo.org/paludis/paludis.git/

@eternaleye

This comment has been minimized.

Show comment
Hide comment
@eternaleye

eternaleye May 9, 2018

This is the relevant bit, but is overconservative - it just checks for build == host (build == run); though it misnames 'host'/'run' as 'target'.

eternaleye commented May 9, 2018

This is the relevant bit, but is overconservative - it just checks for build == host (build == run); though it misnames 'host'/'run' as 'target'.

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 May 9, 2018

Member

@eternaleye err we mean how are (autoconf-based) configure scripts invoked. I see default_src_configure but not where the various env vars it uses are defined.

Member

Ericson2314 commented May 9, 2018

@eternaleye err we mean how are (autoconf-based) configure scripts invoked. I see default_src_configure but not where the various env vars it uses are defined.

@eternaleye

This comment has been minimized.

Show comment
Hide comment
@eternaleye

eternaleye May 9, 2018

@Ericson2314: All the DEFAULT_SRC_CONFIGURE_* variables are set by individual packages. They aren't the defaults for src_configure; they allow packages to parameterize the default implementation of src_configure.

You probably want to look at the definition of econf instead.

eternaleye commented May 9, 2018

@Ericson2314: All the DEFAULT_SRC_CONFIGURE_* variables are set by individual packages. They aren't the defaults for src_configure; they allow packages to parameterize the default implementation of src_configure.

You probably want to look at the definition of econf instead.

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 May 10, 2018

Member

@eternaleye
https://git.exherbo.org/paludis/paludis.git/tree/paludis/repositories/e/ebuild/exheres-0/build_functions.bash?h=cross#n155 huh so --build and --host indeed are always passed if CBUILD and CHOST are defined. Do you all indeed never have configure run programs then? Brave!

Member

Ericson2314 commented May 10, 2018

@eternaleye
https://git.exherbo.org/paludis/paludis.git/tree/paludis/repositories/e/ebuild/exheres-0/build_functions.bash?h=cross#n155 huh so --build and --host indeed are always passed if CBUILD and CHOST are defined. Do you all indeed never have configure run programs then? Brave!

@eternaleye

This comment has been minimized.

Show comment
Hide comment
@eternaleye

eternaleye May 10, 2018

That's a work in progress. The most problematic cases are actually things like gobject-introspection, which is inherently incompatible with cross-compilation at the architectural level. Things like that work in practice when build == host (build == run), but fail when "really" cross-compiling. The ability to ban commands in the environment is the main tool used to fix such issues where possible - for example, we ban unprefixed pkg-config so we can learn what needs patched to respect --host and call ${HOST}-pkg-config instead.

Edit: Also, remember that configure is just shell templated by m4. Exherbo can't "never have configure run programs" - that's in upstream's hands. We can, however, try to detect when they do something incorrect so we can send them a patch - to prefix pkg-config, to use pkg-config instead of archaic foo-config programs, to use the correct include dir, or other such issues.

eternaleye commented May 10, 2018

That's a work in progress. The most problematic cases are actually things like gobject-introspection, which is inherently incompatible with cross-compilation at the architectural level. Things like that work in practice when build == host (build == run), but fail when "really" cross-compiling. The ability to ban commands in the environment is the main tool used to fix such issues where possible - for example, we ban unprefixed pkg-config so we can learn what needs patched to respect --host and call ${HOST}-pkg-config instead.

Edit: Also, remember that configure is just shell templated by m4. Exherbo can't "never have configure run programs" - that's in upstream's hands. We can, however, try to detect when they do something incorrect so we can send them a patch - to prefix pkg-config, to use pkg-config instead of archaic foo-config programs, to use the correct include dir, or other such issues.

@dezgeg

This comment has been minimized.

Show comment
Hide comment
@dezgeg

dezgeg May 11, 2018

Contributor

There was a misunderstanding, by "tests" I mean this particular autoconf macro which can spew the "cannot run test program while cross compiling" error:

# AC_RUN_IFELSE(PROGRAM,
#               [ACTION-IF-TRUE], [ACTION-IF-FALSE],
#               [ACTION-IF-CROSS-COMPILING = RUNTIME-ERROR])
# ----------------------------------------------------------
# Compile, link, and run. Requires that the compiler for the current
# language was checked for, hence do not use this macro in macros looking
# for a compiler.
AC_DEFUN([AC_RUN_IFELSE],
[AC_LANG_COMPILER_REQUIRE()dnl
m4_ifval([$4], [],
	 [AC_DIAGNOSE([cross],
		     [$0 called without default to allow cross compiling])])dnl
AS_IF([test "$cross_compiling" = yes],
  [m4_default([$4],
	   [AC_MSG_FAILURE([cannot run test program while cross compiling])])],
  [_AC_RUN_IFELSE($@)])
])

What will happen on those calls? And in general, how does the cross_compiling autoconf variable get set in Exherbo? Is it just the default autoconf behaviour of setting it to yes if both --build and --host were specified and they were not equal, or no otherwise?

Contributor

dezgeg commented May 11, 2018

There was a misunderstanding, by "tests" I mean this particular autoconf macro which can spew the "cannot run test program while cross compiling" error:

# AC_RUN_IFELSE(PROGRAM,
#               [ACTION-IF-TRUE], [ACTION-IF-FALSE],
#               [ACTION-IF-CROSS-COMPILING = RUNTIME-ERROR])
# ----------------------------------------------------------
# Compile, link, and run. Requires that the compiler for the current
# language was checked for, hence do not use this macro in macros looking
# for a compiler.
AC_DEFUN([AC_RUN_IFELSE],
[AC_LANG_COMPILER_REQUIRE()dnl
m4_ifval([$4], [],
	 [AC_DIAGNOSE([cross],
		     [$0 called without default to allow cross compiling])])dnl
AS_IF([test "$cross_compiling" = yes],
  [m4_default([$4],
	   [AC_MSG_FAILURE([cannot run test program while cross compiling])])],
  [_AC_RUN_IFELSE($@)])
])

What will happen on those calls? And in general, how does the cross_compiling autoconf variable get set in Exherbo? Is it just the default autoconf behaviour of setting it to yes if both --build and --host were specified and they were not equal, or no otherwise?

@eternaleye

This comment has been minimized.

Show comment
Hide comment
@eternaleye

eternaleye May 11, 2018

@dezgeg: AFAIK Exherbo does nothing special there, but I'd need to check. Speaking from experience, AC_RUN_IFELSE isn't especially common for packages to use explicitly, and can often be patched out (or have fallbacks patched in) easily enough - most upstreams take cross fixes well. We do add fallbacks where other autoconf macros use it though.

eternaleye commented May 11, 2018

@dezgeg: AFAIK Exherbo does nothing special there, but I'd need to check. Speaking from experience, AC_RUN_IFELSE isn't especially common for packages to use explicitly, and can often be patched out (or have fallbacks patched in) easily enough - most upstreams take cross fixes well. We do add fallbacks where other autoconf macros use it though.

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 May 11, 2018

Member

Neat patch!

Member

Ericson2314 commented May 11, 2018

Neat patch!

@dezgeg

This comment has been minimized.

Show comment
Hide comment
@dezgeg

dezgeg May 13, 2018

Contributor

Indeed that patch looks like it solves many of those problems in practice. But my general point stands, aka even if you do "cross compilation" with e.g. both build and host set to x86, you still wouldn't detect those problems, you still would have to do e.g. and x86 -> ARM cross compile to actually detect them, right?

Contributor

dezgeg commented May 13, 2018

Indeed that patch looks like it solves many of those problems in practice. But my general point stands, aka even if you do "cross compilation" with e.g. both build and host set to x86, you still wouldn't detect those problems, you still would have to do e.g. and x86 -> ARM cross compile to actually detect them, right?

@eternaleye

This comment has been minimized.

Show comment
Hide comment
@eternaleye

eternaleye May 13, 2018

@dezgeg Yes; there's really no feasible way to do static analysis of autotools-based buildsystems.

eternaleye commented May 13, 2018

@dezgeg Yes; there's really no feasible way to do static analysis of autotools-based buildsystems.

@Ericson2314 Ericson2314 referenced this issue Aug 2, 2018

Merged

Fix androidndk "ensure unaffected test" #44308

0 of 9 tasks complete

@Ericson2314 Ericson2314 modified the milestones: 18.09, 19.03 Aug 6, 2018

@Ericson2314 Ericson2314 referenced this issue Aug 7, 2018

Open

WIP: treewide: Always prefix compilers #44583

0 of 9 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment