Revamp cross-compilation docs

[siraben]

Motivation for this change

I want this to be an interactive process with the community. Cross-compilation is one of Nixpkgs' greatest strengths, but the docs do not do it justice. In particular:

• It has too much technicality at times. Even though cross-compilation is a technical topic, I think many aspects can be simplified for non-specialists.
• Some features are undocumented altogether (see Add documentation for emulators feature in cross infra #106375)
• Instructions on how to add new cross targets is missing yet important (examples: Initial implementation of remarkable1 cross-compile #102503 Initial implementation of cross-compilation to Knuth's MMIX #102766 Initial implementation of vc4 cross-compile #72657)

WIP for now

Things done

• Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
• Built on platform(s)
  • NixOS
  • macOS
  • other Linux distributions
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• 21.11 Release Notes
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

[domenkozar]

Also see https://nix.dev/tutorials/cross-compilation

[siraben]

@domenkozar wow, I didn't know nix.dev covered cross-compilation, thanks! Recently heard the podcast as well :)

[domenkozar]

@domenkozar wow, I didn't know nix.dev covered cross-compilation, thanks! Recently heard the podcast as well :)

It's quite recent, added yesterday :)

Great work with these changes! Looks good to me.

[siraben]

@ianthehenry I saw you mentioned the cross-compilation section of Nixpkgs in your blog post, I agree the way the section is currently structured is confusing as well and I only managed to make heads or tails of it from various sources online by @matthewbauer, discussions with @Ericson2314 and diving head-first into adding cross-compilation support for a few new targets. Would appreciate any suggestions and/or feedback in this WIP PR. You also may want to check out the nix.dev tutorial on cross-compilation.

This section is poignant:

And I think that, if cross-compilation should work by default for as many packages as possible, it is important to communicate to users how they can make that work, in simple language that can be understood by someone (like me) who has not given a lot of thought to cross-compilation before. I feel like the current state of the documentation is probably illegible to people who don’t already know how it works, and I would venture a guess that this means the “average” Nix user (like me) doesn’t know what they should be doing in order to support it. So I’m just not going to think about it, and hope that it works.

[siraben]

@ianthehenry I also highly recommend you join our Matrix channels so you can see what the community is up to and ask questions when they arise.

[ianthehenry]

@siraben You may have seen this post already, but just in case: the first and most confusing explanation of cross-compilation in the manual is actually in Chapter 6. Trying to read that section was the primary source of my frustration in the summary (ctrl-F "offset").

The actual cross-compilation chapter made better reading, for the most part, although the section on bootstrapping completely lost me. That said, I admire any effort to clarify it, and I think the changes you've made so far certainly do increase the legibility a bit.

For me I think that a concrete example would be very helpful -- I tried to cross-compile a trivial package and was unable to. I don't know if I was doing something completely wrong, if I just got unlucky, if Linux is not a reasonable target platform, or what. (I expect that cross-compilation mostly exists for microcontrollers, but that's harder for someone like me (who just wants to see how it works) to experiment with.)

[ghost]

I expect that cross-compilation mostly exists for microcontrollers

There are tons of use cases for cross-compilation. Here are two that you might not have noticed:

1. Cross-compilation allows Hydra to produce trusted bootstrap-files archives for platforms that it lacks physical hardware for. The hashes of these trusted archives get embedded into nixpkgs here. mips64el will be added to this list as soon as libtool: fix /usr/bin/file impurity #166879 merges, and I plan to work on mips32 and powerpc64 after that.

2. Cross-compilation lets you use machines that are readily-accessible with large amounts of memory and CPU (like x86_64) to do builds for architectures that are popular in routers and switches (like mips32/mips64). This makes a big difference in development turnaround time.

There is a proposal (#21471) to make all compiles be cross-compiles, in order to avoid having two separate codepaths for every package. Even if you don't use cross-compilation, planning for it helps clarify build-time/run-time distinctions quite a bit.

[siraben]

Still think it's valuable to update cross-compilation updates, but might not have time this summer to get around to the whole thing...

[siraben]

Maybe we can merge it as-is and incrementally do some of the suggestions at a later PR?

[ghost]

This is wrong; "cross compilation" is ambiguous: it can mean either

build /= host == target

or

build == host /= target

Indeed, this is exactly the reason why nixpkgs doesn't have a stdenv.isCross. It's an ambiguous term.

[ghost]

Suggested change

	Nixpkgs follows the [conventions of GNU autoconf](https://gcc.gnu.org/onlinedocs/gccint/Configure-Terms.html). We distinguish between 3 types of platforms when building a derivation: _build_, _host_, and _target_. In summary, _build_ is the platform on which a package is being built, _host_ is the platform on which it will run. The third attribute, _target_, is relevant only for certain specific compilers and build tools. Using this terminology, we can summarize the different types of compilation as follows:
	| Type of compilation | Condition |
	|----------------------------|-------------------------|
	| native compilation | build == host == target |
	| cross-compilation | build /= host == target |
	| Canadian cross-compilation | build /= host /= target |
	Nixpkgs follows the [conventions of GNU autoconf](https://gcc.gnu.org/onlinedocs/gccint/Configure-Terms.html). We distinguish between 3 types of platforms when building a derivation: _build_, _host_, and _target_. In summary, _build_ is the platform on which a package is being built, _host_ is the platform on which it will run. The third attribute, _target_, is relevant only for certain specific compilers and build tools.

[ghost]

Suggested change

	Using these attributes, the build process of a package can change depending on the situation, for instance, when cross-compiling, one may want to apply specific patches or disable tests.
	Using these attributes, the build process of a package can change depending on the situation.

1. Tests are already disabled when !buildPlatform.canExecute hostPlatform
2. We generally try really hard to avoid conditionally applying patches, because conditionally-applied patches tend to get broken by version bumps and never noticed.

[ghost]

Probably better to actually document lib.systems.emulatorFor than to point to a github issue...