Host Platform build.rs for Cross Build #214

psionic-k · 2021-11-14T10:36:21Z

This PR strips out the localFilters because:

it's useless for independent sources
frequently a source of breakage for users
flakes already respect .gitignore
the more likely use pattern people wanted all along is to filter something out (which can be done during unpack) rather than to select something via regex

I used the static resources example and just built with cargo to get a target dir. Zero effect on the resutls of nix builds. target did not need to be filtered from sources. Flakes 🥳

So Example 3 is Now Cross-Compiling!

The third example's use of a build.rs script is perfect for debugging the host and build platform offsets. We will fix everything. 👊

The cross compiling example with wasi linking can now be compiled on Linux and run with wasmtime

Lets fix....

fix targeting wasm32-unknown-unknown #203
close Allow direct configuration of target #199
fix Build dependencies are built with hostPlatformCpu and hostPlatformFeatures #129
fix Improve localPatterns #36 (local sources already dead)
fix dangling path #202 (drive by fixup)
split test matrix for darwin & linux to get Rust Analyzer back into CI

psionic-k · 2021-11-15T11:49:50Z

Latest changes add explicit target configuration.

I think we need to split up the target for selecting a Rust toolchain and the target configuration for the code output, if these are not the same. Judging by the weird vendor targets people brought up elsewhere, I'm convinced that the Rust toolchain selected by one of its targets doesn't always match what code that toolchain will be used to generate. We can mostly handle this for Rust crates without much extra work I expect.

For C deps, there may be more work in the form of providing a linker or some lib without bringing in a whole stdenv.

buildPhase overrides for mkcrate.nix can probably handle any extra changes necessary for outputing code or linking that go beyond merely passing options to rustc

psionic-k · 2021-11-15T15:17:02Z

There's two places we translate between nix platforms & rust triples. /overlay/lib/real-host-triple.nix and /src/platform.rs Both need some maintenance many targets were added.

I'm working on web assembly right now.

List of triples Rust probably comprehends (at least a bit)
our nix mapping (which is moving to /overlay/lib/rust-triple.nix
our Rust mapping from parsed cargo platforms to nix platforms

psionic-k · 2021-11-16T07:05:12Z

Evidently there is now a toRustTarget function in nixpkgs https://github.com/oxalica/rust-overlay/search?q=toRustTarget

moving static-resources -> cross-compiling This tests the feature correctness of cross builds point to nixpkgs master for the wasi build. Because of an upstream fix to toRustTriple

This stuff was from back when there was some kind of airplain mode. It would replace the remote registry with a local registry. Marginal use if you have the crate sources in your cache.

This change enables overriding the target for rustc, making it build different than whatever `crossPlatform` and `platform` would otherwise go for. The rust toolchain may not have such a target, so this feature is incomplete where the rust toolchain needs to be selected with one target and told to build for another (because it's a target that toolchain can be configured to cross for) Builds for cross should build identically if the target is set explicitly to the same value as the crossSystem config. note about explicit target

This function's entire job is to convert the nix triple to a rust triple. "real" was confusing. "Real" to what? It's also not always used to map a host triple and sometimes was used to map a build triple. The world is a bit better now. Second, the generation of the synthetic Cargo.toml used to build each crate drv in mkcrate.nix was using a combination of rustTriple and the explicit target override. That has been slightly normalized. There is still a problem in pkgsBuild in make-package-set/simple.nix. I don't think the explicit target we set for the host output needs to even touch the depsBuildBuild outputs. The detection for when to enable multiple targets when generating the Cargo.toml for build was changed to compare the two triples we generate. This means even if crossSystem is not set but the explicit target is used, the difference will be picked up and both target entries will be present in the toml. I'm concerned about the CC flags becuase it may mean we provide a triple that Rust understands and the cc cannot. This is especially true if the explicit target is provided by the user.

Rust supports a ton of platforms. At least the stuff with crossSystem examples should work. Some of these like wasm64 are pure fantasy until Rust supports it.

null out explicit target for build packages so that it doesn't become an explicit override

Use nixpkgs-<version>-darwin for darwin builds And logic gate the Rust Analyzer build so that only successful operating systems are used use local cargo2nix in exmaples. This is overriden by CI use darwin nixpkgs for all examples update cachix action Flakes are built in as of 15. Using authenticated requests was on the updated README. Much, much better.

Breaks the linking behavior. Could instead just not st the linker. More testing needed with better examples.

More generated Explicit target from latest overlay breaks them update generated clean this out of the final history update-generated updated generated

They can't build things

psionic-k requested review from ebkalderon, edude03 and dingxiangfei2009 November 14, 2021 10:36

psionic-k added enhancement New feature or request help wanted Extra attention is needed labels Nov 14, 2021

This comment has been minimized.

Sign in to view

psionic-k mentioned this pull request Nov 14, 2021

Bare metal example with cross compile #190

Closed

psionic-k force-pushed the host-platform-build-rs branch from 7887b8c to 9c6fd2f Compare November 15, 2021 10:53

psionic-k mentioned this pull request Nov 15, 2021

Allow direct configuration of target #199

Closed

psionic-k force-pushed the host-platform-build-rs branch from 9c6fd2f to 3e3ba54 Compare November 15, 2021 10:58

psionic-k mentioned this pull request Nov 15, 2021

Build dependencies are built with hostPlatformCpu and hostPlatformFeatures #129

Closed

This comment has been minimized.

Sign in to view

psionic-k mentioned this pull request Nov 16, 2021

crossSystem results in failed componentResolution oxalica/rust-overlay#57

Closed

This comment has been minimized.

Sign in to view

psionic-k force-pushed the host-platform-build-rs branch 2 times, most recently from 036c1ed to 8eb3261 Compare November 19, 2021 09:33

This comment has been minimized.

Sign in to view

psionic-k added 6 commits November 29, 2021 20:42

Cross compiling example

9b96ed0

moving static-resources -> cross-compiling This tests the feature correctness of cross builds point to nixpkgs master for the wasi build. Because of an upstream fix to toRustTriple

Remove unused local registry

0a03451

This stuff was from back when there was some kind of airplain mode. It would replace the remote registry with a local registry. Marginal use if you have the crate sources in your cache.

Update rustTriple map

9bc561d

Rust supports a ton of platforms. At least the stuff with crossSystem examples should work. Some of these like wasm64 are pure fantasy until Rust supports it.

get Rust toolchain with build & output targets

96c9b9d

null out explicit target for build packages so that it doesn't become an explicit override

psionic-k added 4 commits November 29, 2021 20:42

note about test deps

88d3776

Don't emit target configuration for wasi

aeefb0a

Breaks the linking behavior. Could instead just not st the linker. More testing needed with better examples.

update generated

56f4c26

More generated Explicit target from latest overlay breaks them update generated clean this out of the final history update-generated updated generated

psionic-k force-pushed the host-platform-build-rs branch from 9b3ebcf to 56f4c26 Compare November 29, 2021 12:50

Don't stress poor macs too much

f1fc9de

They can't build things

psionic-k marked this pull request as ready for review November 29, 2021 13:05

psionic-k merged commit 51f81fa into master Nov 29, 2021

psionic-k deleted the host-platform-build-rs branch November 29, 2021 13:13

psionic-k mentioned this pull request Nov 30, 2021

CI Improvements #155

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Host Platform build.rs for Cross Build #214

Host Platform build.rs for Cross Build #214

psionic-k commented Nov 14, 2021 •

edited

This comment has been minimized.

This comment has been minimized.

psionic-k commented Nov 15, 2021

psionic-k commented Nov 15, 2021 •

edited

This comment has been minimized.

This comment has been minimized.

psionic-k commented Nov 16, 2021

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Host Platform build.rs for Cross Build #214

Host Platform build.rs for Cross Build #214

Conversation

psionic-k commented Nov 14, 2021 • edited

So Example 3 is Now Cross-Compiling!

This comment has been minimized.

This comment has been minimized.

psionic-k commented Nov 15, 2021

psionic-k commented Nov 15, 2021 • edited

This comment has been minimized.

This comment has been minimized.

psionic-k commented Nov 16, 2021

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

psionic-k commented Nov 14, 2021 •

edited

psionic-k commented Nov 15, 2021 •

edited