Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a zig cc-based image. #880

Merged
merged 1 commit into from
Jul 17, 2022
Merged

Add a zig cc-based image. #880

merged 1 commit into from
Jul 17, 2022

Conversation

Alexhuszagh
Copy link
Contributor

Uses cargo-zigbuild as a backend, and adds configuration options for zig under [build.zig] and [target.(...).zig]. If enabled, and an image override is not provided, cross will always use the zig image.

It supports custom glibc versions by parsing the libc portion of the target, and extracting a the libc version if present. The target, if built-in, is then the triple/libc pair, otherwise, it's just the triple.

The image does not provide runners, bindgen Clang args, or pkg-config paths, since zig cc does not provide the dynamic library loader (ld-linux*.so) required, meaning none of the binaries can be run. For bindgen, zig cc has an unusual directory structure, so there is no traditional sysroot with usr, lib, and include subdirectories. Finally, since we don't have system packages we can work with, exporting a pkg-config path makes little sense.

Closes #860.

@Alexhuszagh
Copy link
Contributor Author

I still need to figure out the best way to add it to CI (it's currently not built as part of the matrix), since it can target numerous other architectures. It might be good to just change the exported triple from zig to a well-known target like aarch64-unknown-linux-gnu, which supports C++, etc.

docker/Dockerfile.zig Outdated Show resolved Hide resolved
src/lib.rs Outdated Show resolved Hide resolved
@Alexhuszagh

This comment was marked as outdated.

bors bot added a commit that referenced this pull request Jun 29, 2022
@Alexhuszagh

This comment was marked as outdated.

bors bot added a commit that referenced this pull request Jun 29, 2022
@bors

This comment was marked as outdated.

@Alexhuszagh

This comment was marked as outdated.

bors bot added a commit that referenced this pull request Jun 30, 2022
@Alexhuszagh
Copy link
Contributor Author

I've changed the format entirely.

cross ... --target now only accepts valid Rust triples, and the config format is as follows:

[target.x86_64-unknown-linux-gnu.zig]
enable = true       # enable use of the zig image
version = "2.17"    # glibc version to use
image = "ghcr.io/cross-rs/zig:local"    # custom image to use

If provided as a bool, it will use the default glibc version:

[target.x86_64-unknown-linux-gnu]
# equivalent to { enable = true }
zig = true

If provided as a string, zig will be automatically enabled:

[target.x86_64-unknown-linux-gnu]
# equivalent to { enable = true, version = "2.17" }
zig = "2.17"

A custom zig image is so users can provide their own, general-purpose zig image if needed. These values can also be provided via environment variables, such as CROSS_BUILD_ZIG, CROSS_BUILD_ZIG_VERSION, and CROSS_BUILD_ZIG_IMAGE, or similar values for specific targets.

@Alexhuszagh Alexhuszagh marked this pull request as ready for review June 30, 2022 16:57
@Alexhuszagh Alexhuszagh requested a review from a team as a code owner June 30, 2022 16:57
@bors

This comment was marked as outdated.

src/lib.rs Outdated Show resolved Hide resolved
docker/zig.sh Outdated Show resolved Hide resolved
src/docker/shared.rs Outdated Show resolved Hide resolved
src/docker/shared.rs Outdated Show resolved Hide resolved
src/lib.rs Outdated Show resolved Hide resolved
src/lib.rs Outdated Show resolved Hide resolved
CHANGELOG.md Outdated Show resolved Hide resolved
ci/test-zig-image.sh Show resolved Hide resolved
@Alexhuszagh
Copy link
Contributor Author

Actually we should provide an ARCH here. I'll mark this as a draft since we need to download zig from a binary anyway.

@Alexhuszagh
Copy link
Contributor Author

Alexhuszagh commented Jun 30, 2022

bors try --target zig

Ok I've changed the build in zig.sh to fetch tarballs when possible, or build from source when we can't. This way, at least we should support numerous different target platforms There's a good chance the bors command fails (due to a permissions error when creating the cache), but weirdly it works on my machine so we'll see....

bors bot added a commit that referenced this pull request Jun 30, 2022
@bors

This comment was marked as outdated.

@Alexhuszagh
Copy link
Contributor Author

Oh this is why it's failing:

$ /usr/bin/docker run --userns host -e 'PKG_CONFIG_ALLOW_CROSS=1' -e 'XARGO_HOME=/xargo' -e 'CARGO_HOME=/cargo' -e 'CARGO_TARGET_DIR=/target' -e 'CROSS_RUNNER=' -e CARGO_HTTP_CHECK_REVOKE -e RUSTFLAGS -e CARGO_NET_RETRY -e CARGO_INCREMENTAL -e 'USER=runner' --rm --user 1001:121 -v /home/runner/.xargo:/xargo:Z -v /home/runner/.cargo:/cargo:Z -v /cargo/bin -v /tmp/tmp.xdLqWsDyv2:/project:Z -v /home/runner/.rustup/toolchains/stable-x86_64-unknown-linux-gnu:/rust:Z,ro -v /tmp/tmp.xdLqWsDyv2/target:/target:Z -w /project ghcr.io/cross-rs/zig:trying sh -c 'PATH=$PATH:/rust/bin cargo-zigbuild build --target aarch64-unknown-linux-gnu --verbose'

Specifically --rm --user 1001:121, and we have RUN adduser --uid 1000 --disabled-password --gecos "" cross in the Dockerfile.

@Alexhuszagh
Copy link
Contributor Author

Alexhuszagh commented Jun 30, 2022

Oh this is why it's failing:

$ /usr/bin/docker run --userns host -e 'PKG_CONFIG_ALLOW_CROSS=1' -e 'XARGO_HOME=/xargo' -e 'CARGO_HOME=/cargo' -e 'CARGO_TARGET_DIR=/target' -e 'CROSS_RUNNER=' -e CARGO_HTTP_CHECK_REVOKE -e RUSTFLAGS -e CARGO_NET_RETRY -e CARGO_INCREMENTAL -e 'USER=runner' --rm --user 1001:121 -v /home/runner/.xargo:/xargo:Z -v /home/runner/.cargo:/cargo:Z -v /cargo/bin -v /tmp/tmp.xdLqWsDyv2:/project:Z -v /home/runner/.rustup/toolchains/stable-x86_64-unknown-linux-gnu:/rust:Z,ro -v /tmp/tmp.xdLqWsDyv2/target:/target:Z -w /project ghcr.io/cross-rs/zig:trying sh -c 'PATH=$PATH:/rust/bin cargo-zigbuild build --target aarch64-unknown-linux-gnu --verbose'

Specifically --rm --user 1001:121, and we have RUN adduser --uid 1000 --disabled-password --gecos "" cross in the Dockerfile.

Ok this has been fixed by not creating a user, but by setting $XDG_CACHE_HOME which will work as long as we're on Linux, which will write it to the target directory (under /target/.zig-cache, to avoid any naming conflicts). If we ever support non-Linux hosts, this may need to be revisited.

@Alexhuszagh Alexhuszagh modified the milestones: v0.2.3, v0.3.0 Jun 30, 2022
bors bot added a commit that referenced this pull request Jul 16, 2022
@Alexhuszagh

This comment was marked as outdated.

bors bot added a commit that referenced this pull request Jul 16, 2022
@bors

This comment was marked as outdated.

@Alexhuszagh

This comment was marked as outdated.

bors bot added a commit that referenced this pull request Jul 17, 2022
@bors

This comment was marked as outdated.

@Alexhuszagh
Copy link
Contributor Author

This won't work but the log will be helpful.
bors try --target zig

bors bot added a commit that referenced this pull request Jul 17, 2022
@bors

This comment was marked as outdated.

@Alexhuszagh
Copy link
Contributor Author

bors try --target zig

bors bot added a commit that referenced this pull request Jul 17, 2022
.github/workflows/ci.yml Outdated Show resolved Hide resolved
@bors
Copy link
Contributor

bors bot commented Jul 17, 2022

try

Build succeeded:

src/lib.rs Show resolved Hide resolved
src/docker/custom.rs Outdated Show resolved Hide resolved
Uses cargo-zigbuild as a backend, and adds configuration options for zig under `[build.zig]` and `[target.(...).zig]`. If enabled, and an image override is not provided, `cross` will always use the `zig` image. The feature can be enabled by providing `zig` as a table, bool, or string.

It supports custom glibc versions by passing the `zig.version` key, and `zig` can be separately enabled or disabled by providing `zig.enable`.

```
[target.x86_64-unknown-linux-gnu.zig]
enable = true       # enable use of the zig image
version = "2.17"    # glibc version to use
image = "ghcr.io/cross-rs/zig:local"    # custom image to use
```

If provided as a bool, it will use the default glibc version:

```
[target.x86_64-unknown-linux-gnu]
\# equivalent to { enable = true }
zig = true
```

If provided as a string, `zig` will be automatically enabled:

```
[target.x86_64-unknown-linux-gnu]
\# equivalent to { enable = true, version = "2.17" }
zig = "2.17"
```

The image does not provide runners, `bindgen` Clang args, or `pkg-config` paths, since `zig cc` does not provide the dynamic library loader (`ld-linux*.so`) required, meaning none of the binaries can be run. For `bindgen`, `zig cc` has an unusual directory structure, so there is no traditional sysroot with `usr`, `lib`, and `include` subdirectories. Finally, since we don't have system packages we can work with, exporting a `pkg-config` path makes little sense.
src/lib.rs Show resolved Hide resolved
Copy link
Member

@Emilgardis Emilgardis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awesome!

@Alexhuszagh
Copy link
Contributor Author

bors r=Emilgardis

@bors
Copy link
Contributor

bors bot commented Jul 17, 2022

Build succeeded:

@bors bors bot merged commit 3f60cc1 into cross-rs:main Jul 17, 2022
@Alexhuszagh Alexhuszagh deleted the zigcc branch July 17, 2022 21:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Provide a zig cc-based image.
2 participants