My native local dist workflow

[kornelski]

I have my own messy release.sh scripts that I'd like to retire, but unfortunately cargo-dist and I live in a different world. In case you're looking for ideas what to add, here's how I build and distribute my software.

I build binaries for macOS, Windows, Debian (.deb), and Linux-MUSL. I'd like to add rpm into the mix.

1. Lack of macOS: Create universal binaries (fat archives) #77 is a dealbreaker for me. Fat binaries are the expected standard on macOS, and having two versions asking for the architecture looks like software made by someone who never used macOS ("how do you do, fellow kids?" vibe). Running lipo is super easy, and Cargo even supports multiple --target args now.

2. macOS executables have to be code signed, unfortunately. Each year Apple gets more and more aggressive blocking unsigned software and telling users it's malware. I don't think cargo-dist does code signing. When building on macOS, once Xcode is set up, it's as easy as codesign -vs "Developer Id" exe. However, signing on GitHub is a massive pain of exporting private keys and certificates.

3. I'd rather not use GitHub's CI. I have my local machines that are faster. I associate GitHub CI with hours lost on pushing tweaks to YAML files without ability to investigate why they're failing. cargo-dist generates a complex workflow file that I can't test without pushing it live. The macOS builders don't have MACOSX_DEPLOYMENT_TARGET set. I'd prefer to build on the latest macOS, and target an older SDK from it.

4. I build with -Z build-std=std,panic_abort -Z build-std-features=panic_immediate_abort and -Zlocation-detail=none, and a separate strip, because I'm dissatisfied with sizes of executables that Rust produces by default. I don't think cargo-dist supports setting the nightly flags — these don't have [profiles] equivalents.

5. My Mac is ARM-based, so I cross-compile from ARM macOS to x86-64 macOS. This is generally very easy thanks to Apple shipping fat binary libraries. In pure-Rust projects it's also relatively easy to cross-compile from Mac to Linux/MUSL. If C code is involved, I use Docker (locally) with aarch64 Linux to build for both aarch64 and cross compile to x86-64 at the same time (x86 Docker images on ARM Macs are slow and crashy.)

6. I use cargo deb and also offer a zip/tarball with a plain executables. I do not offer curl | sh, since there are people who are irrationally offended by that.

[Gankra]

All of this is super reasonable, yeah. Just quickly giving my immediate thoughts on each point:

1 - universal builds

I agree #77 is conceptually easy to add, and can increase it's priority.

cargo-dist is actually perfectly happy to cross macos, and this should work on either mac platform:

cargo dist build --artifacts=local --target=aarch64-apple-darwin --target=x86_64-apple-darwin

In earlier versions we actually defaulted to provisioning only one machine for macos and doing that, in anticipation of universal builds, but no one seemed to really need them with fetching installers handling it for you, and it sucked that it doubled the latency of releases and reduced fault-tolerance. You can however restore the previous behaviour with merge-tasks = true.

2. macOS code signing

Yep, the macOS side of #21 is something I'd love to have an implementation for. This month we put a bunch of energy into the Windows side of it (which it turns out is a huge mess that got way messier in june), and I agree that "signing in the cloud" is painfully more complex than "singing on your local machine".

I expect macOS signing will be a lot simpler, as you say.

3a. I want to run locally

So right up top I'll say "big agree". Here's some notes on how cargo-dist tries to deal with that:

I agree the CI is daunting but that's because we keep adding UX improvements to it that are unrelated to the core functionality. The ultimate ethos of cargo-dist is that every machine is just "install cargo-dist, run cargo-dist exactly once, do the thing the machine-readable manifest it output says to do".

The core behaviour that the CI is trying to run is just this:

1. plan: on any machine (to get the final dist-manifest.json, changelogs, etc):
   • cargo dist plan
2. local builds: for each supported target, get a machine that can build that target, then:
   • cargo dist build --artifacts=local --target=$TARGET
3. global build: copy all results to one machine (so that installers can get hashes and stuff), then:
   • cargo dist build --artifacts=global
4. infra-y stuff like pushing to github releases -- this part is genuinely "Stuck in CI"

This is intentionally designed to be runnable locally with identical results for debugging on your own machines or even using in your own custom infra. The biggest caveat is #236, as well as more infra-y tasks like "upload the results to github" or "push to a homebrew tap". A big focus of this coming month is going to be getting infra backends to N = 2 to break all these assumptions.

bonus context on the above CLI invocations, in case curious: every cargo-dist invocation is aware of the full build graph, and several of our CLI flags say "select a subset of the build to handle". To make this simple and consistent without centralized coordination, I introduced a conceptual split between "local" artifacts which are fundamentally target-specific (like a tarball), and "global" artifacts which are ostensibly multi-platform (like all the fetching installers which check what platform you're on and select the right tarball).

So when we "plan" we list off the work all machines should do in aggregate, and make sure there aren't any issues with that. When we do a "local build" we specifically select "just the local artifacts, and only the ones for this set of targets" to build (building binaries/tarballs, building msi's...). Then we do a single "global build" to get all the platform agnostic builds dealt with (generating curl-sh scripts, generating npm packages, generating homebrew packages, ...).

Actually in previous versions we also ran the "global" stuff totally in parallel but we made it strictly last in in 0.2.0 to let installers have access to things like checksums which can't be predicted.

3b. GitHub has to be live, is brittle

This is also one of my biggest pain-points, and was actually a big focus of the v0.3.0 release we just shipped!

By default we now run the "plan" step on all your PRs (and we made cargo dist plan do way more integrity checks) to catch breakage early and often. That also means you don't even have to "land" cargo-dist support to test it out now.

We also have a new pr-run-mode = upload system that lets you also run all the build steps too, and upload them to a zip hanging off of the worflow. See here for a more details.

4. extra build flags / not in profile

Yeah this stuff is really annoying. On the one hand I can totally just add a config for "extra flags". On the other hand I'm tempted to try to see if encouraging people to use .cargo/config.toml is a better solution. In the past my attempts to have "non-standard" compiler setting config has resulted in more confusion than aide (i tried an alternative to rust-toolchain.toml, did not work out).

5. Interesting Notes On Crossing

Not 100% sure if I have anything salient to say on this, but thanks for the info!

6. deb support

100% happy to add an integration for that, do you have any interesting requirements/configuration or do you do "the default thing" for cargo deb?

[kornelski]

Thank you for your consideration.

Regarding cargo deb — most of it can be configured via [package.metadata.deb],

One exception is that I use different Cargo features on different platforms. When building for Debian, I need to link sys crates dynamically, and macOS/Windows/MUSL need static.