Skip to content

Conversation

@polarathene
Copy link
Contributor

@polarathene polarathene commented May 22, 2025

This PR updates the Dockerfile to use HereDoc syntax, as suggested here. This allows for RUN statements to drop the && \ noise and be easier to grok like a regular shell script.

If you review the first commit that should be fairly straight-forward. I've since noticed other portions of the Dockerfile that could do with some maintenance, but lack time to stagger PRs for each individually, the bulk of those are in the follow-up commits (might be easier to review separately if you rely on a diff).


Additionally:

  • ENV CARGO_REGISTRIES_CRATES_IO_PROTOCOL=sparse can be dropped as this is the default since Rust 1.70.0.

  • Variables use ${VAR_NAME} instead of $VARNAME. I've replaced some ENV with ARG since they're only intended for build-time usage and located them by each associated RUN so that adjusting these won't invalidate all layers.

  • In your recent PR you added some additional LABEL (the existing maintainer label is discouraged btw), one of these you attempt to set the value with a shell command, but these are string values for assignment. If you assumed you could run a subshell because of ${VAR_NAME} usage, that's not a shell ENV but an equivalent feature supported by the Dockerfile format to reference and use ENV / ARG. Thus this value literally stores that as a string:

    LABEL org.opencontainers.image.create="$(date --utc --iso-8601=seconds)"
  • Likewise, no need for this when you have --target option for rustup-init:

    ~/.cargo/bin/rustup target add "${AARCH}-unknown-linux-musl"
  • SHELL is defined early on to change the default /bin/sh used by RUN instructions.

    • This will ensure a build fails early and persists a failed command exit status when a pipe is involved (instead of relying on the exit code from the command after the pipe). It should help with troubleshooting builds if something goes awry.
    • /bin/sh in Debian/Ubuntu symlinks to /usr/bin/dash, I've chosen /usr/bin/bash here which is usually what is expected. On Fedora /bin/sh symlinks to /usr/bin/bash and Alpine to /usr/bin/ash
  • I've removed the comment for protoc version in Jammy having missing features.

  • PATH was set twice, while referencing the previous set PATH which set duplicate paths. The final PATH value became:

    /root/.cargo/bin:/musl/bin:/usr/local/bin:/root/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
    

    I've resolved that. You also switched from Debian to Ubuntu in 2016, where you prepended /usr/local/bin to PATH, but that should be unnecessary as it's already in the PATH.

  • PREFIX is set in the same ENV instruction as LD_LIBRARY_PATH which references it. Thus PREFIX is empty and this likely fails to do what was intended.

    • You'd need to set ENV PREFIX separately before an ENV LD_LIBRARY_PATH.
    • However it's very bad practice to use ENV LD_LIBRARY_PATH, you should prefer to use that explicitly as a prefix to commands rather than a global env.
    • I traced it's addition down to this commit (Oct 2017). The associated PR and referenced issue the PR fixes makes no mention of LD_LIBRARY_PATH being useful, and given that it's been empty since then I've removed that too.
  • In the 2nd commit zlib + sqlite are now handled in separate stages. I also put rustup into it's own stage in the third commit and shifted your labels to the end (while switching the maintainer label for standardized authors one). This has a breaking change where I migrated cargo + rustup to /opt instead of storing them in /root. You don't need to give access to /root home folder this way, but if any one used the image while cache mounting or similar with the /root location hard-coded then this might be an unexpected change for them.

- Remove `ENV CARGO_REGISTRIES_CRATES_IO_PROTOCOL=sparse` it's the default since Rust 1.70.0 (June 2023)
- Revise the rustup `RUN` and relocate relevant changes associated to it with additional PR references for maintainers as docs.
- Handle `PATH` updates properly, remove any redundancy.
- Remove `LD_LIBRARY_PATH`, it was misconfigured and seems redundant?
- Improve the `protoc` + `sccache` bin retrieval + extraction `RUN` instructions.
- Change `zlib` + `sqlite` steps into their own build stages. They each extract the source into their respective `WORKDIR`. No cleanup necessary due to separate stage.
Primary change is rustup `RUN` in it's own separate stage. This may not be as friendly to cache storage? Additionally breaking change that moved rustup and cargo install to `/opt` instead of in `/root`.
Copy link
Owner

@clux clux left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you so much for this. The tracing and documentation on all these flags that's been in here for ages is amazing.

I agree with pretty much all of this. Lots of clear errors on my behalf that's great to clear out and document. I was a bit confused by the benefit of the workstage separation, but I see it actually makes my local build able parallelize the stages. (41s build locally on my ryzen).

@clux clux merged commit d442ed1 into clux:main May 22, 2025
2 of 6 checks passed
@polarathene
Copy link
Contributor Author

I see it actually makes my local build able parallelize the stages. (41s build locally on my ryzen).

Yeah, when the image builder understands it can parallelize the build of multiple stages it will do so. It also has the benefit that updating one will not invalidate the other since they don't depend on each other.

COPY --link is like a rebase, it copies the content to an intermediary FROM scratch stage behind the scenes, caches that and copies that over. Probably not as big of a deal for this Dockerfile but it can be useful sometimes.


I had some additional changes to tackle, but ran out of steam for that today. Seems you merged with the last commit failing the test suite 😓 I'll try tackle that tomorrow for you.

@clux
Copy link
Owner

clux commented May 22, 2025

All good, I did some minor follow-ups so CI has been a bit delayed due to early-cancels, but it looks like all previous failures have just been my failure to deal with logins correctly on PRs.

EDIT: The most recent main timed build at https://github.com/clux/muslrust/actions/runs/15184036234/job/42700189156 is all green.

@polarathene
Copy link
Contributor Author

polarathene commented May 23, 2025

Looks like you tackled everything else I had queued up 💪


In this follow-up commit you replaced there marker from HEREDOC to EOF. Was that only motivated by familiarity/convention? I've been using HEREDOC in Dockerfile examples when I show it so those that are less familiar with it can more easily look it up 😅

If there's another reason to prefer EOF, that'd be good to know too 👀


In this follow-up commit you changed from major pin to major.minor pin of the dockerfile syntax. This is discouraged in the official docs, as you are not setting a minimum AFAIK but locking on that minor series. Better to leave it on the v1 major.

@clux
Copy link
Owner

clux commented May 23, 2025

In this follow-up commit you replaced there marker from HEREDOC to EOF. Was that only motivated by familiarity/convention? I've been using HEREDOC in Dockerfile examples when I show it so those that are less familiar with it can more easily look it up 😅

If there's another reason to prefer EOF, that'd be good to know too 👀

no, not really. just like the idea of using the common heredoc marker we typically encounter outside docker for some code familiarity. a search for dockerfile eof also gives me the heredoc results so probably not a huge deal either way.

In this follow-up commit you changed from major pin to major.minor pin of the dockerfile syntax. This is [discouraged in the official docs(https://docs.docker.com/build/buildkit/frontend/#stable-channel), as you are not setting a minimum AFAIK but locking on that minor series. Better to leave it on the v1 major.

Ah, thanks! I didn't know that.

clux added a commit that referenced this pull request May 23, 2025
as per note in #159 (comment)

Signed-off-by: clux <sszynrae@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants