Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update the version of musl used on *-linux-musl targets to 1.2.3 #107129

Merged
merged 3 commits into from May 6, 2023

Conversation

wesleywiser
Copy link
Member

@wesleywiser wesleywiser commented Jan 20, 2023

Update the version of musl used on our Linux musl targets from 1.1.24 to 1.2.3 as proposed in rust-lang/compiler-team#572. musl 1.2.3 is the latest version of musl and supports the same range of Linux kernels as the 1.1 series. As such, it does not affect the minimum supported version of Linux for any of the musl targets.

One of the major musl 1.2 features is support for time64. This support is both source and ABI compatible with programs built against musl 1.1 and so updating the musl version for these targets should not cause Rust programs to fail to run or compile (a crater run has been completed which demonstrates this for the i686-unknown-linux-musl target).

Once this change reaches stable, the libc crate will then be able to update their definitions to support 64-bit time, matching the default musl 1.2 APIs exactly.

Fixes #91178

@rustbot rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. labels Jan 20, 2023
@compiler-errors

This comment was marked as resolved.

@wesleywiser

This comment was marked as resolved.

@wesleywiser
Copy link
Member Author

@bors try

@bors
Copy link
Contributor

bors commented Jan 20, 2023

⌛ Trying commit 97795b0aa4815fd51e46b1702da9fa34e84863fa with merge a40ec913238f4f188d97c41aa605cb486451acf2...

@bors
Copy link
Contributor

bors commented Jan 21, 2023

☀️ Try build successful - checks-actions
Build commit: a40ec913238f4f188d97c41aa605cb486451acf2 (a40ec913238f4f188d97c41aa605cb486451acf2)

@wesleywiser

This comment was marked as resolved.

@craterbot

This comment was marked as resolved.

@craterbot craterbot added S-waiting-on-crater Status: Waiting on a crater run to be completed. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 23, 2023
@craterbot

This comment was marked as resolved.

@craterbot

This comment was marked as resolved.

@craterbot craterbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-crater Status: Waiting on a crater run to be completed. labels Jan 24, 2023
@wesleywiser
Copy link
Member Author

@craterbot run mode=build-and-test name=musl_1_2_upgrade_i686_unknown_linux_musl start=try#d5830e3d3b571fe52cdf4b7b34be99be74dc65ea+target=i686-unknown-linux-musl end=try#a40ec913238f4f188d97c41aa605cb486451acf2+target=i686-unknown-linux-musl

@craterbot
Copy link
Collaborator

👌 Experiment musl_1_2_upgrade_i686_unknown_linux_musl created and queued.
🔍 You can check out the queue and this experiment's details.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot craterbot added S-waiting-on-crater Status: Waiting on a crater run to be completed. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 26, 2023
@craterbot
Copy link
Collaborator

🚧 Experiment musl_1_2_upgrade_i686_unknown_linux_musl is now running

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot
Copy link
Collaborator

🎉 Experiment musl_1_2_upgrade_i686_unknown_linux_musl is completed!
📊 99 regressed and 94 fixed (253665 total)
📰 Open the full report.

⚠️ If you notice any spurious failure please add them to the blacklist!
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot craterbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-crater Status: Waiting on a crater run to be completed. labels Jan 27, 2023
@wesleywiser
Copy link
Member Author

Crater reports 76 root regressions:

  • 5 are build failures due to running out of disk space during the build.
  • 71 are test failures.
    • 9 are SIGSEGV: invalid memory reference or SIGBUS: access to undefined memory caused by UB in the crate.
      • miri test finds the UB in most of the crates except for a few which use FFI before the test crashes.
    • 32 are assertion failures of some variety
      • Lots of different issues, tests asserting non-deterministic things like hashmap iteration order, using random data generators, tests expecting stdout to be hooked to a terminal, etc.
      • I sampled ~14 failures that looked interesting but did not manually validate the rest. I did not find any actual regressions in the cases I examined.
    • 8 are various kinds of timeout failures, test took too long, etc.
      • I could not reproduce any of these failures.
    • 22 are from OS level errors such as "file not found", "connection refused", etc.
      • All failures in this category were not reproducible or showed the tests as flaky because they periodically failed before and after this change.

I believe the crater results show that updating our musl targets from musl 1.1.24 to musl 1.2.3 will not cause ecosystem breakage which is consistent with the release notes from upstream.

@rustbot rustbot added this to the 1.71.0 milestone May 6, 2023
@rust-timer
Copy link
Collaborator

Finished benchmarking commit (6e96802): comparison URL.

Overall result: ❌ regressions - ACTION NEEDED

Next Steps: If you can justify the regressions found in this perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please open an issue or create a new PR that fixes the regressions, add a comment linking to the newly created issue or PR, and then add the perf-regression-triaged label to this PR.

@rustbot label: +perf-regression
cc @rust-lang/wg-compiler-performance

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
1.7% [0.7%, 2.9%] 4
Regressions ❌
(secondary)
4.9% [0.2%, 8.1%] 8
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) 1.7% [0.7%, 2.9%] 4

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
-3.9% [-3.9%, -3.9%] 1
All ❌✅ (primary) - - 0

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
-2.7% [-3.3%, -2.1%] 3
All ❌✅ (primary) - - 0

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 654.765s -> 656.275s (0.23%)

@rustbot rustbot added the perf-regression Performance regression. label May 6, 2023
RalfJung pushed a commit to RalfJung/miri that referenced this pull request May 7, 2023
Update the version of musl used on `*-linux-musl` targets to 1.2.3

Update the version of musl used on our Linux musl targets from 1.1.24 to 1.2.3 as proposed in rust-lang/compiler-team#572. musl 1.2.3 is the latest version of musl and supports the same range of Linux kernels as the 1.1 series. As such, it does not affect the minimum supported version of Linux for any of the musl targets.

One of the major musl 1.2 features is support for [time64](https://musl.libc.org/time64.html). This support is both source and ABI compatible with programs built against musl 1.1 and so updating the musl version for these targets should not cause Rust programs to fail to run or compile (a [crater run](rust-lang/rust#107129 (comment)) has been completed which demonstrates this for the `i686-unknown-linux-musl` target).

Once this change reaches stable, the `libc` crate will then be able to [update their definitions to support 64-bit time](rust-lang/libc#3068), matching the default musl 1.2 APIs exactly.

Fixes #91178
@wesleywiser
Copy link
Member Author

This PR doesn't contain any changes that would affect x86_64-unknown-linux-gnu so the perf results have to be noise/spurious.

@lqd
Copy link
Member

lqd commented May 8, 2023

I also think it's keccak / cranelift-codegen currently being noisy, and have seen that in other PRs.

image

@rustbot label: +perf-regression-triaged

@rustbot rustbot added the perf-regression-triaged The performance regression has been triaged. label May 8, 2023
@alex
Copy link
Member

alex commented May 9, 2023

This is perhaps a dumb question, but does this mean that, starting with rustc 1.71, binaries compiled by rustc will no longer work on musl 1.1 targets?

@NobodyXu
Copy link
Contributor

This is perhaps a dumb question, but does this mean that, starting with rustc 1.71, binaries compiled by rustc will no longer work on musl 1.1 targets?

AFAIK musl is statically linked anyway, so it probably won't matter

@alex
Copy link
Member

alex commented May 10, 2023

That's the default, but what about -Ctarget-feature=-crt-static builds (i.e., those intended for use on Alpine or other distros where musl is the system libc).

@NobodyXu
Copy link
Contributor

That's the default, but what about -Ctarget-feature=-crt-static builds (i.e., those intended for use on Alpine or other distros where musl is the system libc).

It's rare for people to link to musl libc dynamically, and Alpine just release a new version, upgrading musl libc to 1.2.4 so I guess this is ok.

It's not like there are many other distros there using musl libc and AFAIK Alpine is mostly used for container where upgrading to new version is relatively easy.

https://www.phoronix.com/news/Alpine-Linux-3.18

@nekopsykose
Copy link

iirc the only actual ABI difference is the new functions in 1.2.3, like qsort_r. since musl also internally moved to qsort_r being the default qsort, i'd imagine any code that used qsort (example) would now not run on a 1.1 host indeed since the symbol won't exist in the system, when compiled with the 1.2.3 versioned rust. this would only happen if the rust compiler emitted calls to libc qsort/qsort_r(does it?), same for any other new functions(i don't remember which others).

all other libc function use comes from the libc crate, and not just this change. this only breaks running on 1.1 hosts if the compiler emits new symbols via this alone.

@nekopsykose
Copy link

It's rare for people to link to musl libc dynamically, and Alpine just release a new version, upgrading musl libc to 1.2.4 so I guess this is ok.

the system libc being *newer* is always fine, since there is forward compatibility. it being "rare" or not, or where dynamic musl is used or not, however, does not answer the actual question.

@alex
Copy link
Member

alex commented May 10, 2023

@nekopsykose that's helpful, thanks.

(For fullness of context, my interest is that the Python ecosystem currently has a pre-built build env for binaries that target musl 1.1, but not for 1.2 yet. It seems like this may work for a little bit, until libc begins to take advantage of new symbols, and so getting a 1.2 target for Python wheels will be necessary.)

@nekopsykose
Copy link

until libc begins to take advantage of new symbols, and so getting a 1.2 target for Python wheels will be necessary.)

yep. currently, libc is also on 1.1.X. see rust-lang/libc#3068 for work towards 1.2 there- it will come as a new major version (0.3) when it happens.

@wesleywiser
Copy link
Member Author

My understanding matches that of @nekopsykose's and I don't think this change by itself would cause rustc to generate calls to new symbols introduced in musl 1.2.x so dynamically linking with musl 1.1 should continue to work. That being said, @alex if it's possible for you to test in that musl 1.1 build env with Rust nightly, it would be good to confirm this.

alex added a commit to pyca/cryptography that referenced this pull request May 10, 2023
@alex
Copy link
Member

alex commented May 10, 2023

@wesleywiser Thanks. I can confirm (pyca/cryptography#8902) that we're able to still build and then pass our minimal smoke test on musl 1.1, using rustc nightly (dated 5/9) -- see the musllinux_1_1 builds.

@wesleywiser
Copy link
Member Author

Thanks for testing @alex! Glad to hear it's working 🎉

@apiraino apiraino removed the to-announce Announce this issue on triage meeting label May 25, 2023
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Mar 4, 2024
…cs, r=ehuss

Update platform-support.md with supported musl version

This just reflects the current status quo, there is no actual change here since the update to musl 1.2.3 occurred in rust-lang#107129 and was approved in rust-lang/compiler-team#572.

I also normalized all mentions of musl libc to "musl" (non-capitalized per the project's site and Wikipedia page).

r? `@ehuss`
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Mar 4, 2024
…cs, r=ehuss

Update platform-support.md with supported musl version

This just reflects the current status quo, there is no actual change here since the update to musl 1.2.3 occurred in rust-lang#107129 and was approved in rust-lang/compiler-team#572.

I also normalized all mentions of musl libc to "musl" (non-capitalized per the project's site and Wikipedia page).

r? ``@ehuss``
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Mar 5, 2024
Rollup merge of rust-lang#121994 - wesleywiser:update_musl_version_docs, r=ehuss

Update platform-support.md with supported musl version

This just reflects the current status quo, there is no actual change here since the update to musl 1.2.3 occurred in rust-lang#107129 and was approved in rust-lang/compiler-team#572.

I also normalized all mentions of musl libc to "musl" (non-capitalized per the project's site and Wikipedia page).

r? ``@ehuss``
RalfJung pushed a commit to RalfJung/rust-analyzer that referenced this pull request Apr 20, 2024
Update the version of musl used on `*-linux-musl` targets to 1.2.3

Update the version of musl used on our Linux musl targets from 1.1.24 to 1.2.3 as proposed in rust-lang/compiler-team#572. musl 1.2.3 is the latest version of musl and supports the same range of Linux kernels as the 1.1 series. As such, it does not affect the minimum supported version of Linux for any of the musl targets.

One of the major musl 1.2 features is support for [time64](https://musl.libc.org/time64.html). This support is both source and ABI compatible with programs built against musl 1.1 and so updating the musl version for these targets should not cause Rust programs to fail to run or compile (a [crater run](rust-lang/rust#107129 (comment)) has been completed which demonstrates this for the `i686-unknown-linux-musl` target).

Once this change reaches stable, the `libc` crate will then be able to [update their definitions to support 64-bit time](rust-lang/libc#3068), matching the default musl 1.2 APIs exactly.

Fixes #91178
RalfJung pushed a commit to RalfJung/rust-analyzer that referenced this pull request Apr 27, 2024
Update the version of musl used on `*-linux-musl` targets to 1.2.3

Update the version of musl used on our Linux musl targets from 1.1.24 to 1.2.3 as proposed in rust-lang/compiler-team#572. musl 1.2.3 is the latest version of musl and supports the same range of Linux kernels as the 1.1 series. As such, it does not affect the minimum supported version of Linux for any of the musl targets.

One of the major musl 1.2 features is support for [time64](https://musl.libc.org/time64.html). This support is both source and ABI compatible with programs built against musl 1.1 and so updating the musl version for these targets should not cause Rust programs to fail to run or compile (a [crater run](rust-lang/rust#107129 (comment)) has been completed which demonstrates this for the `i686-unknown-linux-musl` target).

Once this change reaches stable, the `libc` crate will then be able to [update their definitions to support 64-bit time](rust-lang/libc#3068), matching the default musl 1.2 APIs exactly.

Fixes #91178
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-testsuite Area: The testsuite used to check the correctness of rustc disposition-merge This issue / PR is in PFCP or FCP with a disposition to merge it. finished-final-comment-period The final comment period is finished for this PR / Issue. merged-by-bors This PR was explicitly merged by bors. perf-regression Performance regression. perf-regression-triaged The performance regression has been triaged. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

x86_64-unknown-linux-musl: backport musl bug fixes or use new 1.2 version?