strip prefix of temporary file names when it exceeds filesystem name length limit #145005
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rustbot
added
S-waiting-on-review
This comment has been minimized.
This comment has been minimized.
|
This method is also used for generating output files in various places. Mangling the name of output files would break cargo and other build tools that pre-generate the expected filenames based on |
|
@bjorn3 would it be more appropriate to put the filename stripping in temp_path_ext_for_cgu ? those files if I understand correctly are temporary and unlinked after the rustc invocation. |
tardyp
force-pushed
the
lto_big_filesize
branch
2 times, most recently
from
4d6eb1e to
3239eeb
Compare
|
r? bjorn3 feel free to reassign |
bjorn3
reviewed
Aug 7, 2025
| filename[..stripped_len].hash(&mut hasher); | ||
| let hash = hasher.finish::<Hash64>(); | ||
|
|
||
| path.set_file_name(format!("{:x}-{}", hash, &filename[stripped_len..])); |
There was a problem hiding this comment.
The start of the filename has more useful information, but we do absolutely need to preserve file extensions (plural: eg .rcgu.o) for correctness and it isn't trivial to distinguish between file extensions and normal dot separated filename components. So I guess the current implementation is good enough.
bjorn3
reviewed
Aug 7, 2025
...-large-large-large-large-large-large-large-large-large-large-large-large-large-crate-name.rs
Outdated
Show resolved
Hide resolved
tardyp
force-pushed
the
lto_big_filesize
branch
from
3239eeb to
7d98d96
Compare
tardyp
force-pushed
the
lto_big_filesize
branch
2 times, most recently
from
4e616be to
63fd877
Compare
This comment has been minimized.
This comment has been minimized.
…length limit When doing lto, rustc generates filenames that are concatenating many information. In the case of this testcase, it is concatenating crate name and rust file name, plus some hash, and the extension. In some other cases it will concatenate even more information reducing the maximum effective crate name to about 110 chars on linux filesystems where filename max length is 255 This commit is ensuring that the temporary file names are limited in size, while still reasonabily ensuring the unicity (with hashing of the stripped part)
tardyp
force-pushed
the
lto_big_filesize
branch
from
63fd877 to
bc14ad3
Compare
|
@bors r+ |
bors
added
S-waiting-on-bors
Zalathar
added a commit
to Zalathar/rust
that referenced
this pull request
Aug 14, 2025
strip prefix of temporary file names when it exceeds filesystem name length limit When doing lto, rustc generates filenames that are concatenating many information. In the case of this testcase, it is concatenating crate name and rust file name, plus some hash, and the extension. In some other cases it will concatenate even more information reducing the maximum effective crate name to about 110 chars on linux filesystems where filename max length is 255 This commit is ensuring that the temporary file names are limited in size, while still reasonably ensuring the unicity (with hashing of the stripped part) Fix: rust-lang#49914
bors
added a commit
that referenced
this pull request
Aug 14, 2025
Rollup of 13 pull requests Successful merges: - #122661 (Change the desugaring of `assert!` for better error output) - #142372 (Improve `--remap-path-prefix` documentation) - #143075 (compiler: Allow `extern "interrupt" fn() -> !`) - #145005 (strip prefix of temporary file names when it exceeds filesystem name length limit) - #145065 (resolve: Introduce `RibKind::Block`) - #145120 (llvm: Accept new LLVM lifetime format) - #145189 (Weekly `cargo update`) - #145261 (Improve tracing in bootstrap) - #145266 (Reduce some queries around associated items) - #145299 (doc test: fix mpsc.rs try_send doc test) - #145331 (Make std use the edition 2024 prelude) - #145353 (bootstrap: Fix jemalloc 64K page support for aarch64 tools) - #145361 (Suppress wrapper suggestion when expected and actual ty are the same adt and the variant is unresolved) Failed merges: - #145324 (Rename and document `ONLY_HOSTS` in bootstrap) - #145372 (resolve: Miscellaneous cleanups) r? `@ghost` `@rustbot` modify labels: rollup
bors
added a commit
that referenced
this pull request
Aug 14, 2025
Rollup of 13 pull requests Successful merges: - #122661 (Change the desugaring of `assert!` for better error output) - #142372 (Improve `--remap-path-prefix` documentation) - #143075 (compiler: Allow `extern "interrupt" fn() -> !`) - #145005 (strip prefix of temporary file names when it exceeds filesystem name length limit) - #145065 (resolve: Introduce `RibKind::Block`) - #145120 (llvm: Accept new LLVM lifetime format) - #145189 (Weekly `cargo update`) - #145261 (Improve tracing in bootstrap) - #145266 (Reduce some queries around associated items) - #145299 (doc test: fix mpsc.rs try_send doc test) - #145331 (Make std use the edition 2024 prelude) - #145353 (bootstrap: Fix jemalloc 64K page support for aarch64 tools) - #145361 (Suppress wrapper suggestion when expected and actual ty are the same adt and the variant is unresolved) Failed merges: - #145324 (Rename and document `ONLY_HOSTS` in bootstrap) - #145372 (resolve: Miscellaneous cleanups) r? `@ghost` `@rustbot` modify labels: rollup
GuillaumeGomez
added a commit
to GuillaumeGomez/rust
that referenced
this pull request
Aug 14, 2025
strip prefix of temporary file names when it exceeds filesystem name length limit When doing lto, rustc generates filenames that are concatenating many information. In the case of this testcase, it is concatenating crate name and rust file name, plus some hash, and the extension. In some other cases it will concatenate even more information reducing the maximum effective crate name to about 110 chars on linux filesystems where filename max length is 255 This commit is ensuring that the temporary file names are limited in size, while still reasonably ensuring the unicity (with hashing of the stripped part) Fix: rust-lang#49914
bors
added a commit
that referenced
this pull request
Aug 14, 2025
Rollup of 9 pull requests Successful merges: - #118087 (Add Ref/RefMut try_map method) - #140794 (Add information about group a lint belongs to) - #144947 (Fix description of unsigned `checked_exact_div`) - #145005 (strip prefix of temporary file names when it exceeds filesystem name length limit) - #145233 (cfg_select: Support unbraced expressions) - #145243 (take attr style into account in diagnostics) - #145353 (bootstrap: Fix jemalloc 64K page support for aarch64 tools) - #145379 (bootstrap: Support passing `--timings` to cargo) - #145389 ([rustdoc] Revert "rustdoc search: prefer stable items in search results") Failed merges: - #144983 (Rehome 37 `tests/ui/issues/` tests to other subdirectories under `tests/ui/`) - #145065 (resolve: Introduce `RibKind::Block`) r? `@ghost` `@rustbot` modify labels: rollup
Zalathar
added a commit
to Zalathar/rust
that referenced
this pull request
Aug 15, 2025
strip prefix of temporary file names when it exceeds filesystem name length limit When doing lto, rustc generates filenames that are concatenating many information. In the case of this testcase, it is concatenating crate name and rust file name, plus some hash, and the extension. In some other cases it will concatenate even more information reducing the maximum effective crate name to about 110 chars on linux filesystems where filename max length is 255 This commit is ensuring that the temporary file names are limited in size, while still reasonably ensuring the unicity (with hashing of the stripped part) Fix: rust-lang#49914
bors
added a commit
that referenced
this pull request
Aug 15, 2025
Rollup of 22 pull requests Successful merges: - #118087 (Add Ref/RefMut try_map method) - #122661 (Change the desugaring of `assert!` for better error output) - #140740 (Add `-Zindirect-branch-cs-prefix`) - #142640 (Implement autodiff using intrinsics) - #143075 (compiler: Allow `extern "interrupt" fn() -> !`) - #144865 (Fix tail calls to `#[track_caller]` functions) - #144944 (E0793: Clarify that it applies to unions as well) - #144947 (Fix description of unsigned `checked_exact_div`) - #145004 (Couple of minor cleanups) - #145005 (strip prefix of temporary file names when it exceeds filesystem name length limit) - #145012 (Tail call diagnostics to include lifetime info) - #145065 (resolve: Introduce `RibKind::Block`) - #145120 (llvm: Accept new LLVM lifetime format) - #145189 (Weekly `cargo update`) - #145235 (Minor `[const]` tweaks) - #145275 (fix(compiler/rustc_codegen_llvm): apply `target-cpu` attribute) - #145322 (Resolve the prelude import in `build_reduced_graph`) - #145331 (Make std use the edition 2024 prelude) - #145369 (Do not ICE on private type in field of unresolved struct) - #145378 (Add `FnContext` in parser for diagnostic) - #145389 ([rustdoc] Revert "rustdoc search: prefer stable items in search results") - #145392 (coverage: Remove intermediate data structures from mapping creation) r? `@ghost` `@rustbot` modify labels: rollup
Zalathar
added a commit
to Zalathar/rust
that referenced
this pull request
Aug 15, 2025
strip prefix of temporary file names when it exceeds filesystem name length limit When doing lto, rustc generates filenames that are concatenating many information. In the case of this testcase, it is concatenating crate name and rust file name, plus some hash, and the extension. In some other cases it will concatenate even more information reducing the maximum effective crate name to about 110 chars on linux filesystems where filename max length is 255 This commit is ensuring that the temporary file names are limited in size, while still reasonably ensuring the unicity (with hashing of the stripped part) Fix: rust-lang#49914
bors
added a commit
that referenced
this pull request
Aug 15, 2025
Rollup of 21 pull requests Successful merges: - #118087 (Add Ref/RefMut try_map method) - #122661 (Change the desugaring of `assert!` for better error output) - #142640 (Implement autodiff using intrinsics) - #143075 (compiler: Allow `extern "interrupt" fn() -> !`) - #144865 (Fix tail calls to `#[track_caller]` functions) - #144944 (E0793: Clarify that it applies to unions as well) - #144947 (Fix description of unsigned `checked_exact_div`) - #145004 (Couple of minor cleanups) - #145005 (strip prefix of temporary file names when it exceeds filesystem name length limit) - #145012 (Tail call diagnostics to include lifetime info) - #145065 (resolve: Introduce `RibKind::Block`) - #145120 (llvm: Accept new LLVM lifetime format) - #145189 (Weekly `cargo update`) - #145235 (Minor `[const]` tweaks) - #145275 (fix(compiler/rustc_codegen_llvm): apply `target-cpu` attribute) - #145322 (Resolve the prelude import in `build_reduced_graph`) - #145331 (Make std use the edition 2024 prelude) - #145369 (Do not ICE on private type in field of unresolved struct) - #145378 (Add `FnContext` in parser for diagnostic) - #145389 ([rustdoc] Revert "rustdoc search: prefer stable items in search results") - #145392 (coverage: Remove intermediate data structures from mapping creation) r? `@ghost` `@rustbot` modify labels: rollup
rust-timer
added a commit
that referenced
this pull request
Aug 15, 2025
Rollup merge of #145005 - tardyp:lto_big_filesize, r=bjorn3 strip prefix of temporary file names when it exceeds filesystem name length limit When doing lto, rustc generates filenames that are concatenating many information. In the case of this testcase, it is concatenating crate name and rust file name, plus some hash, and the extension. In some other cases it will concatenate even more information reducing the maximum effective crate name to about 110 chars on linux filesystems where filename max length is 255 This commit is ensuring that the temporary file names are limited in size, while still reasonably ensuring the unicity (with hashing of the stripped part) Fix: #49914
github-actions bot
pushed a commit
to rust-lang/rustc-dev-guide
that referenced
this pull request
Aug 18, 2025
Rollup of 21 pull requests Successful merges: - rust-lang/rust#118087 (Add Ref/RefMut try_map method) - rust-lang/rust#122661 (Change the desugaring of `assert!` for better error output) - rust-lang/rust#142640 (Implement autodiff using intrinsics) - rust-lang/rust#143075 (compiler: Allow `extern "interrupt" fn() -> !`) - rust-lang/rust#144865 (Fix tail calls to `#[track_caller]` functions) - rust-lang/rust#144944 (E0793: Clarify that it applies to unions as well) - rust-lang/rust#144947 (Fix description of unsigned `checked_exact_div`) - rust-lang/rust#145004 (Couple of minor cleanups) - rust-lang/rust#145005 (strip prefix of temporary file names when it exceeds filesystem name length limit) - rust-lang/rust#145012 (Tail call diagnostics to include lifetime info) - rust-lang/rust#145065 (resolve: Introduce `RibKind::Block`) - rust-lang/rust#145120 (llvm: Accept new LLVM lifetime format) - rust-lang/rust#145189 (Weekly `cargo update`) - rust-lang/rust#145235 (Minor `[const]` tweaks) - rust-lang/rust#145275 (fix(compiler/rustc_codegen_llvm): apply `target-cpu` attribute) - rust-lang/rust#145322 (Resolve the prelude import in `build_reduced_graph`) - rust-lang/rust#145331 (Make std use the edition 2024 prelude) - rust-lang/rust#145369 (Do not ICE on private type in field of unresolved struct) - rust-lang/rust#145378 (Add `FnContext` in parser for diagnostic) - rust-lang/rust#145389 ([rustdoc] Revert "rustdoc search: prefer stable items in search results") - rust-lang/rust#145392 (coverage: Remove intermediate data structures from mapping creation) r? `@ghost` `@rustbot` modify labels: rollup
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When doing lto, rustc generates filenames that are concatenating many information.
In the case of this testcase, it is concatenating crate name and rust file name, plus some hash, and the extension. In some other cases it will concatenate even more information reducing the maximum effective crate name to about 110 chars on linux filesystems where filename max length is 255
This commit is ensuring that the temporary file names are limited in size, while still reasonably ensuring the unicity (with hashing of the stripped part)
Fix: #49914