rustc: Stabilize `-Zrun-dsymutil` as `-Csplit-debuginfo` #79570

alexcrichton · 2020-11-30T16:46:19Z

This commit adds a new stable codegen option to rustc,
-Csplit-debuginfo. The old -Zrun-dsymutil flag is deleted and now
subsumed by this stable flag. Additionally -Zsplit-dwarf is also
subsumed by this flag but still requires -Zunstable-options to
actually activate. The -Csplit-debuginfo flag takes one of
three values:

off - This indicates that split-debuginfo from the final artifact is
not desired. This is not supported on Windows and is the default on
Unix platforms except macOS. On macOS this means that dsymutil is
not executed.
packed - This means that debuginfo is desired in one location
separate from the main executable. This is the default on Windows
(*.pdb) and macOS (*.dSYM). On other Unix platforms this subsumes
-Zsplit-dwarf=single and produces a *.dwp file.
unpacked - This means that debuginfo will be roughly equivalent to
object files, meaning that it's throughout the build directory
rather than in one location (often the fastest for local development).
This is not the default on any platform and is not supported on Windows.

Each target can indicate its own default preference for how debuginfo is
handled. Almost all platforms default to off except for Windows and
macOS which default to packed for historical reasons.

Some equivalencies for previous unstable flags with the new flags are:

-Zrun-dsymutil=yes -> -Csplit-debuginfo=packed
-Zrun-dsymutil=no -> -Csplit-debuginfo=unpacked
-Zsplit-dwarf=single -> -Csplit-debuginfo=packed
-Zsplit-dwarf=split -> -Csplit-debuginfo=unpacked

Note that -Csplit-debuginfo still requires -Zunstable-options for
non-macOS platforms since split-dwarf support was just implemented in
rustc.

There's some more rationale listed on #79361, but the main gist of the
motivation for this commit is that dsymutil can take quite a long time
to execute in debug builds and provides little benefit. This means that
incremental compile times appear that much worse on macOS because the
compiler is constantly running dsymutil over every single binary it
produces during cargo build (even build scripts!). Ideally rustc would
switch to not running dsymutil by default, but that's a problem left
to get tackled another day.

Closes #79361

rust-highfive · 2020-11-30T16:46:23Z

r? @petrochenkov

(rust-highfive has picked a reviewer for you, use r? to override)

bjorn3 · 2020-11-30T16:51:49Z

off - this is the platform-specific default that platforms other
than Apple use. This defaults to whatever rustc does today, which
unfortunately does mean split-debuginfo for MSVC targets (with PDB
files), but rustc currently has no means of turning that off today.
Note, though, that if this is selected on Apple platforms and
debuginfo is enabled then the final object files are not deleted and
instead preserved on-disk (since the main executable points to them).

I think off should mean debuginfo included in the final linked artifact. For the macOS behavior there should be something like an original-object option.

bjorn3 · 2020-11-30T18:17:48Z

To elaborate, I think there should be three modes:

off: The debuginfo is included in the linked artifact. this is the default on all platforms except macOS and Windows, which don't support this.
packed: The debuginfo is included in a single file/directory that is separate from the linked artifact. On macOS this would be a .dSYM dir. On Windows a .pdb file and with split dwarf, a .dwp file. this is the default for macOS and Windows. For non-linked crate types this is equivalent to unpacked.
unpacked: The debuginfo is included in several debuginfo files separate from the linked artifact. For split dwarf this would be .dwo files. For macOS this would be the original object files. This requires that all the rlibs that the crate depends are not deleted.

This does bring the question of how to handle different -Csplit-debuginfo values for different crates.

rlib\executable/dylib	off	packed	unpacked
off	trivial	debuginfo packer could read debuginfo from rlib. it likely won't on many systems	unpacked requires all compilation artifacts to remain in place, so at runtime the debuginfo could be loaded from the object files in the rlib
packed	see unpacked	see unpacked	see unpacked
unpacked	unsupported as the linker can't add the debuginfo to the link artifact again as far as I know	unpacked and packed are equivalent for rlibs	trivial

briansmith · 2020-11-30T18:33:00Z

This defaults to whatever rustc does today, which
unfortunately does mean split-debuginfo for MSVC targets (with PDB
files), but rustc currently has no means of turning that off today.

I don't understand why the PDB files are "unfortunate." It seems like they are actually the ideal mechanism: they separate out the debug info while avoiding the wasted work that is required in macOS's toolchain.

There's some more rationale listed on #79361, but the main gist of the
motivation for this commit is that dsymutil can take quite a long time
to execute in debug builds and provides little benefit. This means that
incremental compile times appear that much worse on macOS because the
compiler is constantly running dsymutil over every single binary it
produces during cargo build (even build scripts!). Ideally rustc would
switch to not running dsymutil by default, but that's a problem left
to get tackled another day.

I noticed that the mechanism and the motivation don't seem well aligned. The goal is to make builds fast while retaining debug info, but the proposed mechanism is all about where to place the debug info, regardless of performance. It seems likely that in the future different toolchains will evolve so that the fastest thing to do varies by target and/or toolchain. So it seems like there should be a way to say -Cput-debuginfo-wherever-results-in-the-fastest-build (with a better name), and it doesn't seem like any other values are necessary.

bjorn3 · 2020-11-30T18:55:41Z

The three modes of -Csplit-debuginfo each serve a different purpose. off ensures that you always have the debuginfo when you have the executable. This comes at the cost of a bigger executable. packed results in a smaller executable, but requires you to copy the debuginfo file in addition to the executable when you want to have debuginfo. unpacked results in the fastest possible link time, but you can't delete any of the compilation artifacts if you want to have debuginfo.

alexcrichton · 2020-11-30T18:56:20Z

@bjorn3 ah yeah those three modes sound reasonable to me! It would be easy to say that "off" isn't supported on Windows and macOS and it just defaults to "packed". I'm trying to interpret your table though and am having confusion, should one of the row or column headers be crate types? (not sure how to decipher the crate types only mentioned in the corner and having a matrix of debuginfo type x debuginfo type)

@briansmith ah by "unfortunate" I mean that as proposed if you say -Csplit-debuginfo=off you get split debuginfo on Windows, which is weird. I don't mean to try to pass a value judgment on the usefulness of MSVC's PDB splitting. I think it's a good point is that the defaults ideally are "build me the fastest" but given the interaction of debuginfo with other tooling it's not quite as straightforward. For example Rust's RUST_BACKTRACE support until just recently didn't support debuginfo-in-object-files on macOS (and still doesn't on Linux). That's just libstd's own support and depending on the tooling that Rust artifacts are integrated with I'd imagine that a specific strategy would be relied upon. If a project relied on whatever -Cput-debuginfo-wherever-results-in-the-fastest-build happened to do when it was implemented and then we changed it, we still run the risk of breaking projects.

It's still an open question how this is going to get integrated into Cargo. I would ideally like to turn this on by default but there are backwards compatibility concerns.

bjorn3 · 2020-11-30T19:34:20Z

I'm trying to interpret your table though and am having confusion

A row represents the -Csplit-debuginfo value for a dependency. A column represents the -Csplit-debuginfo value for the crate that is being linked.

petrochenkov · 2020-11-30T20:39:44Z

r? @bjorn3, I'll let you exercise your recently gained r+ rights here.

alexcrichton · 2020-12-02T15:15:24Z

@bjorn3 from a theoretical perspective we'd have to handle those various combinations for dependencies and the final executable but I'm not sure those situations come up in practice? If you're linking to a dylib then the compiler doesn't really worry itself with the debuginfo for that dylib itself as that's the responsibility of the dylib itself.

The only case it really matters is with an rlib (an archive) since those objects are being used in the final artifact. For rlibs, however, it seems like they're compiled some way and we'd just try to be compatible with it? I guess I don't really understand why we need to consider mixing and matching different modes with dependencies, since that seems like a linker or toolchain thing to worry about and rustc is just respecting what the user asked for. It may also mean that each option means different things on different platforms?

I think it seems reasonable to have a tri-state as you're proposing, but I can't seem to wrap my head around the "packed" and "unpacked" terminology, I get confused every time I think about it. I like how it's platform-agnostic, though, and we avoid the term "dsymutil" in a cli option ideally.

I think we're basically trying to find names for:

	off	packed	unpacked
Linux	default	`*.dwp` file on link	`*.dwo` files everywhere
macOS	no `dsymutil`, no objects saved	default (`*.dSYM`)	`-Zrun-dsymutil=no` equivalent
MSVC	no pdb from the linker	default (`*.pdb`)	not supported

(I'm not actually sure if MinGW supports split-debuginfo in the same way as Linux since it's DWARF, but I would presume so)

What if "packed" were changed to "on" and "unpacked" were changed to "objects"?

bjorn3 · 2020-12-02T15:36:48Z

from a theoretical perspective we'd have to handle those various combinations for dependencies and the final executable but I'm not sure those situations come up in practice?

Maybe not, but it should be handled in some way. Even if just by giving a warning that debuginfo is unavailable or straight up error.

If you're linking to a dylib then the compiler doesn't really worry itself with the debuginfo for that dylib itself as that's the responsibility of the dylib itself.

Correct

The only case it really matters is with an rlib (an archive) since those objects are being used in the final artifact. For rlibs, however, it seems like they're compiled some way and we'd just try to be compatible with it? I guess I don't really understand why we need to consider mixing and matching different modes with dependencies, since that seems like a linker or toolchain thing to worry about and rustc is just respecting what the user asked for. It may also mean that each option means different things on different platforms?

Rlibs include the debuginfo in a way that would be compatible with linking them using a certain -Csplit-debuginfo. On macOS and I think Windows that would be always including the debuginfo in the object files. On Linux that would be including it in the object files for -Csplit-debuginfo=off. For -Csplit-debuginfo=packed/unpacked, that would be as separate .dwo files. During the linking step the debuginfo would then be stored in the appropriate place. If .dwo files are generated, but -Csplit-debuginfo=no is used for the linked artifact, it would be necessary to stuff the .dwo files into the final artifact. To the best of my knowledge no linker allows this and as such rustc shouldn't allow this combination either. The other way around would require extracting .dwo/.dwp files from the rlib object files, which no toolchain supports either I think, but should be easier to do using something like gimli. I propose that this combination will also be forbidden. Combining unpacked and packed would work just fine as either way .dwo files are included in the rlib on Linux and the debuginfo would be stored in the object files on macOS and I think Windows.

I think it seems reasonable to have a tri-state as you're proposing, but I can't seem to wrap my head around the "packed" and "unpacked" terminology, I get confused every time I think about it. I like how it's platform-agnostic, though, and we avoid the term "dsymutil" in a cli option ideally.

Packed means that all debuginfo for the linked artifact is packed together in a single place. Unpacked means that the debuginfo is scattered all over the place.

For macOS and Windows off should be unsupported as off means that the debuginfo is included in the linked artifact. This is unsupported on these platforms.

What if "packed" were changed to "on" and "unpacked" were changed to "objects"?

unpacked wouldn't necessarily mean that it is stored in object files. In fact I think it should be allowed to store the debuginfo basically anywhere including for example a database at a fixed location with an index based on the full path of the executable or dylib.

alexcrichton · 2020-12-04T18:47:03Z

Sorry I don't really understand what you're asking for about warnings and dealing with mixed split-debuginfo states in dependencies. We don't do any warning or anyhing really with mixed values of debuginfo in dependencies, so why would we do anything with relation to split-debuginfo? Practically this is unlikely to ever really matter since the only "split" is whatever libstd does and what your local build does, and we don't ever want to warn folks about what libstd is doing since you wouldn't be able to silence it.

I still feel that "packed" and "unpacked" aren't the right terms for this, however. I think "on" and "off" are good terms for "in the executable" or "everything next to the executable" (sorry I keep confusing =off as "turn off debuginfo" whereas it's actually "put it all in one place in the executable"). The only other thing is "put it somewhere else that doesn't require assembling into one thing" and while it may not literally be in an object file I feel like *.dwo is similar enough to "object" that "=objects" would be good enough.

In any case though, what would the next steps for this be? I think it's an FCP, right? Could that perhaps be started to see if others have thoughts on this as well?

bjorn3 · 2020-12-04T19:07:53Z

We don't do any warning or anyhing really with mixed values of debuginfo in dependencies, so why would we do anything with relation to split-debuginfo?

Because mixing -Csplit-debuginfo could cause the debuginfo for crates that should include debuginfo to become unavailable. When mixing values of -Cdebuginfo, you explicitly state for which crates you want debuginfo to be available.

I still feel that "packed" and "unpacked" aren't the right terms for this, however. I think "on" and "off" are good terms for "in the executable" or "everything next to the executable" (sorry I keep confusing =off as "turn off debuginfo" whereas it's actually "put it all in one place in the executable"). The only other thing is "put it somewhere else that doesn't require assembling into one thing" and while it may not literally be in an object file I feel like *.dwo is similar enough to "object" that "=objects" would be good enough.

-Csplit-debuginfo=objects to me doesn't say that you can't really remove the target dir without debuginfo becoming unavailable. There is no way to tell the build system what files contain the debuginfo. Cargo uses rustc --crate-name ___ --print file-names to get determine how to get the file names of all output artifacts. This only allows a fixed amount of file names.

bjorn3 · 2020-12-04T19:08:53Z

In any case though, what would the next steps for this be? I think it's an FCP, right?

Makes sense

Could that perhaps be started to see if others have thoughts on this as well?

I don't really know what the procedure for this is.

alexcrichton · 2020-12-07T16:23:13Z

I would consider the availability of debuginfo the toolchain's or tool's issue. If debuginfo is somewhere but can't be loaded then it's either the tool's fault because it isn't reading what the compiler is emitting or it's the compiler's issue because it's not emitting information to find the debuginfo.

Yes it's currently an issue where the object files aren't considered outputs of a compilation. I don't think it's quite right to say that a variable number of ouputs isn't supported, the dep-info file can list all the files necessary.

I'm also not sure how to kick off FCP nowadays, how would one learn this?

bjorn3 · 2020-12-07T17:04:51Z

I would consider the availability of debuginfo the toolchain's or tool's issue. If debuginfo is somewhere but can't be loaded then it's either the tool's fault because it isn't reading what the compiler is emitting or it's the compiler's issue because it's not emitting information to find the debuginfo.

The build system is also in part responsible as it is responsible for managing all compilation artifacts.

Yes it's currently an issue where the object files aren't considered outputs of a compilation. I don't think it's quite right to say that a variable number of ouputs isn't supported, the dep-info file can list all the files necessary.

The dep-info file is only used by cargo for getting the build inputs and not outputs as far as I know. Also the dep-info files don't clearly separate inputs from outputs without any dependencies, as inputs also get an output entry to silence a warning/error from make when an input file gets deleted. Nor do they say what kind of output a file is.

I'm also not sure how to kick off FCP nowadays, how would one learn this?

Found the documentation: https://github.com/rust-lang/rfcbot-rs

nikomatsakis · 2020-12-08T00:05:18Z

@rfcbot fcp merge

It seems like there's still some bikeshedding about the proper term, but I'd like to kick off an FCP for the this stabilization. This seems like a really useful step forward to me!

rfcbot · 2020-12-08T00:05:22Z

`-Zrun-dsymutil` has been recently replaced with `-Csplit-debuginfo` (still nightly-only) See: rust-lang/rust#79570

This commit adds a new `split-debuginfo` option to Cargo compilation profiles which gets forwarded to the `-Csplit-debuginfo` codegen option in rustc. This commit also sets the default, only on macOS, to be `-Csplit-debuginfo=unpacked`. The purpose of this change is to leverage rust-lang/rust#79570 to avoid running `dsymutil` on incremental builds while also preserving a pleasant debugging experience by default. This should lead to much faster incremental build times on macOS since `dsymutil` isn't exactly the speediest tool in the world. This is technically a breaking change in Cargo because we're no longer by-default producing the `*.dSYM` folders on macOS. If those are still desired, however, authors can always run `dsymutil` themselves or otherwise configure `split-debuginfo = 'packed'` in their manifest/profile configuration.

Add split-debuginfo profile option This commit adds a new `split-debuginfo` option to Cargo compilation profiles which gets forwarded to the `-Csplit-debuginfo` codegen option in rustc. This commit also sets the default, only on macOS, to be `-Csplit-debuginfo=unpacked`. The purpose of this change is to leverage rust-lang/rust#79570 to avoid running `dsymutil` on incremental builds while also preserving a pleasant debugging experience by default. This should lead to much faster incremental build times on macOS since `dsymutil` isn't exactly the speediest tool in the world. This is technically a breaking change in Cargo because we're no longer by-default producing the `*.dSYM` folders on macOS. If those are still desired, however, authors can always run `dsymutil` themselves or otherwise configure `split-debuginfo = 'packed'` in their manifest/profile configuration.

alexcrichton · 2021-02-09T22:18:29Z

I've posted about this on internals for the next step hopefully in turning this on by default for Cargo users.

jyn514 · 2021-02-17T02:34:05Z

src/bootstrap/builder.rs

+            if stage == 0 {
+                if self.config.rust_run_dsymutil {
+                    rustflags.arg("-Zrun-dsymutil=yes");
+                } else {
+                    rustflags.arg("-Zrun-dsymutil=no");
+                }


FYI, it would be helpful next time to add // cfg(bootstrap) as a comment, this caused a failure that wasn't caught until testing on Mac: #82076 (comment)

Pkgsrc changes: * Remove one SunOS patch, apparently no longer needed. * Adapt one patch for Darwin, adjust cargo checksum accordingly. * Adjust bootstraps to version 1.50.0. Version 1.51.0 (2021-03-25) ============================ Language -------- - [You can now parameterize items such as functions, traits, and `struct`s by constant values in addition to by types and lifetimes.][79135] Also known as "const generics" E.g. you can now write the following. Note: Only values of primitive integers, `bool`, or `char` types are currently permitted. ```rust struct GenericArray<T, const LENGTH: usize> { inner: [T; LENGTH] } impl<T, const LENGTH: usize> GenericArray<T, LENGTH> { const fn last(&self) -> Option<&T> { if LENGTH == 0 { None } else { Some(&self.inner[LENGTH - 1]) } } } ``` Compiler -------- - [Added the `-Csplit-debuginfo` codegen option for macOS platforms.][79570] This option controls whether debug information is split across multiple files or packed into a single file. **Note** This option is unstable on other platforms. - [Added tier 3\* support for `aarch64_be-unknown-linux-gnu`, `aarch64-unknown-linux-gnu_ilp32`, and `aarch64_be-unknown-linux-gnu_ilp32` targets.][81455] - [Added tier 3 support for `i386-unknown-linux-gnu` and `i486-unknown-linux-gnu` targets.][80662] - [The `target-cpu=native` option will now detect individual features of CPUs.][80749] - [Rust now uses `inline-asm` for stack probes when used with LLVM 11.0.1+][77885] \* Refer to Rust's [platform support page][forge-platform-support] for more information on Rust's tiered platform support. Libraries --------- - [`Box::downcast` is now also implemented for any `dyn Any + Send + Sync` object.][80945] - [`str` now implements `AsMut<str>`.][80279] - [`u64` and `u128` now implement `From<char>`.][79502] - [`Error` is now implemented for `&T` where `T` implements `Error`.][75180] - [`Poll::{map_ok, map_err}` are now implemented for `Poll<Option<Result<T, E>>>`.][80968] - [`unsigned_abs` is now implemented for all signed integer types.][80959] - [`io::Empty` now implements `io::Seek`.][78044] - [`rc::Weak<T>` and `sync::Weak<T>`'s methods such as `as_ptr` are now implemented for `T: ?Sized` types.][80764] Stabilized APIs --------------- - [`Arc::decrement_strong_count`] - [`Arc::increment_strong_count`] - [`Once::call_once_force`] - [`Peekable::next_if_eq`] - [`Peekable::next_if`] - [`Seek::stream_position`] - [`array::IntoIter`] - [`panic::panic_any`] - [`ptr::addr_of!`] - [`ptr::addr_of_mut!`] - [`slice::fill_with`] - [`slice::split_inclusive_mut`] - [`slice::split_inclusive`] - [`slice::strip_prefix`] - [`slice::strip_suffix`] - [`str::split_inclusive`] - [`sync::OnceState`] - [`task::Wake`] Cargo ----- - [Added the `split-debuginfo` profile option to control the -Csplit-debuginfo codegen option.][cargo/9112] - [Added the `resolver` field to `Cargo.toml` to enable the new feature resolver and CLI option behavior.][cargo/8997] Version 2 of the feature resolver will try to avoid unifying features of dependencies where that unification could be unwanted. Such as using the same dependency with a `std` feature in a build scripts and proc-macros, while using the `no-std` feature in the final binary. See the [Cargo book documentation][feature-resolver@2.0] for more information on the feature. Rustdoc ------- - [Rustdoc will now include documentation for methods available from `Deref` traits.][80653] - [You can now provide a `--default-theme` flag which sets the default theme to use for documentation.][79642] Various improvements to intra-doc links: - [You can link to non-path primitives such as `slice`.][80181] - [You can link to associated items.][74489] - [You can now include generic parameters when linking to items, like `Vec<T>`.][76934] Misc ---- - [You can now pass `--include-ignored` to tests (e.g. with `cargo test -- --include-ignored`) to include testing tests marked `#[ignore]`.][80053] Compatibility Notes ------------------- - [WASI platforms no longer use the `wasm-bindgen` ABI, and instead use the wasm32 ABI.][79998] - [`rustc` no longer promotes division, modulo and indexing operations to `const` that could fail.][80579] - [The minimum version of glibc for the following platforms has been bumped to version 2.31 for the distributed artifacts.][81521] - `armv5te-unknown-linux-gnueabi` - `sparc64-unknown-linux-gnu` - `thumbv7neon-unknown-linux-gnueabihf` - `armv7-unknown-linux-gnueabi` - `x86_64-unknown-linux-gnux32` Internal Only ------------- - [Consistently avoid constructing optimized MIR when not doing codegen][80718] [79135]: rust-lang/rust#79135 [74489]: rust-lang/rust#74489 [76934]: rust-lang/rust#76934 [79570]: rust-lang/rust#79570 [80181]: rust-lang/rust#80181 [79642]: rust-lang/rust#79642 [80945]: rust-lang/rust#80945 [80279]: rust-lang/rust#80279 [80053]: rust-lang/rust#80053 [79502]: rust-lang/rust#79502 [75180]: rust-lang/rust#75180 [79135]: rust-lang/rust#79135 [81521]: rust-lang/rust#81521 [80968]: rust-lang/rust#80968 [80959]: rust-lang/rust#80959 [80718]: rust-lang/rust#80718 [80653]: rust-lang/rust#80653 [80579]: rust-lang/rust#80579 [79998]: rust-lang/rust#79998 [78044]: rust-lang/rust#78044 [81455]: rust-lang/rust#81455 [80764]: rust-lang/rust#80764 [80749]: rust-lang/rust#80749 [80662]: rust-lang/rust#80662 [77885]: rust-lang/rust#77885 [cargo/8997]: rust-lang/cargo#8997 [cargo/9112]: rust-lang/cargo#9112 [feature-resolver@2.0]: https://doc.rust-lang.org/nightly/cargo/reference/features.html#feature-resolver-version-2 [`Once::call_once_force`]: https://doc.rust-lang.org/stable/std/sync/struct.Once.html#method.call_once_force [`sync::OnceState`]: https://doc.rust-lang.org/stable/std/sync/struct.OnceState.html [`panic::panic_any`]: https://doc.rust-lang.org/stable/std/panic/fn.panic_any.html [`slice::strip_prefix`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.strip_prefix [`slice::strip_suffix`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.strip_prefix [`Arc::increment_strong_count`]: https://doc.rust-lang.org/nightly/std/sync/struct.Arc.html#method.increment_strong_count [`Arc::decrement_strong_count`]: https://doc.rust-lang.org/nightly/std/sync/struct.Arc.html#method.decrement_strong_count [`slice::fill_with`]: https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.fill_with [`ptr::addr_of!`]: https://doc.rust-lang.org/nightly/std/ptr/macro.addr_of.html [`ptr::addr_of_mut!`]: https://doc.rust-lang.org/nightly/std/ptr/macro.addr_of_mut.html [`array::IntoIter`]: https://doc.rust-lang.org/nightly/std/array/struct.IntoIter.html [`slice::split_inclusive`]: https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.split_inclusive [`slice::split_inclusive_mut`]: https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.split_inclusive_mut [`str::split_inclusive`]: https://doc.rust-lang.org/nightly/std/primitive.str.html#method.split_inclusive [`task::Wake`]: https://doc.rust-lang.org/nightly/std/task/trait.Wake.html [`Seek::stream_position`]: https://doc.rust-lang.org/nightly/std/io/trait.Seek.html#method.stream_position [`Peekable::next_if`]: https://doc.rust-lang.org/nightly/std/iter/struct.Peekable.html#method.next_if [`Peekable::next_if_eq`]: https://doc.rust-lang.org/nightly/std/iter/struct.Peekable.html#method.next_if_eq

Pkgsrc changes: * Add support for the big-endian arm64 NetBSD target (aarch64_be). * On NetBSD/i386, use the i586 (pentium) bootstrap kit variant in preference to i686. * Adjust patches, re-compute line offsets, re-compute crate checksums. * Remove a patch which was either integrated upstream and/or no longer applies. * Bump bootstraps to 1.50.0. * Move conditionals until after bsd.prefs.mk so that they work... * Default to "dist" build target if cross-compiling, but allow also to override via rust.BUILD_TARGET. * Allow overriding MAKE_JOBS_SAFE via rust.MAKE_JOBS_SAFE if you want a different trade-off between occasional breakage and performance. * Adjust platform.mk according to work already done in wip/rust/ * Add a patch to optimize the install.sh script used to install binary bootstraps to not do so many forks; use case/esac and parameter expansion instead of grep, sed and cut. * Drop building documentation for the binary bootstrap kits. This will also impact the lang/rust-bin package. For full documentation, build or install lang/rust as a package. Upstream changes: Version 1.51.0 (2021-03-25) ============================ Language -------- - [You can now parameterize items such as functions, traits, and `struct`s by constant values in addition to by types and lifetimes.][79135] Also known as "const generics" E.g. you can now write the following. Note: Only values of primitive integers, `bool`, or `char` types are currently permitted. ```rust struct GenericArray<T, const LENGTH: usize> { inner: [T; LENGTH] } impl<T, const LENGTH: usize> GenericArray<T, LENGTH> { const fn last(&self) -> Option<&T> { if LENGTH == 0 { None } else { Some(&self.inner[LENGTH - 1]) } } } ``` Compiler -------- - [Added the `-Csplit-debuginfo` codegen option for macOS platforms.][79570] This option controls whether debug information is split across multiple files or packed into a single file. **Note** This option is unstable on other platforms. - [Added tier 3\* support for `aarch64_be-unknown-linux-gnu`, `aarch64-unknown-linux-gnu_ilp32`, and `aarch64_be-unknown-linux-gnu_ilp32` targets.][81455] - [Added tier 3 support for `i386-unknown-linux-gnu` and `i486-unknown-linux-gnu` targets.][80662] - [The `target-cpu=native` option will now detect individual features of CPUs.][80749] \* Refer to Rust's [platform support page][platform-support-doc] for more information on Rust's tiered platform support. Libraries --------- - [`Box::downcast` is now also implemented for any `dyn Any + Send + Sync` object.][80945] - [`str` now implements `AsMut<str>`.][80279] - [`u64` and `u128` now implement `From<char>`.][79502] - [`Error` is now implemented for `&T` where `T` implements `Error`.][75180] - [`Poll::{map_ok, map_err}` are now implemented for `Poll<Option<Result<T,E>>>`.][80968] - [`unsigned_abs` is now implemented for all signed integer types.][80959] - [`io::Empty` now implements `io::Seek`.][78044] - [`rc::Weak<T>` and `sync::Weak<T>`'s methods such as `as_ptr` are now implemented for `T: ?Sized` types.][80764] - [`Div` and `Rem` by their `NonZero` variant is now implemented for all unsigned integers.][79134] Stabilized APIs --------------- - [`Arc::decrement_strong_count`] - [`Arc::increment_strong_count`] - [`Once::call_once_force`] - [`Peekable::next_if_eq`] - [`Peekable::next_if`] - [`Seek::stream_position`] - [`array::IntoIter`] - [`panic::panic_any`] - [`ptr::addr_of!`] - [`ptr::addr_of_mut!`] - [`slice::fill_with`] - [`slice::split_inclusive_mut`] - [`slice::split_inclusive`] - [`slice::strip_prefix`] - [`slice::strip_suffix`] - [`str::split_inclusive`] - [`sync::OnceState`] - [`task::Wake`] - [`VecDeque::range`] - [`VecDeque::range_mut`] Cargo ----- - [Added the `split-debuginfo` profile option to control the -Csplit-debuginfo codegen option.][cargo/9112] - [Added the `resolver` field to `Cargo.toml` to enable the new feature resolver and CLI option behavior.][cargo/8997] Version 2 of the feature resolver will try to avoid unifying features of dependencies where that unification could be unwanted. Such as using the same dependency with a `std` feature in a build scripts and proc-macros, while using the `no-std` feature in the final binary. See the [Cargo book documentation][feature-resolver@2.0] for more information on the feature. Rustdoc ------- - [Rustdoc will now include documentation for methods available from _nested_ `Deref` traits.][80653] - [You can now provide a `--default-theme` flag which sets the default theme to use for documentation.][79642] Various improvements to intra-doc links: - [You can link to non-path primitives such as `slice`.][80181] - [You can link to associated items.][74489] - [You can now include generic parameters when linking to items, like `Vec<T>`.][76934] Misc ---- - [You can now pass `--include-ignored` to tests (e.g. with `cargo test -- --include-ignored`) to include testing tests marked `#[ignore]`.][80053] Compatibility Notes ------------------- - [WASI platforms no longer use the `wasm-bindgen` ABI, and instead use the wasm32 ABI.][79998] - [`rustc` no longer promotes division, modulo and indexing operations to `const` that could fail.][80579] - [The minimum version of glibc for the following platforms has been bumped to version 2.31 for the distributed artifacts.][81521] - `armv5te-unknown-linux-gnueabi` - `sparc64-unknown-linux-gnu` - `thumbv7neon-unknown-linux-gnueabihf` - `armv7-unknown-linux-gnueabi` - `x86_64-unknown-linux-gnux32` - [`atomic::spin_loop_hint` has been deprecated.][80966] It's recommended to use `hint::spin_loop` instead. Internal Only ------------- - [Consistently avoid constructing optimized MIR when not doing codegen][80718] [79135]: rust-lang/rust#79135 [74489]: rust-lang/rust#74489 [76934]: rust-lang/rust#76934 [79570]: rust-lang/rust#79570 [80181]: rust-lang/rust#80181 [79642]: rust-lang/rust#79642 [80945]: rust-lang/rust#80945 [80279]: rust-lang/rust#80279 [80053]: rust-lang/rust#80053 [79502]: rust-lang/rust#79502 [75180]: rust-lang/rust#75180 [79135]: rust-lang/rust#79135 [81521]: rust-lang/rust#81521 [80968]: rust-lang/rust#80968 [80959]: rust-lang/rust#80959 [80718]: rust-lang/rust#80718 [80653]: rust-lang/rust#80653 [80579]: rust-lang/rust#80579 [79998]: rust-lang/rust#79998 [78044]: rust-lang/rust#78044 [81455]: rust-lang/rust#81455 [80764]: rust-lang/rust#80764 [80749]: rust-lang/rust#80749 [80662]: rust-lang/rust#80662 [79134]: rust-lang/rust#79134 [80966]: rust-lang/rust#80966 [cargo/8997]: rust-lang/cargo#8997 [cargo/9112]: rust-lang/cargo#9112 [feature-resolver@2.0]: https://doc.rust-lang.org/nightly/cargo/reference/features.html#feature-resolver-version-2 [`Once::call_once_force`]: https://doc.rust-lang.org/stable/std/sync/struct.Once.html#method.call_once_force [`sync::OnceState`]: https://doc.rust-lang.org/stable/std/sync/struct.OnceState.html [`panic::panic_any`]: https://doc.rust-lang.org/stable/std/panic/fn.panic_any.html [`slice::strip_prefix`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.strip_prefix [`slice::strip_suffix`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.strip_prefix [`Arc::increment_strong_count`]: https://doc.rust-lang.org/nightly/std/sync/struct.Arc.html#method.increment_strong_count [`Arc::decrement_strong_count`]: https://doc.rust-lang.org/nightly/std/sync/struct.Arc.html#method.decrement_strong_count [`slice::fill_with`]: https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.fill_with [`ptr::addr_of!`]: https://doc.rust-lang.org/nightly/std/ptr/macro.addr_of.html [`ptr::addr_of_mut!`]: https://doc.rust-lang.org/nightly/std/ptr/macro.addr_of_mut.html [`array::IntoIter`]: https://doc.rust-lang.org/nightly/std/array/struct.IntoIter.html [`slice::split_inclusive`]: https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.split_inclusive [`slice::split_inclusive_mut`]: https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.split_inclusive_mut [`str::split_inclusive`]: https://doc.rust-lang.org/nightly/std/primitive.str.html#method.split_inclusive [`task::Wake`]: https://doc.rust-lang.org/nightly/std/task/trait.Wake.html [`Seek::stream_position`]: https://doc.rust-lang.org/nightly/std/io/trait.Seek.html#method.stream_position [`Peekable::next_if`]: https://doc.rust-lang.org/nightly/std/iter/struct.Peekable.html#method.next_if [`Peekable::next_if_eq`]: https://doc.rust-lang.org/nightly/std/iter/struct.Peekable.html#method.next_if_eq [`VecDeque::range`]: https://doc.rust-lang.org/nightly/std/collections/struct.VecDeque.html#method.range [`VecDeque::range_mut`]: https://doc.rust-lang.org/nightly/std/collections/struct.VecDeque.html#method.range_mut

luser · 2021-09-22T14:39:36Z

unpacked - This means that debuginfo will be roughly equivalent to
object files, meaning that it's throughout the build directory
rather than in one location (often the fastest for local development).
This is not the default on any platform and is not supported on Windows.

I know this is an old PR but I was catching up on this and I wanted to note that there are two MSVC features that are related and could potentially be lumped under this feature:

link.exe supports a /DEBUG:FASTLINK option, which makes it operate roughly the same as Apple's ld without running dsymutil: it produces a simple PDB during the link that is just an index to the debug info contained in the PDB files for each compilation unit. It comes with quite a few caveats, several of which are detailed in this LLVM blog post.
cl.exe supports a /Z7 option to include debug info directly in the generated .obj file instead of in a .pdb file. The linker can still produce a .pdb file from this debug info during linking. While I was at Mozilla we made the Firefox build use this option because it plays nicer with sccache caching. (I admit I don't know where debug info for .rlib compilation on Windows winds up currently. In a PDB next to the rlib? Inside the rlib?)

bjorn3 · 2021-09-22T15:02:17Z

I admit I don't know where debug info for .rlib compilation on Windows winds up currently. In a PDB next to the rlib? Inside the rlib?

AFAIK inside the object files inside an rlib.

In rust-lang#79570, `-Z split-dwarf-kind={none,single,split}` was replaced by `-C split-debuginfo={off,packed,unpacked}`. `-C split-debuginfo`'s packed and unpacked aren't exact parallels to single and split, respectively. On Unix, `-C split-debuginfo=packed` will put debuginfo into object files and package debuginfo into a DWARF package file (`.dwp`) and `-C split-debuginfo=unpacked` will put debuginfo into dwarf object files and won't package it. In the initial implementation of Split DWARF, split mode wrote sections which did not require relocation into a DWARF object (`.dwo`) file which was ignored by the linker and then packaged those DWARF objects into DWARF packages (`.dwp`). In single mode, sections which did not require relocation were written into object files but ignored by the linker and were not packaged. However, both split and single modes could be packaged or not, the primary difference in behaviour was where the debuginfo sections that did not require link-time relocation were written (in a DWARF object or the object file). This commit re-introduces a `-Z split-dwarf-kind` flag, which can be used to pick between split and single modes when `-C split-debuginfo` is used to enable Split DWARF (either packed or unpacked). Signed-off-by: David Wood <david.wood@huawei.com>

…-multiple-dwarves, r=nagisa cg: split dwarf for crate dependencies Fixes rust-lang#81024. - In rust-lang#79570, `-Z split-dwarf-kind={none,single,split}` was replaced by `-C split-debuginfo={off,packed,unpacked}`. `-C split-debuginfo`'s packed and unpacked aren't exact parallels to single and split, respectively. On Unix, `-C split-debuginfo=packed` will put debuginfo in object files and package debuginfo into a DWARF package file (`.dwp`) and `-C split-debuginfo=unpacked` will put debuginfo in dwarf object files and won't package it. In the initial implementation of Split DWARF, split mode wrote sections which did not require relocation into a DWARF object (`.dwo`) file which was ignored by the linker and then packaged those DWARF objects into DWARF packages (`.dwp`). In single mode, sections which did not require relocation were written into object files but ignored by the linker and were not packaged. However, both split and single modes could be packaged or not, the primary difference in behaviour was where the debuginfo sections that did not require link-time relocation were written (in a DWARF object or the object file). In the first commit of this PR, I re-introduce a `-Z split-dwarf-kind` flag, which can be used to pick between split and single modes when `-C split-debuginfo` is used to enable Split DWARF (either packed or unpacked). - Split DWARF packaging requires all of the object files to exist, including those in dependencies. ~~Therefore, the second commit of this PR makes rustc keep all objects or dwarf objects for unpacked mode and if the crate is a dependency in packed mode (determined by heuristic: if no linking is taking place), then objects or dwarf objects are kept. Objects are kept if `-Z split-dwarf-kind` is `SplitDwarfKind::Single`, and dwarf objects if `SplitDwarfKind::Split`.~~ ~~There are other approaches that could be taken to supporting packed Split DWARF with crate dependencies but this seemed like the least complicated and was contained to only rustc. Other potential approaches are described in rust-lang#81024 (comment), I'm happy to change the approach I've taken here if it isn't what we're looking for.~~ See rust-lang#89819 (comment) for the current approach. - ~~There's still a dependency on `llvm-dwp` after this change, which [we probably want to move away from](https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/llvm-dwp.20is.20not.20recommended) but that seems out-of-scope for this PR. Ideally, Split DWARF (in packed or unpacked modes) will be usable on nightly after this lands. If there aren't any bugs reported then it's possible we could allow Split DWARF to be used on stable after this change, it depends whether or not switching away from `llvm-dwp` later would break any guarantees, or whether we'd want to change how we handle this cross-crate case in future.~~ See rust-lang#89819 (comment). r? `@nagisa` cc `@alexcrichton`

rust-highfive assigned petrochenkov Nov 30, 2020

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Nov 30, 2020

alexcrichton mentioned this pull request Nov 30, 2020

Tracking issue for -Zrun-dsymutil #79361

Closed

jyn514 added A-debuginfo Area: Debugging information in compiled programs (DWARF, PDB, etc.) needs-fcp This change is insta-stable, so needs a completed FCP to proceed. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Nov 30, 2020

rust-highfive assigned bjorn3 and unassigned petrochenkov Nov 30, 2020

alexcrichton closed this Dec 4, 2020

alexcrichton reopened this Dec 4, 2020

bjorn3 mentioned this pull request Dec 7, 2020

cg_llvm: split dwarf support #77117

Merged

4 tasks

rfcbot added proposed-final-comment-period Proposed to merge/close by relevant subteam, see T-<team> label. Will enter FCP once signed off. disposition-merge This issue / PR is in PFCP or FCP with a disposition to merge it. labels Dec 8, 2020

Divoolej mentioned this pull request Jan 31, 2021

Update config_fast_builds for latest nightly bevyengine/bevy#1362

Merged

cart pushed a commit to bevyengine/bevy that referenced this pull request Jan 31, 2021

Update config_fast_builds for latest nightly (#1362)

83777e0

`-Zrun-dsymutil` has been recently replaced with `-Csplit-debuginfo` (still nightly-only) See: rust-lang/rust#79570

jyn514 mentioned this pull request Feb 17, 2021

Update the bootstrap compiler #82076

Merged

jyn514 reviewed Feb 17, 2021

View reviewed changes

calavera mentioned this pull request Apr 22, 2021

Make symbols stripping work on MacOS X #82037

Merged

davidtwco mentioned this pull request Oct 12, 2021

cg: split dwarf for crate dependencies #89819

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

rustc: Stabilize `-Zrun-dsymutil` as `-Csplit-debuginfo` #79570

rustc: Stabilize `-Zrun-dsymutil` as `-Csplit-debuginfo` #79570

alexcrichton commented Nov 30, 2020 •

edited

rust-highfive commented Nov 30, 2020

bjorn3 commented Nov 30, 2020

bjorn3 commented Nov 30, 2020

briansmith commented Nov 30, 2020

bjorn3 commented Nov 30, 2020

alexcrichton commented Nov 30, 2020

bjorn3 commented Nov 30, 2020

petrochenkov commented Nov 30, 2020

alexcrichton commented Dec 2, 2020

bjorn3 commented Dec 2, 2020

alexcrichton commented Dec 4, 2020

bjorn3 commented Dec 4, 2020

bjorn3 commented Dec 4, 2020

alexcrichton commented Dec 7, 2020

bjorn3 commented Dec 7, 2020

nikomatsakis commented Dec 8, 2020

rfcbot commented Dec 8, 2020 •

edited by estebank

alexcrichton commented Feb 9, 2021

jyn514 Feb 17, 2021

luser commented Sep 22, 2021

bjorn3 commented Sep 22, 2021

rustc: Stabilize -Zrun-dsymutil as -Csplit-debuginfo #79570

rustc: Stabilize -Zrun-dsymutil as -Csplit-debuginfo #79570

Conversation

alexcrichton commented Nov 30, 2020 • edited

rust-highfive commented Nov 30, 2020

bjorn3 commented Nov 30, 2020

bjorn3 commented Nov 30, 2020

briansmith commented Nov 30, 2020

bjorn3 commented Nov 30, 2020

alexcrichton commented Nov 30, 2020

bjorn3 commented Nov 30, 2020

petrochenkov commented Nov 30, 2020

alexcrichton commented Dec 2, 2020

bjorn3 commented Dec 2, 2020

alexcrichton commented Dec 4, 2020

bjorn3 commented Dec 4, 2020

bjorn3 commented Dec 4, 2020

alexcrichton commented Dec 7, 2020

bjorn3 commented Dec 7, 2020

nikomatsakis commented Dec 8, 2020

rfcbot commented Dec 8, 2020 • edited by estebank

alexcrichton commented Feb 9, 2021

jyn514 Feb 17, 2021

Choose a reason for hiding this comment

luser commented Sep 22, 2021

bjorn3 commented Sep 22, 2021

rustc: Stabilize `-Zrun-dsymutil` as `-Csplit-debuginfo` #79570

rustc: Stabilize `-Zrun-dsymutil` as `-Csplit-debuginfo` #79570

alexcrichton commented Nov 30, 2020 •

edited

rfcbot commented Dec 8, 2020 •

edited by estebank