many unexpected duplicates in --print native-static-libs output in some configurations

[cpu]

Hi there,

In rustls-ffi we use RUSTFLAGS="--print native-static-libs" cargo build to try and keep our crate's linking instructions up-to-date across supported platforms.

We've noticed that in some build configurations the output can change in surprising ways.

For instance, using rustc nightly (1.70.0) on Windows (x86_64-pc-windows-msvc) in a --debug configuration driven by cmake we see many consecutive duplicate libraries in the --print-native-static-libs output.

I understand duplicates are to be expected, and can be meaningful on some platforms, but seeing so many consecutive repeats seems unexpected and unlikely to be meaningful for the build process.

Is this perhaps a regression? Or are we making a mistake by relying on the stability of this output across versions/build configurations? This feature is documented as having a known prefix to "make it easier to fetch the output." leading to the idea that it is expected to be machine consumable.

Code

I tried this code:

RUSTFLAGS="--print native-static-libs" cargo build 

I expected to see this happen:

A list produced, with some duplicates (e.g. multiple kernel32.lib), but not many consecutive duplicates. E.g.:

advapi32.lib credui.lib kernel32.lib secur32.lib legacy_stdio_definitions.lib kernel32.lib advapi32.lib bcrypt.lib kernel32.lib ntdll.lib userenv.lib ws2_32.lib kernel32.lib ws2_32.lib kernel32.lib msvcrt.lib

Instead, this happened:

A list with many consecutive duplicates is produced:

advapi32.lib credui.lib kernel32.lib secur32.lib legacy_stdio_definitions.lib kernel32.lib advapi32.lib advapi32.lib bcrypt.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib kernel32.lib ntdll.lib ntdll.lib ntdll.lib ntdll.lib userenv.lib ws2_32.lib ws2_32.lib ws2_32.lib ws2_32.lib ws2_32.lib ws2_32.lib ws2_32.lib ws2_32.lib ws2_32.lib ws2_32.lib ws2_32.lib ws2_32.lib ws2_32.lib ws2_32.lib ws2_32.lib ws2_32.lib ws2_32.lib ws2_32.lib ws2_32.lib ws2_32.lib ws2_32.lib ws2_32.lib ws2_32.lib ws2_32.lib kernel32.lib ws2_32.lib kernel32.lib kernel32.lib msvcrt.lib

Version it worked on

rustc nightly (1.70.0) on Windows (x86_64-pc-windows-msvc) in a --release configuration.

Version with regression

rustc nightly (1.70.0) on Windows (x86_64-pc-windows-msvc) in a --debug configuration.

Extra links

Project: https://github.com/rustls/rustls-ffi
CMakeLists.txt: https://github.com/rustls/rustls-ffi/blob/2508f98ebb39be6103c78183e42ba94d70d9dd07/CMakeLists.txt
Build task: https://github.com/rustls/rustls-ffi/blob/2508f98ebb39be6103c78183e42ba94d70d9dd07/.github/workflows/test.yaml#L73-L93

[ChrisDenton]

Hm, I thought we usually removed consecutive duplicates but apparently only in release. A workaround would be to do so manually. For the msvc targets all duplicates can be removed so long as the order is preserved. This isn't true of many other targets though.

It would be much better to have a proper fix though.

[cpu]

Thanks for the quick reply :-)

Hm, I thought we usually removed consecutive duplicates but apparently only in release. A workaround would be to do so manually.

That's the approach we landed on for the short term 👍

For the msvc targets all duplicates can be removed so long as the order is preserved. This isn't true of many other targets though.

Great, thanks for clarifying.

[apiraino]

WG-prioritization assigning priority (Zulip discussion).

@rustbot label -I-prioritize +P-medium

[Urgau]

@rustbot claim

[cpu]

Thanks @Urgau !