Project fails to link when using dylibs without the -Zshare-generics flag

[alexkornitzer]

I have opened this issue under the recommendation of @michaelwoerister who has been helping me track down the linking issues I have been having with dylibs.

As reported in previous issues which I piggybacked, the example provided will build on 1.36.0 but nothing newer: https://github.com/AlexKornitzer/dylib-errors/tree/error/consul

To quote @michaelwoerister, the -Zshare-generics flag has potentially been identified as the root cause:

OK, with the latest version of the error/consul branch I can reproduce. Interestingly the error goes away when compiling with RUSTFLAGS=-Zshare-generics=no.
The error also isn't present for me when compiling with cargo build --release (which is expected because --release implies -Zshare-generics=no)

Hence when building with --release the linking issues go away.

This is potentially related to #64319 and was originally being tracked in #64340.

[michaelwoerister]

Nominating for prioritization.

[pnkfelix]

pre-triage: Explicitly leaving nominated (and not prioritized); lets figure out the priority of this as a group tomorrow.

[michaelwoerister]

Alright, I have identified the cause of this problem. Consider the following subset of the repro's crate graph:

    tokio_timer.rlib         regex_syntax.rlib
         ^                     ^      ^
         |                     |      |
         +-----------+---------+      |
                     |             env_logger.rlib
                 shared.so            ^
                     ^                |
                     |                |
                     +-------+--------+
                             | 
                         serverctl.exe

Both tokio_timer and regex_syntax contain a reusable instance of Cell<isize>::set and we get a linker error that the symbol for Cell<isize>::set is undefined when trying to link serverctl. The problem is that:

• when compiling env_logger the compiler only knows about the instance in regex_syntax, but
• only the version from tokio_timer is reexported from shared.so, so that
• the regex_syntax version cannot be found when the env_logger objects are being linked as part of serverctl.

-Zshare-generics deterministically choose an instance if multiple instances are available but the set to choose from can be different for different crates in the graph. That is fine if we make sure that these sets can only grow but in the case of dylibs the set is shrunk again because we don't re-export all available instances.

I'm pretty sure that's also the cleanest fix: Re-export all instances from dylibs.

[pnkfelix]

@michaelwoerister Question: how can this be a stable-to-stable regression when it depends on -Zshare-generics, an unstable command-line option?

[pnkfelix]

I'll put this on the explicit agenda for tomorrow.

[alexkornitzer]

@pnkfelix, thats my fault for putting in a bad issue title basically in 1.36.0 dylibs will link fine without the generics flag. In 1.37.0 and higher they will only compile with --release or the -Zshare-generics flags. Hence, regression from stable to stable.

[michaelwoerister]

@pnkfelix Yes, -Zshare-generics defaults to true for debug builds and on stable there is no way to opt out (because the -Z flag is not available).

[alexkornitzer]

Now that the PR is merged, will report back once the nightly is live so that I can test on a non toy example.

[michaelwoerister]

Excellent. Thank you.

[alexkornitzer]

Just tried the latest nightly and problem solved, thanks again for sorting it out :D

[michaelwoerister]

That's great to hear. Thanks for the reproduction case, it really helped!