Regression: linking errors since latest nightly

[tomaka]

Unfortunately I don't have any repro case, as my repro case is my big closed source project that I don't want to be public. Some other smaller projects of mine which use the same libraries and which I thought would trigger the same issue in fact do not trigger the same issue.
After some discussion on IRC, I decided to open an issue despite having no repro to show.

With the 2016-07-03 nightly (commit 696b703), the project compiles fine.
But with the 2016-07-08 nightly (commit 5e18b4b), I'm getting linking errors:

note: libencoding-b89c0661000f249b.rlib(encoding-b89c0661000f249b.1.o) : error L
NK2001: unresolved external symbol _ZN8encoding5types8Encoding11whatwg_name17h8a
002847172e0962E
libencoding-b89c0661000f249b.rlib(encoding-b89c0661000f249b.1.o) : error LNK2001
: unresolved external symbol _ZN8encoding5types8Encoding6encode17hc2acf21d2f0b5a
ecE
libencoding-b89c0661000f249b.rlib(encoding-b89c0661000f249b.1.o) : error LNK2001
: unresolved external symbol _ZN8encoding5types8Encoding9encode_to17h6b8d0fb5b17
6ec15E
libencoding-b89c0661000f249b.rlib(encoding-b89c0661000f249b.1.o) : error LNK2001
: unresolved external symbol _ZN8encoding5types8Encoding6decode17hcc02e59307da6d
60E
libencoding-b89c0661000f249b.rlib(encoding-b89c0661000f249b.1.o) : error LNK2001
: unresolved external symbol _ZN8encoding5types8Encoding9decode_to17h8ed258278c1
7edfbE
libencoding-b89c0661000f249b.rlib(encoding-b89c0661000f249b.1.o) : error LNK2001
: unresolved external symbol _ZN8encoding5types8Encoding11whatwg_name17h2074370d
10042f2aE
libencoding-b89c0661000f249b.rlib(encoding-b89c0661000f249b.1.o) : error LNK2001
: unresolved external symbol _ZN8encoding5types8Encoding6encode17hca510ee43466fc
10E
libencoding-b89c0661000f249b.rlib(encoding-b89c0661000f249b.1.o) : error LNK2001
: unresolved external symbol _ZN8encoding5types8Encoding9encode_to17ha87086244a6
f2cbcE
libencoding-b89c0661000f249b.rlib(encoding-b89c0661000f249b.1.o) : error LNK2001
: unresolved external symbol _ZN8encoding5types8Encoding6decode17h0667439a4d87c3
8bE
libencoding-b89c0661000f249b.rlib(encoding-b89c0661000f249b.1.o) : error LNK2001
: unresolved external symbol _ZN8encoding5types8Encoding9decode_to17h2e8fed91f82
31f7fE
libencoding-b89c0661000f249b.rlib(encoding-b89c0661000f249b.1.o) : error LNK2001
: unresolved external symbol _ZN8encoding5types8Encoding6encode17h254265be7b9110
6eE
libencoding-b89c0661000f249b.rlib(encoding-b89c0661000f249b.1.o) : error LNK2001
: unresolved external symbol _ZN8encoding5types8Encoding9encode_to17h3a460bcb9f9
fafd2E
libencoding-b89c0661000f249b.rlib(encoding-b89c0661000f249b.1.o) : error LNK2001
: unresolved external symbol _ZN8encoding5types8Encoding6decode17hcaedeaf591cc63
ddE

... (many more) ...

This is with the MSVC 64bits version of Rust.

According to IRC, the culprit is likely #33890

[jonas-schievink]

cc @michaelwoerister

[tomaka]

After some investigation, the linking errors are the names of the five provided methods of the Encoding trait:

https://github.com/lifthrasiir/rust-encoding/blob/058aeac4c622337b97fce52e3ae34df5fb9451f1/src/types.rs#L192
https://github.com/lifthrasiir/rust-encoding/blob/058aeac4c622337b97fce52e3ae34df5fb9451f1/src/types.rs#L204
https://github.com/lifthrasiir/rust-encoding/blob/058aeac4c622337b97fce52e3ae34df5fb9451f1/src/types.rs#L210
https://github.com/lifthrasiir/rust-encoding/blob/058aeac4c622337b97fce52e3ae34df5fb9451f1/src/types.rs#L249
https://github.com/lifthrasiir/rust-encoding/blob/058aeac4c622337b97fce52e3ae34df5fb9451f1/src/types.rs#L258

And also these four functions as well:

https://github.com/lifthrasiir/rust-encoding/blob/4cfff6b875cc0b76f83aca50a79d7051d5c8a89a/src/codec/simpchinese.rs#L90-L95

Interestingly, there are other chunks of code that are very similar (they implement Encoding on a struct) but that don't trigger a linking error.

(note: don't look at the master branch, as the crate seems to be in the middle of a rework)

EDIT: removed wrong stuff

[tomaka]

Couldn't reproduce on Linux. This is likely specific to MSVC.

[michaelwoerister]

@tomaka Thanks that's all already very valuable information.

It seems that you are compiling things with multiple codegen units. Does the error also occur if you with just one codegen unit?

[tomaka]

@michaelwoerister Good point, I had totally forgotten about codegen units. Compiling with codegen-units = 1 fixes the problem.

[michaelwoerister]

@tomaka OK, that narrows the problem down somewhat.

[michaelwoerister]

• Are all the linker errors for default methods?
• Do the errors occur while compiling libencoding or when compiling something that depends on libencoding?

[tomaka]

Are all the linker errors for default methods?

All the errors are the ones I linked above. The 5 default methods and the 4 methods of the trait implementation (of which only 1 overwrites a default method).

Do the errors occur while compiling libencoding or when compiling something that depends on libencoding?

No. encoding is only pulled because it is an indirect dependency of one of my crates named "browser", which is itself a dependency of the big project. However compiling an example of "browser" works fine.

That's why I don't really know where to look for to find a repro.

[tomaka]

If you wish so I can also upload the project to a private location.

[michaelwoerister]

@tomaka That would be helpful.

[michaelwoerister]

OK, I was able to reproduce this. It seems to have to do something with constants storing trait-objects (EncodingRefs in this case). The collector does not properly recognize that methods referenced in vtables might need to be available in the same codegen-unit in that case. I need to investigate a little further still.

[michaelwoerister]

Update
It seems that the above analysis was indeed correct: Constants mess up the collector's inlining map and thus the partitioner does not know that it has to place copies of some methods in another codegen unit. A fix for this either involves properly representing constants as translation items or adding special handling to the collector.
Setting -Ccodegen-units=1 for affected crates should be a reliable work-around until this is fixed.

[pnkfelix]

triage: P-high

[michaelwoerister]

I'll see that I'll take care of this next week.

[brson]

@mw status update?

This will go to beta on 8/18.

[michaelwoerister]

There were a few refactorings I needed to do before being able to fix this. These have landed by now, so I expect that the actual fix should be ready before beta.

[brson]

Sweet!

[nikomatsakis]

Update from @rust-lang/compiler meeting: fix is in the queue, upping priority since this is a regression.

[michaelwoerister]

So, this should be fixed (via #35534) but there is no nightly containing the fix yet.

[michaelwoerister]

OK, the latest nightly contains the fix. Could you please test whether it solves your problem? (and close the issue if it does)

[tomaka]

Thanks, seems to work!