rustc: Allow cdylibs to link against dylibs

[maurer]

Previously, rustc mandated that cdylibs could only link against rlibs as dependencies (not dylibs).
This commit disables that restriction and tests that it works in a simple case.

I don't have a windows rustc dev environment, so I guessed at the MSVC test code, I'm hoping the CI can run that for me.

Additionally, we might want to consider emitting (through cargo or rustc) some metadata to help C users of a cdylib figure out where all the dylibs they need are. I don't think that should be needed to land this change, as it will still be usable by homogeneous build systems without it.

My new test was templated off the tests/run-make-fulldeps/cdylib test. It seemed more appropriate to have it as a separate test, since both foo.rs and bar.rs would need to be replicated to make that test cover both cases, but I can do that if it would be preferred.

If I'm doing anything out of order/process, please let me know; this is only my second change to rustc and the prior one was trivial.

r? alexcrichton

[rust-highfive]

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @zackmdavis (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

[maurer]

r? @alexcrichton
(Sorry, got syntax wrong the first time.)

[alexcrichton]

@bors: r+

Thanks!

[bors]

📌 Commit 518cb70 has been approved by alexcrichton

[bors]

⌛ Testing commit 518cb70 with merge e7a420e...

[rust-highfive]

The job x86_64-apple of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

2020-01-23T16:42:15.3767560Z status: exit code: 2
2020-01-23T16:42:15.3768310Z command: "make"
2020-01-23T16:42:15.3768630Z stdout:
2020-01-23T16:42:15.3769550Z ------------------------------------------
2020-01-23T16:42:15.3770930Z DYLD_LIBRARY_PATH="/Users/runner/runners/2.164.6/work/1/s/build/x86_64-apple-darwin/test/run-make-fulldeps/cdylib-dylib-linkage/cdylib-dylib-linkage:/Users/runner/runners/2.164.6/work/1/s/build/x86_64-apple-darwin/stage2/lib:" '/Users/runner/runners/2.164.6/work/1/s/build/x86_64-apple-darwin/stage2/bin/rustc' --out-dir /Users/runner/runners/2.164.6/work/1/s/build/x86_64-apple-darwin/test/run-make-fulldeps/cdylib-dylib-linkage/cdylib-dylib-linkage -L /Users/runner/runners/2.164.6/work/1/s/build/x86_64-apple-darwin/test/run-make-fulldeps/cdylib-dylib-linkage/cdylib-dylib-linkage  -C prefer-dynamic bar.rs
2020-01-23T16:42:15.3772390Z DYLD_LIBRARY_PATH="/Users/runner/runners/2.164.6/work/1/s/build/x86_64-apple-darwin/test/run-make-fulldeps/cdylib-dylib-linkage/cdylib-dylib-linkage:/Users/runner/runners/2.164.6/work/1/s/build/x86_64-apple-darwin/stage2/lib:" '/Users/runner/runners/2.164.6/work/1/s/build/x86_64-apple-darwin/stage2/bin/rustc' --out-dir /Users/runner/runners/2.164.6/work/1/s/build/x86_64-apple-darwin/test/run-make-fulldeps/cdylib-dylib-linkage/cdylib-dylib-linkage -L /Users/runner/runners/2.164.6/work/1/s/build/x86_64-apple-darwin/test/run-make-fulldeps/cdylib-dylib-linkage/cdylib-dylib-linkage  foo.rs
2020-01-23T16:42:15.3774050Z /Users/runner/runners/2.164.6/work/1/s/clang+llvm-9.0.0-x86_64-darwin-apple/bin/clang -ffunction-sections -fdata-sections -fPIC --target=x86_64-apple-darwin -stdlib=libc++  foo.c -lfoo -lbar -l -L /Users/runner/runners/2.164.6/work/1/s/build/x86_64-apple-darwin/test/run-make-fulldeps/cdylib-dylib-linkage/cdylib-dylib-linkage -L /Users/runner/runners/2.164.6/work/1/s/build/x86_64-apple-darwin/stage2/lib/rustlib/x86_64-apple-darwin/lib -o /Users/runner/runners/2.164.6/work/1/s/build/x86_64-apple-darwin/test/run-make-fulldeps/cdylib-dylib-linkage/cdylib-dylib-linkage/foo
2020-01-23T16:42:15.3775210Z ------------------------------------------
2020-01-23T16:42:15.3775370Z stderr:
2020-01-23T16:42:15.3776100Z ------------------------------------------
2020-01-23T16:42:15.3776100Z ------------------------------------------
2020-01-23T16:42:15.3776860Z ld: library not found for -lfoo
2020-01-23T16:42:15.3777700Z clang-9: error: linker command failed with exit code 1 (use -v to see invocation)
2020-01-23T16:42:15.3778720Z make: *** [/Users/runner/runners/2.164.6/work/1/s/build/x86_64-apple-darwin/test/run-make-fulldeps/cdylib-dylib-linkage/cdylib-dylib-linkage/foo] Error 1
2020-01-23T16:42:15.3779980Z ------------------------------------------
2020-01-23T16:42:15.3780040Z 
2020-01-23T16:42:15.3780100Z 
2020-01-23T16:42:15.3780140Z 
---
2020-01-23T16:42:15.3782240Z thread 'main' panicked at 'Some tests failed', src/tools/compiletest/src/main.rs:387:22
2020-01-23T16:42:15.3782360Z note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
2020-01-23T16:42:15.3782430Z 
2020-01-23T16:42:15.3782460Z 
2020-01-23T16:42:15.3790290Z command did not execute successfully: "/Users/runner/runners/2.164.6/work/1/s/build/x86_64-apple-darwin/stage0-tools-bin/compiletest" "/Users/runner/runners/2.164.6/work/1/s/build/x86_64-apple-darwin/stage0-tools-bin/compiletest" "--compile-lib-path" "/Users/runner/runners/2.164.6/work/1/s/build/x86_64-apple-darwin/stage2/lib" "--run-lib-path" "/Users/runner/runners/2.164.6/work/1/s/build/x86_64-apple-darwin/stage2/lib/rustlib/x86_64-apple-darwin/lib" "--rustc-path" "/Users/runner/runners/2.164.6/work/1/s/build/x86_64-apple-darwin/stage2/bin/rustc" "--rustdoc-path" "/Users/runner/runners/2.164.6/work/1/s/build/x86_64-apple-darwin/stage2/bin/rustdoc" "--src-base" "/Users/runner/runners/2.164.6/work/1/s/src/test/run-make-fulldeps" "--build-base" "/Users/runner/runners/2.164.6/work/1/s/build/x86_64-apple-darwin/test/run-make-fulldeps" "--stage-id" "stage2-x86_64-apple-darwin" "--mode" "run-make" "--target" "x86_64-apple-darwin" "--host" "x86_64-apple-darwin" "--llvm-filecheck" "/Users/runner/runners/2.164.6/work/1/s/build/x86_64-apple-darwin/llvm/build/bin/FileCheck" "--nodejs" "/usr/local/bin/node" "--host-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/Users/runner/runners/2.164.6/work/1/s/build/x86_64-apple-darwin/native/rust-test-helpers" "--target-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/Users/runner/runners/2.164.6/work/1/s/build/x86_64-apple-darwin/native/rust-test-helpers" "--docck-python" "/usr/local/bin/python2.7" "--lldb-python" "/usr/bin/python" "--lldb-version" "lldb-902.0.79.2\n  Swift-4.1\n" "--lldb-python-dir" "/Applications/Xcode_9.3.app/Contents/SharedFrameworks/LLDB.framework/Resources/Python" "--llvm-version" "9.0.1-rust-1.42.0-nightly\n" "--cc" "/Users/runner/runners/2.164.6/work/1/s/clang+llvm-9.0.0-x86_64-darwin-apple/bin/clang" "--cxx" "/Users/runner/runners/2.164.6/work/1/s/clang+llvm-9.0.0-x86_64-darwin-apple/bin/clang++" "--cflags" "-ffunction-sections -fdata-sections -fPIC --target=x86_64-apple-darwin -stdlib=libc++" "--llvm-components" "aarch64 aarch64asmparser aarch64codegen aarch64desc aarch64disassembler aarch64info aarch64utils aggressiveinstcombine all all-targets analysis arm armasmparser armcodegen armdesc armdisassembler arminfo armutils asmparser asmprinter binaryformat bitreader bitstreamreader bitwriter codegen core coroutines coverage debuginfocodeview debuginfodwarf debuginfogsym debuginfomsf debuginfopdb demangle dlltooldriver engine executionengine fuzzmutate globalisel gtest gtest_main hexagon hexagonasmparser hexagoncodegen hexagondesc hexagondisassembler hexagoninfo instcombine instrumentation interpreter ipo irreader jitlink libdriver lineeditor linker lto mc mca mcdisassembler mcjit mcparser mips mipsasmparser mipscodegen mipsdesc mipsdisassembler mipsinfo mirparser msp430 msp430asmparser msp430codegen msp430desc msp430disassembler msp430info native nativecodegen nvptx nvptxcodegen nvptxdesc nvptxinfo objcarcopts object objectyaml option orcjit passes powerpc powerpcasmparser powerpccodegen powerpcdesc powerpcdisassembler powerpcinfo profiledata remarks riscv riscvasmparser riscvcodegen riscvdesc riscvdisassembler riscvinfo riscvutils runtimedyld scalaropts selectiondag sparc sparcasmparser sparccodegen sparcdesc sparcdisassembler sparcinfo support symbolize systemz systemzasmparser systemzcodegen systemzdesc systemzdisassembler systemzinfo tablegen target testingsupport textapi transformutils vectorize webassembly webassemblyasmparser webassemblycodegen webassemblydesc webassemblydisassembler webassemblyinfo windowsmanifest x86 x86asmparser x86codegen x86desc x86disassembler x86info x86utils xray" "--llvm-cxxflags" "-I/Users/runner/runners/2.164.6/work/1/s/src/llvm-project/llvm/include -I/Users/runner/runners/2.164.6/work/1/s/build/x86_64-apple-darwin/llvm/build/include -std=c++11 -stdlib=libc++  -fno-exceptions -fno-rtti -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS" "--ar" "ar" "--llvm-bin-dir" "/Users/runner/runners/2.164.6/work/1/s/build/x86_64-apple-darwin/llvm/build/bin" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
2020-01-23T16:42:15.3792880Z 
2020-01-23T16:42:15.3792960Z 
2020-01-23T16:42:15.3804420Z failed to run: /Users/runner/runners/2.164.6/work/1/s/build/bootstrap/debug/bootstrap test
2020-01-23T16:42:15.3805920Z Build completed unsuccessfully in 1:57:58
2020-01-23T16:42:15.3805920Z Build completed unsuccessfully in 1:57:58
2020-01-23T16:42:15.3923710Z == clock drift check ==
2020-01-23T16:42:15.3998370Z   local time: Thu Jan 23 16:42:15 UTC 2020
2020-01-23T16:42:15.4700210Z   network time: Thu, 23 Jan 2020 16:42:15 GMT
2020-01-23T16:42:15.4704270Z == end clock drift check ==
2020-01-23T16:42:15.4784500Z 
2020-01-23T16:42:15.5036960Z ##[error]Bash exited with code '1'.
2020-01-23T16:42:15.5167040Z ##[section]Starting: Checkout rust-lang/rust@auto to s
2020-01-23T16:42:15.5184760Z ==============================================================================
2020-01-23T16:42:15.5184860Z Task         : Get sources
2020-01-23T16:42:15.5184940Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

[bors]

💔 Test failed - checks-azure

[maurer]

@alexcrichton
I switched to using a dylib glob rather than doing it manually, which will hopefully fix the mac build. I don't think I have the power to retrigger bors myself though (or to trigger only the mac build to check if it worked first, as I don't have easy access to an OSX machine).

[alexcrichton]

@bors: r+

[bors]

📌 Commit 72aaa3a has been approved by alexcrichton

[bors]

⌛ Testing commit 72aaa3a with merge 80a65bc...

[bors]

☀️ Test successful - checks-azure
Approved by: alexcrichton
Pushing 80a65bc to master...