bootstrap fails on NetBSD

[MartinHusemann]

Trying to boostrap rust on NetBSD/amd64 fails:

    Compiling toml v0.4.5
    Compiling bootstrap v0.0.0 (file:///work/rust/rust/src/bootstrap)
 error: Shared object "libproc_macro-33f3fe1937fc08f8.so" not found
    --> /work/rust/rust/src/bootstrap/lib.rs:123:1
     |
 123 | extern crate serde_derive;
     | ^^^^^^^^^^^^^^^^^^^^^^^^^^
 error: Could not compile `bootstrap`.

[Mark-Simulacrum]

This looks like a problem with the rustc being used... @alexcrichton Might this mean that we don't have working beta for NetBSD currently?

[alexcrichton]

Maybe? No idea.

[MartinHusemann]

I started from scratch, so it downloaded the official beta. How can I help analysing?
Where is the libproc_macro reference coming from?
(@jakllsch - could you help?)

[Mark-Simulacrum]

Bootstrap depends on libproc_macro through serde_derive. I'll try and get some instructions for how to diagnose further typed up soon.

[jakllsch]

I bet it's something to do with std::env::current_exe() not actually being relaible on NetBSD.

If I let the bootstrap build fail, then execute:

ln -s /sd0a/jakllsch/rust/obj/build/x86_64-unknown-netbsd/stage0/lib/*.so build/bootstrap/debug/deps/
ln -s /sd0a/jakllsch/rust/obj/build/x86_64-unknown-netbsd/stage0/lib/*.rlib build/bootstrap/debug/deps/

After this resuming the bootstrap build succeeds, and it seems to be progressing to the rest of the build.

[MartinHusemann]

Thanks, I'll look at the NetBSD side of std::env::current_exe(), maybe we can do something to improve its reliability.

[MartinHusemann]

I instrumented access to /proc/curproc/exe and am pretty sure that this is not the reason for failure.
I get:

 > ../rust/x.py build |& tee /tmp/rust.log
Updating submodules
   Compiling bootstrap v0.0.0 (file:///work/rust/rust/src/bootstrap)
error: Shared object "libproc_macro-33f3fe1937fc08f8.so" not found
   --> /work/rust/rust/src/bootstrap/lib.rs:123:1
    |
123 | extern crate serde_derive;
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^

error: Could not compile `bootstrap`.

and during that try, the kernel logs:

rustc: exe is /work/rust/build/build/x86_64-unknown-netbsd/stage0/bin/rustc
cargo: exe is /work/rust/build/build/x86_64-unknown-netbsd/stage0/bin/cargo
rustc: exe is /work/rust/build/build/x86_64-unknown-netbsd/stage0/bin/rustc

so to me it looks like current_exe() is working just fine.

[Mark-Simulacrum]

If you run something like find build -name 'libproc_macro*', what output do you get? I'm trying to see if the file actually doesn't exist, or whether we're just not finding it?

[MartinHusemann]

I have it twice (they are identitcal):

build/x86_64-unknown-netbsd/stage0/lib/rustlib/x86_64-unknown-netbsd/lib/libproc_macro-33f3fe1937fc08f8.so
build/x86_64-unknown-netbsd/stage0/lib/libproc_macro-33f3fe1937fc08f8.so

I traced the rustc process and extracted all attempts to open .so files.

The fun part is at the end. While it properly looks at the various paths for
others, it only trys /usr/lib for libproc_macro.

The .so has a $ORIGIN RPATH:

Dynamic section at offset 0x3e988 contains 29 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libsyntax-053c42242cf7fae8.so]
 0x0000000000000001 (NEEDED)             Shared library: [librustc_errors-3a43c9700ef50f60.so]
 0x0000000000000001 (NEEDED)             Shared library: [libsyntax_pos-c0a71a00b1235fd2.so]
 0x0000000000000001 (NEEDED)             Shared library: [libstd-61b5b9f1c88c205d.so]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.12]
 0x000000000000000f (RPATH)              Library rpath: [$ORIGIN/../lib]

I think we try to avoid that on NetBSD in general.

Many other .so have that RPATH, but libserde_derive does not:

  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libproc_macro-33f3fe1937fc08f8.so]
 0x0000000000000001 (NEEDED)             Shared library: [libstd-61b5b9f1c88c205d.so]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.12]
 0x000000000000000c (INIT)               0x12bd0
 0x000000000000000d (FINI)               0x1d0ee0

Martin

rust-so-access.txt

[Mark-Simulacrum]

I'm not familiar enough with the dylib loading process unfortunately to be able to comment on this -- it seems like perhaps our loading calls are wrong...

[MartinHusemann]

The runtime loading itself is fine, something in the build process is broken so libserde_derive*.so does not get the expected RPATH (but maybe that is what you meant and I misunderstood)

[MartinHusemann]

Would you happen to have any idea who could give me a pointer where to look?
Is this a bug in the toml file in the crate or in cargo?

[jakllsch]

I see where src/bootstrap/bin/rustc.rs has bits that insert the $ORIGIN rpath. I also see where src/bootstrap/tool.rs#389 vaguely suggests that it might not be using the aforementioned bootstrap rustc wrapper when it builds cargo and its dependencies.

Unfortunately, the bootstrap stuff is a bit too abstract for me to fully comprehend at this point.

Is it possible that we're trying to use a serde_derive and/or proc_macro binary library object from the stage0 bootstrap kit to compile src/bootstrap stuff, instead of building these libraries with the provided stage0 kit specifically for the new local build of bootstrap? Or is it possible that we are already building these as fresh dependencies, but we're failing to find the correct instance (that is, the fresh one) of the binary?