New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build instruction profiler runtime as part of compiler-rt #38608

Closed
wants to merge 1 commit into
base: master
from

Conversation

Projects
None yet
@whitequark
Contributor

whitequark commented Dec 26, 2016

This is the only change that is needed to the core compiler in order to produce (mostly) usable branch coverage information. This PR will address rust-lang/rfcs#646 (note that the generated information is not really compatible with gcov, one has to use llvm-cov to work with it).

I believe that it is safe to unconditionally enable the profiler builtins in compiler-rt as they are written to work in any hosted environment--MSVC/Win32, GNU/Win32, Apple, Linux, FreeBSD, plus a catch-all for anything else based on POSIX.

The InstrProfilingPlatform{Darwin,Linux,Other}.c files are unconditionally built because they define away the entire contents on platforms other than the supported one.

@rust-highfive

This comment has been minimized.

Show comment
Hide comment
@rust-highfive

rust-highfive Dec 26, 2016

Collaborator

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @alexcrichton (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

Collaborator

rust-highfive commented Dec 26, 2016

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @alexcrichton (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

@whitequark

This comment has been minimized.

Show comment
Hide comment
@whitequark

whitequark Dec 26, 2016

Contributor

One thing I'm not sure about is the old buildsystem. This PR only updates rustbuild. I haven't even looked into enabling this with the old buildsystem.

Contributor

whitequark commented Dec 26, 2016

One thing I'm not sure about is the old buildsystem. This PR only updates rustbuild. I haven't even looked into enabling this with the old buildsystem.

@whitequark

This comment has been minimized.

Show comment
Hide comment
@whitequark

whitequark Dec 26, 2016

Contributor

See https://users.rust-lang.org/t/howto-generating-a-branch-coverage-report/8524 for what can be done with stock rustc after this is merged.

Contributor

whitequark commented Dec 26, 2016

See https://users.rust-lang.org/t/howto-generating-a-branch-coverage-report/8524 for what can be done with stock rustc after this is merged.

@est31

This comment has been minimized.

Show comment
Hide comment
@est31

est31 Dec 26, 2016

Contributor

@japaric how will this affect #37802 ? Should the profiler runtime be implemented in rust as well? Or should the dependency on compiler-rt be kept?

Contributor

est31 commented Dec 26, 2016

@japaric how will this affect #37802 ? Should the profiler runtime be implemented in rust as well? Or should the dependency on compiler-rt be kept?

@japaric

This comment has been minimized.

Show comment
Hide comment
@japaric

japaric Dec 26, 2016

Member

@est31 Not all the intrinsics have been ported to Rust so the compiler-builtins crate still compiles some C code to fill the gaps. We could do the same here: continue compiling the C files added here and, in the future, port them to Rust.

Member

japaric commented Dec 26, 2016

@est31 Not all the intrinsics have been ported to Rust so the compiler-builtins crate still compiles some C code to fill the gaps. We could do the same here: continue compiling the C files added here and, in the future, port them to Rust.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Dec 28, 2016

Member

Thanks for the PR! I've long wanted to integrate the various runtimes of compiler-rt into rustc in a more first-class fashion. I've been particularly interested in sanitizers like ASAN and TSAN historically for unsafe code, but cross-platform profiling would also be awesome!

I think, though, that we may want to keep this library separate from the standard compiler-builtins library. That is, I think we'd want to ensure that you can pull in compiler-bulitins without the profiling runtime, and then the profiling runtime is only linked in when necessary. There's been some efforts to do this in the past but unfortunately haven't gotten far off the ground.

I'd prefer to not have us on the hook for implementing these intrinsics in Rust, as well. Although compiler-rt has a very stable ABI and set of symbols, I suspect that the profiling runtime probably doesn't? I wouldn't want to have to track upstream LLVM changes whenever we update LLVM :(

Member

alexcrichton commented Dec 28, 2016

Thanks for the PR! I've long wanted to integrate the various runtimes of compiler-rt into rustc in a more first-class fashion. I've been particularly interested in sanitizers like ASAN and TSAN historically for unsafe code, but cross-platform profiling would also be awesome!

I think, though, that we may want to keep this library separate from the standard compiler-builtins library. That is, I think we'd want to ensure that you can pull in compiler-bulitins without the profiling runtime, and then the profiling runtime is only linked in when necessary. There's been some efforts to do this in the past but unfortunately haven't gotten far off the ground.

I'd prefer to not have us on the hook for implementing these intrinsics in Rust, as well. Although compiler-rt has a very stable ABI and set of symbols, I suspect that the profiling runtime probably doesn't? I wouldn't want to have to track upstream LLVM changes whenever we update LLVM :(

@whitequark

This comment has been minimized.

Show comment
Hide comment
@whitequark

whitequark Dec 29, 2016

Contributor

I think, though, that we may want to keep this library separate from the standard compiler-builtins library. That is, I think we'd want to ensure that you can pull in compiler-bulitins without the profiling runtime, and then the profiling runtime is only linked in when necessary.

Why is this needed at all? -ffunction-sections is a thing, isn't it?

I'd prefer to not have us on the hook for implementing these intrinsics in Rust, as well. Although compiler-rt has a very stable ABI and set of symbols, I suspect that the profiling runtime probably doesn't? I wouldn't want to have to track upstream LLVM changes whenever we update LLVM :(

I think you will have to, unless you're fine with rustc being stuck with an inferior profiler.

Contributor

whitequark commented Dec 29, 2016

I think, though, that we may want to keep this library separate from the standard compiler-builtins library. That is, I think we'd want to ensure that you can pull in compiler-bulitins without the profiling runtime, and then the profiling runtime is only linked in when necessary.

Why is this needed at all? -ffunction-sections is a thing, isn't it?

I'd prefer to not have us on the hook for implementing these intrinsics in Rust, as well. Although compiler-rt has a very stable ABI and set of symbols, I suspect that the profiling runtime probably doesn't? I wouldn't want to have to track upstream LLVM changes whenever we update LLVM :(

I think you will have to, unless you're fine with rustc being stuck with an inferior profiler.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Dec 29, 2016

Member

Yes -ffunction-sections should strip out objects if we don't use them but it's still possible to reference the symbols and it may be a breaking change if we ever remove them (e.g. move them to a separate library). Additionally this becomes an instant requirement for all platforms whereas I doubt all platforms that Rust supports support the profiling options here. By starting out as a separate library we can piecemeal enable support where possible.

Member

alexcrichton commented Dec 29, 2016

Yes -ffunction-sections should strip out objects if we don't use them but it's still possible to reference the symbols and it may be a breaking change if we ever remove them (e.g. move them to a separate library). Additionally this becomes an instant requirement for all platforms whereas I doubt all platforms that Rust supports support the profiling options here. By starting out as a separate library we can piecemeal enable support where possible.

@whitequark

This comment has been minimized.

Show comment
Hide comment
@whitequark

whitequark Dec 29, 2016

Contributor

By starting out as a separate library we can piecemeal enable support where possible.

This is something I would normally get very much behind, but...

Additionally this becomes an instant requirement for all platforms whereas I doubt all platforms that Rust supports support the profiling options here.

... the main reason why I so eagerly submitted this PR is that it is my understanding is that the compiler-rt profiling runtine works on any platform with libc. If I am proven wrong with regards to this then we should definitely go your way.

Contributor

whitequark commented Dec 29, 2016

By starting out as a separate library we can piecemeal enable support where possible.

This is something I would normally get very much behind, but...

Additionally this becomes an instant requirement for all platforms whereas I doubt all platforms that Rust supports support the profiling options here.

... the main reason why I so eagerly submitted this PR is that it is my understanding is that the compiler-rt profiling runtine works on any platform with libc. If I am proven wrong with regards to this then we should definitely go your way.

@whitequark

This comment has been minimized.

Show comment
Hide comment
@whitequark

whitequark Dec 29, 2016

Contributor

That said--would you like me to separate it into perhaps a libprofiler_builtins library, structured like libcompiler_builtins? If you do then I can just do that.

Contributor

whitequark commented Dec 29, 2016

That said--would you like me to separate it into perhaps a libprofiler_builtins library, structured like libcompiler_builtins? If you do then I can just do that.

@whitequark

This comment has been minimized.

Show comment
Hide comment
@whitequark

whitequark Dec 29, 2016

Contributor

it's still possible to reference the symbols and it may be a breaking change if we ever remove them (e.g. move them to a separate library)

Do I understand you correctly here? In other words, is the problem that...

  • the Rust runtime will export these symbols unconditionally;
  • thus someone can reference them via extern "C".

I don't see how this is not true right now. For example, someone can reference __divdi3, but there's no obligation on part of LLVM or rustc to keep this symbol around as it is an implementation detail. The profiler symbols also start with __ and so they have the same contract. The only change I see as needed here is to have this contract stated explicitly somewhere.

Contributor

whitequark commented Dec 29, 2016

it's still possible to reference the symbols and it may be a breaking change if we ever remove them (e.g. move them to a separate library)

Do I understand you correctly here? In other words, is the problem that...

  • the Rust runtime will export these symbols unconditionally;
  • thus someone can reference them via extern "C".

I don't see how this is not true right now. For example, someone can reference __divdi3, but there's no obligation on part of LLVM or rustc to keep this symbol around as it is an implementation detail. The profiler symbols also start with __ and so they have the same contract. The only change I see as needed here is to have this contract stated explicitly somewhere.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Dec 29, 2016

Member

Yes I believe the theory is that compiler-rt and all various libraries in that project work on all platforms, but in reality that's been very far from the truth. Just getting the libraries to build is a nightmare, much less actually working correctly. Until it's actually tested, I'd hazard a guess that it only works well on x86_64 linux, and probably OSX as well.

I think that there's basically an open design space here. I do think that we'd want a separate library like profiler-builtins but exactly how you request that from the compiler is something I'm not so sure about. This is likely to be a platform-specific tool and the compiler needs to cope with that somehow as well. I haven't personally thought through all of the design questions here, but you can find some prior art on old PRs.

I don't see how this is not true right now. For example, someone can reference __divdi3, but there's no obligation on part of LLVM or rustc to keep this symbol around as it is an implementation detail.

We can't remove symbols like __divdi3 for correctness. We don't need to add this for correctness. That means we can take a more principled approach here.

Member

alexcrichton commented Dec 29, 2016

Yes I believe the theory is that compiler-rt and all various libraries in that project work on all platforms, but in reality that's been very far from the truth. Just getting the libraries to build is a nightmare, much less actually working correctly. Until it's actually tested, I'd hazard a guess that it only works well on x86_64 linux, and probably OSX as well.

I think that there's basically an open design space here. I do think that we'd want a separate library like profiler-builtins but exactly how you request that from the compiler is something I'm not so sure about. This is likely to be a platform-specific tool and the compiler needs to cope with that somehow as well. I haven't personally thought through all of the design questions here, but you can find some prior art on old PRs.

I don't see how this is not true right now. For example, someone can reference __divdi3, but there's no obligation on part of LLVM or rustc to keep this symbol around as it is an implementation detail.

We can't remove symbols like __divdi3 for correctness. We don't need to add this for correctness. That means we can take a more principled approach here.

@whitequark

This comment has been minimized.

Show comment
Hide comment
@whitequark

whitequark Dec 29, 2016

Contributor

Until it's actually tested, I'd hazard a guess that it only works well on x86_64 linux, and probably OSX as well.

This is likely to be a platform-specific tool

I am confused as to how you expect the profiler runtime to function. Its job is to increment a counter then load a file, update a field and save it back. What exactly in it is so platform-specific? LLVM doesn't attempt to be compatible with any existing tools, apparently not even gcov, so you are supposed to use it with llvm-cov only, which uses the same representation on any platform...

Contributor

whitequark commented Dec 29, 2016

Until it's actually tested, I'd hazard a guess that it only works well on x86_64 linux, and probably OSX as well.

This is likely to be a platform-specific tool

I am confused as to how you expect the profiler runtime to function. Its job is to increment a counter then load a file, update a field and save it back. What exactly in it is so platform-specific? LLVM doesn't attempt to be compatible with any existing tools, apparently not even gcov, so you are supposed to use it with llvm-cov only, which uses the same representation on any platform...

@whitequark

This comment has been minimized.

Show comment
Hide comment
@whitequark

whitequark Dec 29, 2016

Contributor

I see you reference sanitizers; the profiler runtime of course has a huge difference from sanitizers, which indeed have to use many platform-specific tricks to run.

Contributor

whitequark commented Dec 29, 2016

I see you reference sanitizers; the profiler runtime of course has a huge difference from sanitizers, which indeed have to use many platform-specific tricks to run.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Dec 31, 2016

Member

I am not privvy to the internals of LLVM profiling runtime, I have just had historical experience that "cross platform libraries" like compiler-rt are anything but. If it's not platform-specific then that's great!

Again though my preference is to not have this come by default in compiler-rt. At the very least using this should require some form of feature gate as it's entering as unstable, and then we can flesh out bugs and exactly how it works before stabilizing.

I feel that if we want to land this we also want documentation one way or another. From this PR I don't even know how to make use of the support added.

Member

alexcrichton commented Dec 31, 2016

I am not privvy to the internals of LLVM profiling runtime, I have just had historical experience that "cross platform libraries" like compiler-rt are anything but. If it's not platform-specific then that's great!

Again though my preference is to not have this come by default in compiler-rt. At the very least using this should require some form of feature gate as it's entering as unstable, and then we can flesh out bugs and exactly how it works before stabilizing.

I feel that if we want to land this we also want documentation one way or another. From this PR I don't even know how to make use of the support added.

@whitequark

This comment has been minimized.

Show comment
Hide comment
@whitequark

whitequark Dec 31, 2016

Contributor

Again though my preference is to not have this come by default in compiler-rt. At the very least using this should require some form of feature gate as it's entering as unstable, and then we can flesh out bugs and exactly how it works before stabilizing.

Ack. How about this plan:

  • Extract profiler builtins into a separate library, libprofiler_builtins.
  • Add a rustc codegen flag, -C profiling, which requires an unstable rustc to be used.
  • Add a test.
  • Add documentation (where?).

I feel that if we want to land this we also want documentation one way or another. From this PR I don't even know how to make use of the support added.

I wrote a fairly complete guide for this in https://users.rust-lang.org/t/howto-generating-a-branch-coverage-report/8524. I'll of course introduce it into the Rust documentation if someone points me to where it should live.

Contributor

whitequark commented Dec 31, 2016

Again though my preference is to not have this come by default in compiler-rt. At the very least using this should require some form of feature gate as it's entering as unstable, and then we can flesh out bugs and exactly how it works before stabilizing.

Ack. How about this plan:

  • Extract profiler builtins into a separate library, libprofiler_builtins.
  • Add a rustc codegen flag, -C profiling, which requires an unstable rustc to be used.
  • Add a test.
  • Add documentation (where?).

I feel that if we want to land this we also want documentation one way or another. From this PR I don't even know how to make use of the support added.

I wrote a fairly complete guide for this in https://users.rust-lang.org/t/howto-generating-a-branch-coverage-report/8524. I'll of course introduce it into the Rust documentation if someone points me to where it should live.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Jan 3, 2017

Member

Yeah that sounds like a good approach to me, modulo probably -Z instead of -C. We can add documentation either in-tree, in tests, or in the nightly portion of the book. It just needs to go somewhere.

Member

alexcrichton commented Jan 3, 2017

Yeah that sounds like a good approach to me, modulo probably -Z instead of -C. We can add documentation either in-tree, in tests, or in the nightly portion of the book. It just needs to go somewhere.

@whitequark

This comment has been minimized.

Show comment
Hide comment
@whitequark

whitequark Jan 3, 2017

Contributor

Why -Z?

Contributor

whitequark commented Jan 3, 2017

Why -Z?

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Jan 3, 2017

Member

-Z == unstable

Member

alexcrichton commented Jan 3, 2017

-Z == unstable

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Jan 17, 2017

Member

ping @whitequark, any update on moving these to unstable features?

Member

alexcrichton commented Jan 17, 2017

ping @whitequark, any update on moving these to unstable features?

@whitequark

This comment has been minimized.

Show comment
Hide comment
@whitequark

whitequark Jan 18, 2017

Contributor

Will update soon. I've changed my overall plans somewhat as LLVM's built-in profiler has a much better output than gcov+lcov and is portable, but I don't yet know how it works.

Contributor

whitequark commented Jan 18, 2017

Will update soon. I've changed my overall plans somewhat as LLVM's built-in profiler has a much better output than gcov+lcov and is portable, but I don't yet know how it works.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Feb 4, 2017

Member

ping @whitequark, any updates with your new plans?

Member

alexcrichton commented Feb 4, 2017

ping @whitequark, any updates with your new plans?

@whitequark

This comment has been minimized.

Show comment
Hide comment
@whitequark

whitequark Feb 5, 2017

Contributor

@alexcrichton How do I add libprofiler_builtins in the build system? Looks like everything else is a dependency of libstd, but we're specifically trying to avoid that.

Contributor

whitequark commented Feb 5, 2017

@alexcrichton How do I add libprofiler_builtins in the build system? Looks like everything else is a dependency of libstd, but we're specifically trying to avoid that.

@whitequark

This comment has been minimized.

Show comment
Hide comment
@whitequark

whitequark Feb 5, 2017

Contributor

I've found this in bootstrap/lib.rs:

/// The various "modes" of invoking Cargo.
///
/// These entries currently correspond to the various output directories of the
/// build system, with each mod generating output in a different directory.
#[derive(Clone, Copy)]
pub enum Mode {
    /// This cargo is going to build the standard library, placing output in the
    /// "stageN-std" directory.
    Libstd,

    /// This cargo is going to build libtest, placing output in the
    /// "stageN-test" directory.
    Libtest,

    /// This cargo is going to build librustc and compiler libraries, placing
    /// output in the "stageN-rustc" directory.
    Librustc,

    /// This cargo is going to some build tool, placing output in the
    /// "stageN-tools" directory.
    Tool,
}

but none of the modes seem fit. Should I just add a new one?

Contributor

whitequark commented Feb 5, 2017

I've found this in bootstrap/lib.rs:

/// The various "modes" of invoking Cargo.
///
/// These entries currently correspond to the various output directories of the
/// build system, with each mod generating output in a different directory.
#[derive(Clone, Copy)]
pub enum Mode {
    /// This cargo is going to build the standard library, placing output in the
    /// "stageN-std" directory.
    Libstd,

    /// This cargo is going to build libtest, placing output in the
    /// "stageN-test" directory.
    Libtest,

    /// This cargo is going to build librustc and compiler libraries, placing
    /// output in the "stageN-rustc" directory.
    Librustc,

    /// This cargo is going to some build tool, placing output in the
    /// "stageN-tools" directory.
    Tool,
}

but none of the modes seem fit. Should I just add a new one?

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Feb 5, 2017

Member

Ideally it'd be in a crate rather than the build system, but is it not possible to do so?

Member

alexcrichton commented Feb 5, 2017

Ideally it'd be in a crate rather than the build system, but is it not possible to do so?

@whitequark

This comment has been minimized.

Show comment
Hide comment
@whitequark

whitequark Feb 6, 2017

Contributor

@alexcrichton I'm confused. Do you mean it would be in a crate completely external to rustc?

Contributor

whitequark commented Feb 6, 2017

@alexcrichton I'm confused. Do you mean it would be in a crate completely external to rustc?

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Feb 6, 2017

Member

Ideally, yeah, it'd follow the pattern of compiler-builtins, the sanitizer runtimes, panic runtimes, allocators, etc. If necessary the compiler can have a small amount of logic, but all runtime support goes into a dedicated crate.

Member

alexcrichton commented Feb 6, 2017

Ideally, yeah, it'd follow the pattern of compiler-builtins, the sanitizer runtimes, panic runtimes, allocators, etc. If necessary the compiler can have a small amount of logic, but all runtime support goes into a dedicated crate.

@whitequark

This comment has been minimized.

Show comment
Hide comment
@whitequark

whitequark Feb 7, 2017

Contributor

@alexcrichton Ah, I realized that, but I didn't know how to make the build system (bootstrap crate, that is) build this new crate. But the PR you linked to clears it.

Contributor

whitequark commented Feb 7, 2017

@alexcrichton Ah, I realized that, but I didn't know how to make the build system (bootstrap crate, that is) build this new crate. But the PR you linked to clears it.

@whitequark

This comment has been minimized.

Show comment
Hide comment
@whitequark

whitequark Feb 13, 2017

Contributor

I rewrote the patch but there's some weird issue that prevents me from bootstrapping a compiler now:

  = note: /home/whitequark/Work/rust/build/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/debug/deps/librustc_llvm-880e12bf8a3af75a.so: undefined reference to `ffi_type_float'
          /home/whitequark/Work/rust/build/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/debug/deps/librustc_llvm-880e12bf8a3af75a.so: undefined reference to `ffi_type_sint64'
          /home/whitequark/Work/rust/build/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/debug/deps/librustc_llvm-880e12bf8a3af75a.so: undefined reference to `ffi_type_pointer'
          /home/whitequark/Work/rust/build/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/debug/deps/librustc_llvm-880e12bf8a3af75a.so: undefined reference to `ffi_type_sint32'
          /home/whitequark/Work/rust/build/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/debug/deps/librustc_llvm-880e12bf8a3af75a.so: undefined reference to `ffi_type_void'
          /home/whitequark/Work/rust/build/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/debug/deps/librustc_llvm-880e12bf8a3af75a.so: undefined reference to `ffi_type_sint8'
          /home/whitequark/Work/rust/build/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/debug/deps/librustc_llvm-880e12bf8a3af75a.so: undefined reference to `ffi_call'
          /home/whitequark/Work/rust/build/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/debug/deps/librustc_llvm-880e12bf8a3af75a.so: undefined reference to `ffi_type_sint16'
          /home/whitequark/Work/rust/build/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/debug/deps/librustc_llvm-880e12bf8a3af75a.so: undefined reference to `ffi_type_double'
          /home/whitequark/Work/rust/build/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/debug/deps/librustc_llvm-880e12bf8a3af75a.so: undefined reference to `ffi_prep_cif'
Contributor

whitequark commented Feb 13, 2017

I rewrote the patch but there's some weird issue that prevents me from bootstrapping a compiler now:

  = note: /home/whitequark/Work/rust/build/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/debug/deps/librustc_llvm-880e12bf8a3af75a.so: undefined reference to `ffi_type_float'
          /home/whitequark/Work/rust/build/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/debug/deps/librustc_llvm-880e12bf8a3af75a.so: undefined reference to `ffi_type_sint64'
          /home/whitequark/Work/rust/build/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/debug/deps/librustc_llvm-880e12bf8a3af75a.so: undefined reference to `ffi_type_pointer'
          /home/whitequark/Work/rust/build/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/debug/deps/librustc_llvm-880e12bf8a3af75a.so: undefined reference to `ffi_type_sint32'
          /home/whitequark/Work/rust/build/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/debug/deps/librustc_llvm-880e12bf8a3af75a.so: undefined reference to `ffi_type_void'
          /home/whitequark/Work/rust/build/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/debug/deps/librustc_llvm-880e12bf8a3af75a.so: undefined reference to `ffi_type_sint8'
          /home/whitequark/Work/rust/build/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/debug/deps/librustc_llvm-880e12bf8a3af75a.so: undefined reference to `ffi_call'
          /home/whitequark/Work/rust/build/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/debug/deps/librustc_llvm-880e12bf8a3af75a.so: undefined reference to `ffi_type_sint16'
          /home/whitequark/Work/rust/build/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/debug/deps/librustc_llvm-880e12bf8a3af75a.so: undefined reference to `ffi_type_double'
          /home/whitequark/Work/rust/build/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/debug/deps/librustc_llvm-880e12bf8a3af75a.so: undefined reference to `ffi_prep_cif'
@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Feb 13, 2017

Member

@whitequark are you using an external LLVM? In some cases if that's done then LLVM doesn't report that it needs -lffi to link, which may be the problem you're seeing here.

Member

alexcrichton commented Feb 13, 2017

@whitequark are you using an external LLVM? In some cases if that's done then LLVM doesn't report that it needs -lffi to link, which may be the problem you're seeing here.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Feb 13, 2017

Member

Looking good, thanks @whitequark! Could you also document how these are used? Ideally we could also include a test in tree which exercises this feature!

Member

alexcrichton commented Feb 13, 2017

Looking good, thanks @whitequark! Could you also document how these are used? Ideally we could also include a test in tree which exercises this feature!

@whitequark

This comment has been minimized.

Show comment
Hide comment
@whitequark

whitequark Feb 13, 2017

Contributor

are you using an external LLVM? In some cases if that's done then LLVM doesn't report that it needs -lffi to link, which may be the problem you're seeing here.

Indeed that's the case. I'll use internal LLVM instead.

Could you also document how these are used? Ideally we could also include a test in tree which exercises this feature!

Sure, I was planning to do that anyway, just once it actually builds...

Contributor

whitequark commented Feb 13, 2017

are you using an external LLVM? In some cases if that's done then LLVM doesn't report that it needs -lffi to link, which may be the problem you're seeing here.

Indeed that's the case. I'll use internal LLVM instead.

Could you also document how these are used? Ideally we could also include a test in tree which exercises this feature!

Sure, I was planning to do that anyway, just once it actually builds...

@bors

This comment has been minimized.

Show comment
Hide comment
@bors

bors Feb 15, 2017

Contributor

☔️ The latest upstream changes (presumably #39633) made this pull request unmergeable. Please resolve the merge conflicts.

Contributor

bors commented Feb 15, 2017

☔️ The latest upstream changes (presumably #39633) made this pull request unmergeable. Please resolve the merge conflicts.

@japaric

This comment has been minimized.

Show comment
Hide comment
@japaric

japaric Feb 17, 2017

Member

What's the stabilization story for this feature?

From @whitequark's post on urlo, it seems that this is already usable on stable thanks to -C passes (why is that even stable? It lets you use LLVM internals). The only speed bump being building libprofiler.a but that's not hard either because the source is in the rust-src component.

Here's a repro script and this is the stable rustc invocation:

$ RUSTFLAGS="-C passes=insert-gcov-profiling" \
      cargo rustc -- \
            -C link-arg=-lprofiler \
            -C link-arg=-L$(pwd)

$ target/debug/hello
Hello, world!

# output
$ ls main.gc*
main.gcda  main.gcno

$ rustc -V
rustc 1.15.1 (021bd294c 2017-02-08)
Member

japaric commented Feb 17, 2017

What's the stabilization story for this feature?

From @whitequark's post on urlo, it seems that this is already usable on stable thanks to -C passes (why is that even stable? It lets you use LLVM internals). The only speed bump being building libprofiler.a but that's not hard either because the source is in the rust-src component.

Here's a repro script and this is the stable rustc invocation:

$ RUSTFLAGS="-C passes=insert-gcov-profiling" \
      cargo rustc -- \
            -C link-arg=-lprofiler \
            -C link-arg=-L$(pwd)

$ target/debug/hello
Hello, world!

# output
$ ls main.gc*
main.gcda  main.gcno

$ rustc -V
rustc 1.15.1 (021bd294c 2017-02-08)
@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Feb 21, 2017

Member

@japaric There's not really much we can do about that, but I wouldn't necessarily consider that a "first class support" path for this. I'd want the stability story to say that whatever the first class story is here (e.g. not hacking around the compiler) would be gated, and eventually stabilized.

Member

alexcrichton commented Feb 21, 2017

@japaric There's not really much we can do about that, but I wouldn't necessarily consider that a "first class support" path for this. I'd want the stability story to say that whatever the first class story is here (e.g. not hacking around the compiler) would be gated, and eventually stabilized.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Mar 2, 2017

Member

ping @whitequark, any update on this?

Member

alexcrichton commented Mar 2, 2017

ping @whitequark, any update on this?

@whitequark

This comment has been minimized.

Show comment
Hide comment
@whitequark

whitequark Apr 17, 2017

Contributor

Yes, still working, though extremely overloaded.

Contributor

whitequark commented Apr 17, 2017

Yes, still working, though extremely overloaded.

@whitequark

This comment has been minimized.

Show comment
Hide comment
@whitequark

whitequark Apr 23, 2017

Contributor

This PR is essentially done.

Contributor

whitequark commented Apr 23, 2017

This PR is essentially done.

@alexcrichton

Looking great to me, thanks @whitequark!

Some thoughts:

  • Do you want to enable this on some builders? It looks relatively lightweight so testing it on OSX/Linux/Windows seems reasonable, and it'd be nice to distribute as well I think, right?
  • I think the test may want a gate as the profiler isn't always enabled?
  • Could you fill out the profile.md unstable book template? Basically just some "hello world" docs on how to use it.
  • Could you file an issue to track this with? If you cc me I'll tag it.
}
});
if uses_std {

This comment has been minimized.

@alexcrichton

alexcrichton Apr 24, 2017

Member

Mentioned above, we can remove this clause, right?

(just execute the code below if -Z profile is provided)

@alexcrichton

alexcrichton Apr 24, 2017

Member

Mentioned above, we can remove this clause, right?

(just execute the code below if -Z profile is provided)

// available in LLVM 3.9+; so we rely on LLVM chopping off the extension
// and replacing it with gcno/gcda, instead.
path_to_mdstring(debug_context.llcontext,
&scc.output_filenames().with_extension("gcno")),

This comment has been minimized.

@alexcrichton

alexcrichton Apr 24, 2017

Member

Should this all happen conditionally? Only if -Zprofile is enabled?

@alexcrichton

alexcrichton Apr 24, 2017

Member

Should this all happen conditionally? Only if -Zprofile is enabled?

This comment has been minimized.

@whitequark

whitequark Apr 25, 2017

Contributor

The !llvm.gcov named metadata is only ever parsed by the GCOVProfiling pass, so there doesn't seem to be any harm in doing this unconditionally...

@whitequark

whitequark Apr 25, 2017

Contributor

The !llvm.gcov named metadata is only ever parsed by the GCOVProfiling pass, so there doesn't seem to be any harm in doing this unconditionally...

This comment has been minimized.

@alexcrichton

alexcrichton Apr 25, 2017

Member

Hm perhaps this could be behind a guard for clarity? I'd be wary of an update to LLVM randomly starting to read this by accident...

@alexcrichton

alexcrichton Apr 25, 2017

Member

Hm perhaps this could be behind a guard for clarity? I'd be wary of an update to LLVM randomly starting to read this by accident...

let gcov_cu_info = [
// Ideally we would be using the three-element form of !llvm.gcov metadata,
// which allows us to specify gcno/gcda files explicitly, but that's only
// available in LLVM 3.9+; so we rely on LLVM chopping off the extension

This comment has been minimized.

@alexcrichton

alexcrichton Apr 24, 2017

Member

I think we should be using 3.9+, right? (and soon 4.0!)

Is this using a different form for compatibility with older llvm version? (if so, I wouldn't worry about that and just use the newer version)

@alexcrichton

alexcrichton Apr 24, 2017

Member

I think we should be using 3.9+, right? (and soon 4.0!)

Is this using a different form for compatibility with older llvm version? (if so, I wouldn't worry about that and just use the newer version)

This comment has been minimized.

@whitequark

whitequark Apr 25, 2017

Contributor

I think we should be using 3.9+, right? (and soon 4.0!)

Is that really the case? The HEAD in the LLVM submodule is pre-3.9 isn't it? (Or at least it did not have the functionality I tried to rely on here.) Also, it is my understanding that the LLVM version check only rejects LLVM <3.7, so someone could still make their own build with that.

@whitequark

whitequark Apr 25, 2017

Contributor

I think we should be using 3.9+, right? (and soon 4.0!)

Is that really the case? The HEAD in the LLVM submodule is pre-3.9 isn't it? (Or at least it did not have the functionality I tried to rely on here.) Also, it is my understanding that the LLVM version check only rejects LLVM <3.7, so someone could still make their own build with that.

This comment has been minimized.

@alexcrichton

alexcrichton Apr 25, 2017

Member

Yes someone can make their own build, but we're not obligated to support literally everything on all LLVM versions, it's fine if profiling just doesn't work on previous LLVM versions (no one will slight us)

@alexcrichton

alexcrichton Apr 25, 2017

Member

Yes someone can make their own build, but we're not obligated to support literally everything on all LLVM versions, it's fine if profiling just doesn't work on previous LLVM versions (no one will slight us)

@whitequark

This comment has been minimized.

Show comment
Hide comment
@whitequark

whitequark Apr 25, 2017

Contributor

Do you want to enable this on some builders? It looks relatively lightweight so testing it on OSX/Linux/Windows seems reasonable, and it'd be nice to distribute as well I think, right?

How do I do this?

Contributor

whitequark commented Apr 25, 2017

Do you want to enable this on some builders? It looks relatively lightweight so testing it on OSX/Linux/Windows seems reasonable, and it'd be nice to distribute as well I think, right?

How do I do this?

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Apr 25, 2017

Member

Oh all builders are configured in .travis.yml and appveyor.yml, you can explore, for example, the sanitizer configuration there.

Member

alexcrichton commented Apr 25, 2017

Oh all builders are configured in .travis.yml and appveyor.yml, you can explore, for example, the sanitizer configuration there.

@marco-c

This comment has been minimized.

Show comment
Hide comment
@marco-c

marco-c Apr 25, 2017

Contributor

It would be nice if you could somehow support some kind of option to generate gcno/gcda files with a different extension or with a prefix/suffix.
This would make it easier to aggregate code coverage data when you compile part of your sources with GCC and part of them with LLVM.

The gcno/gcda formats of GCC and LLVM aren't compatible as far as I can tell and I couldn't find a way to detect if a file was generated by GCC or LLVM (so I can't know if I have to use gcov or llvm-cov to parse it).

Contributor

marco-c commented Apr 25, 2017

It would be nice if you could somehow support some kind of option to generate gcno/gcda files with a different extension or with a prefix/suffix.
This would make it easier to aggregate code coverage data when you compile part of your sources with GCC and part of them with LLVM.

The gcno/gcda formats of GCC and LLVM aren't compatible as far as I can tell and I couldn't find a way to detect if a file was generated by GCC or LLVM (so I can't know if I have to use gcov or llvm-cov to parse it).

@bors

This comment has been minimized.

Show comment
Hide comment
@bors

bors Apr 28, 2017

Contributor

☔️ The latest upstream changes (presumably #41508) made this pull request unmergeable. Please resolve the merge conflicts.

Contributor

bors commented Apr 28, 2017

☔️ The latest upstream changes (presumably #41508) made this pull request unmergeable. Please resolve the merge conflicts.

@whitequark

This comment has been minimized.

Show comment
Hide comment
@whitequark

whitequark Apr 30, 2017

Contributor

It would be nice if you could somehow support some kind of option to generate gcno/gcda files with a different extension or with a prefix/suffix.
This would make it easier to aggregate code coverage data when you compile part of your sources with GCC and part of them with LLVM.

Okay. This can only be done with LLVM 4.0+ but since @alexcrichton says it's fine I can do that.

Bikeshed: is rsgcno/rsgcda fine?

Contributor

whitequark commented Apr 30, 2017

It would be nice if you could somehow support some kind of option to generate gcno/gcda files with a different extension or with a prefix/suffix.
This would make it easier to aggregate code coverage data when you compile part of your sources with GCC and part of them with LLVM.

Okay. This can only be done with LLVM 4.0+ but since @alexcrichton says it's fine I can do that.

Bikeshed: is rsgcno/rsgcda fine?

Add profiling support, through the rustc -Z profile flag.
When -Z profile is passed, the GCDAProfiling LLVM pass is added
to the pipeline, which uses debug information to instrument the IR.
After compiling with -Z profile, the $(OUT_DIR)/$(CRATE_NAME).gcno
file is created, containing initial profiling information.
After running the program built, the $(OUT_DIR)/$(CRATE_NAME).gcda
file is created, containing branch counters.

The created *.gcno and *.gcda files can be processed using
the "llvm-cov gcov" and "lcov" tools. The profiling data LLVM
generates does not faithfully follow the GCC's format for *.gcno
and *.gcda files, and so it will probably not work with other tools
(such as gcov itself) that consume these files.
@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton May 1, 2017

Member

Sounds reasonable to me!

Member

alexcrichton commented May 1, 2017

Sounds reasonable to me!

@marco-c

This comment has been minimized.

Show comment
Hide comment
@marco-c

marco-c May 4, 2017

Contributor

Okay. This can only be done with LLVM 4.0+ but since @alexcrichton says it's fine I can do that.

Bikeshed: is rsgcno/rsgcda fine?

Actually, I've noticed the files generated by LLVM have a different version than the files generated by GCC. So I can also use the version to identify them.

Contributor

marco-c commented May 4, 2017

Okay. This can only be done with LLVM 4.0+ but since @alexcrichton says it's fine I can do that.

Bikeshed: is rsgcno/rsgcda fine?

Actually, I've noticed the files generated by LLVM have a different version than the files generated by GCC. So I can also use the version to identify them.

@carols10cents

This comment has been minimized.

Show comment
Hide comment
@carols10cents

carols10cents May 8, 2017

Member

Looks like there was a legitimate test failure? My brain isn't pattern-matching it to one of the spurious failures, at least:

[00:47:51] failures:
[00:47:51] 
[00:47:51] ---- [run-make] run-make/profile stdout ----
[00:47:51] 	
[00:47:51] error: make failed
[00:47:51] status: exit code: 2
[00:47:51] command: "make"
[00:47:51] stdout:
[00:47:51] ------------------------------------------
[00:47:51] make[1]: Entering directory '/checkout/src/test/run-make/profile'
[00:47:51] LD_LIBRARY_PATH="/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/profile.stage2-x86_64-unknown-linux-gnu:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-tools/x86_64-unknown-linux-gnu/release/deps:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib:" '/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc' --out-dir /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/profile.stage2-x86_64-unknown-linux-gnu -L /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/profile.stage2-x86_64-unknown-linux-gnu  -g -Z profile test.rs
[00:47:51] Makefile:4: recipe for target 'all' failed
[00:47:51] make[1]: Leaving directory '/checkout/src/test/run-make/profile'
[00:47:51] 
[00:47:51] ------------------------------------------
[00:47:51] stderr:
[00:47:51] ------------------------------------------
[00:47:51] make[1]: warning: jobserver unavailable: using -j1.  Add '+' to parent make rule.
[00:47:51] error[E0463]: can't find crate for `profiler_builtins`
[00:47:51] 
[00:47:51] error: aborting due to previous error
[00:47:51] 
[00:47:51] make[1]: *** [all] Error 101
[00:47:51] 
[00:47:51] ------------------------------------------
[00:47:51] 
[00:47:51] thread '[run-make] run-make/profile' panicked at 'explicit panic', /checkout/src/tools/compiletest/src/runtest.rs:2469
[00:47:51] note: Run with `RUST_BACKTRACE=1` for a backtrace.
[00:47:51] 
[00:47:51] 
[00:47:51] failures:
[00:47:51]     [run-make] run-make/profile
[00:47:51] 
[00:47:51] test result: FAILED. 150 passed; 1 failed; 0 ignored; 0 measured
[00:47:51] 
[00:47:51] 
[00:47:51] 
[00:47:51] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools/x86_64-unknown-linux-gnu/release/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--rustdoc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "--src-base" "/checkout/src/test/run-make" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--mode" "run-make" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-3.7/bin/FileCheck" "--host-rustcflags" "-Crpath -O" "--target-rustcflags" "-Crpath -O -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--gdb" "/usr/bin/gdb" "--llvm-version" "3.7.1\n" "--cc" "cc" "--cxx" "c++" "--cflags" "-ffunction-sections -fdata-sections -fPIC -m64" "--llvm-components" "aarch64 aarch64asmparser aarch64asmprinter aarch64codegen aarch64desc aarch64disassembler aarch64info aarch64utils all all-targets amdgpu amdgpuasmparser amdgpuasmprinter amdgpucodegen amdgpudesc amdgpuinfo amdgpuutils analysis arm armasmparser armasmprinter armcodegen armdesc armdisassembler arminfo asmparser asmprinter bitreader bitwriter bpf bpfasmprinter bpfcodegen bpfdesc bpfinfo codegen core cppbackend cppbackendcodegen cppbackendinfo debuginfodwarf debuginfopdb engine executionengine hexagon hexagoncodegen hexagondesc hexagondisassembler hexagoninfo instcombine instrumentation interpreter ipa ipo irreader libdriver lineeditor linker lto mc mcdisassembler mcjit mcparser mips mipsasmparser mipsasmprinter mipscodegen mipsdesc mipsdisassembler mipsinfo mirparser msp430 msp430asmprinter msp430codegen msp430desc msp430info native nativecodegen nvptx nvptxasmprinter nvptxcodegen nvptxdesc nvptxinfo objcarcopts object option orcjit passes powerpc powerpcasmparser powerpcasmprinter powerpccodegen powerpcdesc powerpcdisassembler powerpcinfo profiledata runtimedyld scalaropts selectiondag sparc sparcasmparser sparcasmprinter sparccodegen sparcdesc sparcdisassembler sparcinfo support systemz systemzasmparser systemzasmprinter systemzcodegen systemzdesc systemzdisassembler systemzinfo tablegen target transformutils vectorize x86 x86asmparser x86asmprinter x86codegen x86desc x86disassembler x86info x86utils xcore xcoreasmprinter xcorecodegen xcoredesc xcoredisassembler xcoreinfo" "--llvm-cxxflags" "-I/usr/lib/llvm-3.7/include  -DNDEBUG -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -g -O2 -fomit-frame-pointer -std=c++11 -fvisibility-inlines-hidden -fno-exceptions -fPIC -ffunction-sections -fdata-sections -Wcast-qual" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" ""
[00:47:51] expected success, got: exit code: 101
[00:47:51] 
[00:47:51] 
[00:47:51] Build completed unsuccessfully in 0:09:38
[00:47:51] make: *** [check] Error 1
[00:47:51] Makefile:54: recipe for target 'check' failed

@whitequark is this a real failure?

Member

carols10cents commented May 8, 2017

Looks like there was a legitimate test failure? My brain isn't pattern-matching it to one of the spurious failures, at least:

[00:47:51] failures:
[00:47:51] 
[00:47:51] ---- [run-make] run-make/profile stdout ----
[00:47:51] 	
[00:47:51] error: make failed
[00:47:51] status: exit code: 2
[00:47:51] command: "make"
[00:47:51] stdout:
[00:47:51] ------------------------------------------
[00:47:51] make[1]: Entering directory '/checkout/src/test/run-make/profile'
[00:47:51] LD_LIBRARY_PATH="/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/profile.stage2-x86_64-unknown-linux-gnu:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-tools/x86_64-unknown-linux-gnu/release/deps:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib:" '/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc' --out-dir /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/profile.stage2-x86_64-unknown-linux-gnu -L /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/profile.stage2-x86_64-unknown-linux-gnu  -g -Z profile test.rs
[00:47:51] Makefile:4: recipe for target 'all' failed
[00:47:51] make[1]: Leaving directory '/checkout/src/test/run-make/profile'
[00:47:51] 
[00:47:51] ------------------------------------------
[00:47:51] stderr:
[00:47:51] ------------------------------------------
[00:47:51] make[1]: warning: jobserver unavailable: using -j1.  Add '+' to parent make rule.
[00:47:51] error[E0463]: can't find crate for `profiler_builtins`
[00:47:51] 
[00:47:51] error: aborting due to previous error
[00:47:51] 
[00:47:51] make[1]: *** [all] Error 101
[00:47:51] 
[00:47:51] ------------------------------------------
[00:47:51] 
[00:47:51] thread '[run-make] run-make/profile' panicked at 'explicit panic', /checkout/src/tools/compiletest/src/runtest.rs:2469
[00:47:51] note: Run with `RUST_BACKTRACE=1` for a backtrace.
[00:47:51] 
[00:47:51] 
[00:47:51] failures:
[00:47:51]     [run-make] run-make/profile
[00:47:51] 
[00:47:51] test result: FAILED. 150 passed; 1 failed; 0 ignored; 0 measured
[00:47:51] 
[00:47:51] 
[00:47:51] 
[00:47:51] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools/x86_64-unknown-linux-gnu/release/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--rustdoc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "--src-base" "/checkout/src/test/run-make" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--mode" "run-make" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-3.7/bin/FileCheck" "--host-rustcflags" "-Crpath -O" "--target-rustcflags" "-Crpath -O -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--gdb" "/usr/bin/gdb" "--llvm-version" "3.7.1\n" "--cc" "cc" "--cxx" "c++" "--cflags" "-ffunction-sections -fdata-sections -fPIC -m64" "--llvm-components" "aarch64 aarch64asmparser aarch64asmprinter aarch64codegen aarch64desc aarch64disassembler aarch64info aarch64utils all all-targets amdgpu amdgpuasmparser amdgpuasmprinter amdgpucodegen amdgpudesc amdgpuinfo amdgpuutils analysis arm armasmparser armasmprinter armcodegen armdesc armdisassembler arminfo asmparser asmprinter bitreader bitwriter bpf bpfasmprinter bpfcodegen bpfdesc bpfinfo codegen core cppbackend cppbackendcodegen cppbackendinfo debuginfodwarf debuginfopdb engine executionengine hexagon hexagoncodegen hexagondesc hexagondisassembler hexagoninfo instcombine instrumentation interpreter ipa ipo irreader libdriver lineeditor linker lto mc mcdisassembler mcjit mcparser mips mipsasmparser mipsasmprinter mipscodegen mipsdesc mipsdisassembler mipsinfo mirparser msp430 msp430asmprinter msp430codegen msp430desc msp430info native nativecodegen nvptx nvptxasmprinter nvptxcodegen nvptxdesc nvptxinfo objcarcopts object option orcjit passes powerpc powerpcasmparser powerpcasmprinter powerpccodegen powerpcdesc powerpcdisassembler powerpcinfo profiledata runtimedyld scalaropts selectiondag sparc sparcasmparser sparcasmprinter sparccodegen sparcdesc sparcdisassembler sparcinfo support systemz systemzasmparser systemzasmprinter systemzcodegen systemzdesc systemzdisassembler systemzinfo tablegen target transformutils vectorize x86 x86asmparser x86asmprinter x86codegen x86desc x86disassembler x86info x86utils xcore xcoreasmprinter xcorecodegen xcoredesc xcoredisassembler xcoreinfo" "--llvm-cxxflags" "-I/usr/lib/llvm-3.7/include  -DNDEBUG -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -g -O2 -fomit-frame-pointer -std=c++11 -fvisibility-inlines-hidden -fno-exceptions -fPIC -ffunction-sections -fdata-sections -Wcast-qual" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" ""
[00:47:51] expected success, got: exit code: 101
[00:47:51] 
[00:47:51] 
[00:47:51] Build completed unsuccessfully in 0:09:38
[00:47:51] make: *** [check] Error 1
[00:47:51] Makefile:54: recipe for target 'check' failed

@whitequark is this a real failure?

@whitequark

This comment has been minimized.

Show comment
Hide comment
@whitequark

whitequark May 8, 2017

Contributor

@carols10cents Yes; I haven't gated the test on the profiler being enabled. I'll fix that once I get around to the rest of the issues.

Contributor

whitequark commented May 8, 2017

@carols10cents Yes; I haven't gated the test on the profiler being enabled. I'll fix that once I get around to the rest of the issues.

@bors

This comment has been minimized.

Show comment
Hide comment
@bors

bors May 9, 2017

Contributor

☔️ The latest upstream changes (presumably #41709) made this pull request unmergeable. Please resolve the merge conflicts.

Contributor

bors commented May 9, 2017

☔️ The latest upstream changes (presumably #41709) made this pull request unmergeable. Please resolve the merge conflicts.

@carols10cents

This comment has been minimized.

Show comment
Hide comment
@carols10cents

carols10cents May 15, 2017

Member

friendly ping @whitequark, just want to keep this on your radar!

Member

carols10cents commented May 15, 2017

friendly ping @whitequark, just want to keep this on your radar!

@whitequark

This comment has been minimized.

Show comment
Hide comment
@whitequark

whitequark May 19, 2017

Contributor

@carols10cents I'm ill and temporarily unable to work. I'll go back to this PR right once I clear the higher priority parts of my backlog.

Contributor

whitequark commented May 19, 2017

@carols10cents I'm ill and temporarily unable to work. I'll go back to this PR right once I clear the higher priority parts of my backlog.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Jun 1, 2017

Member

Ok this has been inactive for a bit so I'm going to close to help clear out the queue, but @whitequark please feel free to resubmit if you get the chance! (hope you're feeling better!)

Member

alexcrichton commented Jun 1, 2017

Ok this has been inactive for a bit so I'm going to close to help clear out the queue, but @whitequark please feel free to resubmit if you get the chance! (hope you're feeling better!)

@marco-c marco-c referenced this pull request Jun 4, 2017

Merged

Build instruction profiler runtime as part of compiler-rt #42433

3 of 3 tasks complete

bors added a commit that referenced this pull request Jun 7, 2017

Auto merge of #42433 - marco-c:profiling, r=alexcrichton
Build instruction profiler runtime as part of compiler-rt

r? @alexcrichton

This is #38608 with some fixes.

Still missing:
- [x] testing with profiler enabled on some builders (on which ones? Should I add the option to some of the already existing configurations, or create a new configuration?);
- [x] enabling distribution (on which builders?);
- [x] documentation.

bors added a commit that referenced this pull request Jun 7, 2017

Auto merge of #42433 - marco-c:profiling, r=alexcrichton
Build instruction profiler runtime as part of compiler-rt

r? @alexcrichton

This is #38608 with some fixes.

Still missing:
- [x] testing with profiler enabled on some builders (on which ones? Should I add the option to some of the already existing configurations, or create a new configuration?);
- [x] enabling distribution (on which builders?);
- [x] documentation.
@jonathandturner

This comment has been minimized.

Show comment
Hide comment
@jonathandturner

jonathandturner Jun 8, 2017

Contributor

@alexcrichton - do we have a tracking bug for this work that I can link to? It's one of the Stylo asks.

Contributor

jonathandturner commented Jun 8, 2017

@alexcrichton - do we have a tracking bug for this work that I can link to? It's one of the Stylo asks.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
Member

alexcrichton commented Jun 8, 2017

@jonathandturner

This comment has been minimized.

Show comment
Hide comment
@jonathandturner

jonathandturner Jun 8, 2017

Contributor

@alexcrichton - thanks!

Contributor

jonathandturner commented Jun 8, 2017

@alexcrichton - thanks!

bors added a commit that referenced this pull request Jun 13, 2017

Auto merge of #42433 - marco-c:profiling, r=alexcrichton
Build instruction profiler runtime as part of compiler-rt

r? @alexcrichton

This is #38608 with some fixes.

Still missing:
- [x] testing with profiler enabled on some builders (on which ones? Should I add the option to some of the already existing configurations, or create a new configuration?);
- [x] enabling distribution (on which builders?);
- [x] documentation.

bors added a commit that referenced this pull request Jun 13, 2017

Auto merge of #42433 - marco-c:profiling, r=alexcrichton
Build instruction profiler runtime as part of compiler-rt

r? @alexcrichton

This is #38608 with some fixes.

Still missing:
- [x] testing with profiler enabled on some builders (on which ones? Should I add the option to some of the already existing configurations, or create a new configuration?);
- [x] enabling distribution (on which builders?);
- [x] documentation.

bors added a commit that referenced this pull request Jun 14, 2017

Auto merge of #42433 - marco-c:profiling, r=alexcrichton
Build instruction profiler runtime as part of compiler-rt

r? @alexcrichton

This is #38608 with some fixes.

Still missing:
- [x] testing with profiler enabled on some builders (on which ones? Should I add the option to some of the already existing configurations, or create a new configuration?);
- [x] enabling distribution (on which builders?);
- [x] documentation.

bors added a commit that referenced this pull request Jun 14, 2017

Auto merge of #42433 - marco-c:profiling, r=alexcrichton
Build instruction profiler runtime as part of compiler-rt

r? @alexcrichton

This is #38608 with some fixes.

Still missing:
- [x] testing with profiler enabled on some builders (on which ones? Should I add the option to some of the already existing configurations, or create a new configuration?);
- [x] enabling distribution (on which builders?);
- [x] documentation.

bors added a commit that referenced this pull request Jun 14, 2017

Auto merge of #42433 - marco-c:profiling, r=alexcrichton
Build instruction profiler runtime as part of compiler-rt

r? @alexcrichton

This is #38608 with some fixes.

Still missing:
- [x] testing with profiler enabled on some builders (on which ones? Should I add the option to some of the already existing configurations, or create a new configuration?);
- [x] enabling distribution (on which builders?);
- [x] documentation.

@whitequark whitequark deleted the whitequark:profiling branch Jul 27, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment