Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable Rust + Sanitizers on Travis #381

Merged
merged 6 commits into from Oct 30, 2018

Conversation

Projects
None yet
3 participants
@alexcrichton
Copy link
Contributor

commented Oct 2, 2018

After discussing with the networking team folks in Mexico this is a series of commits which should get the Travis build green for testing Rust with C code that's been compiled with sanitizers. It turned out that there was a slew of issues that needed to be fixed! Many of them are detailed in the commits but some high level ones are:

  • Rust nightly is used to pick up rust-lang/rust#54681 and rust-lang/rust#54675
  • The link_rust.sh script is remove din favor of RUSTFLAGS used to carry arguments
  • Jemalloc is avoided when testing as it appears incompatible with ASan.
  • Some minor cleanup of crate types was done

Some issue links this is relevant to:

Please let me know if anything should be updated here as well! I'd definitely be down for adding more comments here and there.

alexcrichton added some commits Oct 2, 2018

Reenable hardening options with Rust on Travis
Previously the sanitizers are forcibly disabled as they were found to be
incompatible with Rust code. The nightly channel of Rust, however, now
has some fixes which should make this disabling no longer necessary.
Remove the `link_rust.sh.in` script
This is no longer necessary with upstream rust-lang/rust changes as well
as some local tweaks. Namely:

* The `-fsanitize=address`-style options are now passed via `-C
  link-args` through `RUSTFLAGS`. This obviates the need for the shell
  script.
* The `-C default-linker-libraries`, disabling `-nodefaultlibs`, is
  passed through `RUSTFLAGS`, which is necessary to ensure that
  `-fsanitize=address` links correctly.
* The `-C linker` option is passed to ensure we're using the same C
  compiler as normal C code, although it has a bit of hackery to only
  get the `gcc` out of `gcc -std=c99`
Fix segfaults related to sanitizers+jemalloc
It looks to be the case that Rust's standard allocator, jemalloc, is
incompatible with sanitizers. The incompatibility, for whatever reason,
seems to cause segfaults at runtime when jemalloc is linked with
sanitizers.

Without actually trying to figure out what's going on here this commit
instead takes the hammer of "let's remove jemalloc when testing". The
`tor_allocate` crate now by default switches to the system allocator
(eventually this will want to be the tor allocator). Most crates then
link to `tor_allocate` ot pick this up, but the `smartlist` crate had to
manually switch to the system allocator in testing and the `external`
crate had to be sure to link to `tor_allocate`.

The final gotcha here is that this patch also switches to
unconditionally passing `--target` to Cargo. For weird and arcane
reasons passing `--target` with the host target of the compiler (which
Cargo otherwise uses as the default) is different than not passing
`--target` at all. This ensure that our custom `RUSTFLAGS` with
sanitizer options doesn't make its way into build scripts, just the
final testing artifacts.
Remove `[features]` from workspace Cargo.toml
Unfortunately Cargo doesn't actually parse these! Cargo should probably
print a warning saying they're not used...
Remove rlib+staticlib configuration for Rust crates
Only the final crate needs to be a `staticlib`, no need for all the
intermediate steps to produce staticlibs!
@alexcrichton

This comment has been minimized.

Copy link
Contributor Author

commented Oct 2, 2018

I also did some investigation into the test_linking_hack feature and left some comments on the commit, but if y'all would like I can certainly open a ticket as well with some more details! (and/or copy that comment to a more appropriate location)

@coveralls

This comment has been minimized.

Copy link

commented Oct 2, 2018

Pull Request Test Coverage Report for Build 2624

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 5323 unchanged lines in 29 files lost coverage.
  • Overall coverage increased (+0.06%) to 62.03%

Files with Coverage Reduction New Missed Lines %
src/lib/fs/storagedir.c 1 87.34%
src/core/or/or.h 1 88.89%
src/feature/hs/hs_common.c 1 83.28%
src/core/or/channelpadding.c 5 97.27%
src/lib/tls/tortls.c 6 96.55%
src/lib/evloop/procmon.c 7 76.74%
src/core/or/channel.c 12 83.43%
src/ext/tinytest.c 12 41.84%
src/feature/nodelist/torcert.c 12 95.22%
src/core/proto/proto_socks.c 14 92.74%
Totals Coverage Status
Change from base Build 2353: 0.06%
Covered Lines: 44078
Relevant Lines: 71059

💛 - Coveralls
Only pass `-C default-linker-libraries` with sanitizers
This'll help retain test compatibility until 1.31.0 is released!

@torproject-pusher torproject-pusher merged commit 8285784 into torproject:master Oct 30, 2018

2 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

@alexcrichton alexcrichton deleted the alexcrichton:fix-rust-again branch Oct 30, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.