New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OSX support tracking issue #152

Open
dataf3l opened this Issue Oct 3, 2018 · 22 comments

Comments

Projects
None yet
3 participants
@dataf3l
Copy link

dataf3l commented Oct 3, 2018

This is a tracking issue for OSX support on tarpaulin

  • Figure out why my OSX tests don't have the __debug_info section
  • Update nix and libc to add ptrace support
  • Update tarpaulin to use updated nix
  • Fix other compile issues
  • Refactor ASLR disabling in OSX (not done via personality ffi call)
    • Figure out if this has worked (GDB uses same method so it should)
  • Look into the IPC ports that are used to send things like termination signals. They may be necessary
  • Investigate and update PTRACE_EVENT based code. OSX doesn't support PTRACE_EVENT
    • I've stuck in ptrace event currently but returns -1 so probably doing nothing
  • Look at how poke/peek differs with the user struct in OSX vs linux and update code if needed
  • General tidy up and refactor

Resources/links

2018/12/05 - gets into initialising the breakpoints to track coverage but none are ever hit and no exit signal.

Original issue text

localhost:src b$ rustc --version rustc 1.30.0-nightly (721913067 2018-08-26)

localhost:src b$ RUSTFLAGS="--cfg procmacro2_semver_exempt" cargo install cargo-tarpaulin
Updating registry https://github.com/rust-lang/crates.io-index
Installing cargo-tarpaulin v0.6.7
Compiling serde v1.0.78
Compiling semver-parser v0.7.0
Compiling proc-macro2 v0.4.19
Compiling version_check v0.1.4
Compiling pkg-config v0.3.14
Compiling cc v1.0.24
Compiling unicode-xid v0.1.0
Compiling num-traits v0.2.5
Compiling void v1.0.2
Compiling build_const v0.2.1
Compiling matches v0.1.8
Compiling libc v0.2.43
Compiling stable_deref_trait v1.1.1
Compiling rand_core v0.2.1
Compiling scopeguard v0.3.3
Compiling ryu v0.2.6
Compiling unicode-normalization v0.1.7
Compiling failure_derive v0.1.2
Compiling ucd-util v0.1.1
Compiling regex v1.0.5
Compiling nodrop v0.1.12
Compiling memoffset v0.2.1
Compiling percent-encoding v1.0.1
Compiling utf8-ranges v1.0.1
Compiling core-foundation-sys v0.6.1
Compiling crossbeam-utils v0.5.0
Compiling rustc-demangle v0.1.9
Compiling cfg-if v0.1.5
Compiling itoa v0.4.3
Compiling unicode-width v0.1.5
Compiling fnv v1.0.6
Compiling bitflags v1.0.4
Compiling quick-error v1.2.2
Compiling same-file v1.0.3
Compiling byteorder v1.2.6
Compiling adler32 v1.0.3
Compiling nix v0.11.0
Compiling strsim v0.7.0
Compiling termcolor v1.0.3
Compiling remove_dir_all v0.5.1
Compiling hex v0.3.2
Compiling vec_map v0.8.1
Compiling plain v0.2.3
Compiling ansi_term v0.11.0
Compiling glob v0.2.11
Compiling md5 v0.3.8
Compiling termcolor v0.3.6
Compiling shell-escape v0.1.4
Compiling fallible-iterator v0.1.5
Compiling lazycell v0.6.0
Compiling crossbeam v0.3.2
Compiling home v0.3.3
Compiling unreachable v1.0.0
Compiling unicode-bidi v0.3.4
Compiling crc v1.8.1
Compiling owning_ref v0.3.3
Compiling lazy_static v1.1.0
Compiling rand v0.5.5
Compiling memchr v2.0.2
Compiling commoncrypto-sys v0.2.0
Compiling atty v0.2.11
Compiling fs2 v0.4.3
Compiling num_cpus v1.8.0
Compiling memmap v0.6.2
Compiling arrayvec v0.4.7
Compiling regex-syntax v0.6.2
Compiling socket2 v0.3.8
Compiling log v0.4.5
Compiling filetime v0.2.1
Compiling encoding_rs v0.8.6
Compiling uuid v0.6.5
Compiling textwrap v0.10.0
Compiling humantime v1.1.1
Compiling walkdir v2.2.5
Compiling parity-wasm v0.31.3
Compiling backtrace v0.3.9
Compiling smallvec v0.6.5
Compiling cmake v0.1.33
Compiling libz-sys v1.0.20
Compiling openssl-sys v0.9.35
Compiling curl-sys v0.4.8
Compiling miniz-sys v0.1.10
Compiling lock_api v0.1.3
Compiling aho-corasick v0.6.8
Compiling commoncrypto v0.2.0
Compiling gimli v0.16.1
Compiling jobserver v0.1.11
Compiling tar v0.4.16
Compiling clap v2.32.0
Compiling tempfile v3.0.3
Compiling libssh2-sys v0.2.10
Compiling libgit2-sys v0.7.7
Compiling num-traits v0.1.43
Compiling quote v0.6.8
Compiling crypto-hash v0.3.1
Compiling core-foundation v0.6.1
Compiling thread_local v0.3.6
Compiling crossbeam-epoch v0.6.0
Compiling enum_primitive v0.1.1
Compiling semver v0.9.0
Compiling serde_json v1.0.27
Compiling toml v0.4.6
Compiling serde_ignored v0.0.4
Compiling syn v0.14.9
Compiling syn v0.15.3
Compiling gzip-header v0.1.2
Compiling flate2 v1.0.2
Compiling curl v0.4.14
Compiling rustc_version v0.2.3
Compiling idna v0.1.5
Compiling deflate v0.7.18
Compiling parking_lot_core v0.3.1
Compiling url v1.7.1
Compiling synstructure v0.9.0
Compiling scroll_derive v0.9.4
Compiling parking_lot v0.6.4
Compiling globset v0.4.2
Compiling env_logger v0.5.13
Compiling serde_derive v1.0.78
Compiling crossbeam-channel v0.2.5
Compiling scroll v0.9.0
Compiling goblin v0.0.17
Compiling ignore v0.4.4
Compiling git2 v0.7.5
Compiling failure v0.1.2
Compiling quick-xml v0.12.3
Compiling git2-curl v0.8.1
Compiling object v0.10.0
Compiling crates-io v0.17.0
Compiling coveralls-api v0.3.3
Compiling cargo v0.29.0
Compiling cargo-tarpaulin v0.6.7
error[E0432]: unresolved import nix::sys::ptrace
--> src/ptrace_control.rs:3:15
|
3 | use nix::sys::ptrace::*;
| ^^^^^^ Could not find ptrace in sys

error[E0408]: variable PTRACE_EVENT_FORK is not bound in all patterns
--> src/statemachine.rs:356:37
|
356 | PTRACE_EVENT_FORK | PTRACE_EVENT_VFORK => {
| ----------------- ^^^^^^^^^^^^^^^^^^ pattern doesn't bind PTRACE_EVENT_FORK
| |
| variable not in all patterns

error[E0408]: variable PTRACE_EVENT_VFORK is not bound in all patterns
--> src/statemachine.rs:356:17
|
356 | PTRACE_EVENT_FORK | PTRACE_EVENT_VFORK => {
| ^^^^^^^^^^^^^^^^^ ------------------ variable not in all patterns
| |
| pattern doesn't bind PTRACE_EVENT_VFORK

error[E0433]: failed to resolve. Use of undeclared type or module Options
--> src/ptrace_control.rs:13:28
|
13 | let options: Options = Options::PTRACE_O_TRACESYSGOOD |
| ^^^^^^^ Use of undeclared type or module Options

error[E0433]: failed to resolve. Use of undeclared type or module Options
--> src/ptrace_control.rs:14:9
|
14 | Options::PTRACE_O_TRACEEXEC | Options::PTRACE_O_TRACEEXIT |
| ^^^^^^^ Use of undeclared type or module Options

error[E0433]: failed to resolve. Use of undeclared type or module Options
--> src/ptrace_control.rs:14:39
|
14 | Options::PTRACE_O_TRACEEXEC | Options::PTRACE_O_TRACEEXIT |
| ^^^^^^^ Use of undeclared type or module Options

error[E0433]: failed to resolve. Use of undeclared type or module Options
--> src/ptrace_control.rs:15:9
|
15 | Options::PTRACE_O_TRACECLONE | Options::PTRACE_O_TRACEFORK |
| ^^^^^^^ Use of undeclared type or module Options

error[E0433]: failed to resolve. Use of undeclared type or module Options
--> src/ptrace_control.rs:15:40
|
15 | Options::PTRACE_O_TRACECLONE | Options::PTRACE_O_TRACEFORK |
| ^^^^^^^ Use of undeclared type or module Options

error[E0433]: failed to resolve. Use of undeclared type or module Options
--> src/ptrace_control.rs:16:9
|
16 | Options::PTRACE_O_TRACEVFORK;
| ^^^^^^^ Use of undeclared type or module Options

error[E0433]: failed to resolve. Use of undeclared type or module Request
--> src/ptrace_control.rs:31:16
|
31 | ptrace(Request::PTRACE_SINGLESTEP, pid, ptr::null_mut(), ptr::null_mut())
| ^^^^^^^ Use of undeclared type or module Request

error[E0433]: failed to resolve. Use of undeclared type or module Request
--> src/ptrace_control.rs:38:16
|
38 | ptrace(Request::PTRACE_PEEKDATA, pid, address as * mut c_void, ptr::null_mut())
| ^^^^^^^ Use of undeclared type or module Request

error[E0433]: failed to resolve. Use of undeclared type or module Request
--> src/ptrace_control.rs:47:16
|
47 | ptrace(Request::PTRACE_POKEDATA, pid, address as * mut c_void, data as * mut c_void)
| ^^^^^^^ Use of undeclared type or module Request

error[E0433]: failed to resolve. Use of undeclared type or module Request
--> src/ptrace_control.rs:54:16
|
54 | ptrace(Request::PTRACE_PEEKUSER, pid, RIP as * mut c_void, ptr::null_mut())
| ^^^^^^^ Use of undeclared type or module Request

error[E0433]: failed to resolve. Use of undeclared type or module Request
--> src/ptrace_control.rs:61:16
|
61 | ptrace(Request::PTRACE_POKEUSER, pid, RIP as * mut c_void, pc as * mut c_void)
| ^^^^^^^ Use of undeclared type or module Request

error[E0412]: cannot find type Persona in this scope
--> src/personality.rs:12:26
|
12 | const ADDR_NO_RANDOMIZE: Persona = 0x004_0000;
| ^^^^^^^ not found in this scope

error[E0412]: cannot find type Persona in this scope
--> src/personality.rs:13:20
|
13 | const GET_PERSONA: Persona = 0xFFFF_FFFF;
| ^^^^^^^ not found in this scope

error[E0412]: cannot find type Persona in this scope
--> src/personality.rs:24:25
|
24 | fn personality(persona: Persona) -> Result<c_int> {
| ^^^^^^^ not found in this scope

error[E0412]: cannot find type Options in this scope
--> src/ptrace_control.rs:13:18
|
13 | let options: Options = Options::PTRACE_O_TRACESYSGOOD |
| ^^^^^^^ did you mean Option?

error[E0425]: cannot find function setoptions in this scope
--> src/ptrace_control.rs:17:5
|
17 | setoptions(pid, options)
| ^^^^^^^^^^ did you mean options?

error[E0425]: cannot find function detach in this scope
--> src/ptrace_control.rs:21:5
|
21 | detach(pid)
| ^^^^^^ not found in this scope

error[E0425]: cannot find function cont in this scope
--> src/ptrace_control.rs:25:5
|
25 | cont(pid, sig)
| ^^^^ not found in this scope

error[E0425]: cannot find function ptrace in this scope
--> src/ptrace_control.rs:31:9
|
31 | ptrace(Request::PTRACE_SINGLESTEP, pid, ptr::null_mut(), ptr::null_mut())
| ^^^^^^ not found in this scope
help: possible candidate is found in another module, you can import it into scope
|
1 | use nix::libc::ptrace;
|

error[E0425]: cannot find function ptrace in this scope
--> src/ptrace_control.rs:38:9
|
38 | ptrace(Request::PTRACE_PEEKDATA, pid, address as * mut c_void, ptr::null_mut())
| ^^^^^^ not found in this scope
help: possible candidate is found in another module, you can import it into scope
|
1 | use nix::libc::ptrace;
|

error[E0425]: cannot find function ptrace in this scope
--> src/ptrace_control.rs:47:9
|
47 | ptrace(Request::PTRACE_POKEDATA, pid, address as * mut c_void, data as * mut c_void)
| ^^^^^^ not found in this scope
help: possible candidate is found in another module, you can import it into scope
|
1 | use nix::libc::ptrace;
|

error[E0425]: cannot find function ptrace in this scope
--> src/ptrace_control.rs:54:9
|
54 | ptrace(Request::PTRACE_PEEKUSER, pid, RIP as * mut c_void, ptr::null_mut())
| ^^^^^^ not found in this scope
help: possible candidate is found in another module, you can import it into scope
|
1 | use nix::libc::ptrace;
|

error[E0425]: cannot find function ptrace in this scope
--> src/ptrace_control.rs:61:9
|
61 | ptrace(Request::PTRACE_POKEUSER, pid, RIP as * mut c_void, pc as * mut c_void)
| ^^^^^^ not found in this scope
help: possible candidate is found in another module, you can import it into scope
|
1 | use nix::libc::ptrace;
|

error[E0425]: cannot find function traceme in this scope
--> src/ptrace_control.rs:66:5
|
66 | traceme()
| ^^^^^^^ not found in this scope

error[E0425]: cannot find function getevent in this scope
--> src/ptrace_control.rs:70:5
|
70 | getevent(pid)
| ^^^^^^^^ not found in this scope

error[E0599]: no associated item named __WALL found for type nix::sys::wait::WaitPidFlag in the current scope
--> src/statemachine.rs:231:75
|
231 | let wait = waitpid(Pid::from_raw(-1), Some(WaitPidFlag::WNOHANG | WaitPidFlag::__WALL));
| ^^^^^^^^^^^^^^^^^^^ associated item not found in nix::sys::wait::WaitPidFlag

error[E0599]: no variant named PtraceEvent found for type nix::sys::wait::WaitStatus in the current scope
--> src/statemachine.rs:251:13
|
251 | WaitStatus::PtraceEvent(c,s,e) => {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ variant not found in nix::sys::wait::WaitStatus

error: aborting due to 30 previous errors

Some errors occurred: E0408, E0412, E0425, E0432, E0433, E0599.
For more information about an error, try rustc --explain E0408.
error: failed to compile cargo-tarpaulin v0.6.7, intermediate artifacts can be found at /var/folders/26/ghqp1zl50yq3bp47zljpksk80000gn/T/cargo-install5pqpq9

Caused by:
Could not compile cargo-tarpaulin.

To learn more, run the command again with --verbose.

@xd009642

This comment has been minimized.

Copy link
Owner

xd009642 commented Oct 3, 2018

So tarpaulin currently only supports linux, if anything else works it's more luck that intent. OSX compared to linux doesn't have a complete ptrace implementation so the nix crate doesn't expose the ptrace bindings.

If anyone was interested in adding OSX support they'd need to either replace ptrace with appropriate mac system calls or contribute to nix to add the parts of ptrace mac exports and for the rest of it add the appropriate system calls.

I don't have any Macs so while I'm open to more OS support this is something I'd appreciate help on.

@dataf3l

This comment has been minimized.

Copy link

dataf3l commented Oct 3, 2018

Do you know of any tools that can be used on OSX for coverage reports?,
Also, I have a mac and can help test whatever you guys code, but I'm not smart enough to do the syscalls on OSX, I need to know more about that, first.
I can also provide ssh access I guess, via Hamachi, if somebody wants to to connect to my machine and work remotely on it, but I assume that wouldn't work out, since the people who are experts on osx syscalls and stuff like that probably already have and love a mac.

would a donation to the project help?

@xd009642

This comment has been minimized.

Copy link
Owner

xd009642 commented Oct 4, 2018

You might be able to get it working with docker on mac - though I'm not really that familiar with docker so maybe not... A good start if you wanted to help for the ptrace calls on mac. It should just be a case of looking at the ptrace docs for OSX finding what's supported and changing the cfgs in https://github.com/nix-rust/nix/ to match. Once that's done and merged in it we can see what fails to compile and go from there.

As for donations I hadn't considered project donations before, though I'm probably going to be rather busy with my day job till the end of the year so that probably won't change anything in reality.

@xd009642

This comment has been minimized.

Copy link
Owner

xd009642 commented Oct 7, 2018

Oh realised I missed your first question on tools that can be used for coverage, there is https://github.com/SimonKagstrom/kcov which works on linux, freebsd and OSX.

I've just opened a PR on rust-nix/nix nix-rust/nix#949 for ptrace support in OSX and BSD. So once that's done, released and I update tarpaulin for the new nix version tarpaulin might work on OSX or there might be other things that need adapting. I'll ping you when there' something to test though!

Edit: Notes for me -

@xd009642

This comment has been minimized.

Copy link
Owner

xd009642 commented Oct 8, 2018

@dataf3l could you clone https://github.com/xd009642/nix/ checkout the branch issue-947 and let me know the output of the following tests sys::test_ptrace::test_ptrace sys::test_ptrace::test_ptrace_cont? This is part of adding OSX support to some of my dependencies and the travis runs are failing so we're trying to figure out if it's a travis qemu issue or an actual apple issue

@dataf3l

This comment has been minimized.

Copy link

dataf3l commented Oct 8, 2018

@xd009642 Unfortunately, it looks like it some of the tests didn't pass...

Build output

``` localhost:~ b$ cd work/ localhost:work b$ git clone https://github.com/xd009642/nix.git -b issue-947 Cloning into 'nix'... remote: Enumerating objects: 16, done. remote: Counting objects: 100% (16/16), done. remote: Compressing objects: 100% (13/13), done. remote: Total 7722 (delta 4), reused 6 (delta 3), pack-reused 7706 Receiving objects: 100% (7722/7722), 2.02 MiB | 990.00 KiB/s, done. Resolving deltas: 100% (5583/5583), done. localhost:work b$ cd nix/ localhost:nix b$ ls bors.toml build.rs Cargo.toml CHANGELOG.md ci CONTRIBUTING.md CONVENTIONS.md Cross.toml LICENSE README.md RELEASE_PROCEDURE.md src test localhost:nix b$ git status . On branch issue-947 Your branch is up to date with 'origin/issue-947'.

nothing to commit, working tree clean
localhost:nix b$ cargo build
Updating registry https://github.com/rust-lang/crates.io-index
Updating git repository https://github.com/rust-lang/libc
Compiling libc v0.2.43 (https://github.com/rust-lang/libc#41944d5c)
Compiling nix v0.11.0 (file:///Users/b/work/nix)
Compiling cfg-if v0.1.5
Compiling void v1.0.2
Compiling bitflags v1.0.4
Finished dev [unoptimized + debuginfo] target(s) in 23.98s
localhost:nix b$ cargo test
Downloading rand_core v0.2.2
Downloading rand_core v0.3.0
Compiling rand_core v0.3.0
Compiling version_check v0.1.5
Compiling libc v0.2.43
Compiling remove_dir_all v0.5.1
Compiling byteorder v1.2.6
Compiling rand_core v0.2.2
Compiling lazy_static v1.1.0
Compiling iovec v0.1.2
Compiling rand v0.5.5
Compiling bytes v0.4.10
Compiling tempfile v3.0.4
Compiling nix v0.11.0 (file:///Users/b/work/nix)
warning: unused macro definition
--> test/test_mount.rs:209:1
|
209 | / macro_rules! run_tests {
210 | | ( $($test_fn:ident),* ) => {{
211 | | println!();
212 | |
... |
220 | | }}
221 | | }
| |_^
|
= note: #[warn(unused_macros)] on by default

warning: unused macro definition
--> test/test.rs:12:1
|
12 | / macro_rules! skip_if_not_root {
13 | | ($name:expr) => {
14 | | use nix::unistd::Uid;
15 | | use std;
... |
24 | | };
25 | | }
| |_^
|
= note: #[warn(unused_macros)] on by default

warning: unused import: std::mem
--> test/sys/test_ptrace.rs:8:5
|
8 | use std::mem;
| ^^^^^^^^
|
= note: #[warn(unused_imports)] on by default

warning: unused import: nix::sys::signal::SigSet
--> test/test_poll.rs:2:5
|
2 | use nix::sys::signal::SigSet;
| ^^^^^^^^^^^^^^^^^^^^^^^^

warning: unused imports: TimeSpec, TimeValLike
--> test/test_poll.rs:3:22
|
3 | use nix::sys::time::{TimeSpec, TimeValLike};
| ^^^^^^^^ ^^^^^^^^^^^

warning: use of deprecated item 'nix::unistd::pipe2': pipe2(2) is not actually atomic on these platforms. Use pipe(2) and fcntl(2) instead
--> test/test_unistd.rs:386:22
|
386 | let (fd0, fd1) = pipe2(OFlag::O_CLOEXEC).unwrap();
| ^^^^^
|
= note: #[warn(deprecated)] on by default

warning: unused std::result::Result which must be used
--> test/test_stat.rs:177:5
|
177 | utimes(&fullpath, &TimeVal::seconds(9990), &TimeVal::seconds(5550));
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: #[warn(unused_must_use)] on by default
= note: this Result may be an Err variant, which should be handled

Finished dev [unoptimized + debuginfo] target(s) in 13.20s
 Running target/debug/deps/nix-f5796617f4709c01

running 35 tests
test sys::event::test_struct_kevent ... ok
test ifaddrs::tests::test_getifaddrs ... ok
test sys::select::tests::fdset_clear ... ok
test sys::select::tests::fdset_insert ... ok
test sys::select::tests::fdset_remove ... ok
test sys::select::tests::test_select ... ok
test sys::select::tests::test_select_nfds ... ok
test sys::select::tests::test_select_nfds2 ... ok
test sys::signal::tests::test_clear ... ok
test sys::select::tests::fdset_highest ... ok
test sys::signal::tests::test_contains ... ok
test sys::signal::tests::test_extend ... ok
test sys::signal::tests::test_from_str_invalid_value ... ok
test sys::signal::tests::test_sigaction ... ok
test sys::signal::tests::test_thread_signal_block ... ok
test sys::signal::tests::test_thread_signal_get_mask ... ok
test sys::signal::tests::test_from_str_round_trips ... ok
test sys::signal::tests::test_thread_signal_set_mask ... ok
test sys::signal::tests::test_thread_signal_swap ... ok
test sys::signal::tests::test_thread_signal_unblock ... ok
test sys::socket::addr::tests::test_macos_loopback_datalink_addr ... ok
test sys::socket::addr::tests::test_macos_tap_datalink_addr ... ok
test sys::socket::sockopt::test::is_socket_type_dgram ... ok
test sys::statvfs::test::fstatvfs_call ... ok
test sys::statvfs::test::statvfs_call ... ok
test sys::socket::sockopt::test::is_socket_type_unix ... ok
test sys::time::test::test_timespec ... ok
test sys::time::test::test_timespec_neg ... ok
test sys::time::test::test_timespec_fmt ... ok
test sys::time::test::test_timespec_ord ... ok
test sys::time::test::test_timeval ... ok
test sys::time::test::test_timeval_neg ... ok
test sys::time::test::test_timeval_fmt ... ok
test sys::time::test::test_timeval_ord ... ok
test sys::utsname::test::test_uname_darwin ... ok

test result: ok. 35 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

 Running target/debug/deps/test-fc2b5d0af9911a3b

running 98 tests
test sys::test_aio::test_accessors ... ok
test sys::test_aio::test_aio_suspend ... ok
test sys::test_aio::test_read_bytes_mut_small ... ok
test sys::test_aio::test_aio_cancel_all ... ok
test sys::test_aio::test_fsync_error ... ok
test sys::test_aio::test_read_error ... ok
test sys::test_aio::test_read_immutable_buffer ... ok
test sys::test_aio::test_write_error ... ok
test sys::test_aio::test_fsync ... ok
test sys::test_aio::test_cancel ... ok
test sys::test_ioctl::bsd::test_op_none ... ok
test sys::test_ioctl::bsd::test_op_read ... ok
test sys::test_ioctl::bsd::test_op_read_64 ... ok
test sys::test_ioctl::bsd::test_op_read_write ... ok
test sys::test_ioctl::bsd::test_op_read_write_64 ... ok
test sys::test_ioctl::bsd::test_op_write ... ok
test sys::test_ioctl::bsd::test_op_write_64 ... ok
test sys::test_pthread::test_pthread_self ... ok
test sys::test_aio::test_read_into_mut_slice ... ok
test sys::test_aio::test_read ... ok
test sys::test_ptrace::test_ptrace ... FAILED
test sys::test_aio::test_write_bytes ... ok
test sys::test_aio::test_read_into_pointer ... ok
test sys::test_aio::test_write_from_pointer ... ok
test sys::test_aio::test_write ... ok
test sys::test_select::test_pselect_nfds2 ... ok
test sys::test_ptrace::test_ptrace_cont ... FAILED
test sys::test_signal::test_kill_none ... ok
test sys::test_signal::test_old_sigaction_flags ... ok
test sys::test_signal::test_sigprocmask_noop ... ok
test sys::test_socket::test_inetv4_addr_to_sock_addr ... ok
test sys::test_socket::test_inetv6_addr_to_sock_addr ... ok
test sys::test_socket::test_getsockname ... ok
test sys::test_socket::test_path_to_sock_addr ... ok
test sys::test_socket::test_scm_rights ... ok
test sys::test_socket::test_socketpair ... ok
test sys::test_socket::test_scm_rights_single_cmsg_multiple_fds ... ok
test sys::test_socket::test_sendmsg_empty_cmsgs ... ok
test sys::test_socket::test_syscontrol ... ok
test sys::test_termios::test_cfmakeraw ... ok
test sys::test_termios::test_tcgetattr_ebadf ... ok
test sys::test_termios::test_local_flags ... ok
test sys::test_sockopt::test_so_buf ... ok
test sys::test_termios::test_output_flags ... ok
test sys::test_socket::test_unixdomain ... ok
test sys::test_aio::test_write_sigev_signal ... ok
test sys::test_select::test_pselect ... ok
test sys::test_signal::test_sigprocmask ... ok
test sys::test_termios::test_tcgetattr_pty ... ok
test sys::test_termios::test_tcgetattr_enotty ... ok
test sys::test_uio::test_pwrite ... ok
test sys::test_wait::test_waitstatus_from_raw ... ok
test sys::test_uio::test_readv ... ok
test test_dir::ebadf ... ok
test sys::test_wait::test_wait_exit ... ok
test sys::test_uio::test_pread ... ok
test sys::test_wait::test_wait_signal ... ok
test sys::test_uio::test_writev ... ok
test sys::test_wait::test_waitstatus_pid ... ok
test test_dir::rewind ... ok
test test_net::test_if_nametoindex ... ok
test test_poll::test_poll_debug ... ok
test test_dir::read ... ok
test test_fcntl::test_openat ... ok
test test_pty::test_open_ptty_pair ... ok
test test_pty::test_openpty ... ok
test test_fcntl::test_readlink ... ok
test test_pty::test_openpty_with_termios ... ok
test test_pty::test_explicit_close ... ok
test test_sendfile::test_sendfile_darwin ... ok
test test_stat::test_fchmod ... ok
test test_stat::test_fchmodat ... ok
test test_stat::test_fstatat ... ok
test test_stat::test_futimens ... ok
test test_stat::test_stat_and_fstat ... ok
test test_stat::test_utimes ... ok
test test_stat::test_utimensat ... ok
test test_stat::test_stat_fstat_lstat ... ok
test test_unistd::test_fchdir ... ok
test test_unistd::test_fpathconf_limited ... ok
test test_unistd::test_getpid ... ok
test test_unistd::test_getsid ... ok
test test_unistd::test_lseek ... ok
test test_unistd::test_mkfifo_directory ... ok
test test_unistd::test_mkfifo ... ok
test test_unistd::test_mkstemp_directory ... ok
test test_unistd::test_mkstemp ... ok
test test_unistd::test_pathconf_limited ... ok
test test_unistd::test_pipe ... ok
test test_unistd::test_pipe2 ... ok
test test_unistd::test_sysconf_limited ... ok
test test_unistd::test_getcwd ... ok
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
test test_unistd::test_execve ... ok
test test_unistd::test_fork_and_waitpid ... ok
test test_unistd::test_wait ... ok
test test_poll::test_poll ... ok
test test_unistd::test_alarm ... ok
test test_unistd::test_canceling_alarm ... ok

failures:

---- sys::test_ptrace::test_ptrace stdout ----
thread 'sys::test_ptrace::test_ptrace' panicked at 'assertion failed: err == Error::Sys(Errno::EPERM) || err == Error::Sys(Errno::ENOSYS)', test/sys/test_ptrace.rs:15:5

---- sys::test_ptrace::test_ptrace_cont stdout ----
thread 'sys::test_ptrace::test_ptrace_cont' panicked at 'called Result::unwrap() on an Err value: Sys(ENOTSUP)', libcore/result.rs:983:5

failures:
sys::test_ptrace::test_ptrace
sys::test_ptrace::test_ptrace_cont

test result: FAILED. 96 passed; 2 failed; 0 ignored; 0 measured; 0 filtered out

error: test failed, to rerun pass '--test test'
localhost:nix b$

</p>
</details>
@xd009642

This comment has been minimized.

Copy link
Owner

xd009642 commented Oct 8, 2018

Okay in src/sys/ptrace/bsd.rs in the function attach could you replace Request::PT_ATTACH with Request::PT_ATTACHEXC? Okay never mind I expect that to fail.

@dataf3l

This comment has been minimized.

Copy link

dataf3l commented Oct 9, 2018

Still no dice:

Build output

localhost:nix b$ vi src/sys/ptrace/bsd.rs localhost:nix b$ cargo clean localhost:nix b$ cargo build Compiling libc v0.2.43 (https://github.com/rust-lang/libc#41944d5c) Compiling nix v0.11.0 (file:///Users/b/work/nix) Compiling bitflags v1.0.4 Compiling cfg-if v0.1.5 Compiling void v1.0.2 Finished dev [unoptimized + debuginfo] target(s) in 8.92s localhost:nix b$ localhost:nix b$ localhost:nix b$ brew install sphinx-doc Updating Homebrew... ==> Auto-updated Homebrew! Updated 2 taps (homebrew/core and homebrew/cask). ==> Updated Formulae aws-elasticbeanstalk futhark jhipster php-code-sniffer swiftgen terragrunt

==> Downloading https://homebrew.bintray.com/bottles/sphinx-doc-1.8.1.high_sierra.bottle.tar.gz
######################################################################## 100.0%
==> Pouring sphinx-doc-1.8.1.high_sierra.bottle.tar.gz
==> Caveats
sphinx-doc is keg-only, which means it was not symlinked into /usr/local,
because this formula is mainly used internally by other formulae.
Users are advised to use pip to install sphinx-doc.

If you need to have sphinx-doc first in your PATH run:
echo 'export PATH="/usr/local/opt/sphinx-doc/bin:$PATH"' >> ~/.bash_profile

==> Summary
🍺 /usr/local/Cellar/sphinx-doc/1.8.1: 2,695 files, 46.3MB
localhost:nix b$ cargo test
Compiling version_check v0.1.5
Compiling rand_core v0.3.0
Compiling libc v0.2.43
Compiling byteorder v1.2.6
Compiling remove_dir_all v0.5.1
Compiling rand_core v0.2.2
Compiling lazy_static v1.1.0
Compiling rand v0.5.5
Compiling iovec v0.1.2
Compiling bytes v0.4.10
Compiling tempfile v3.0.4
Compiling nix v0.11.0 (file:///Users/b/work/nix)
warning: unused macro definition
--> test/test_mount.rs:209:1
|
209 | / macro_rules! run_tests {
210 | | ( $($test_fn:ident),* ) => {{
211 | | println!();
212 | |
... |
220 | | }}
221 | | }
| |_^
|
= note: #[warn(unused_macros)] on by default

warning: unused macro definition
--> test/test.rs:12:1
|
12 | / macro_rules! skip_if_not_root {
13 | | ($name:expr) => {
14 | | use nix::unistd::Uid;
15 | | use std;
... |
24 | | };
25 | | }
| |_^
|
= note: #[warn(unused_macros)] on by default

warning: unused import: std::mem
--> test/sys/test_ptrace.rs:8:5
|
8 | use std::mem;
| ^^^^^^^^
|
= note: #[warn(unused_imports)] on by default

warning: unused import: nix::sys::signal::SigSet
--> test/test_poll.rs:2:5
|
2 | use nix::sys::signal::SigSet;
| ^^^^^^^^^^^^^^^^^^^^^^^^

warning: unused imports: TimeSpec, TimeValLike
--> test/test_poll.rs:3:22
|
3 | use nix::sys::time::{TimeSpec, TimeValLike};
| ^^^^^^^^ ^^^^^^^^^^^

warning: use of deprecated item 'nix::unistd::pipe2': pipe2(2) is not actually atomic on these platforms. Use pipe(2) and fcntl(2) instead
--> test/test_unistd.rs:386:22
|
386 | let (fd0, fd1) = pipe2(OFlag::O_CLOEXEC).unwrap();
| ^^^^^
|
= note: #[warn(deprecated)] on by default

warning: unused std::result::Result which must be used
--> test/test_stat.rs:177:5
|
177 | utimes(&fullpath, &TimeVal::seconds(9990), &TimeVal::seconds(5550));
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: #[warn(unused_must_use)] on by default
= note: this Result may be an Err variant, which should be handled

Finished dev [unoptimized + debuginfo] target(s) in 10.49s
 Running target/debug/deps/nix-f5796617f4709c01

running 35 tests
test sys::event::test_struct_kevent ... ok
test sys::select::tests::fdset_insert ... ok
test sys::select::tests::fdset_clear ... ok
test sys::select::tests::test_select ... ok
test sys::select::tests::test_select_nfds2 ... ok
test ifaddrs::tests::test_getifaddrs ... ok
test sys::select::tests::test_select_nfds ... ok
test sys::select::tests::fdset_remove ... ok
test sys::signal::tests::test_clear ... ok
test sys::signal::tests::test_contains ... ok
test sys::signal::tests::test_extend ... ok
test sys::signal::tests::test_from_str_invalid_value ... ok
test sys::select::tests::fdset_highest ... ok
test sys::signal::tests::test_sigaction ... ok
test sys::signal::tests::test_thread_signal_block ... ok
test sys::signal::tests::test_thread_signal_get_mask ... ok
test sys::signal::tests::test_from_str_round_trips ... ok
test sys::signal::tests::test_thread_signal_set_mask ... ok
test sys::signal::tests::test_thread_signal_swap ... ok
test sys::signal::tests::test_thread_signal_unblock ... ok
test sys::socket::addr::tests::test_macos_loopback_datalink_addr ... ok
test sys::socket::addr::tests::test_macos_tap_datalink_addr ... ok
test sys::socket::sockopt::test::is_socket_type_dgram ... ok
test sys::statvfs::test::fstatvfs_call ... ok
test sys::statvfs::test::statvfs_call ... ok
test sys::time::test::test_timespec ... ok
test sys::time::test::test_timespec_neg ... ok
test sys::time::test::test_timespec_ord ... ok
test sys::time::test::test_timespec_fmt ... ok
test sys::socket::sockopt::test::is_socket_type_unix ... ok
test sys::time::test::test_timeval ... ok
test sys::time::test::test_timeval_fmt ... ok
test sys::time::test::test_timeval_neg ... ok
test sys::time::test::test_timeval_ord ... ok
test sys::utsname::test::test_uname_darwin ... ok

test result: ok. 35 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

 Running target/debug/deps/test-fc2b5d0af9911a3b

running 98 tests
test sys::test_aio::test_accessors ... ok
test sys::test_aio::test_aio_suspend ... ok
test sys::test_aio::test_fsync_error ... ok
test sys::test_aio::test_read_error ... ok
test sys::test_aio::test_read_immutable_buffer ... ok
test sys::test_aio::test_cancel ... ok
test sys::test_aio::test_aio_cancel_all ... ok
test sys::test_aio::test_read_bytes_mut_small ... ok
test sys::test_aio::test_read ... ok
test sys::test_aio::test_fsync ... ok
test sys::test_aio::test_write_error ... ok
test sys::test_ioctl::bsd::test_op_none ... ok
test sys::test_ioctl::bsd::test_op_read ... ok
test sys::test_aio::test_read_into_mut_slice ... ok
test sys::test_aio::test_read_into_pointer ... ok
test sys::test_ioctl::bsd::test_op_read_64 ... ok
test sys::test_ioctl::bsd::test_op_read_write ... ok
test sys::test_ioctl::bsd::test_op_read_write_64 ... ok
test sys::test_ioctl::bsd::test_op_write ... ok
test sys::test_aio::test_write ... ok
test sys::test_ioctl::bsd::test_op_write_64 ... ok
test sys::test_pthread::test_pthread_self ... ok
test sys::test_ptrace::test_ptrace ... FAILED
test sys::test_signal::test_old_sigaction_flags ... ok
test sys::test_select::test_pselect_nfds2 ... ok
test sys::test_signal::test_kill_none ... ok
test sys::test_signal::test_sigprocmask_noop ... ok
test sys::test_ptrace::test_ptrace_cont ... FAILED
test sys::test_socket::test_inetv4_addr_to_sock_addr ... ok
test sys::test_socket::test_inetv6_addr_to_sock_addr ... ok
test sys::test_socket::test_path_to_sock_addr ... ok
test sys::test_socket::test_scm_rights ... ok
test sys::test_socket::test_scm_rights_single_cmsg_multiple_fds ... ok
test sys::test_socket::test_sendmsg_empty_cmsgs ... ok
test sys::test_socket::test_socketpair ... ok
test sys::test_socket::test_syscontrol ... ok
test sys::test_socket::test_getsockname ... ok
test sys::test_termios::test_cfmakeraw ... ok
test sys::test_sockopt::test_so_buf ... ok
test sys::test_termios::test_local_flags ... ok
test sys::test_termios::test_tcgetattr_ebadf ... ok
test sys::test_socket::test_unixdomain ... ok
test sys::test_termios::test_output_flags ... ok
test sys::test_termios::test_tcgetattr_pty ... ok
test sys::test_termios::test_tcgetattr_enotty ... ok
test sys::test_uio::test_pwrite ... ok
test sys::test_uio::test_pread ... ok
test sys::test_uio::test_readv ... ok
test sys::test_wait::test_wait_exit ... ok
test sys::test_wait::test_waitstatus_from_raw ... ok
test sys::test_wait::test_wait_signal ... ok
test sys::test_uio::test_writev ... ok
test test_dir::ebadf ... ok
test sys::test_aio::test_write_bytes ... ok
test sys::test_aio::test_write_from_pointer ... ok
test sys::test_wait::test_waitstatus_pid ... ok
test test_net::test_if_nametoindex ... ok
test sys::test_aio::test_write_sigev_signal ... ok
test sys::test_select::test_pselect ... ok
test test_dir::rewind ... ok
test test_fcntl::test_openat ... ok
test sys::test_signal::test_sigprocmask ... ok
test test_poll::test_poll_debug ... ok
test test_pty::test_open_ptty_pair ... ok
test test_fcntl::test_readlink ... ok
test test_pty::test_openpty ... ok
test test_dir::read ... ok
test test_pty::test_openpty_with_termios ... ok
test test_pty::test_explicit_close ... ok
test test_sendfile::test_sendfile_darwin ... ok
test test_stat::test_fchmodat ... ok
test test_stat::test_fstatat ... ok
test test_stat::test_fchmod ... ok
test test_stat::test_stat_and_fstat ... ok
test test_stat::test_futimens ... ok
test test_stat::test_stat_fstat_lstat ... ok
test test_stat::test_utimes ... ok
test test_stat::test_utimensat ... ok
test test_unistd::test_fchdir ... ok
test test_unistd::test_getpid ... ok
test test_unistd::test_fpathconf_limited ... ok
test test_unistd::test_getsid ... ok
test test_unistd::test_lseek ... ok
test test_unistd::test_mkfifo_directory ... ok
test test_unistd::test_mkfifo ... ok
test test_unistd::test_mkstemp_directory ... ok
test test_unistd::test_mkstemp ... ok
test test_unistd::test_pathconf_limited ... ok
test test_unistd::test_pipe ... ok
test test_unistd::test_pipe2 ... ok
test test_unistd::test_sysconf_limited ... ok
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
test test_unistd::test_getcwd ... ok
test test_unistd::test_execve ... ok
test test_unistd::test_fork_and_waitpid ... ok
test test_unistd::test_wait ... ok
test test_poll::test_poll ... ok
test test_unistd::test_alarm ... ok
test test_unistd::test_canceling_alarm ... ok

failures:

---- sys::test_ptrace::test_ptrace stdout ----
thread 'sys::test_ptrace::test_ptrace' panicked at 'assertion failed: err == Error::Sys(Errno::EPERM) || err == Error::Sys(Errno::ENOSYS)', test/sys/test_ptrace.rs:15:5

---- sys::test_ptrace::test_ptrace_cont stdout ----
thread 'sys::test_ptrace::test_ptrace_cont' panicked at 'called Result::unwrap() on an Err value: Sys(ENOTSUP)', libcore/result.rs:983:5

failures:
sys::test_ptrace::test_ptrace
sys::test_ptrace::test_ptrace_cont

test result: FAILED. 96 passed; 2 failed; 0 ignored; 0 measured; 0 filtered out

error: test failed, to rerun pass '--test test'
localhost:nix b$ git diff
diff --git a/src/sys/ptrace/bsd.rs b/src/sys/ptrace/bsd.rs
index ee9b6eb..45c8fa2 100644
--- a/src/sys/ptrace/bsd.rs
+++ b/src/sys/ptrace/bsd.rs
@@ -77,7 +77,7 @@ pub fn traceme() -> Result<()> {
///
/// Attaches to the process specified in pid, making it a tracee of the calling process.
pub fn attach(pid: Pid) -> Result<()> {

  • unsafe { ptrace_other(Request::PT_ATTACH, pid, ptr::null_mut(), 0).map(|_| ()) }
  • unsafe { ptrace_other(Request::PT_ATTACHEXC, pid, ptr::null_mut(), 0).map(|_| ()) }
    }

/// Detaches the current running process, as with ptrace(PT_DETACH, ...)
localhost:nix b$

@dataf3l

This comment has been minimized.

Copy link

dataf3l commented Oct 9, 2018

I guess I could setup a cron job to clone the repo and run the tests on this machine, like daily, if that is something you guys would find useful.

@xd009642

This comment has been minimized.

Copy link
Owner

xd009642 commented Oct 9, 2018

No that's alright I got it more working some point last night and affirmed that travis is fine for testing. Thanks for offering though! Once this is in and released I'll be able to apply some changes to tarpaulin and may ask you to test them

@dataf3l

This comment has been minimized.

Copy link

dataf3l commented Oct 9, 2018

@xd009642

This comment has been minimized.

Copy link
Owner

xd009642 commented Oct 22, 2018

There's now a tracking branch for this. https://github.com/xd009642/tarpaulin/tree/osx-support I don't yet expect it to work but the ptrace compiler errors should be gone (in theory)

@xd009642 xd009642 changed the title It doesn't build in OSX OSX support tracking issue Oct 23, 2018

@xd009642

This comment has been minimized.

Copy link
Owner

xd009642 commented Oct 23, 2018

So tarpaulin now builds but doesn't work on OSX. I'm going to edit your first comment to add a checklist of things that need doing so it actually shows up towards the issue progress and to generally keep things tidy. Gonna try and keep the original text in a collapsable markdown block for posterity though

@dataf3l

This comment has been minimized.

Copy link

dataf3l commented Oct 24, 2018

I like it because it makes my comment look more professional.
Screw the past, bring on the future! :)

@xd009642

This comment has been minimized.

Copy link
Owner

xd009642 commented Dec 5, 2018

So as an update since I've been working on this issue this week. I appear to have ported over most of the ptrace stuff launch a test add the breakpoints to instrument it but after initialising the test it times out.

This makes me think the trace isn't being passed down to the child threads and rustc_test does spawn a thread for each test so it can kill it if it runs too long. Hopefully, once I solve this issue I might see some coverage results come out (if I'm really lucky and everything else works to some degree).

@xd009642

This comment has been minimized.

Copy link
Owner

xd009642 commented Dec 7, 2018

@dataf3l it would be helpful if you could take the latest build off the osx-support branch and create the default lib project cargo new osxtest --lib run tarpaulin with verbose and then use ddbug on the test binary it attempts to process. After Launching test in the tarpaulin verbose printout it prints the path to the test binary.

Also, if the outputs big upload it in a text file or link to a gist

@andrewtj

This comment has been minimized.

Copy link

andrewtj commented Dec 13, 2018

@xd009642 I've just wandered in from "This Week in Rust" and have zero context for this issue, but I gave it a go:

$ ddbug osxtest-21111c0e5489441f
ERROR 2018-12-13T00:24:24Z: ddbug: osxtest-21111c0e5489441f: unable to locate debug file: dSYM not found

osxtest-21111c0e5489441f.dSYM exists in that folder for whatever it's worth.

@xd009642

This comment has been minimized.

Copy link
Owner

xd009642 commented Dec 13, 2018

Oh that's actually super useful! Could you run ddbug on the dsym file?

@xd009642

This comment has been minimized.

Copy link
Owner

xd009642 commented Dec 13, 2018

Awesome that's what I expected to see! I've been trying to debug and develop this via travis and apples online docs which has been a bit of a mare. I've pushed a change to the osx-support branch and if results come out we should have the first beta of OSX support.

If more work is needed (I have a feeling there is more) and you're interested in working on the tarpaulin code I've got a developer wiki that explains some of how the project works https://github.com/xd009642/tarpaulin/wiki/Developers if there are any missing parts or issues I do also have a tracking issue for the wiki since it's relatively new 👍

@andrewtj

This comment has been minimized.

Copy link

andrewtj commented Dec 14, 2018

That overview's given me the gist of how this is supposed to work 👍. At the minute osx-support fails at the mmap call because the path it's provided is a bundle (folder). With the change below to test_loader::open_symbols_file the mmap call succeeds:

 #[cfg(target_os = "macos")]
 fn open_symbols_file(test: &Path) -> io::Result<File> {
-    let d_sym = test.with_extension("dSYM");
+    let mut d_sym = test.with_extension("dSYM");
+    d_sym.push("Contents/Resources/DWARF");
+    d_sym.push(test.file_stem().unwrap());
     println!("Test file {}", d_sym.display());
     File::open(&d_sym)
 }

Which leads to:

[...]
Processing compilation units
Getting the entry points
Processed all compilation units
trace children -1
Failed to trace child threads
Number of traces 6
Finished instrumenting code

I guess this could be a privilege issue (codesigning/entitlements) or some macOS API deviation. I've got to shelf this for the moment but I'll try and circle back to this in the next few days.

@xd009642

This comment has been minimized.

Copy link
Owner

xd009642 commented Dec 14, 2018

So the mach-o loading side of thing looks like it's working with that change as it's found the traces from the DWARF tables. This is in the update loop actually attempting to trace. I think this is a case of src/process_handling/bsd.rs the trace_children function being incorrect as ptrace isn't tracing spawned threads.

Apple doesn't expose the constants but provides more ptrace functionality than they advertise so I hard coded it myself to see if it would work. So looks like back to the apple ptrace drawing board for that

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment