Skip to content

/rust

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Segfault update thread-local panic count on macOS in empty unit test #50867

Closed
jdm opened this Issue May 18, 2018 · 28 comments

Comments

Assignees
No one assigned
Labels
A-runtime C-bug O-macos T-compiler regression-from-stable-to-nightly
Projects
None yet
Milestone
No milestone
7 participants
@jdm @kennytm @goffrie @alexcrichton @SimonSapin @kamyuentse @cuviper
@jdm
Copy link
Contributor

jdm commented May 18, 2018

text_util.rs:

#[test]
fn test_transform_compress_none() {
}
$ rustup override set nightly-2018-04-30
info: using existing install for 'nightly-2018-04-30-x86_64-apple-darwin'
info: override toolchain for '/Users/servo/test' set to 'nightly-2018-04-30-x86_64-apple-darwin'
$ rustc text_util.rs --test
$ ./text_util

running 1 test
test test_transform_compress_none ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

$ rustup override set nightly-2018-05-03
info: using existing install for 'nightly-2018-05-03-x86_64-apple-darwin'
info: override toolchain for '/Users/servo/test' set to 'nightly-2018-05-03-x86_64-apple-darwin'

  nightly-2018-05-03-x86_64-apple-darwin unchanged - rustc 1.27.0-nightly (8a37c75a3 2018-05-02)

server:test servo$ rustc text_util.rs --test
server:test servo$ ./text_util

running 1 test
Segmentation fault: 11

@jdm jdm added the regression-from-stable-to-nightly label May 18, 2018

@jdm

This comment has been minimized.

Sign in to view
Copy link
Contributor Author

jdm commented May 18, 2018 

# lldb /Users/servo/test/text_util
(lldb) target create "/Users/servo/test/text_util"
2018-05-18 12:50:51.381 lldb[48794:3611387704] Metadata.framework [Error]: couldn't get the client port
Current executable set to '/Users/servo/test/text_util' (x86_64).
(lldb) r
Process 49139 launched: '/Users/servo/test/text_util' (x86_64)

running 1 test
Process 49139 stopped
* thread #2: tid = 0xd741685b, 0x000000010005d312 text_util`std::panicking::panicking::hbf60d20c3ea01bc0 [inlined] core::ptr::swap_nonoverlapping_bytes::h5ceec518bedbc8b5 + 28 at ptr.rs:237, name = 'test_transform_compress_none', stop reason = EXC_BAD_ACCESS (code=EXC_I386_GPFLT)
    frame #0: 0x000000010005d312 text_util`std::panicking::panicking::hbf60d20c3ea01bc0 [inlined] core::ptr::swap_nonoverlapping_bytes::h5ceec518bedbc8b5 + 28 at ptr.rs:237
(lldb) bt
* thread #2: tid = 0xd741685b, 0x000000010005d312 text_util`std::panicking::panicking::hbf60d20c3ea01bc0 [inlined] core::ptr::swap_nonoverlapping_bytes::h5ceec518bedbc8b5 + 28 at ptr.rs:237, name = 'test_transform_compress_none', stop reason = EXC_BAD_ACCESS (code=EXC_I386_GPFLT)
  * frame #0: 0x000000010005d312 text_util`std::panicking::panicking::hbf60d20c3ea01bc0 [inlined] core::ptr::swap_nonoverlapping_bytes::h5ceec518bedbc8b5 + 28 at ptr.rs:237
    frame #1: 0x000000010005d2f6 text_util`std::panicking::panicking::hbf60d20c3ea01bc0 [inlined] core::ptr::swap_nonoverlapping::h135f714f781a3d4a at ptr.rs:187
    frame #2: 0x000000010005d2f6 text_util`std::panicking::panicking::hbf60d20c3ea01bc0 [inlined] core::mem::swap::ha8a5fb0906ce1619 at mem.rs:634
    frame #3: 0x000000010005d2f6 text_util`std::panicking::panicking::hbf60d20c3ea01bc0 [inlined] core::mem::replace::hc5999657bde25f00 at mem.rs:691
    frame #4: 0x000000010005d2f6 text_util`std::panicking::panicking::hbf60d20c3ea01bc0 [inlined] _$LT$std..thread..local..LocalKey$LT$T$GT$$GT$::init::h03c530c70809d764 at local.rs:270
    frame #5: 0x000000010005d2f6 text_util`std::panicking::panicking::hbf60d20c3ea01bc0 [inlined] _$LT$std..thread..local..LocalKey$LT$T$GT$$GT$::try_with::h1ddab9e8e4e02c71 + 34 at local.rs:296
    frame #6: 0x000000010005d2d4 text_util`std::panicking::panicking::hbf60d20c3ea01bc0 [inlined] _$LT$std..thread..local..LocalKey$LT$T$GT$$GT$::with::h7ad9abd4ee126f73 at local.rs:248
    frame #7: 0x000000010005d2d4 text_util`std::panicking::panicking::hbf60d20c3ea01bc0 [inlined] std::panicking::update_panic_count::h8ff9c302601a529d at panicking.rs:240
    frame #8: 0x000000010005d2d4 text_util`std::panicking::panicking::hbf60d20c3ea01bc0 + 4 at panicking.rs:317
    frame #9: 0x00000001000176b3 text_util`std::sys_common::backtrace::__rust_begin_short_backtrace::h5ce7af51dff78437 [inlined] std::thread::panicking::ha8ec11437d5dda69 + 563 at mod.rs:650
    frame #10: 0x00000001000176ae text_util`std::sys_common::backtrace::__rust_begin_short_backtrace::h5ce7af51dff78437 [inlined] std::sys_common::poison::Flag::borrow::h4e2e930fbcd29fe9 at poison.rs:36
    frame #11: 0x00000001000176ae text_util`std::sys_common::backtrace::__rust_begin_short_backtrace::h5ce7af51dff78437 [inlined] _$LT$std..sync..mutex..MutexGuard$LT$$u27$mutex$C$$u20$T$GT$$GT$::new::hf958532833c3eeee at mutex.rs:426
    frame #12: 0x00000001000176ae text_util`std::sys_common::backtrace::__rust_begin_short_backtrace::h5ce7af51dff78437 [inlined] _$LT$std..sync..mutex..Mutex$LT$T$GT$$GT$::lock::h08aa85ea884e0a79 + 9 at mutex.rs:231
    frame #13: 0x00000001000176a5 text_util`std::sys_common::backtrace::__rust_begin_short_backtrace::h5ce7af51dff78437 [inlined] test::run_test::run_test_inner::_$u7b$$u7b$closure$u7d$$u7d$::h293891f148241b3f + 508 at lib.rs:1414
    frame #14: 0x00000001000174a9 text_util`std::sys_common::backtrace::__rust_begin_short_backtrace::h5ce7af51dff78437 + 41 at backtrace.rs:136
    frame #15: 0x000000010002e6d8 text_util`std::panicking::try::do_call::h317793a359ef8eae [inlined] std::thread::Builder::spawn::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::hce2a129863ba037b + 40 at mod.rs:409
    frame #16: 0x000000010002e6c2 text_util`std::panicking::try::do_call::h317793a359ef8eae [inlined] _$LT$std..panic..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once::ha38aec20aaf8023b at panic.rs:305
    frame #17: 0x000000010002e6c2 text_util`std::panicking::try::do_call::h317793a359ef8eae + 18 at panicking.rs:310
    frame #18: 0x0000000100068f9f text_util`__rust_maybe_catch_panic + 31 at lib.rs:105
    frame #19: 0x0000000100018fa5 text_util`_$LT$F$u20$as$u20$alloc..boxed..FnBox$LT$A$GT$$GT$::call_box::hbda32c8bc14400fa [inlined] std::panicking::try::hd50d66bdaa9b80f6 + 57 at panicking.rs:289
    frame #20: 0x0000000100018f6c text_util`_$LT$F$u20$as$u20$alloc..boxed..FnBox$LT$A$GT$$GT$::call_box::hbda32c8bc14400fa [inlined] std::panic::catch_unwind::h23f85cf47b3beb69 + 7 at panic.rs:374
    frame #21: 0x0000000100018f65 text_util`_$LT$F$u20$as$u20$alloc..boxed..FnBox$LT$A$GT$$GT$::call_box::hbda32c8bc14400fa [inlined] std::thread::Builder::spawn::_$u7b$$u7b$closure$u7d$$u7d$::hb9992794009509d0 + 55 at mod.rs:408
    frame #22: 0x0000000100018f2e text_util`_$LT$F$u20$as$u20$alloc..boxed..FnBox$LT$A$GT$$GT$::call_box::hbda32c8bc14400fa + 46 at boxed.rs:638
    frame #23: 0x000000010005a178 text_util`std::sys_common::thread::start_thread::h3a2a1586806f3926 [inlined] _$LT$alloc..boxed..Box$LT$alloc..boxed..FnBox$LT$A$C$$u20$Output$u3d$R$GT$$u20$$u2b$$u20$$u27$a$GT$$u20$as$u20$core..ops..function..FnOnce$LT$A$GT$$GT$::call_once::he272aed47e96b55c + 136 at boxed.rs:648
    frame #24: 0x000000010005a175 text_util`std::sys_common::thread::start_thread::h3a2a1586806f3926 + 133 at thread.rs:24
    frame #25: 0x000000010004c0a9 text_util`std::sys::unix::thread::Thread::new::thread_start::h657bd8405a5d884c + 9 at thread.rs:90
    frame #26: 0x00007fff95aed05a libsystem_pthread.dylib`_pthread_body + 131
    frame #27: 0x00007fff95aecfd7 libsystem_pthread.dylib`_pthread_start + 176
    frame #28: 0x00007fff95aea3ed libsystem_pthread.dylib`thread_start + 13

This was referenced May 18, 2018

Merged
Open
@jdm

This comment has been minimized.

Sign in to view
Copy link
Contributor Author

jdm commented May 18, 2018

There are no nightlies between 4-30 and 5-03, unfortunately. Based on the commit log from github, I believe 221b7ca...58eeeb4 is the range of commits that could be responsible.

@kennytm kennytm added the O-macos label May 18, 2018

@kennytm

This comment has been minimized.

Sign in to view
Copy link
Member

kennytm commented May 18, 2018 •
edited

I can't reproduce the error with rustc 1.27.0-nightly (2f2a11dfc 2018-05-16).

Edit: No repro with the mentioned nightly either.

rustc 1.27.0-nightly (8a37c75a3 2018-05-02)

running 1 test
test test_transform_compress_none ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
@jdm

This comment has been minimized.

Sign in to view
Copy link
Contributor Author

jdm commented May 18, 2018 •
edited

It reproduces consistently on three particular macOS build machines that Servo uses. I can't actually reproduce it locally. I am attempting to bisect rustc on an affected machine to figure out the cause.
@kennytm

This comment has been minimized.

Sign in to view
Copy link
Member

kennytm commented May 18, 2018

@jdm What are the macOS and Xcode versions of those build machines? Maybe it's caused by Xcode 7 again.
@jdm

This comment has been minimized.

Sign in to view
Copy link
Contributor Author

jdm commented May 18, 2018 

$ uname -a
Darwin server.blueoceanstudios.us 14.5.0 Darwin Kernel Version 14.5.0: Sun Sep 25 22:07:15 PDT 2016; root:xnu-2782.50.9~1/RELEASE_X86_64 x86_64
$ xcodebuild -version
Xcode 7.2.1
Build version 7C1002
@kennytm

This comment has been minimized.

Sign in to view
Copy link
Member

kennytm commented May 18, 2018

(Darwin 14.5.0 = OS X Yosemite 10.10.5)

@cuviper cuviper added A-runtime T-compiler C-bug labels May 19, 2018

@goffrie

This comment has been minimized.

Sign in to view
Copy link
Contributor

goffrie commented May 19, 2018

@jdm what instruction is it failing on? My guess is it's a movdqa, which would make this the same issue as e.g. #48866, caused by Xcode <8 having broken TLS alignment.
@jdm

This comment has been minimized.

Sign in to view
Copy link
Contributor Author

jdm commented May 19, 2018 •
edited 

(lldb) dis
text_util`std::panicking::panicking::hbf60d20c3ea01bc0:
    0x10005c880 <+0>:  pushq  %rbp
    0x10005c881 <+1>:  movq   %rsp, %rbp
    0x10005c884 <+4>:  subq   $0x10, %rsp
    0x10005c888 <+8>:  leaq   0x6a349(%rip), %rdi       ; std::panicking::update_panic_count::PANIC_COUNT::__getit::__KEY::hfd55c7d72b45f2cb
    0x10005c88f <+15>: callq  *(%rdi)
    0x10005c891 <+17>: cmpq   $0x1, (%rax)
    0x10005c895 <+21>: jne    0x10005c8a6               ; <+38> [inlined] core::ptr::swap_nonoverlapping_bytes::h5ceec518bedbc8b5 at ptr.rs:187
    0x10005c897 <+23>: leaq   0x6a33a(%rip), %rdi       ; std::panicking::update_panic_count::PANIC_COUNT::__getit::__KEY::hfd55c7d72b45f2cb
    0x10005c89e <+30>: callq  *(%rdi)
    0x10005c8a0 <+32>: movq   0x8(%rax), %rcx
    0x10005c8a4 <+36>: jmp    0x10005c8c7               ; <+71> [inlined] core::ptr::swap_nonoverlapping_bytes::h5ceec518bedbc8b5 at ptr.rs:187
    0x10005c8a6 <+38>: movl   $0x1, %eax
    0x10005c8ab <+43>: movd   %rax, %xmm0
    0x10005c8b0 <+48>: movdqa %xmm0, -0x10(%rbp)
    0x10005c8b5 <+53>: leaq   0x6a31c(%rip), %rdi       ; std::panicking::update_panic_count::PANIC_COUNT::__getit::__KEY::hfd55c7d72b45f2cb
    0x10005c8bc <+60>: callq  *(%rdi)
    0x10005c8be <+62>: movaps -0x10(%rbp), %xmm0
->  0x10005c8c2 <+66>: movaps %xmm0, (%rax)
    0x10005c8c5 <+69>: xorl   %ecx, %ecx
    0x10005c8c7 <+71>: leaq   0x6a30a(%rip), %rdi       ; std::panicking::update_panic_count::PANIC_COUNT::__getit::__KEY::hfd55c7d72b45f2cb
    0x10005c8ce <+78>: callq  *(%rdi)
    0x10005c8d0 <+80>: movq   %rcx, 0x8(%rax)
    0x10005c8d4 <+84>: testq  %rcx, %rcx
    0x10005c8d7 <+87>: setne  %al
    0x10005c8da <+90>: addq   $0x10, %rsp
    0x10005c8de <+94>: popq   %rbp
    0x10005c8df <+95>: retq
(lldb) x $rax
0x1003000c8: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
0x1003000d8: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................

@SimonSapin SimonSapin referenced this issue May 21, 2018

Closed

Tracking issue for upgrading rustc Nightly #20844

@alexcrichton

This comment has been minimized.

Sign in to view
Copy link
Member

alexcrichton commented May 21, 2018

FWIW this historically has looked to be an issue with OSX's linker. Newer versions of Xcode appear to have a fixed linker as this bug doesn't manifest when Rust programs are linked with newer versions of Xcode
@SimonSapin

This comment has been minimized.

Sign in to view
Copy link
Contributor

SimonSapin commented May 27, 2018 •
edited

While the underlying bug might be in the linker, programs that used to compile correctly with an older rustc version don’t anymore. https://forge.rust-lang.org/platform-support.html lists "OSX (10.7+, Lion+)" as tier 1, but XCode 8 is only available on 10.11.5+. Are we dropping support for older versions?
@alexcrichton

This comment has been minimized.

Sign in to view
Copy link
Member

alexcrichton commented May 27, 2018

@SimonSapin oh I'm not saying we shouldn't fix this, it's moreso that at this point in time no one knows how to fix this other than (a) using an updated Xcode or (b) compiling with MACOSX_DEPLOYMENT_TARGET=10.6. If there's a better fix we'd definitely add it!
@SimonSapin

This comment has been minimized.

Sign in to view
Copy link
Contributor

SimonSapin commented May 28, 2018

(b) compiling with MACOSX_DEPLOYMENT_TARGET=10.6

This sounds very very good to know, thanks!

It’s much easier than upgrading the OS on Servo’s CI. I’ll try it today.
@SimonSapin

This comment has been minimized.

Sign in to view
Copy link
Contributor

SimonSapin commented May 28, 2018

As to knowing how to fix it, bisecting to a single PR would be a start. Is there an easier way to use CI builds than forging S3 URLs and extracting a couple tarballs manually?
@kennytm

This comment has been minimized.

Sign in to view
Copy link
Member

kennytm commented May 28, 2018

@SimonSapin Both cargo-bisect-rustc and bisect-rust allow bisecting by commit hash, though if the regression happens 90 to 168 days ago the tarballs may have already been deleted from S3.
@SimonSapin

This comment has been minimized.

Sign in to view
Copy link
Contributor

SimonSapin commented May 28, 2018 •
edited 

MACOSX_DEPLOYMENT_TARGET=10.6 rustc +nightly-2018-05-03 --test text_util.rs

error: linking with `cc` failed: exit code: 1
  |
  = note: "cc" "-m64" "-L" "/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib" "text_util.text_util0.rcgu.o" "text_util.text_util1.rcgu.o" "text_util.text_util2.rcgu.o" "text_util.text_util3.rcgu.o" "text_util.text_util4.rcgu.o" "text_util.text_util5.rcgu.o" "text_util.text_util6.rcgu.o" "text_util.text_util7.rcgu.o" "text_util.text_util8.rcgu.o" "-o" "text_util" "text_util.crate.allocator.rcgu.o" "-Wl,-dead_strip" "-nodefaultlibs" "-L" "/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib" "/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libtest-84ff69afa4c635e2.rlib" "/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libterm-5f0300267beca837.rlib" "/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libgetopts-daf9ce0941a89e3a.rlib" "/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libstd-1bc2364238ee1293.rlib" "/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libpanic_unwind-aea109835d0cd8df.rlib" "/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc_jemalloc-175b8696befeaef5.rlib" "/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libunwind-5938b59795070128.rlib" "/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc_system-df19d11d01029b6d.rlib" "/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liblibc-4f078042f251608d.rlib" "/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc-62469b0229d06da4.rlib" "/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libcore-372f0f2eba567b16.rlib" "/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libcompiler_builtins-4ac07626707bbf32.rlib" "-l" "System" "-l" "resolv" "-l" "pthread" "-l" "c" "-l" "m"
  = note: ld: warning: object file (/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc_jemalloc-175b8696befeaef5.rlib(jemalloc.pic.o)) was built for newer OSX version (10.7) than being linked (10.6)
          ld: warning: object file (/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libstd-1bc2364238ee1293.rlib(state.o)) was built for newer OSX version (10.7) than being linked (10.6)
          ld: warning: object file (/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libstd-1bc2364238ee1293.rlib(fileline.o)) was built for newer OSX version (10.7) than being linked (10.6)
          ld: warning: object file (/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc_jemalloc-175b8696befeaef5.rlib(arena.pic.o)) was built for newer OSX version (10.7) than being linked (10.6)
          ld: warning: object file (/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc_jemalloc-175b8696befeaef5.rlib(base.pic.o)) was built for newer OSX version (10.7) than being linked (10.6)
          ld: warning: object file (/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc_jemalloc-175b8696befeaef5.rlib(chunk.pic.o)) was built for newer OSX version (10.7) than being linked (10.6)
          ld: warning: object file (/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc_jemalloc-175b8696befeaef5.rlib(chunk_dss.pic.o)) was built for newer OSX version (10.7) than being linked (10.6)
          ld: warning: object file (/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc_jemalloc-175b8696befeaef5.rlib(ctl.pic.o)) was built for newer OSX version (10.7) than being linked (10.6)
          ld: warning: object file (/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc_jemalloc-175b8696befeaef5.rlib(huge.pic.o)) was built for newer OSX version (10.7) than being linked (10.6)
          ld: warning: object file (/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc_jemalloc-175b8696befeaef5.rlib(mutex.pic.o)) was built for newer OSX version (10.7) than being linked (10.6)
          ld: warning: object file (/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc_jemalloc-175b8696befeaef5.rlib(util.pic.o)) was built for newer OSX version (10.7) than being linked (10.6)
          ld: warning: object file (/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc_jemalloc-175b8696befeaef5.rlib(tsd.pic.o)) was built for newer OSX version (10.7) than being linked (10.6)
          ld: warning: object file (/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc_jemalloc-175b8696befeaef5.rlib(tcache.pic.o)) was built for newer OSX version (10.7) than being linked (10.6)
          ld: warning: object file (/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc_jemalloc-175b8696befeaef5.rlib(stats.pic.o)) was built for newer OSX version (10.7) than being linked (10.6)
          ld: warning: object file (/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc_jemalloc-175b8696befeaef5.rlib(pages.pic.o)) was built for newer OSX version (10.7) than being linked (10.6)
          ld: warning: object file (/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc_jemalloc-175b8696befeaef5.rlib(prof.pic.o)) was built for newer OSX version (10.7) than being linked (10.6)
          ld: warning: object file (/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc_jemalloc-175b8696befeaef5.rlib(quarantine.pic.o)) was built for newer OSX version (10.7) than being linked (10.6)
          ld: warning: object file (/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc_jemalloc-175b8696befeaef5.rlib(witness.pic.o)) was built for newer OSX version (10.7) than being linked (10.6)
          ld: warning: object file (/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libstd-1bc2364238ee1293.rlib(alloc.o)) was built for newer OSX version (10.7) than being linked (10.6)
          ld: warning: object file (/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libstd-1bc2364238ee1293.rlib(macho.o)) was built for newer OSX version (10.7) than being linked (10.6)
          ld: warning: object file (/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libstd-1bc2364238ee1293.rlib(posix.o)) was built for newer OSX version (10.7) than being linked (10.6)
          ld: warning: object file (/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc_jemalloc-175b8696befeaef5.rlib(bitmap.pic.o)) was built for newer OSX version (10.7) than being linked (10.6)
          ld: warning: object file (/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc_jemalloc-175b8696befeaef5.rlib(chunk_mmap.pic.o)) was built for newer OSX version (10.7) than being linked (10.6)
          ld: warning: object file (/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc_jemalloc-175b8696befeaef5.rlib(ckh.pic.o)) was built for newer OSX version (10.7) than being linked (10.6)
          ld: warning: object file (/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc_jemalloc-175b8696befeaef5.rlib(extent.pic.o)) was built for newer OSX version (10.7) than being linked (10.6)
          ld: warning: object file (/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc_jemalloc-175b8696befeaef5.rlib(nstime.pic.o)) was built for newer OSX version (10.7) than being linked (10.6)
          ld: warning: object file (/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc_jemalloc-175b8696befeaef5.rlib(rtree.pic.o)) was built for newer OSX version (10.7) than being linked (10.6)
          ld: warning: object file (/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libstd-1bc2364238ee1293.rlib(dwarf.o)) was built for newer OSX version (10.7) than being linked (10.6)
          ld: warning: object file (/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libstd-1bc2364238ee1293.rlib(read.o)) was built for newer OSX version (10.7) than being linked (10.6)
          ld: warning: object file (/Users/servo/.rustup/toolchains/nightly-2018-05-03-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libstd-1bc2364238ee1293.rlib(sort.o)) was built for newer OSX version (10.7) than being linked (10.6)
          ld: targeted OS version does not support use of thread local variables in __ZN3std11collections4hash3map11RandomState3new4KEYS7__getit17hd8ede3cfb6c10733E for architecture x86_64
          clang: error: linker command failed with exit code 1 (use -v to see invocation)
          

error: aborting due to previous error
@SimonSapin

This comment has been minimized.

Sign in to view
Copy link
Contributor

SimonSapin commented May 28, 2018 

$ cat issue50867/src/main.rs 
fn main() {
    println!("Hello, world!");
}

#[test]
fn test_transform_compress_none() {
}
$ rustc +nightly-2018-04-30 -Vv|grep hash
commit-hash: 79252ff4e25d82f9fe856cb66f127b79cdace163
$ rustc +nightly-2018-05-03 -Vv|grep hash         
commit-hash: 8a37c75a3a661385cc607d934c70e86a9eaf5fd7

79252ff...8a37c75

$ cargo +nightly run --manifest-path cargo-bisect-rustc/Cargo.toml -- \
  --start 79252ff4e25d82f9fe856cb66f127b79cdace163 \
  --end 8a37c75a3a661385cc607d934c70e86a9eaf5fd7 \
  --test-dir issue50867/ -- test
[…]
searched toolchains 79252ff4e25d82f9fe856cb66f127b79cdace163 through 8a37c75a3a661385cc607d934c70e86a9eaf5fd7
regression in 5f3994f58f41818bf31efeda6af41e8060978ca5

5f3994f is #50329, "Set opt-level to 3"
@alexcrichton

This comment has been minimized.

Sign in to view
Copy link
Member

alexcrichton commented May 28, 2018

Awesome thanks for the bisection @SimonSapin! Also cc @Zoxc for that commit.

For the build error you had above, was that a clean build? I think you'll have to recompile some of the C objects.

For the bisection result my guess would be that it's a legitimate optimization being performed which the linker or something along the way doesn't handle. I'm not sure what though? I would personally be ok backing out the opt-level=3 change as well.
@SimonSapin

This comment has been minimized.

Sign in to view
Copy link
Contributor

SimonSapin commented May 28, 2018

The libstd-[…] was built for newer OSX version (10.7) than being linked (10.6) / targeted OS version does not support use of thread local variables in __ZN3std[…] error above was when running rustc directly, without Cargo (with MACOSX_DEPLOYMENT_TARGET=10.6).
@SimonSapin

This comment has been minimized.

Sign in to view
Copy link
Contributor

SimonSapin commented May 28, 2018

I would personally be ok backing out the opt-level=3 change as well.

I think this would probably work around the bug, but adding a test so that it doesn’t regress again is tricky since the Travis-CI config uses newer XCode than versions that are affected according to #50867 (comment).

SimonSapin added a commit to SimonSapin/rust that referenced this issue May 29, 2018

@SimonSapin
Revert "Set opt-level to 3" 
This reverts commit aad9840.

Level 3 (possibly indirectly, the underlying bug might be in XCode’s linker)
causes unit tests to sefault when compiled with some versions of XCode:
rust-lang#50867

It also appears to cause some segfaults on Windows:
rust-lang#50329 (comment)

… and regressions in some rustc performance benchmarks:
rust-lang#50329 (comment)
01e8616

@SimonSapin SimonSapin referenced this issue May 29, 2018

Merged

Revert "Set opt-level to 3" #51165

SimonSapin added a commit to SimonSapin/rust that referenced this issue May 29, 2018

@SimonSapin
Change the comment on `opt-level = 2` to point to rust-lang#50867
5067d2f
@kennytm

This comment has been minimized.

Sign in to view
Copy link
Member

kennytm commented May 29, 2018 •
edited

Note: On LLVM 6, -O3 adds two additional passes to -O2:

Maybe we could isolate to one of these passes but we need to find a faster reproducer besides recompiling the whole compiler to stage1.

bors added a commit that referenced this issue May 29, 2018

@bors
Auto merge of #51165 - SimonSapin:opt2, r=alexcrichton 
Revert "Set opt-level to 3"

This reverts commit aad9840.

Level 3 (possibly indirectly, the underlying bug might be in XCode’s linker) causes unit tests to sefault when compiled with some versions of XCode: #50867

It also appears to cause some segfaults on Windows: #50329 (comment), and regressions in some rustc performance benchmarks: #50329 (comment)
Loading status checks…
524ad9b
@SimonSapin

This comment has been minimized.

Sign in to view
Copy link
Contributor

SimonSapin commented May 30, 2018

Reverting in #51165 has worked around the issue for now. Should we keep this open for finding a more targeted work-around?

$ cargo +nightly-2018-04-30 test --manifest-path issue50867/Cargo.toml  
    Finished dev [unoptimized + debuginfo] target(s) in 0.35 secs
     Running issue50867/target/debug/deps/issue50867-89d6e487ff6882a0

running 1 test
test test_transform_compress_none ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

$ cargo +nightly-2018-05-03 test --manifest-path issue50867/Cargo.toml 
    Finished dev [unoptimized + debuginfo] target(s) in 0.38 secs
     Running issue50867/target/debug/deps/issue50867-eff7d2b2573ec7f4

running 1 test
error: process didn't exit successfully: `/Users/servo/simon/simon/issue50867/target/debug/deps/issue50867-eff7d2b2573ec7f4` (signal: 11, SIGSEGV: invalid memory reference)

$ cargo +nightly-2018-05-30 test --manifest-path issue50867/Cargo.toml              
    Finished dev [unoptimized + debuginfo] target(s) in 0.12s
     Running issue50867/target/debug/deps/issue50867-efe70058af46c68e

running 1 test
test test_transform_compress_none ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
@alexcrichton

This comment has been minimized.

Sign in to view
Copy link
Member

alexcrichton commented May 30, 2018

Can you reproduce on today's nightly on older OSX machines when compiling with LTO? If so I think that's definitely a reason to keep this issue
@SimonSapin

This comment has been minimized.

Sign in to view
Copy link
Contributor

SimonSapin commented May 30, 2018

Adding this:

[profile.test]
opt-level = 3
lto = true

does cause segfaults to come back:

$ cargo +nightly-2018-05-30 test --manifest-path issue50867/Cargo.toml --release -v
   Compiling issue50867 v0.1.0 (file:///Users/servo/simon/simon/issue50867)
     Running `rustc --crate-name issue50867 src/main.rs --emit=dep-info,link -C opt-level=3 --test -C metadata=4a4ef17a80c6bc85 -C extra-filename=-4a4ef17a80c6bc85 --out-dir /Users/servo/simon/simon/issue50867/target/release/deps -L dependency=/Users/servo/simon/simon/issue50867/target/release/deps`
    Finished release [optimized] target(s) in 0.36s
     Running `/Users/servo/simon/simon/issue50867/target/release/deps/issue50867-4a4ef17a80c6bc85`

running 1 test
test test_transform_compress_none ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

But I’m not seeing anything in the rustc command for LTO…
@alexcrichton

This comment has been minimized.

Sign in to view
Copy link
Member

alexcrichton commented May 30, 2018

It's weird but you'll want to configure the bench profile
@SimonSapin

This comment has been minimized.

Sign in to view
Copy link
Contributor

SimonSapin commented May 30, 2018

Alright, this time I get -C lto but still no segfault:

   Compiling issue50867 v0.1.0 (file:///Users/servo/simon/simon/issue50867)
     Running `rustc --crate-name issue50867 src/main.rs --emit=dep-info,link -C opt-level=3 -C lto --test -C metadata=c4cb84c5ba2c6b20 -C extra-filename=-c4cb84c5ba2c6b20 --out-dir /Users/servo/simon/simon/issue50867/target/release/deps -L dependency=/Users/servo/simon/simon/issue50867/target/release/deps`
    Finished release [optimized] target(s) in 8.83s
     Running `/Users/servo/simon/simon/issue50867/target/release/deps/issue50867-c4cb84c5ba2c6b20`

running 1 test
test test_transform_compress_none ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
@alexcrichton

This comment has been minimized.

Sign in to view
Copy link
Member

alexcrichton commented May 30, 2018

bah! In that case seems safe to close as we can't reproduce easily then

@alexcrichton alexcrichton closed this May 30, 2018

@kamyuentse

This comment has been minimized.

Sign in to view
Copy link

kamyuentse commented May 31, 2018

I encountered a similar situation on macOS with the XCode 9.3 and rustc 1.28.0-nightly (5d0631a64 2018-05-30)

(lldb) bt
* thread #2, name = 'bench_httparse_short', stop reason = EXC_BAD_ACCESS (code=EXC_I386_GPFLT)
  * frame #0: 0x0000000100050217 parse-45f0c687ce1cb6ef`std::sys_common::thread_info::set::h653837fe234bdc81 [inlined] core::ptr::swap_nonoverlapping_bytes::h0b169aa8ef3bf4f2 at ptr.rs:221 [opt]
    frame #1: 0x000000010005020e parse-45f0c687ce1cb6ef`std::sys_common::thread_info::set::h653837fe234bdc81 [inlined] core::ptr::swap_nonoverlapping::hca32daf491eb7d79 at ptr.rs:187 [opt]
    frame #2: 0x000000010005020e parse-45f0c687ce1cb6ef`std::sys_common::thread_info::set::h653837fe234bdc81 [inlined] core::mem::swap::he62a32c44c5d3a12 at mem.rs:634 [opt]
    frame #3: 0x000000010005020e parse-45f0c687ce1cb6ef`std::sys_common::thread_info::set::h653837fe234bdc81 [inlined] core::mem::replace::hee354da570f8b6a1 at mem.rs:691 [opt]
    frame #4: 0x000000010005020e parse-45f0c687ce1cb6ef`std::sys_common::thread_info::set::h653837fe234bdc81 [inlined] _$LT$std..thread..local..LocalKey$LT$T$GT$$GT$::init::h8241a6fabfe63004 at local.rs:270 [opt]
    frame #5: 0x000000010005020e parse-45f0c687ce1cb6ef`std::sys_common::thread_info::set::h653837fe234bdc81 [inlined] _$LT$std..thread..local..LocalKey$LT$T$GT$$GT$::try_with::h019266c217631df0 at local.rs:296 [opt]
    frame #6: 0x00000001000501ba parse-45f0c687ce1cb6ef`std::sys_common::thread_info::set::h653837fe234bdc81 [inlined] _$LT$std..thread..local..LocalKey$LT$T$GT$$GT$::with::h977b626335ede091 at local.rs:248 [opt]
    frame #7: 0x00000001000501ba parse-45f0c687ce1cb6ef`std::sys_common::thread_info::set::h653837fe234bdc81 at thread_info.rs:47 [opt]
    frame #8: 0x000000010002cb88 parse-45f0c687ce1cb6ef`_$LT$F$u20$as$u20$alloc..boxed..FnBox$LT$A$GT$$GT$::call_box::h6f23826ab9e7d513 [inlined] std::thread::Builder::spawn::_$u7b$$u7b$closure$u7d$$u7d$::h0bb9a60731c9f04c at mod.rs:406 [opt]
    frame #9: 0x000000010002cb1b parse-45f0c687ce1cb6ef`_$LT$F$u20$as$u20$alloc..boxed..FnBox$LT$A$GT$$GT$::call_box::h6f23826ab9e7d513 at boxed.rs:638 [opt]
    frame #10: 0x00000001000422b8 parse-45f0c687ce1cb6ef`std::sys::unix::thread::Thread::new::thread_start::h0c1394726addc699 [inlined] _$LT$alloc..boxed..Box$LT$alloc..boxed..FnBox$LT$A$C$$u20$Output$u3d$R$GT$$u20$$u2b$$u20$$u27$a$GT$$u20$as$u20$core..ops..function..FnOnce$LT$A$GT$$GT$::call_once::hbdc518a2bdf15123 at boxed.rs:648 [opt]
    frame #11: 0x00000001000422b5 parse-45f0c687ce1cb6ef`std::sys::unix::thread::Thread::new::thread_start::h0c1394726addc699 [inlined] std::sys_common::thread::start_thread::h8d1134075e93d8df at thread.rs:24 [opt]
    frame #12: 0x0000000100042240 parse-45f0c687ce1cb6ef`std::sys::unix::thread::Thread::new::thread_start::h0c1394726addc699 at thread.rs:90 [opt]
    frame #13: 0x00007fff5ef8d661 libsystem_pthread.dylib`_pthread_body + 340
    frame #14: 0x00007fff5ef8d50d libsystem_pthread.dylib`_pthread_start + 377
    frame #15: 0x00007fff5ef8cbf9 libsystem_pthread.dylib`thread_start + 13

    0x10005020c <+108>: jne    0x10005026e               ; <+206> [inlined] _$LT$core..cell..Cell$LT$T$GT$$GT$::get::h6870a5734ed219a9 at cell.rs:1013
    0x10005020e <+110>: leaq   0x5f513(%rip), %rdi       ; std::sys_common::thread_info::THREAD_INFO::__getit::__KEY::hc2d4dd6884ed9a0e
    0x100050215 <+117>: callq  *(%rdi)
->  0x100050217 <+119>: vmovaps (%rax), %ymm0
    0x10005021b <+123>: vmovaps 0x3bc5d(%rip), %ymm1
    0x100050223 <+131>: vmovaps %ymm1, (%rax)
    0x100050227 <+135>: movq   0x20(%rax), %rax

project: https://github.com/kamyuentse/httparse.git branch: rewrite

compile with RUSTFLAGS="-C target-feature=+avx2,+bmi" cargo bench --features=nightly --no-run

@BurntSushi BurntSushi referenced this issue Jun 17, 2018

Closed

Segmentation fault: 11 error on OSX in release mode #489

@SimonSapin SimonSapin referenced this issue Jun 24, 2018

Closed

println! segfaults with nightly-2018-06-24 on macOS 10.10 #51758

@kennytm kennytm referenced this issue Jun 28, 2018

Merged

Do not allow LLVM to increase a TLS's alignment on macOS. #51828

bors added a commit that referenced this issue Jun 28, 2018

@bors
Auto merge of #51828 - kennytm:no-simd-swap-for-mac, r=<try> 
[DO NOT MERGE] Do not allow LLVM to increase a TLS's alignment on macOS.

This addresses the various TLS segfault on macOS 10.10.

Fix #51794.
Fix #51758.
Fix #50867.
Fix #48866.
Fix #46355.
Fix #44056.
Loading status checks…
29f834a

Mark-Simulacrum added a commit to Mark-Simulacrum/rust that referenced this issue Jun 30, 2018

@Mark-Simulacrum
Rollup merge of rust-lang#51828 - kennytm:no-simd-swap-for-mac, r=ale… 
…xcrichton

Do not allow LLVM to increase a TLS's alignment on macOS.

This addresses the various TLS segfault on macOS 10.10.

Fix rust-lang#51794.
Fix rust-lang#51758.
Fix rust-lang#50867.
Fix rust-lang#48866.
Fix rust-lang#46355.
Fix rust-lang#44056.
Verified
This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
8c30769

bors added a commit that referenced this issue Jun 30, 2018

@bors
Auto merge of #51828 - kennytm:no-simd-swap-for-mac, r=alexcrichton 
Do not allow LLVM to increase a TLS's alignment on macOS.

This addresses the various TLS segfault on macOS 10.10.

Fix #51794.
Fix #51758.
Fix #50867.
Fix #48866.
Fix #46355.
Fix #44056.
Loading status checks…
157328e

bors added a commit that referenced this issue Jun 30, 2018

@bors
Auto merge of #51828 - kennytm:no-simd-swap-for-mac, r=alexcrichton 
Do not allow LLVM to increase a TLS's alignment on macOS.

This addresses the various TLS segfault on macOS 10.10.

Fix #51794.
Fix #51758.
Fix #50867.
Fix #48866.
Fix #46355.
Fix #44056.
Loading status checks…
96b4733

@kennytm kennytm referenced this issue Jul 10, 2018

Merged

Set opt-level = 3 the third time. #52212

bors added a commit that referenced this issue Jul 11, 2018

@bors
Auto merge of #52212 - kennytm:o3-again, r=<try> 
Set opt-level = 3 the third time.

This PR reverts #51165 (set -O2 to fix #50867),
which reverted #50329 (set -O3),
which was second attempt of #48204 (set -O3, closed due to Windows segfault that is fixed now),
which reverted #42123 (set -O2 to fix spurious Windows segfaults),
which reverted #41967 (set -O3).

Last time we've found that setting -O3 regressed the wall time of NLL (#50329 (comment)), so we may need another perf run to confirm. I'd like to check this *after* the LLVM 7 upgrade #51966 has been merged, so marking this as <kbd>S-blocked</kbd> for now.
Loading status checks…
25ff830

bors added a commit that referenced this issue Jul 12, 2018

@bors
Auto merge of #52212 - kennytm:o3-again, r=alexcrichton 
Set opt-level = 3 the third time.

This PR reverts #51165 (set -O2 for fixing #50867),
which reverted #50329 (set -O3),
which was second attempt of #48204 (set -O3, closed due to Windows segfault that is fixed now),
which reverted #42123 (set -O2 to fix spurious Windows segfaults),
which reverted #41967 (set -O3).

Since we have found the root cause of #50867, this optimization could be tried again.

Last time we've found that setting -O3 regressed the wall time of NLL (#50329 (comment)), so we may need another perf run to confirm. I'd like to check this *after* the LLVM 7 upgrade #51966 has been merged, so marking this as <kbd>S-blocked</kbd> for now.
Loading status checks…
ece70aa

bors added a commit that referenced this issue Jul 14, 2018

@bors
Auto merge of #52212 - kennytm:o3-again, r=alexcrichton 
Set opt-level = 3 the third time.

This PR reverts #51165 (set -O2 for fixing #50867),
which reverted #50329 (set -O3),
which was second attempt of #48204 (set -O3, closed due to Windows segfault that is fixed now),
which reverted #42123 (set -O2 to fix spurious Windows segfaults),
which reverted #41967 (set -O3).

Since we have found the root cause of #50867, this optimization could be tried again.

Last time we've found that setting -O3 regressed the wall time of NLL (#50329 (comment)), so we may need another perf run to confirm. I'd like to check this *after* the LLVM 7 upgrade #51966 has been merged, so marking this as <kbd>S-blocked</kbd> for now.
Loading status checks…
1731f0a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.