Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Illegal instruction with x86-64 jemalloc on Rosetta #2014

Open
pitrou opened this issue Jan 13, 2021 · 7 comments
Open

Illegal instruction with x86-64 jemalloc on Rosetta #2014

pitrou opened this issue Jan 13, 2021 · 7 comments

Comments

@pitrou
Copy link

pitrou commented Jan 13, 2021

In Apache Arrow we bundle a private build of jemalloc 5.2.1. When we compile a x86-64 version of Arrow on a Apple M1 machine and then run the resulting code using Rosetta, the process crashes at startup:

$ sudo lldb ./build-debug/debug/arrow-public-api-test 
Password:
(lldb) target create "./build-debug/debug/arrow-public-api-test"
Current executable set to '/Users/ursa/antoine/arrow/cpp/build-debug/debug/arrow-public-api-test' (x86_64).
(lldb) run
Process 67189 launched: '/Users/ursa/antoine/arrow/cpp/build-debug/debug/arrow-public-api-test' (x86_64)
libarrow.300.dylib was compiled with optimization - stepping may behave oddly; variables may not be available.
Process 67189 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0)
    frame #0: 0x0000000109e73b03 libarrow.300.dylib`je_arrow_private_je_tsd_tcache_data_init.cold.1 at tcache.c:49:23 [opt]
   46  			tbin = tcache_small_bin_get(tcache, binind);
   47  		} else {
   48  			tbin = tcache_large_bin_get(tcache, binind);
-> 49  		}
   50  		if (tbin->low_water > 0) {
   51  			/*
   52  			 * Flush (ceiling) 3/4 of the objects below the low water mark.
Target 0: (arrow-public-api-test) stopped.

Traceback:

* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0)
  * frame #0: 0x0000000109e73b03 libarrow.300.dylib`je_arrow_private_je_tsd_tcache_data_init.cold.1 at tcache.c:49:23 [opt]
    frame #1: 0x0000000109d43e8e libarrow.300.dylib`je_arrow_private_je_tsd_tcache_data_init [inlined] iaalloc at jemalloc_internal_inlines_c.h:0 [opt]
    frame #2: 0x0000000109d43e80 libarrow.300.dylib`je_arrow_private_je_tsd_tcache_data_init [inlined] ipallocztm(tsdn=<unavailable>, usize=<unavailable>, alignment=<unavailable>, zero=<unavailable>, tcache=<unavailable>, is_internal=<unavailable>, arena=<unavailable>) at jemalloc_internal_inlines_c.h:81 [opt]
    frame #3: 0x0000000109d43e80 libarrow.300.dylib`je_arrow_private_je_tsd_tcache_data_init(tsd=<unavailable>) at tcache.c:451 [opt]
    frame #4: 0x0000000109d43d28 libarrow.300.dylib`je_arrow_private_je_tsd_tcache_enabled_data_init(tsd=0x000000010a077a50) at tcache.c:402:3 [opt]
    frame #5: 0x0000000109d450fa libarrow.300.dylib`je_arrow_private_je_tsd_fetch_slow(tsd=0x000000010a077a50, minimal=<unavailable>) at tsd.c:0 [opt]
    frame #6: 0x0000000109d45854 libarrow.300.dylib`je_arrow_private_je_malloc_tsd_boot0 [inlined] tsd_fetch_impl(init=true, minimal=false) at tsd.h:354:10 [opt]
    frame #7: 0x0000000109d4582d libarrow.300.dylib`je_arrow_private_je_malloc_tsd_boot0 [inlined] tsd_fetch at tsd.h:380 [opt]
    frame #8: 0x0000000109d4582d libarrow.300.dylib`je_arrow_private_je_malloc_tsd_boot0 at tsd.c:429 [opt]
    frame #9: 0x0000000109d069b2 libarrow.300.dylib`jemalloc_constructor [inlined] malloc_init_hard at jemalloc.c:1756:8 [opt]
    frame #10: 0x0000000109d06911 libarrow.300.dylib`jemalloc_constructor [inlined] malloc_init at jemalloc.c:223 [opt]
    frame #11: 0x0000000109d06911 libarrow.300.dylib`jemalloc_constructor at jemalloc.c:3775 [opt]
    frame #12: 0x0000000200059079 dyld`ImageLoaderMachO::doModInitFunctions(ImageLoader::LinkContext const&) + 559
    frame #13: 0x0000000200059478 dyld`ImageLoaderMachO::doInitialization(ImageLoader::LinkContext const&) + 40
    frame #14: 0x0000000200053d1a dyld`ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int, char const*, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) + 492
    frame #15: 0x0000000200053c85 dyld`ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int, char const*, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) + 343
    frame #16: 0x0000000200053c85 dyld`ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int, char const*, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) + 343
    frame #17: 0x0000000200051b82 dyld`ImageLoader::processInitializers(ImageLoader::LinkContext const&, unsigned int, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) + 188
    frame #18: 0x0000000200051c22 dyld`ImageLoader::runInitializers(ImageLoader::LinkContext const&, ImageLoader::InitializerTimingList&) + 82
    frame #19: 0x000000020003e62f dyld`dyld::initializeMainExecutable() + 199
    frame #20: 0x00000002000449a4 dyld`dyld::_main(macho_header const*, unsigned long, int, char const**, char const**, char const**, unsigned long*) + 8288
    frame #21: 0x000000020003d22b dyld`dyldbootstrap::start(dyld3::MachOLoaded const*, int, char const**, dyld3::MachOLoaded const*, unsigned long*) + 457
    frame #22: 0x000000020003d025 dyld`_dyld_start + 37

The disassembly shows a ud2 instruction:

libarrow.300.dylib`je_arrow_private_je_tsd_tcache_data_init.cold.1:
    0x109e739b0 <+0>:   pushq  %rbp
    0x109e739b1 <+1>:   movq   %rsp, %rbp
    0x109e739b4 <+4>:   pushq  %r15
    0x109e739b6 <+6>:   pushq  %r14
    0x109e739b8 <+8>:   pushq  %rbx
    0x109e739b9 <+9>:   subq   $0x188, %rsp              ; imm = 0x188 
    0x109e739c0 <+16>:  movq   %rdx, %r14
    0x109e739c3 <+19>:  movq   %rsi, %r15
    0x109e739c6 <+22>:  testq  %rdi, %rdi
    0x109e739c9 <+25>:  jne    0x109e739dc               ; <+44> at tcache.c
    0x109e739cb <+27>:  leaq   -0x198(%rbp), %rbx
    0x109e739d2 <+34>:  movq   %rbx, %rdi
    0x109e739d5 <+37>:  callq  0x109d31130               ; je_arrow_private_je_rtree_ctx_data_init at rtree.c:309
    0x109e739da <+42>:  jmp    0x109e739e3               ; <+51> at tcache.c:147:36
    0x109e739dc <+44>:  movq   %rdi, %rbx
    0x109e739df <+47>:  addq   $0x30, %rbx
    0x109e739e3 <+51>:  movl   %r14d, %ecx
    0x109e739e6 <+54>:  andl   $0xf, %ecx
    0x109e739e9 <+57>:  shlq   $0x4, %rcx
    0x109e739ed <+61>:  movq   (%rbx,%rcx), %rsi
    0x109e739f1 <+65>:  cmpq   %r14, %rsi
    0x109e739f4 <+68>:  jne    0x109e73a00               ; <+80> at tcache.c
    0x109e739f6 <+70>:  movq   0x8(%rbx,%rcx), %rax
    0x109e739fb <+75>:  jmp    0x109e73b00               ; <+336> at tcache.c:62:1
    0x109e73a00 <+80>:  leaq   (%rbx,%rcx), %r9
    0x109e73a04 <+84>:  cmpq   %r14, 0x100(%rbx)
    0x109e73a0b <+91>:  jne    0x109e73a2c               ; <+124> at tcache.c:377:3
    0x109e73a0d <+93>:  movq   0x108(%rbx), %rax
    0x109e73a14 <+100>: movq   %rsi, 0x100(%rbx)
    0x109e73a1b <+107>: movq   0x8(%rbx,%rcx), %rdx
    0x109e73a20 <+112>: movq   %rdx, 0x108(%rbx)
    0x109e73a27 <+119>: jmp    0x109e73af8               ; <+328> at tcache.c
    0x109e73a2c <+124>: cmpq   %r14, 0x110(%rbx)
    0x109e73a33 <+131>: jne    0x109e73a41               ; <+145> at tcache.c:377:3
    0x109e73a35 <+133>: leaq   0x110(%rbx), %r8
    0x109e73a3c <+140>: pushq  $0x1
    0x109e73a3e <+142>: popq   %rdi
    0x109e73a3f <+143>: jmp    0x109e73abd               ; <+269> at tcache.c:377:3
    0x109e73a41 <+145>: cmpq   %r14, 0x120(%rbx)
    0x109e73a48 <+152>: jne    0x109e73a56               ; <+166> at tcache.c:377:3
    0x109e73a4a <+154>: leaq   0x120(%rbx), %r8
    0x109e73a51 <+161>: pushq  $0x2
    0x109e73a53 <+163>: popq   %rdi
    0x109e73a54 <+164>: jmp    0x109e73abd               ; <+269> at tcache.c:377:3
    0x109e73a56 <+166>: cmpq   %r14, 0x130(%rbx)
    0x109e73a5d <+173>: jne    0x109e73a6b               ; <+187> at tcache.c:377:3
    0x109e73a5f <+175>: leaq   0x130(%rbx), %r8
    0x109e73a66 <+182>: pushq  $0x3
    0x109e73a68 <+184>: popq   %rdi
    0x109e73a69 <+185>: jmp    0x109e73abd               ; <+269> at tcache.c:377:3
    0x109e73a6b <+187>: cmpq   %r14, 0x140(%rbx)
    0x109e73a72 <+194>: jne    0x109e73a80               ; <+208> at tcache.c:377:3
    0x109e73a74 <+196>: leaq   0x140(%rbx), %r8
    0x109e73a7b <+203>: pushq  $0x4
    0x109e73a7d <+205>: popq   %rdi
    0x109e73a7e <+206>: jmp    0x109e73abd               ; <+269> at tcache.c:377:3
    0x109e73a80 <+208>: cmpq   %r14, 0x150(%rbx)
    0x109e73a87 <+215>: jne    0x109e73a95               ; <+229> at tcache.c:377:3
    0x109e73a89 <+217>: leaq   0x150(%rbx), %r8
    0x109e73a90 <+224>: pushq  $0x5
    0x109e73a92 <+226>: popq   %rdi
    0x109e73a93 <+227>: jmp    0x109e73abd               ; <+269> at tcache.c:377:3
    0x109e73a95 <+229>: cmpq   %r14, 0x160(%rbx)
    0x109e73a9c <+236>: jne    0x109e73aaa               ; <+250> at tcache.c:377:3
    0x109e73a9e <+238>: leaq   0x160(%rbx), %r8
    0x109e73aa5 <+245>: pushq  $0x6
    0x109e73aa7 <+247>: popq   %rdi
    0x109e73aa8 <+248>: jmp    0x109e73abd               ; <+269> at tcache.c:377:3
    0x109e73aaa <+250>: cmpq   %r14, 0x170(%rbx)
    0x109e73ab1 <+257>: jne    0x109e73b05               ; <+341> at tcache.c:381:9
    0x109e73ab3 <+259>: leaq   0x170(%rbx), %r8
    0x109e73aba <+266>: pushq  $0x7
    0x109e73abc <+268>: popq   %rdi
    0x109e73abd <+269>: movl   %edi, %esi
    0x109e73abf <+271>: shlq   $0x4, %rsi
    0x109e73ac3 <+275>: movq   0x108(%rbx,%rsi), %rax
    0x109e73acb <+283>: decl   %edi
    0x109e73acd <+285>: shlq   $0x4, %rdi
    0x109e73ad1 <+289>: movq   0x100(%rbx,%rdi), %rdx
    0x109e73ad9 <+297>: movq   %rdx, (%r8)
    0x109e73adc <+300>: movq   0x108(%rbx,%rdi), %rdx
    0x109e73ae4 <+308>: movq   %rdx, 0x108(%rbx,%rsi)
    0x109e73aec <+316>: movups (%r9), %xmm0
    0x109e73af0 <+320>: movups %xmm0, 0x100(%rbx,%rdi)
    0x109e73af8 <+328>: movq   %r14, (%r9)
    0x109e73afb <+331>: movq   %rax, 0x8(%rbx,%rcx)
    0x109e73b00 <+336>: movq   (%rax), %rax
->  0x109e73b03 <+339>: ud2    
    0x109e73b05 <+341>: leaq   0x216584(%rip), %rsi      ; je_arrow_private_je_extents_rtree
    0x109e73b0c <+348>: pushq  $0x1
    0x109e73b0e <+350>: popq   %r8
    0x109e73b10 <+352>: movq   %r15, %rdi
    0x109e73b13 <+355>: movq   %rbx, %rdx
    0x109e73b16 <+358>: movq   %r14, %rcx
    0x109e73b19 <+361>: xorl   %r9d, %r9d
    0x109e73b1c <+364>: callq  0x109d310d0               ; je_arrow_private_je_rtree_leaf_elm_lookup_hard at rtree.c:224
    0x109e73b21 <+369>: jmp    0x109e73b00               ; <+336> at tcache.c:62:1
    0x109e73b23 <+371>: nopw   %cs:(%rax,%rax)
    0x109e73b2d <+381>: nopl   (%rax)
@pitrou
Copy link
Author

pitrou commented Jan 13, 2021

If I rebuild jemalloc with --enable-debug, I get a different crash:

$ sudo lldb ./build-debug/debug/arrow-public-api-test 
(lldb) target create "./build-debug/debug/arrow-public-api-test"
Current executable set to '/Users/ursa/antoine/arrow/cpp/build-debug/debug/arrow-public-api-test' (x86_64).
(lldb) run
Process 83607 launched: '/Users/ursa/antoine/arrow/cpp/build-debug/debug/arrow-public-api-test' (x86_64)
Process 83607 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=2, address=0x10a971380)
    frame #0: 0x0000000109d5430d libarrow.300.dylib`atomic_store_p(a=0x000000010a971380, val=0x0000000000000027, mo=atomic_memory_order_release) at atomic.h:62:1
   59  	#  define JEMALLOC_ATOMIC_U64
   60  	#endif
   61  	
-> 62  	JEMALLOC_GENERATE_ATOMICS(void *, p, LG_SIZEOF_PTR)
   63  	
   64  	/*
   65  	 * There's no actual guarantee that sizeof(bool) == 1, but it's true on the only
Target 0: (arrow-public-api-test) stopped.

Traceback:

* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=2, address=0x10a971380)
  * frame #0: 0x0000000109d5430d libarrow.300.dylib`atomic_store_p(a=0x000000010a971380, val=0x0000000000000027, mo=atomic_memory_order_release) at atomic.h:62:1
    frame #1: 0x0000000109d54292 libarrow.300.dylib`rtree_leaf_elm_write(tsdn=0x000000010a0c7e98, rtree=0x000000010a0de2c0, elm=0x000000010a971380, extent=0x000000011320a740, szind=39, slab=false) at rtree.h:301:2
    frame #2: 0x0000000109d5407f libarrow.300.dylib`extent_rtree_write_acquired(tsdn=0x000000010a0c7e98, elm_a=0x000000010a971380, elm_b=0x000000010a9713b8, extent=0x000000011320a740, szind=39, slab=false) at extent.c:723:2
    frame #3: 0x0000000109d5826c libarrow.300.dylib`extent_register_impl(tsdn=0x000000010a0c7e98, extent=0x000000011320a740, gdump_add=true) at extent.c:800:2
    frame #4: 0x0000000109d4e0e2 libarrow.300.dylib`extent_register(tsdn=0x000000010a0c7e98, extent=0x000000011320a740) at extent.c:816:9
    frame #5: 0x0000000109d4dfee libarrow.300.dylib`extent_alloc_wrapper_hard(tsdn=0x000000010a0c7e98, arena=0x00000001132009c0, r_extent_hooks=0x0000000304151a78, new_addr=0x0000000000000000, size=28672, pad=4096, alignment=64, slab=false, szind=39, zero=0x0000000304151acf, commit=0x0000000304151a67) at extent.c:1522:6
    frame #6: 0x0000000109d4da1d libarrow.300.dylib`je_arrow_private_je_extent_alloc_wrapper(tsdn=0x000000010a0c7e98, arena=0x00000001132009c0, r_extent_hooks=0x0000000304151a78, new_addr=0x0000000000000000, size=28672, pad=4096, alignment=64, slab=false, szind=39, zero=0x0000000304151acf, commit=0x0000000304151a67) at extent.c:1551:12
    frame #7: 0x0000000109d0be56 libarrow.300.dylib`je_arrow_private_je_arena_extent_alloc_large(tsdn=0x000000010a0c7e98, arena=0x00000001132009c0, usize=28672, alignment=64, zero=0x0000000304151acf) at arena.c:448:12
    frame #8: 0x0000000109d5c1ae libarrow.300.dylib`je_arrow_private_je_large_palloc(tsdn=0x000000010a0c7e98, arena=0x00000001132009c0, usize=28672, alignment=64, zero=true) at large.c:47:43
    frame #9: 0x0000000109d5bfab libarrow.300.dylib`je_arrow_private_je_large_malloc(tsdn=0x000000010a0c7e98, arena=0x00000001132009c0, usize=28672, zero=true) at large.c:17:9
    frame #10: 0x0000000109d0e205 libarrow.300.dylib`je_arrow_private_je_arena_palloc(tsdn=0x000000010a0c7e98, arena=0x00000001132009c0, usize=28672, alignment=64, zero=true, tcache=0x0000000000000000) at arena.c:1544:10
    frame #11: 0x0000000109d8c419 libarrow.300.dylib`ipallocztm(tsdn=0x000000010a0c7e98, usize=28672, alignment=64, zero=true, tcache=0x0000000000000000, is_internal=true, arena=0x00000001132009c0) at jemalloc_internal_inlines_c.h:78:8
    frame #12: 0x0000000109d8bf0e libarrow.300.dylib`je_arrow_private_je_tsd_tcache_data_init(tsd=0x000000010a0c7e98) at tcache.c:451:22
    frame #13: 0x0000000109d8bd6f libarrow.300.dylib`je_arrow_private_je_tsd_tcache_enabled_data_init(tsd=0x000000010a0c7e98) at tcache.c:402:3
    frame #14: 0x0000000109d92a66 libarrow.300.dylib`tsd_data_init(tsd=0x000000010a0c7e98) at tsd.c:236:9
    frame #15: 0x0000000109d92761 libarrow.300.dylib`je_arrow_private_je_tsd_fetch_slow(tsd=0x000000010a0c7e98, minimal=false) at tsd.c:289:5
    frame #16: 0x0000000109d94176 libarrow.300.dylib`tsd_fetch_impl(init=true, minimal=false) at tsd.h:354:10
    frame #17: 0x0000000109d93200 libarrow.300.dylib`tsd_fetch at tsd.h:380:9
    frame #18: 0x0000000109d93169 libarrow.300.dylib`je_arrow_private_je_malloc_tsd_boot0 at tsd.c:429:8
    frame #19: 0x0000000109d096f1 libarrow.300.dylib`malloc_init_hard at jemalloc.c:1756:8
    frame #20: 0x0000000109cff0ea libarrow.300.dylib`malloc_init at jemalloc.c:223:41
    frame #21: 0x0000000109cff4d9 libarrow.300.dylib`jemalloc_constructor at jemalloc.c:3775:2
    frame #22: 0x0000000200059079 dyld`ImageLoaderMachO::doModInitFunctions(ImageLoader::LinkContext const&) + 559
    frame #23: 0x0000000200059478 dyld`ImageLoaderMachO::doInitialization(ImageLoader::LinkContext const&) + 40
    frame #24: 0x0000000200053d1a dyld`ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int, char const*, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) + 492
    frame #25: 0x0000000200053c85 dyld`ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int, char const*, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) + 343
    frame #26: 0x0000000200053c85 dyld`ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int, char const*, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) + 343
    frame #27: 0x0000000200051b82 dyld`ImageLoader::processInitializers(ImageLoader::LinkContext const&, unsigned int, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) + 188
    frame #28: 0x0000000200051c22 dyld`ImageLoader::runInitializers(ImageLoader::LinkContext const&, ImageLoader::InitializerTimingList&) + 82
    frame #29: 0x000000020003e62f dyld`dyld::initializeMainExecutable() + 199
    frame #30: 0x00000002000449a4 dyld`dyld::_main(macho_header const*, unsigned long, int, char const**, char const**, char const**, unsigned long*) + 8288
    frame #31: 0x000000020003d22b dyld`dyldbootstrap::start(dyld3::MachOLoaded const*, int, char const**, dyld3::MachOLoaded const*, unsigned long*) + 457
    frame #32: 0x000000020003d025 dyld`_dyld_start + 37

Disassembly:

libarrow.300.dylib`atomic_store_p:
    0x109d542a0 <+0>:   pushq  %rbp
    0x109d542a1 <+1>:   movq   %rsp, %rbp
    0x109d542a4 <+4>:   subq   $0x30, %rsp
    0x109d542a8 <+8>:   movq   %rdi, -0x8(%rbp)
    0x109d542ac <+12>:  movq   %rsi, -0x10(%rbp)
    0x109d542b0 <+16>:  movl   %edx, -0x14(%rbp)
    0x109d542b3 <+19>:  movq   -0x8(%rbp), %rax
    0x109d542b7 <+23>:  movl   -0x14(%rbp), %edi
    0x109d542ba <+26>:  movq   %rax, -0x20(%rbp)
    0x109d542be <+30>:  callq  0x109d50d30               ; atomic_enum_to_builtin at atomic_gcc_atomic.h:17
    0x109d542c3 <+35>:  leaq   -0x10(%rbp), %rcx
    0x109d542c7 <+39>:  movl   %eax, %edx
    0x109d542c9 <+41>:  subl   $0x3, %eax
    0x109d542cc <+44>:  movq   %rcx, -0x28(%rbp)
    0x109d542d0 <+48>:  movl   %edx, -0x2c(%rbp)
    0x109d542d3 <+51>:  je     0x109d54302               ; <+98> at atomic.h
    0x109d542d9 <+57>:  jmp    0x109d542de               ; <+62> at atomic.h
    0x109d542de <+62>:  movl   -0x2c(%rbp), %eax
    0x109d542e1 <+65>:  subl   $0x5, %eax
    0x109d542e4 <+68>:  je     0x109d54315               ; <+117> at atomic.h
    0x109d542ea <+74>:  jmp    0x109d542ef               ; <+79> at atomic.h
    0x109d542ef <+79>:  movq   -0x28(%rbp), %rax
    0x109d542f3 <+83>:  movq   (%rax), %rcx
    0x109d542f6 <+86>:  movq   -0x20(%rbp), %rdx
    0x109d542fa <+90>:  movq   %rcx, (%rdx)
    0x109d542fd <+93>:  jmp    0x109d54323               ; <+131> at atomic.h:62:1
    0x109d54302 <+98>:  movq   -0x28(%rbp), %rax
    0x109d54306 <+102>: movq   (%rax), %rcx
    0x109d54309 <+105>: movq   -0x20(%rbp), %rdx
->  0x109d5430d <+109>: movq   %rcx, (%rdx)
    0x109d54310 <+112>: jmp    0x109d54323               ; <+131> at atomic.h:62:1
    0x109d54315 <+117>: movq   -0x28(%rbp), %rax
    0x109d54319 <+121>: movq   (%rax), %rcx
    0x109d5431c <+124>: movq   -0x20(%rbp), %rdx
    0x109d54320 <+128>: xchgq  %rcx, (%rdx)
    0x109d54323 <+131>: addq   $0x30, %rsp
    0x109d54327 <+135>: popq   %rbp
    0x109d54328 <+136>: retq   
    0x109d54329 <+137>: nopl   (%rax)

@davidtgoldblatt
Copy link
Member

I don't have an M1 to test on, but while I work on trying to finagle some ssh access, is there an easy repro you could share?

@pitrou
Copy link
Author

pitrou commented Jan 19, 2021 •
edited
Loading

Ok, first, you should make sure that a x86-64 cmake is installed (it is probably possible to compile a x86-64 target from a native arm64 cmake, but I haven't found the right incantation). This is possible using a x86 Homebrew install (which will give you a /usr/local/bin/cmake).
Then:

$ git clone https://github.com/apache/arrow.git
$ cp arrow/cpp
$ mkdir build
$ cd build
$ <path/to/x86/cmake> .. -DCMAKE_BUILD_TYPE=Debug -DARROW_BUILD_TESTS=ON -DARROW_DEPENDENCY_SOURCE=bundled
$ make -j4
$ file ./debug/arrow-misc-test  # check that this is x86-64
$ ./debug/arrow-misc-test

The last command should probably crash with a "illegal instruction".

@devnexen
Copy link
Contributor

I do not know if it helps but Rosetta does not translate AVX* instructions.

@pitrou
Copy link
Author

pitrou commented Jan 19, 2021

When trying to debug, I didn't see a AVX instruction but a ud2 instruction (which sounds a bit weird).

@devnexen
Copy link
Contributor

yes

@BusyJay BusyJay mentioned this issue Aug 15, 2022
Open
@veprbl veprbl mentioned this issue Sep 21, 2022
Closed
@niyue niyue mentioned this issue Oct 25, 2022
Open
@bkchr bkchr mentioned this issue Jan 18, 2023
Closed
@ehuss ehuss mentioned this issue Oct 7, 2023
Open
@sagudev sagudev mentioned this issue Dec 21, 2023
Closed
@lukaso
Copy link

lukaso commented Mar 2, 2024

Here's the full commands to do what @pitrou suggested. However this did not create a crash on an M3 Macbook.

arch -x86_64 /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"\n
arch -x86_64 /bin/zsh
/usr/local/bin/brew install cmake
git clone https://github.com/apache/arrow.git
cd arrow/cpp
mkdir build
cd build
/usr/local/bin/cmake .. -DCMAKE_BUILD_TYPE=Debug -DARROW_BUILD_TESTS=ON -DARROW_DEPENDENCY_SOURCE=BUNDLED
make -j4
file ./debug/arrow-misc-test # you should see x86_64 in the output
./debug/arrow-misc-test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@lukaso @davidtgoldblatt @pitrou @devnexen