UPX is way, way broken by Darwin 16 / OS X Sierra. #4

Closed
geoff-codes opened this Issue Sep 15, 2016 · 109 comments

Projects

None yet
@geoff-codes
geoff-codes commented Sep 15, 2016 edited

I'm not sure if this is already on your radar (and if so, well, I'd recommend you just let this issue sit open here until/unless there's a fix or workaround), but:

OS X Sierra (I refuse to say macOS) really abhors UPX. I don't think I've seen anything quite like this, but they've actually taken the time to code in some diagnostics that call you out by name.

Please do note I do say broken by, not merely broken on.

This is a one of maybe a dozen programs I have (i.e., built by other developers) which now act this very stubbornly act this way.

I should note:

  • SIP is disabled.
  • SecAssessment is (apparently) disabled. In layman's terms: Gatekeeper is "off", i.e. spctl --master-disable.
  • I'm not sure if these even do anything any more, but I also threw debug=0x14e amfi_get_out_of_my_way=0x1 cs_enforcement_disable=0x1 in my boot-args.
  • Again, I'm not really sure if there's any particular relevance, but there's only a couple of kernel security.mac switches that the system will let you change, and there was no smoking gun I could obviously find in there, either, but FWIW, I set sudo sysctl security.mac.qtn.sandbox_enforce=0.

Some program:

Process:               program [63584]
Path:                  /Volumes/VOLUME/*/program.app/Contents/MacOS/program
Identifier:            org.website.program
Version:               ???
Code Type:             X86-64 (Native)
Parent Process:        ??? [1]
Responsible:           program [63584]
User ID:               501

Date/Time:             2016-09-15 07:53:21.269 -0700
OS Version:            Mac OS X 10.12 (16A320)
Report Version:        12
Anonymous UUID:        8833382B-065F-9020-2102-BC778676C039


Time Awake Since Boot: 11000 seconds

System Integrity Protection: disabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_CRASH (SIGKILL)
Exception Codes:       0x0000000000000000, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Termination Reason:    EXEC, [0xc] This UPX compressed binary contains an invalid Mach-O header and cannot be loaded.

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib          0x00007fffa0485bb2 __posix_spawn + 10
1   libsystem_kernel.dylib          0x00007fffa0480ef2 posix_spawn + 386
2   xpcproxy                        0x0000000106bcbd75 0x106bc9000 + 11637
3   xpcproxy                        0x0000000106bcc992 0x106bc9000 + 14738
4   libdyld.dylib                   0x00007fffa0357255 start + 1

Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x000000000000000d  rbx: 0x0000000000000000  rcx: 0x00007fff59036388  rdx: 0x00007fff590363a0
  rdi: 0x0000000000000000  rsi: 0x00007f90400008e5  rbp: 0x00007fff59036430  rsp: 0x00007fff59036388
   r8: 0x00007f903f5020e0   r9: 0x00007f903f5020e0  r10: 0x00007f903f500250  r11: 0x0000000000000202
  r12: 0x00007f903f500250  r13: 0x00007f90400008e5  r14: 0x00007fff59036950  r15: 0x00007fff59036958
  rip: 0x00007fffa0485bb2  rfl: 0x0000000000000203  cr2: 0x00007fffa90270a8

Logical CPU:     0
Error Code:      0x020000f4
Trap Number:     133


Binary Images:
       0x106bc9000 -        0x106bcdffb  xpcproxy (2.0.0 - 972.1.5) <625A730D-D6CA-3B26-ABFC-E0FD9005BAB6> /usr/libexec/xpcproxy
       0x10f553000 -        0x10f5901c7  dyld (421.1) <A525EAEA-AF86-30C2-B360-3D093B4F0828> /usr/lib/dyld
    0x7fff9edc3000 -     0x7fff9edc4ff3  libSystem.B.dylib (1238) <168B3C56-081B-3998-9A44-681EB4C6828F> /usr/lib/libSystem.B.dylib
    0x7fff9eed4000 -     0x7fff9eed4ff3  libauto.dylib (187) <5BBF6A00-CC76-389D-84E7-CA88EDADE683> /usr/lib/libauto.dylib
    0x7fff9eef5000 -     0x7fff9ef4bff7  libc++.1.dylib (307.4) <BEE86868-F831-384C-919E-2B286ACFE87C> /usr/lib/libc++.1.dylib
    0x7fff9ef4c000 -     0x7fff9ef76fff  libc++abi.dylib (307.2) <1CEF8ABB-7E6D-3C2F-8E0A-E7884478DD23> /usr/lib/libc++abi.dylib
    0x7fff9fa65000 -     0x7fff9fe35d97  libobjc.A.dylib (706) <F9AFE665-A3A2-3285-9495-19803A565861> /usr/lib/libobjc.A.dylib
    0x7fffa027b000 -     0x7fffa027fff7  libcache.dylib (79) <84E55656-FDA9-3B29-9E4F-BE31B2C0AA3C> /usr/lib/system/libcache.dylib
    0x7fffa0280000 -     0x7fffa028afff  libcommonCrypto.dylib (60092.1.2) <79F738D8-0AD7-3DEA-AF80-E0F8B90B74E3> /usr/lib/system/libcommonCrypto.dylib
    0x7fffa028b000 -     0x7fffa0292fff  libcompiler_rt.dylib (62) <486BDE52-81B4-3446-BD72-23977CAE556F> /usr/lib/system/libcompiler_rt.dylib
    0x7fffa0293000 -     0x7fffa029bfff  libcopyfile.dylib (138) <0DA49B77-56EC-362D-98FF-FA78CFD986D6> /usr/lib/system/libcopyfile.dylib
    0x7fffa029c000 -     0x7fffa031eff3  libcorecrypto.dylib (442.1.3) <1CA8B679-810E-3C04-945A-4AC19372CDC7> /usr/lib/system/libcorecrypto.dylib
    0x7fffa031f000 -     0x7fffa0351fff  libdispatch.dylib (703.1.4) <12B1D95B-283D-325D-85AB-29A2FFD36987> /usr/lib/system/libdispatch.dylib
    0x7fffa0352000 -     0x7fffa0357ff3  libdyld.dylib (421.1) <CDFBDC9C-418C-369D-B433-F64B0630E640> /usr/lib/system/libdyld.dylib
    0x7fffa0358000 -     0x7fffa0358ffb  libkeymgr.dylib (28) <09CD7CA6-46D2-3A9F-B9F1-7C4CA5CA0D68> /usr/lib/system/libkeymgr.dylib
    0x7fffa0366000 -     0x7fffa0366fff  liblaunch.dylib (972.1.5) <DC31FDEA-FD81-335E-BAA8-5A7395D20772> /usr/lib/system/liblaunch.dylib
    0x7fffa0367000 -     0x7fffa036cfff  libmacho.dylib (894) <1EAE5ADD-490C-3B1F-9F97-447BA8E0E90F> /usr/lib/system/libmacho.dylib
    0x7fffa036d000 -     0x7fffa036fff3  libquarantine.dylib (85) <78EF62D8-C890-3FC0-937A-C2FD8CEF8992> /usr/lib/system/libquarantine.dylib
    0x7fffa0370000 -     0x7fffa0371ffb  libremovefile.dylib (45) <C4FC07FF-ED86-382E-B06F-33C34718080C> /usr/lib/system/libremovefile.dylib
    0x7fffa0372000 -     0x7fffa038aff7  libsystem_asl.dylib (349.1.1) <F0987490-8427-367F-B302-A05A7D61FEBF> /usr/lib/system/libsystem_asl.dylib
    0x7fffa038b000 -     0x7fffa038bff7  libsystem_blocks.dylib (67) <B8C3701D-5A91-3D35-999D-2DC8D5393525> /usr/lib/system/libsystem_blocks.dylib
    0x7fffa038c000 -     0x7fffa0419fe7  libsystem_c.dylib (1158.1.2) <5F260836-48E4-3F57-8553-62D2DA228A1F> /usr/lib/system/libsystem_c.dylib
    0x7fffa041a000 -     0x7fffa041dffb  libsystem_configuration.dylib (888.1.2) <67BB9D8B-2430-38AD-81A7-F0EC924B2590> /usr/lib/system/libsystem_configuration.dylib
    0x7fffa041e000 -     0x7fffa0421fff  libsystem_coreservices.dylib (41.1) <11F22E6C-0DCB-3699-A4F0-C99E301E56E9> /usr/lib/system/libsystem_coreservices.dylib
    0x7fffa0422000 -     0x7fffa043affb  libsystem_coretls.dylib (121.1.1) <8F7E9B12-400D-3276-A9C5-4546B0258554> /usr/lib/system/libsystem_coretls.dylib
    0x7fffa043b000 -     0x7fffa0441fff  libsystem_dnssd.dylib (765.1.2) <C5FF2025-C60B-39C6-B205-6BF1BC51D1B3> /usr/lib/system/libsystem_dnssd.dylib
    0x7fffa0442000 -     0x7fffa046bfff  libsystem_info.dylib (503) <C686B834-5E7D-382C-AF6E-44AB78EE83E2> /usr/lib/system/libsystem_info.dylib
    0x7fffa046c000 -     0x7fffa048eff7  libsystem_kernel.dylib (3789.1.32) <5C68A0D7-C3C9-3E52-B983-EDE9A28AB6FC> /usr/lib/system/libsystem_kernel.dylib
    0x7fffa048f000 -     0x7fffa04d6fe7  libsystem_m.dylib (3121.4) <E3370D16-EBAA-3C7F-AC56-2D6EAD7DB0A4> /usr/lib/system/libsystem_m.dylib
    0x7fffa04d7000 -     0x7fffa04f5ff7  libsystem_malloc.dylib (116) <3DD17B88-B7A4-38B9-9E95-AB88E1C3B647> /usr/lib/system/libsystem_malloc.dylib
    0x7fffa04f6000 -     0x7fffa054cfff  libsystem_network.dylib (856.1.8) <A8973360-956A-33BF-9971-14D59C84E0D1> /usr/lib/system/libsystem_network.dylib
    0x7fffa054d000 -     0x7fffa0556ff3  libsystem_networkextension.dylib (563.1.11) <1C551832-9360-36DD-A7BA-52B55A171720> /usr/lib/system/libsystem_networkextension.dylib
    0x7fffa0557000 -     0x7fffa0560ffb  libsystem_notify.dylib (165) <AF77D471-6B13-36BA-B144-7E7DDB9DBA9F> /usr/lib/system/libsystem_notify.dylib
    0x7fffa0561000 -     0x7fffa0569fe7  libsystem_platform.dylib (126.1.2) <884DDF42-3CAE-334A-82CE-965617130FB1> /usr/lib/system/libsystem_platform.dylib
    0x7fffa056a000 -     0x7fffa0574fff  libsystem_pthread.dylib (218.1.3) <050AE77B-4F4B-334A-A5BA-CF0D10AF5304> /usr/lib/system/libsystem_pthread.dylib
    0x7fffa0575000 -     0x7fffa0578fff  libsystem_sandbox.dylib (592.1.3) <90FEF628-316D-3F84-9EF4-F7F510A4FBBB> /usr/lib/system/libsystem_sandbox.dylib
    0x7fffa0579000 -     0x7fffa057afff  libsystem_secinit.dylib (24) <A54B8FEF-E792-3C54-8E0B-E80A376662F2> /usr/lib/system/libsystem_secinit.dylib
    0x7fffa057b000 -     0x7fffa0582fff  libsystem_symptoms.dylib (532.1.1) <B26F656E-94F9-3834-9B03-51C4FF11D1BF> /usr/lib/system/libsystem_symptoms.dylib
    0x7fffa0583000 -     0x7fffa05a3ff7  libsystem_trace.dylib (518.1.16) <C3CDF7BC-CA3D-34F5-ADF8-46AAAB0B47F8> /usr/lib/system/libsystem_trace.dylib
    0x7fffa05a4000 -     0x7fffa05a9ffb  libunwind.dylib (35.3) <9F7C2AD8-A9A7-3DE4-828D-B0F0F166AAA0> /usr/lib/system/libunwind.dylib
    0x7fffa05aa000 -     0x7fffa05d3ff7  libxpc.dylib (972.1.5) <2A901937-48E1-3BF2-83F1-2431156D82B7> /usr/lib/system/libxpc.dylib

External Modification Summary:
  Calls made by other processes targeting this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by all processes on this machine:
    task_for_pid: 5497
    thread_create: 0
    thread_set_state: 0

VM Region Summary:
ReadOnly portion of Libraries: Total=119.1M resident=0K(0%) swapped_out_or_unallocated=119.1M(100%)
Writable regions: Total=26.4M written=0K(0%) resident=0K(0%) swapped_out=0K(0%) unallocated=26.4M(100%)

                                VIRTUAL   REGION 
REGION TYPE                        SIZE    COUNT (non-coalesced) 
===========                     =======  ======= 
Kernel Alloc Once                    8K        2 
MALLOC                            18.2M        8 
MALLOC guard page                   16K        4 
STACK GUARD                       56.0M        2 
Stack                             8192K        2 
VM_ALLOCATE                          4K        2 
__DATA                            1460K       43 
__LINKEDIT                       111.3M        4 
__TEXT                            8084K       42 
shared memory                       12K        4 
===========                     =======  ======= 
TOTAL                            202.8M      103 

Model: MacBookPro11,2, BootROM MBP...
...

Additionally:

  • The same occurs with a clean build of upx from this new repo, on both i386 and amd64.

Process:               upx [59680]
Path:                  /Users/USER/*/upx
Identifier:            upx
Version:               ???
Code Type:             X86-64 (Native)
Parent Process:        fish [2114]
Responsible:           upxx [59680]
User ID:               501

Date/Time:             2016-09-15 06:58:07.394 -0700
OS Version:            Mac OS X 10.12 (16A320)
Report Version:        12
Anonymous UUID:        8833382B-065F-9020-2102-BC778676C039


Time Awake Since Boot: 7600 seconds

System Integrity Protection: disabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_CRASH (SIGKILL)
Exception Codes:       0x0000000000000000, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Termination Reason:    EXEC, [0xc] This UPX compressed binary contains an invalid Mach-O header and cannot be loaded.

Application Specific Information:
crashed on child side of fork pre-exec

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib          0x00007fffa0486816 execve + 10
1   fish                            0x000000010d29e292 safe_launch_process(process_t*, char const*, char const* const*, char const* const*) + 34
2   fish                            0x000000010d29e182 exec_job(parser_t&, job_t*) + 10434
3   fish                            0x000000010d2ccc4a parse_execution_context_t::run_1_job(parse_node_t const&, block_t const*) + 1802
4   fish                            0x000000010d2cd005 parse_execution_context_t::run_job_list(parse_node_t const&, block_t const*) + 245
5   fish                            0x000000010d2d1b2d parse_execution_context_t::eval_node_at_offset(unsigned int, block_t const*, io_chain_t const&) + 365
6   fish                            0x000000010d2de800 parser_t::eval_block_node(unsigned int, io_chain_t const&, block_type_t) + 336
7   fish                            0x000000010d2de5a5 parser_t::eval_acquiring_tree(std::__1::basic_string<wchar_t, std::__1::char_traits<wchar_t>, std::__1::allocator<wchar_t> > const&, io_chain_t const&, block_type_t, moved_ref<parse_node_tree_t>) + 373
8   fish                            0x000000010d2dde8f parser_t::eval(std::__1::basic_string<wchar_t, std::__1::char_traits<wchar_t>, std::__1::allocator<wchar_t> > const&, io_chain_t const&, block_type_t) + 111
9   fish                            0x000000010d2ebae1 reader_run_command(parser_t&, std::__1::basic_string<wchar_t, std::__1::char_traits<wchar_t>, std::__1::allocator<wchar_t> > const&) + 417
10  fish                            0x000000010d2f413f reader_read(int, io_chain_t const&) + 1375
11  fish                            0x000000010d304711 main + 5681
12  libdyld.dylib                   0x00007fffa0357255 start + 1

Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x000000000000000d  rbx: 0x00007fc716e0e3a0  rcx: 0x00007fff529a5458  rdx: 0x00007fc717026400
  rdi: 0x00007fff529a5ac1  rsi: 0x00007fc716e0e3a0  rbp: 0x00007fff529a58b0  rsp: 0x00007fff529a5458
   r8: 0x0000000000000303   r9: 0x0000000000000000  r10: 0x0000000000000000  r11: 0x0000000000000202
  r12: 0x00007fff529a5ac1  r13: 0x00007fc717026400  r14: 0x00007fc717026400  r15: 0x00007fff529a5ac1
  rip: 0x00007fffa0486816  rfl: 0x0000000000000203  cr2: 0x000000010d3295d8

Logical CPU:     0
Error Code:      0x0200003b
Trap Number:     133


Binary Images:
       0x10d258000 -        0x10d328ff3 +fish (0) <9EE6BE98-CD04-31D0-9C71-3BD3671623D1> /usr/local/bin/fish
       0x10d371000 -        0x10d3ceffb +libpcre2-32.0.dylib (0) <B11362C5-ECB1-33E3-B97D-5D086DBCE667> /usr/local/opt/pcre2/lib/libpcre2-32.0.dylib
       0x110b65000 -        0x110ba21c7  dyld (421.1) <A525EAEA-AF86-30C2-B360-3D093B4F0828> /usr/lib/dyld
    0x7fff9edc3000 -     0x7fff9edc4ff3  libSystem.B.dylib (1238) <168B3C56-081B-3998-9A44-681EB4C6828F> /usr/lib/libSystem.B.dylib
    0x7fff9eed4000 -     0x7fff9eed4ff3  libauto.dylib (187) <5BBF6A00-CC76-389D-84E7-CA88EDADE683> /usr/lib/libauto.dylib
    0x7fff9eef5000 -     0x7fff9ef4bff7  libc++.1.dylib (307.4) <BEE86868-F831-384C-919E-2B286ACFE87C> /usr/lib/libc++.1.dylib
    0x7fff9ef4c000 -     0x7fff9ef76fff  libc++abi.dylib (307.2) <1CEF8ABB-7E6D-3C2F-8E0A-E7884478DD23> /usr/lib/libc++abi.dylib
    0x7fff9f9ba000 -     0x7fff9f9ebff3  libncurses.5.4.dylib (51) <6B88562D-86DB-3EFA-8C55-0148C30DC642> /usr/lib/libncurses.5.4.dylib
    0x7fff9fa65000 -     0x7fff9fe35d97  libobjc.A.dylib (706) <F9AFE665-A3A2-3285-9495-19803A565861> /usr/lib/libobjc.A.dylib
    0x7fffa027b000 -     0x7fffa027fff7  libcache.dylib (79) <84E55656-FDA9-3B29-9E4F-BE31B2C0AA3C> /usr/lib/system/libcache.dylib
    0x7fffa0280000 -     0x7fffa028afff  libcommonCrypto.dylib (60092.1.2) <79F738D8-0AD7-3DEA-AF80-E0F8B90B74E3> /usr/lib/system/libcommonCrypto.dylib
    0x7fffa028b000 -     0x7fffa0292fff  libcompiler_rt.dylib (62) <486BDE52-81B4-3446-BD72-23977CAE556F> /usr/lib/system/libcompiler_rt.dylib
    0x7fffa0293000 -     0x7fffa029bfff  libcopyfile.dylib (138) <0DA49B77-56EC-362D-98FF-FA78CFD986D6> /usr/lib/system/libcopyfile.dylib
    0x7fffa029c000 -     0x7fffa031eff3  libcorecrypto.dylib (442.1.3) <1CA8B679-810E-3C04-945A-4AC19372CDC7> /usr/lib/system/libcorecrypto.dylib
    0x7fffa031f000 -     0x7fffa0351fff  libdispatch.dylib (703.1.4) <12B1D95B-283D-325D-85AB-29A2FFD36987> /usr/lib/system/libdispatch.dylib
    0x7fffa0352000 -     0x7fffa0357ff3  libdyld.dylib (421.1) <CDFBDC9C-418C-369D-B433-F64B0630E640> /usr/lib/system/libdyld.dylib
    0x7fffa0358000 -     0x7fffa0358ffb  libkeymgr.dylib (28) <09CD7CA6-46D2-3A9F-B9F1-7C4CA5CA0D68> /usr/lib/system/libkeymgr.dylib
    0x7fffa0366000 -     0x7fffa0366fff  liblaunch.dylib (972.1.5) <DC31FDEA-FD81-335E-BAA8-5A7395D20772> /usr/lib/system/liblaunch.dylib
    0x7fffa0367000 -     0x7fffa036cfff  libmacho.dylib (894) <1EAE5ADD-490C-3B1F-9F97-447BA8E0E90F> /usr/lib/system/libmacho.dylib
    0x7fffa036d000 -     0x7fffa036fff3  libquarantine.dylib (85) <78EF62D8-C890-3FC0-937A-C2FD8CEF8992> /usr/lib/system/libquarantine.dylib
    0x7fffa0370000 -     0x7fffa0371ffb  libremovefile.dylib (45) <C4FC07FF-ED86-382E-B06F-33C34718080C> /usr/lib/system/libremovefile.dylib
    0x7fffa0372000 -     0x7fffa038aff7  libsystem_asl.dylib (349.1.1) <F0987490-8427-367F-B302-A05A7D61FEBF> /usr/lib/system/libsystem_asl.dylib
    0x7fffa038b000 -     0x7fffa038bff7  libsystem_blocks.dylib (67) <B8C3701D-5A91-3D35-999D-2DC8D5393525> /usr/lib/system/libsystem_blocks.dylib
    0x7fffa038c000 -     0x7fffa0419fe7  libsystem_c.dylib (1158.1.2) <5F260836-48E4-3F57-8553-62D2DA228A1F> /usr/lib/system/libsystem_c.dylib
    0x7fffa041a000 -     0x7fffa041dffb  libsystem_configuration.dylib (888.1.2) <67BB9D8B-2430-38AD-81A7-F0EC924B2590> /usr/lib/system/libsystem_configuration.dylib
    0x7fffa041e000 -     0x7fffa0421fff  libsystem_coreservices.dylib (41.1) <11F22E6C-0DCB-3699-A4F0-C99E301E56E9> /usr/lib/system/libsystem_coreservices.dylib
    0x7fffa0422000 -     0x7fffa043affb  libsystem_coretls.dylib (121.1.1) <8F7E9B12-400D-3276-A9C5-4546B0258554> /usr/lib/system/libsystem_coretls.dylib
    0x7fffa043b000 -     0x7fffa0441fff  libsystem_dnssd.dylib (765.1.2) <C5FF2025-C60B-39C6-B205-6BF1BC51D1B3> /usr/lib/system/libsystem_dnssd.dylib
    0x7fffa0442000 -     0x7fffa046bfff  libsystem_info.dylib (503) <C686B834-5E7D-382C-AF6E-44AB78EE83E2> /usr/lib/system/libsystem_info.dylib
    0x7fffa046c000 -     0x7fffa048eff7  libsystem_kernel.dylib (3789.1.32) <5C68A0D7-C3C9-3E52-B983-EDE9A28AB6FC> /usr/lib/system/libsystem_kernel.dylib
    0x7fffa048f000 -     0x7fffa04d6fe7  libsystem_m.dylib (3121.4) <E3370D16-EBAA-3C7F-AC56-2D6EAD7DB0A4> /usr/lib/system/libsystem_m.dylib
    0x7fffa04d7000 -     0x7fffa04f5ff7  libsystem_malloc.dylib (116) <3DD17B88-B7A4-38B9-9E95-AB88E1C3B647> /usr/lib/system/libsystem_malloc.dylib
    0x7fffa04f6000 -     0x7fffa054cfff  libsystem_network.dylib (856.1.8) <A8973360-956A-33BF-9971-14D59C84E0D1> /usr/lib/system/libsystem_network.dylib
    0x7fffa054d000 -     0x7fffa0556ff3  libsystem_networkextension.dylib (563.1.11) <1C551832-9360-36DD-A7BA-52B55A171720> /usr/lib/system/libsystem_networkextension.dylib
    0x7fffa0557000 -     0x7fffa0560ffb  libsystem_notify.dylib (165) <AF77D471-6B13-36BA-B144-7E7DDB9DBA9F> /usr/lib/system/libsystem_notify.dylib
    0x7fffa0561000 -     0x7fffa0569fe7  libsystem_platform.dylib (126.1.2) <884DDF42-3CAE-334A-82CE-965617130FB1> /usr/lib/system/libsystem_platform.dylib
    0x7fffa056a000 -     0x7fffa0574fff  libsystem_pthread.dylib (218.1.3) <050AE77B-4F4B-334A-A5BA-CF0D10AF5304> /usr/lib/system/libsystem_pthread.dylib
    0x7fffa0575000 -     0x7fffa0578fff  libsystem_sandbox.dylib (592.1.3) <90FEF628-316D-3F84-9EF4-F7F510A4FBBB> /usr/lib/system/libsystem_sandbox.dylib
    0x7fffa0579000 -     0x7fffa057afff  libsystem_secinit.dylib (24) <A54B8FEF-E792-3C54-8E0B-E80A376662F2> /usr/lib/system/libsystem_secinit.dylib
    0x7fffa057b000 -     0x7fffa0582fff  libsystem_symptoms.dylib (532.1.1) <B26F656E-94F9-3834-9B03-51C4FF11D1BF> /usr/lib/system/libsystem_symptoms.dylib
    0x7fffa0583000 -     0x7fffa05a3ff7  libsystem_trace.dylib (518.1.16) <C3CDF7BC-CA3D-34F5-ADF8-46AAAB0B47F8> /usr/lib/system/libsystem_trace.dylib
    0x7fffa05a4000 -     0x7fffa05a9ffb  libunwind.dylib (35.3) <9F7C2AD8-A9A7-3DE4-828D-B0F0F166AAA0> /usr/lib/system/libunwind.dylib
    0x7fffa05aa000 -     0x7fffa05d3ff7  libxpc.dylib (972.1.5) <2A901937-48E1-3BF2-83F1-2431156D82B7> /usr/lib/system/libxpc.dylib

External Modification Summary:
  Calls made by other processes targeting this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by all processes on this machine:
    task_for_pid: 3690
    thread_create: 0
    thread_set_state: 0

VM Region Summary:
ReadOnly portion of Libraries: Total=120.7M resident=0K(0%) swapped_out_or_unallocated=120.7M(100%)
Writable regions: Total=81.5M written=0K(0%) resident=0K(0%) swapped_out=0K(0%) unallocated=81.5M(100%)

                                VIRTUAL   REGION
REGION TYPE                        SIZE    COUNT (non-coalesced)
===========                     =======  =======
Kernel Alloc Once                    8K        2
MALLOC                            73.5M       13
MALLOC guard page                   16K        4
STACK GUARD                       56.0M        2
Stack                             8192K        2
VM_ALLOCATE                          4K        2
__DATA                            1500K       45
__LINKEDIT                       111.5M        5
__TEXT                            9476K       44
shared memory                       12K        4
===========                     =======  =======
TOTAL                            259.7M      113

I think this is actually from after I started trying to hack on your code a bit, seeing if I could exorcise some of the stranger load command and MH_WHATEVER flags, which made no difference, at which point I realized this is probably something related the stubs, no?

I valiantly did try to build rebuild them, but no, I definitely do not have the right tools to do so. Although, in my defense, I'm pretty sure the issue is not that my copy off the stubtools is outdated, as the message says.

In any case, can I reasonably infer that this probably just boils down to the fact that they just aren't wanting to allow anything that "fully static", and not fully PIE? I think I might have come across a comment somewhere in this repo mentioning PIC or address randomization or something, but I've been up all night and I could have been imagining that or something.

In any case, I'm petty darn sure there's little point in my continuing to try to examine this myself any further. I would assume its reasonable to think that at some point there will be a version of UPX that works on a Mac again. But I wanted to ask, on a scale of probably-not-gonna-happen to you-dreamin-son, I was wondering if it you think it might be theoretically possible to seghack (swap out segments?) or otherwise patch (maybe using that crazy new LLVM macho-disassembly framework) to get existing UPX executables to run again?

Otherwise, maybe sometime next week I'll do a writeup on my awful kludge of a workaround.

The one sentence version is 1. grab an older Recovery Partition update pkg from the Apple SW update repos, extract it, do a block level copy of the partition into a mounted virtual disk...

@gingerbeardman

I came here to post about this, but not in so many words :)

@egraether

👍

@egraether egraether referenced this issue in CoatiSoftware/CoatiBugTracker Sep 16, 2016
Closed

Crash when opening app on macOS Sierra #191

@Anubis88

I also wanted to report this. I hope this gets fixed soon.

@markus-oberhumer
Contributor

We're aware of this issue - please see

https://sourceforge.net/p/upx/bugs/248/

@jreiser
Contributor
jreiser commented Sep 17, 2016

We're aware of the problem. I have a new version which compresses /bin/date so that the result runs on Sierra. The code is not quite ready for checkin. Lack of documentation from Apple has slowed progress. Yes, Sierra can run [some] non-PIE MH_EXECUTABLE files.

"...to get existing UPX executables to run again?" Decompress them: "upx -d my_app".

@sampl3x
sampl3x commented Sep 19, 2016 edited

All the apps that give the killed 9 error i try to do a upx -d gives:

upx: Patcher: NotPackedException: not packed by UPX

@jreiser
Contributor
jreiser commented Sep 19, 2016

If the app was code signed after compression by UPX, then try removing the code signature before decompressing. If nothing else, a truncated copy can be done by using 'dd' and some parameters provided by output from "otool -hl". Code signing adds a large amount of data to the end of the file. UPX looks near the end of the file to find its marker for clues about where the compressed data lives.

@sampl3x
sampl3x commented Sep 19, 2016

Jreiser tnx for your quick replay...

How can i see it codesigned, is there a tool for that?
How can i remove the code signing, Is there a tool for that?

@jreiser
Contributor
jreiser commented Sep 19, 2016

"otool -hl my_app" lists all the Mach-O headers including LC_CODE_SIGNATURE if present. Data member .fileoff indicates the start, and .filesize the end; and the region should be at the end of the file. Use 'dd' to make a copy that stops before the signature. Complain to Apple that the code signing tool does not have the feature of removing a signature.

@jreiser
Contributor
jreiser commented Sep 20, 2016

Call for testers: current git tip, commit c678ccd, works for me: compress and run a copy of /bin/date on Sierra.

$ upx -f -o date.upx /bin/date
     28544 ->     17008   59.59%   Mach/AMD64   date.upx
$ ls -l /bin/date date.upx
-rwxr-xr-x  1 root     wheel  28544 Sep  6 21:43 /bin/date
-rwxr-xr-x  1 jreiser  staff  17008 Sep 19 21:16 date.upx
$ sum /bin/date date.upx
9372 28 /bin/date
62796 17 date.upx

The compressed version also runs after code signing. What does NOT work yet is decompression (upx -d). That's next.

@rasky
rasky commented Sep 20, 2016

Was willing to try, but compilation fails on my Mac:

g++ -O2 -fno-strict-aliasing -fwrapv -Wall -W -Wcast-align -Wcast-qual -Wpointer-arith -Wshadow -Wwrite-strings -Werror -o p_armpe.o -c p_armpe.cpp
In file included from p_armpe.cpp:34:
./p_armpe.h:70:18: error: 'PackArmPe::processImports' hides overloaded virtual function [-Werror,-Woverloaded-virtual]
    virtual void processImports(unsigned, unsigned);
                 ^
./pefile.h:430:22: note: hidden overloaded virtual function 'PeFile32::processImports' declared here: different number of parameters (0 vs 2)
    virtual unsigned processImports();
                     ^
1 error generated.
make: *** [p_armpe.o] Error 1
@sampl3x
sampl3x commented Sep 20, 2016

What about files created with UPX a year ago, they still no run on Sierra right?

@jreiser
Contributor
jreiser commented Sep 20, 2016

@sampl3x: Correct, Sierra still does not run old compressed output; I didn't change Sierra. The workaround is: de-compress what old upx produced, then re-compress with new upx. The typical problem is that the old compressed output was then code signed, which hides the location of the compressed data from the de-compressor in the old upx; and the Apple code sign utility does not have the feature of removing the signature that it installed. As I explained in this thread before, you can remove the signature yourself using "otool -hl" to find out where it is, then 'dd' to make a copy that stops before the signature. I will see if the decompressor in the new upx effectively can do this for you.

@sampl3x
sampl3x commented Sep 20, 2016 edited

jreieser, tnx for the explanation.

i try to remove the signature but i don't know how with the otool.
I tried your command line and get the following output example:

otool -hl patcher
patcher:
Mach header
magic cputype cpusubtype caps filetype ncmds sizeofcmds flags
0xfeedfacf 16777223 3 0x80 2 21 3768 0x00000085
Load command 0
cmd LC_SEGMENT_64

all way down to then end:

cmd LC_CODE_SIGNATURE

cmdsize 16
dataoff 276912
datasize 10592

And then what to do with DD with this output??

@jreiser
Contributor
jreiser commented Sep 20, 2016

@sampl3x: First check that the signature is at the end of the file: "ls -l" should be equal to (.dataoff + .datasize). Then "dd if=my_app of=foo bs=276912 count=1" to produce new file 'foo' which contains the first 276912 bytes; that is, everything before the signature. Then "upx -d foo".

@sampl3x
sampl3x commented Sep 20, 2016

Ill try it.

also i tried this: https://www.dropbox.com/s/wmeeodg91d1qef2/stripcodesig.zip

Seems to work also but still UPX says its not an upx compressed file.

@RyuX51
RyuX51 commented Sep 20, 2016 edited

3.92 (commit 3f7c1f9) works when trying to compress and run date, where 3.91 was not. Thank you.
Also still getting NotPackedException: not packed by UPX when trying to decompress old ones after removing the code signature.

@sampl3x
sampl3x commented Sep 20, 2016

I dont know how to upgrade UPX using brew. brew update doesnt update upx to 3.92.

@RyuX51
RyuX51 commented Sep 20, 2016 edited

No, 3.92 is still in development. You have it to build from the sources to test it. Or be patient a little more, it's already been merged into master. :)

@jreiser
Contributor
jreiser commented Sep 21, 2016

"upx -d" to decompress packed Mach-O executables now works for me, for both old (upx 3.91, pre-Sierra) and new (soon-to-be upx 3.92, Sierra) compressed files, and both when code signed after compression or not signed. The commit is ad6914b which is now on branch 'devel'. The diff for that commit gives clues how upx detects where the compressed data lives.

If "upx -d my.app" does not work for you then please post the output from "otool -hl my.app".

@sampl3x
sampl3x commented Sep 21, 2016 edited

Is it possible you create a mac binary so i can test it?
Or how i can add the branch devel to brew..

@RyuX51
RyuX51 commented Sep 21, 2016

The output is from the untouched signed executable:

signed:
Mach header
magic cputype cpusubtype caps filetype ncmds sizeofcmds flags
0xfeedfacf 16777223 3 0x80 2 6 648 0x00000001
Load command 0
cmd LC_SEGMENT_64
cmdsize 72
segname __PAGEZERO
vmaddr 0x0000000000000000
vmsize 0x0000000000001000
fileoff 0
filesize 0
maxprot 0x00000000
initprot 0x00000000
nsects 0
flags 0x0
Load command 1
cmd LC_SEGMENT_64
cmdsize 152
segname __XHDR
vmaddr 0x0000000000001000
vmsize 0x0000000000001000
fileoff 0
filesize 4096
maxprot 0x00000007
initprot 0x00000007
nsects 1
flags 0x0
Section
sectname __xhdr
segname __XHDR
addr 0x0000000000001298
size 0x0000000000000000
offset 664
align 2^2 (4)
reloff 0
nreloc 0
flags 0x00000000
reserved1 0
reserved2 0
Load command 2
cmd LC_SEGMENT_64
cmdsize 152
segname __TEXT
vmaddr 0x000000010001b000
vmsize 0x0000000000008cac
fileoff 0
filesize 35608
maxprot 0x00000007
initprot 0x00000007
nsects 1
flags 0x0
Section
sectname __text
segname __TEXT
addr 0x000000010001b2c0
size 0x00000000000089ec
offset 704
align 2^2 (4)
reloff 0
nreloc 0
flags 0x00000000
reserved1 0
reserved2 0
Load command 3
cmd LC_SEGMENT_64
cmdsize 72
segname __LINKEDIT
vmaddr 0x0000000100024000
vmsize 0x0000000000003000
fileoff 36864
filesize 9840
maxprot 0x00000007
initprot 0x00000001
nsects 0
flags 0x0
Load command 4
cmd LC_UNIXTHREAD
cmdsize 184
flavor x86_THREAD_STATE64
count x86_THREAD_STATE64_COUNT
rax 0x0000000000000000 rbx 0x0000000000000000 rcx 0x0000000000000000
rdx 0x0000000000000000 rdi 0x0000000000000000 rsi 0x0000000000000000
rbp 0x0000000000000000 rsp 0x0000000000000000 r8 0x0000000000000000
r9 0x0000000000000000 r10 0x0000000000000000 r11 0x0000000000000000
r12 0x0000000000000000 r13 0x0000000000000000 r14 0x0000000000000000
r15 0x0000000000000000 rip 0x000000010002344c
rflags 0x0000000000000000 cs 0x0000000000000000 fs 0x0000000000000000
gs 0x0000000000000000
Load command 5
cmd LC_CODE_SIGNATURE
cmdsize 16
dataoff 36864
datasize 9840

@jreiser
Contributor
jreiser commented Sep 21, 2016

@RyuX51 I'd like to look at the file. Can you put it somewhere I can download it (such as dropbox, etc.) then send me the filename at my address given in the sources? Or, at only about 47KB, it might be small enough for email. Thanks.

@sampl3x
sampl3x commented Sep 21, 2016

jreiser can you share the UPX 3.9.2 mac binary?

@jreiser
Contributor
jreiser commented Sep 21, 2016

@sampl3x I develop on Linux. It will take a while to generate a mac binary.

@sampl3x
sampl3x commented Sep 21, 2016

Oke no problem, maybe someone else can tell me how to create it on a Mac so i can test it.

@gingerbeardman

Thanks for the binary

chmod +x upx.out
./upx.out
@sampl3x
sampl3x commented Sep 21, 2016

Tnx jreiser!

@Bicet
Bicet commented Sep 21, 2016

I still have the problem with @jreiser's file :)
NotPackedException: not packed by UPX

thank you anyway

@warking
warking commented Sep 21, 2016

@sampl3x your executable "patcher" may NOT be UPX compressed at all. check with this:

hexdump -C path_to_your_patcher | grep -C 1 UPX

@sampl3x
sampl3x commented Sep 21, 2016

@warking I tried your command but but get no output.

Like Bicet i still get NotPackedException: not packed by UPX.

But when i run the app on Sierra i get: This UPX compressed binary contains an invalid Mach-O header and cannot be loaded.

@RyuX51
RyuX51 commented Sep 21, 2016 edited

This excludes me then, too. There is no UPX_DATA or even UPX in the hexdump. I feel stupid never questioning if the file really is UPX compressed. The crash report from geoff-codes looks nearly identically to mine so I was curious and wanted to lend a hand but ended up wasting your time instead. I apologise.
So Termination Reason: EXEC, [0xc] This UPX compressed binary contains an invalid Mach-O header and cannot be loaded. is misleading, because Sierra falsely assumes an UPX compressed file where there is none.
Have a nice day. :)

@Bicet
Bicet commented Sep 21, 2016

So Termination Reason: EXEC, [0xc] This UPX compressed binary contains an invalid Mach-O header and cannot be loaded. is misleading, because Sierra falsely assumes an UPX compressed file where there is none.

this seems a good question...

@jreiser
Contributor
jreiser commented Sep 21, 2016

@RyuX51 : Thank you for apologizing, but to me it seems possible that some "pirate" has deliberately removed the "UPX!" string that marks the pointer to the compressed data, and UPX does not look more closely. Your output from "otool -hl" is so close to what UPX generates that I would like to examine the file carefully. Would you please send it to me?

@RyuX51
RyuX51 commented Sep 21, 2016

Of course, I'll send it right away.

@markus-oberhumer
Contributor

Please wait with further testing until the upx-testsuite is functional.

@jreiser
Contributor
jreiser commented Sep 22, 2016

@RyuX51: Yes, a "pirate" has zeroed the markers, and the UPX copyright notice. Of course some redundancy remains (actual execution does find the compressed data) but it will take more work for "upx -d" to locate the input for the offline decompression.

@sampl3x
sampl3x commented Sep 22, 2016

@jreiser that's why we get NotPackedException: not packed by UPX and it looks like it was not packed with UPX.

I hope you can find a way to decompress it still..

@RyuX51
RyuX51 commented Sep 22, 2016

@sampl3x Why would he do that? He just found out that this bug is not os related but a issue appearing in illegal patchers because some pirate overwrote the UPX markers. This is clearly nothing he should be asked to care of since UPX is working fine. With this discovery and the separation from https://sourceforge.net/p/upx/bugs/248/ (which has already been solved) this issue should be considered done.

@sbarex
sbarex commented Sep 22, 2016

yes the "pirated" binary has substitute the UPX! signature with SPLK string. With a hex editor you can change any occurrence of SPLK string with UPX! then you can decompress the executable.
The the app works (but show that the file was changed and cannot run...)

@sampl3x
sampl3x commented Sep 22, 2016 edited

I ask this because then we can unpack them and run them in Sierra.

@sampl3x
sampl3x commented Sep 22, 2016

@sbarex when i do a hexdump -C patcher | grep -C 1 SPLK i found no string, so can you tell me how to do that?

@RyuX51
RyuX51 commented Sep 22, 2016 edited

There was an issue with Sierra and it has been taken care of. This issue here has nothing to do with it but has been mistaken for it. The developers of UPX are not responsible to repair binaries that have been meddled with afterwards and they should not even be asked to do so. Do you disagree?
Don't be so desperate, just wait until a new version of your patcher is released. It should not take long. The piracy wheel spins fast.

@sbarex
sbarex commented Sep 22, 2016

We are OT! But the problem is not on patcher file. There is another file to unpack...

@RyuX51
RyuX51 commented Sep 22, 2016 edited

@jreiser

To put something slightly productive in:
In commit 544ec21 there was added in Makefile:

ifeq ($(findstring clang,$(CXX)),)
CXXFLAGS += -fno-delete-null-pointer-checks
endif

which stops building with the error:

clang: error: optimization flag '-fno-delete-null-pointer-checks' is not supported

Furthermore I have to remove in conf.h:

if !defined(UCL_VERSION) || (UCL_VERSION < 0x010300L)

error "please upgrade your UCL installation"

endif

because it breaks with

./conf.h:136:6: error: "please upgrade your UCL installation"

I need to remove both these parts to be able to build with clang-800.0.38 on macOS.

@markus-oberhumer
Contributor

@RyuX51 Try using "make CXX=clang" instead.

Also, UPX needs UCL 1.03, so what is your UCL_VERSION?

@RyuX51
RyuX51 commented Sep 22, 2016 edited

make CXX=clang breaks with because of undefined symbols for architecture x86_64 which seems to be connected with NLZMA. (Whole output if requested.) My UCL version is (still) 1.03.
You don't need to fix this for me, I just wanted to mention this because commit 3f7c1f9 worked with a simple make (and setting UPX_UCLDIR, of course).

@dylib
dylib commented Sep 22, 2016

O' Apple what have you done now? This entire fiasco could have been avoided had they not jumped the gun again. It's a recurring theme it seems though, label everything this, when it's not even that. It's too bad we've got to clean up the mess each time this flawed logic trickles down upon us.

@sampl3x
sampl3x commented Sep 22, 2016

So what did Apple do to prevent us from running binaries packed with UPX <3.91?

@RyuX51
RyuX51 commented Sep 22, 2016

@sampl3x Mr. Reiser already answered this exact question to you:

The typical problem is that the old compressed output was then code signed, which hides the location of the compressed data from the de-compressor in the old upx; and the Apple code sign utility does not have the feature of removing the signature that it installed.

@sampl3x
sampl3x commented Sep 22, 2016

So the codesigning is not supported anymore in Sierra?

@sampl3x
sampl3x commented Sep 24, 2016

yes the "pirated" binary has substitute the UPX! signature with SPLK string. With a hex editor you can change any occurrence of SPLK string with UPX! then you can decompress the executable.
The the app works (but show that the file was changed and cannot run...)

I cant find the SPLK string in the binary, any update on the upx -d option that unpacks the file even if there is signature?

@grgarside

@RyuX51 Using that version of upx, I now get ‘CantUnpackException: file corrupted’ — is this a different problem?

@Bicet
Bicet commented Sep 24, 2016

@RyuX51 Using that version of upx, I now get ‘CantUnpackException: file corrupted’ — is this a different problem?

same here :)

@RyuX51
RyuX51 commented Sep 24, 2016 edited

I don't think so. It decompresses the binary I sent him. Maybe (for now?) only this binary because this was the example file. Was really surprised though that he takes on this issue...

@jreiser
Contributor
jreiser commented Sep 24, 2016

Obviously I need more examples for decompression. The upx source code has one address for me, the git metadata has another. If your file is 128KiB or less, then please send it. If your file is larger, then please use 'dd' to create two 64KiB pieces: one from the beginning, and one from the end (and please tell me the total length, too.)

@jreiser
Contributor
jreiser commented Sep 24, 2016

trial mac binary for testing decompression: https://www.dropbox.com/s/xufsyqc1uo8smxf/upx.out?dl=0

@RyuX51
RyuX51 commented Sep 24, 2016 edited

I'm pretty sure you won't lack samples soon after asking for them here... ^__^

@aerge
aerge commented Sep 24, 2016 edited

@jreiser e: link removed
Really appreciate your help with this!

@jreiser
Contributor
jreiser commented Sep 24, 2016

https://www.dropbox.com/s/x765t3i42p7hr8b/upx.out?dl=0 now works for I386, too.

@dylib
dylib commented Sep 24, 2016 edited

Using the following on El Capitan (OS X 10.11.6):

https://www.dropbox.com/s/x765t3i42p7hr8b/upx.out?dl=0 now works for I386, too.

I get the following results:

$ ./upx.out PackTest.app/Contents/MacOS/PackTest 
                       Ultimate Packer for eXecutables
                          Copyright (C) 1996 - 2016
UPX 3.92-BETA   Markus Oberhumer, Laszlo Molnar & John Reiser   Sep 22nd 2016

        File size         Ratio      Format      Name
   --------------------   ------   -----------   -----------
     14968 ->      8816   58.90%   macho/amd64   PackTest                      

Packed 1 file.

When trying to execute the resulting app:

9/24/16 10:40:15.370 AM syspolicyd[754]: assessment denied for .app
com.apple.message.domain: com.apple.security.assessment.outcome2
com.apple.message.signature2: bundle:com.foobar.PackTest
com.apple.message.signature3: .app
com.apple.message.signature5: 1.0
com.apple.message.signature4: 1
com.apple.message.signature: denied:no usable signature
SenderMachUUID: 035F8EDC-510F-3824-B171-4AA92313E2C0

9/24/16 10:40:34.173 AM ReportCrash[15108]: com.apple.message.domain: com.apple.crashreporter.writereport.crash
com.apple.message.signature: PackTest
com.apple.message.signature2: com.foobar.PackTest ||| ??? (???)
com.apple.message.signature3: 3EE7A6CD011434BFCD14DAFACE4E9269
com.apple.message.result: YES
com.apple.message.summarize: YES
SenderMachUUID: BAB8DC6C-25DC-37CF-9484-4DEF8FBB8555

I will add that the app is not codesigned, but launches fine when not packed, file here:

https://www.dropbox.com/s/2btwh7er931460x/PackTest.dmg?dl=0

Crash report:

Process:               PackTest [15218]
Path:                  /PackTest.app/Contents/MacOS/PackTest
Identifier:            com.foobar.PackTest
Version:               ???
Code Type:             X86-64 (Native)
Parent Process:        ??? [1]
Responsible:           PackTest [15218]
User ID:               501

Date/Time:             2016-09-24 11:00:15.370 -0700
OS Version:            Mac OS X 10.11.6 (15G1004)
Report Version:        11
Anonymous UUID:        EDCDE8EC-BD7D-34C0-81FE-F96E6789E207

Sleep/Wake UUID:       B586D5D1-0E23-44A2-B771-4D643EACC65B

Time Awake Since Boot: 71000 seconds
Time Since Wake:       68000 seconds

System Integrity Protection: disabled

Crashed Thread:        Unknown

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       KERN_INVALID_ADDRESS at 0x00007fff77bffc79
Exception Note:        EXC_CORPSE_NOTIFY

Backtrace not available

Unknown thread crashed with X86 Thread State (64-bit):
  rax: 0x00000001000013fe  rbx: 0x0000000000000013  rcx: 0x0000000018000000  rdx: 0x1800000001000000
  rdi: 0x0000002a00000004  rsi: 0x0000000000fffffc  rbp: 0x00007fff5fbff3a8  rsp: 0x00007fff5fbff308
   r8: 0x0000000000000000   r9: 0x00007fff5fbff4b0  r10: 0x0000000000000000  r11: 0x0000000000000202
  r12: 0x0000000100004000  r13: 0x00007fff77bffc79  r14: 0x000000000000039b  r15: 0x0000000000000010
  rip: 0x00000000f0000c04  rfl: 0x0000000000010293  cr2: 0x00007fff77bffc79

Logical CPU:     2
Error Code:      0x00000004
Trap Number:     14

Binary images description not available

External Modification Summary:
  Calls made by other processes targeting this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by all processes on this machine:
    task_for_pid: 201850
    thread_create: 1
    thread_set_state: 75

Unpacking/Packing looks to work just fine — even with "zeroed" binaries. It's the actual packed binaries that still don't seem to execute which is problematic.

@jreiser
Contributor
jreiser commented Sep 24, 2016

@dylib thank you for the report. I have downloaded the .dmg and begun looking; progress might be slower for a while.

@jreiser
Contributor
jreiser commented Sep 25, 2016

Bug fix to allow longer Mach-O commands, but I still see "No Info.plist file in application bundle or no NSPrincipalClass in the Info.plist file, exiting". https://developer.apple.com/library/content/documentation/General/Reference/InfoPlistKeyReference/Articles/AboutInformationPropertyListFiles.html says

Important: The property list editor in Xcode displays human-readable strings (instead of the actual key name) for many keys by default. To display the actual key names as they appear in the Info.plist file, Control-click any of the keys in the editor window and enable the Show Raw Keys/Values item in the contextual menu.

For sure the UPX-compressed output must reside in the same directory as the original file, and the Info.plist may even use some "absolute name" such as a filesystem UUID instead of path and directory names. Also, try using the code signing tool to add a UUID to the compressed file.

@sampl3x
sampl3x commented Sep 25, 2016

I also get upx.out: patcher: CantUnpackException: file corrupted

@jreiser
Contributor
jreiser commented Sep 25, 2016

@sampl3x Please send me the file? If that's not possible then please send the first 256 lines of output from either one of these dumps:

hexdump -C my_app
od -Ax -tx4 my_app
@jreiser
Contributor
jreiser commented Sep 25, 2016

@sampl3x After reconstructing the file from the 'od' (the file size is 287504 bytes and contains the LC_UUID of 6D30BC5F-7E51-3890-B450-D9C8B6AD19E4) and looking at it carefully, I conclude that it was not packed by UPX. There are no traces of UPX anywhere in the file. Running the file under the debugger lldb: it invokes, loads 193 modules according to "(lldb) target modules list", and makes a bad memory reference:

* thread #1: tid = 0x10526, 0x00007fffb143d68d libobjc.A.dylib`_mapStrHash(_NXMapTable*, void const*) + 11, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=50, address=0x100038fe0)
    frame #0: 0x00007fffb143d68d libobjc.A.dylib`_mapStrHash(_NXMapTable*, void const*) + 11
libobjc.A.dylib`_mapStrHash:
->  0x7fffb143d68d <+11>: movb   (%rsi), %al

Please find someone to help you in person. The posts here have been much less useful than expected.

@jreiser
Contributor
jreiser commented Sep 25, 2016

https://www.dropbox.com/s/f1xqw40me5wlqj5/upx.out?dl=0 fixes a bug when more than 2048 bytes in Mach-O headers. (@dylib in particular)

@sampl3x
sampl3x commented Sep 26, 2016

@jreiser i can unpack now the files! The patcher file didn't worked but the other files are working now.

@corv45
corv45 commented Sep 26, 2016

got it to work thanks

@corv45
corv45 commented Sep 26, 2016

Thank amao

@asmaloney asmaloney referenced this issue in CloudCompare/CloudCompare Sep 28, 2016
Closed

CC crashes on startup with MacOS Sierra #439

@dylib
dylib commented Sep 29, 2016 edited

https://www.dropbox.com/s/f1xqw40me5wlqj5/upx.out?dl=0 fixes a bug when more than 2048 bytes in Mach-O headers. (@dylib in particular)

@jreiser: The version you've supplied above appears to work for some packed x86_64 (non-codesigned) binaries running on 10.11.6 (El Capitan) and 10.12.0 (Sierra) without the crash/issues from before. If you need anything else in terms of testing candidates, etc. just let me know and I'll be happy to provide it.

If the .app was previously codesigned and the binary is then packed trying to run it will result in:

9/29/16 7:20:59.062 AM syspolicyd[754]: assessment denied for .app
com.apple.message.domain: com.apple.security.assessment.outcome2
com.apple.message.signature2: bundle:com.foobar
com.apple.message.signature3: .app
com.apple.message.signature5: 1.0
com.apple.message.signature4: 1
com.apple.message.signature: denied:no usable signature

Trying to re-codesign the .app with the now packed binary will result in:

app: object file format unrecognized, invalid, or unsuitable

So it appears the main issue now would be the signature becoming invalid, or the packed .app simply not being able to be codesigned, although for now stripping the signature seems to work ( at least partially for the time being ).

@jreiser: PS: Thank you very much for all the work you've put into resolving this issue — it's been seriously appreciated!

@solidsnack
solidsnack commented Oct 1, 2016 edited

Here's a shell function to automate the unsigning process that @jreiser mentions above:

function unsign {
  local binary output

  binary="${1:?'Please supply a path to a packed binary as the 1st argument.'}"
  output="${2:?'Please provide an output path as the 2nd argument.'}"

  offset="$(otool -hl "$binary" |
            sed -n '/LC_CODE_SIGNATURE/,$ { /dataoff/ p ;}' |
            egrep -o '[0-9]+$')"

  echo "Unsigning binary (removing signature at offset $offset)." >&2
  dd if="$binary" count=1 bs="$offset" of="$output"
}

Use it like so:

unsign  /tmp/TheApp.app/Contents/MacOS/TheApp /tmp/TheApp
upx -f -o /tmp/TheApp.app/Contents/MacOS/TheApp -d /tmp/TheApp
@jreiser
Contributor
jreiser commented Oct 2, 2016

@dylib https://www.dropbox.com/s/kszfp2p8a1c1mwc/upx.out?dl=0 : I can compress /bin/date (which is already signed by Apple), run the compressed output, codesign that myself, run the compressed-then-signed output, and decompress both.

@dylib
dylib commented Oct 2, 2016

@jreiser I tested the following .app combinations with UPX (universal binaries, x86_64, i386, apps which include .dylibs), and it looks like the latest version resolved everything that was critical.

Type                    Codesigned     Re-Codesigned      Packed      Unpacked     Result
Uni .app bundle             ✕                ✕              ✓             ✓          ✅
Uni .app bundle             ✓                ✕              ✓             ✓          ✅
Uni .app bundle+            ✕                ✕              ✓             ✓          ✅
Uni .app bundle+            ✓                ✓              ✓             ✓          ✅
x86_64 .app bundle          ✕                ✕              ✓             ✓          ✅
x86_64 .app bundle          ✓                ✓              ✓             ✓          ✅
x86_64 .app bundle+         ✕                ✕              ✓             ✓          ✅
x86_64 .app bundle+         ✓                ✓              ✓             ✓          ✅

+ indicates .app bundle included .dylib

There might be a few scenarios that I might not have considered, although it looks like everything is now working on 10.12. :)

@gubatron gubatron added a commit to frostwire/frostwire that referenced this issue Oct 12, 2016
@gubatron gubatron [desktop] fwplayer_osx upx encryption was broken for macOS sierra.
The solution was to re-compress with a beta macOS upx binary by @jreiser himself
see: upx/upx#4
#issuecomment-251871590 (published 7 days ago)

This binary has also been signed.
b93be7a
@vszakats vszakats referenced this issue in Homebrew/homebrew-core Oct 12, 2016
Merged

upx: project moved to github, update urls #5850

2 of 4 tasks complete
@sromocki

Is there an ETA on the next release?

@zmwangx zmwangx referenced this issue in Homebrew/homebrew-core Oct 25, 2016
Closed

Formulae without Sierra bottles #5488

70 of 156 tasks complete
@zmwangx zmwangx added a commit to zmwangx/homebrew-core that referenced this issue Oct 25, 2016
@zmwangx zmwangx upx: slap on an el_capitan MaximumMacOSRequirement
upx 3.91 is completely broken on Sierra, and there's no easy
rescue. Need to wait for the next release, for which no one seems to
have an ETA.

See:
- https://sourceforge.net/p/upx/bugs/248/
- upx/upx#4
f54b4d4
@zmwangx zmwangx referenced this issue in Homebrew/homebrew-core Oct 25, 2016
Merged

upx: slap on an el_capitan MaximumMacOSRequirement #6264

4 of 4 tasks complete
@BrewTestBot BrewTestBot added a commit to BrewTestBot/homebrew-core that referenced this issue Oct 25, 2016
@zmwangx @BrewTestBot zmwangx + BrewTestBot upx: slap on an el_capitan MaximumMacOSRequirement
upx 3.91 is completely broken on Sierra, and there's no easy
rescue. Need to wait for the next release, for which no one seems to
have an ETA.

See:
- https://sourceforge.net/p/upx/bugs/248/
- upx/upx#4
848adb4
@zmwangx zmwangx added a commit to Homebrew/homebrew-core that referenced this issue Oct 26, 2016
@zmwangx zmwangx upx: slap on an el_capitan MaximumMacOSRequirement (#6264)
upx 3.91 is completely broken on Sierra, and there's no easy
rescue. Need to wait for the next release, for which no one seems to
have an ETA.

See:
- https://sourceforge.net/p/upx/bugs/248/
- upx/upx#4
1ad90de
@asmaloney

@markus-oberhumer Do you have a rough estimate for the next release with this fix? I have two product releases on hold for this.

Thanks!

@markus-oberhumer
Contributor

I need green light from @jreiser and @ml1050 before a new release. I think there are still some open issues, though. In case of hurry you can use the current "devel" branch - Sierra issues may work perfectly for your binary (otherwise please file a bug report).

@asmaloney

Understood - thank you for the update!

@korczis
korczis commented Nov 13, 2016

Is there any ETA for fix?

@jreiser
Contributor
jreiser commented Nov 13, 2016

Functional fixes (UPX and compressed executables run on MacOS "Sierra") have been committed to the 'devel' branch as early as Sept.21. Various updates have made de-compression work in more cases, including for "pirated" outputs which have tried to hide the use of UPX. Several pre-compiled binaries for x86_64 MacOS have been made available; look for "dropbox" in my posts above. The latest is today's:
https://www.dropbox.com/s/g9gzvm819t55tse/upx-2016-11-13.out?dl=0
Please try it, and report your experience here.

@markus-oberhumer markus-oberhumer added this to the v3.92 milestone Nov 17, 2016
@felice64
felice64 commented Nov 17, 2016 edited

I tested it on this but the result is "permission denied"

MacBook-Pro-di-F:~ F$ /Users/F/Downloads/upx-2016-11-14.out -d /Users/F/Downloads/AutoCAD\ 2016\ for\ Mac\ +\ something/xf-adsk2016.dmg
-bash: /Users/F/Downloads/upx-2016-11-14.out: Permission denied
MacBook-Pro-di-F:~ F$

@NickSC
NickSC commented Nov 17, 2016

chmod +x /Users/F/Downloads/upx-2016-11-14.out

@jreiser
Contributor
jreiser commented Nov 19, 2016

On 11/17/2016 04:33 AM, Felice wrote:

Thanks. The problem is solved but now Iget this:

Exception Type: EXC_CRASH (SIGKILL)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Exception Note: EXC_CORPSE_NOTIFY

Termination Reason: EXEC, [0xc] This UPX compressed binary contains an invalid Mach-O header and cannot be loaded.

Please tell us which version of MacOS you are running (Sierra? El Capitan? ...).
What Version number does (apple) > About This Mac say?

What program was running when the "EXC_CRASH (SIGKILL)" occurred?

Please try compressing /bin/date, then running the compressed version:
upx-2016-11-14.out -f -o foo /bin/date
./foo

If a program crashes, then please run this command, and send a copy+paste of the output:
otool -hl the_program_which_crashes
If the crashing program was compressed by upx, then please also run
otool -hl the_never_compressed_program_file
and send the output.

All that additional information will help us understand your environment
and provide clues about what is going wrong and how to fix it.

Thank you,

@jsssw2
jsssw2 commented Nov 19, 2016

@jreiser I'm trying to launch ableton and it keeps giving me the mac 0 thing, on the sierra os. Any ideas?

@jreiser
Contributor
jreiser commented Nov 19, 2016

@jssw2 Please provide more context. "launch ableton": Which ableton? The Live 9 suite with free 30-day trial, or something else? "the mac 0 thing": Please describe the symptom in more detail. Which program (such as the pathname), which invocation method (command line or double-click or ...), which parameters, where does the symptom appear (stderr, system log, ...), what is the exact quote of the complaint? Then please run "otool -hl executable_filename" on the program which complains, and on the corresponding original, never-compressed executable. If you don't have 'otool' then please run "od -Ax -tx4 executable_filename | sed 256q" instead. All this information is reasonably necessary to identify, find, and fix problems.

@jsssw2
jsssw2 commented Nov 20, 2016

@jreiser Its live 9 suite pirated, Crashed Thread: 0 Dispatch queue: com.apple.main-thread

Exception Type: EXC_CRASH (SIGKILL)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Exception Note: EXC_CORPSE_NOTIFY
Termination Reason: EXEC, [0xc] This UPX compressed binary contains an invalid Mach-O header and cannot be loaded.

Thats what I'm seeing

@jreiser
Contributor
jreiser commented Nov 20, 2016

@jsssw2 Take the file that was compressed by a previous version of upx, de-compress it using the new version ("upx-2016-11-14 -d ableton"), re-compress it ("upx-2016-11-14 ableton"), and then it should work. If not, then please provide the info requested 2 comments above (namely: output from otool or od, for both the de-compressed and re-compressed files.)

@jsssw2
jsssw2 commented Nov 20, 2016

@jreiser wait where do i get the new version of upx and how do i change it ? I have a mac btw.

@markus-oberhumer
Contributor

@jsssw2 We're not interested into supporting any "hacked" or "pirate" executables, so please stop talking about those.

@jreiser
Contributor
jreiser commented Nov 21, 2016

fixed on devel branch

@jreiser jreiser closed this Nov 21, 2016
@sephethus

How do I install from devel branch?

@jreiser
Contributor
jreiser commented Nov 21, 2016 edited

There is a pre-compiled binary upx-2016-11-14.out that was announced a week ago; look back in this Issue.
High-level overview: README.SRC and other files at the top level.

git clone https://github.com/upx/upx  # 52MiB
cd upx;  git checkout devel
cd src
export UPX_UCLDIR=...  # wherever you put ucl-1.03
make
@dylib
dylib commented Nov 21, 2016

@felice64: upx does not unpack .dmg files; rtfm!

@sephethus

@jreiser why do I get the error: No rule to make target .depend', needed byc_file.o'. Stop.

Seems to be a file missing.

@jreiser
Contributor
jreiser commented Nov 21, 2016

@sephethus Works for me, in a new directory. In src/Makefile:

:g/BUILD_USE_DEPEND/p   # vi command to show all lines with that string
BUILD_USE_DEPEND    ?= 1     # defaults to 1
ifeq ($(BUILD_USE_DEPEND),1)  # two tests
ifeq ($(BUILD_USE_DEPEND),1)

Actual lines from Makefile:

ifeq ($(BUILD_USE_DEPEND),1)
./.depend: $(sort $(wildcard $(srcdir)/*.cpp $(srcdir)/*.h)) $(MAKEFILE_LIST)
        @rm -f $@
        @echo "Updating $@"
        @$(strip $(CXX) $(call ee,CPPFLAGS) $(call ee,CXXFLAGS) -MM) $(filter %.cpp,$^) > $@

My versions:

$ make --version
GNU Make 3.81
   [[snip]]
This program built for i386-apple-darwin11.3.0
$ c++ --version
Apple LLVM version 8.0.0 (clang-800.0.42.1)
Target: x86_64-apple-darwin16.3.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
$ 
@dylib
dylib commented Nov 29, 2016 edited

@sephethus: In the src/Makefile try commenting out:

CXXFLAGS += -fno-delete-null-pointer-checks

then it should build correctly, otherwise you might get:

clang: error: optimization flag '-fno-delete-null-pointer-checks' is not supported
make: *** Deleting file `.depend'
make: *** No rule to make target `.depend', needed by `c_file.o'.  Stop.
@korczis
korczis commented Nov 29, 2016
@lucasts
lucasts commented Nov 30, 2016

Worked it out decompressing and compressing(both with devel bin)

@arctelix
arctelix commented Dec 7, 2016 edited

I'm getting the following errors when running make on the devel branch:

c++ -O2 -fno-strict-aliasing -fwrapv -funsigned-char -Wall -W -Wcast-align -Wcast-qual -Wmissing-declarations -Wpointer-arith -Wshadow -Wvla -Wwrite-strings -Werror -o upx.out c_file.o c_init.o c_none.o c_screen.o compress.o compress_lzma.o compress_ucl.o compress_zlib.o except.o file.o filter.o filteri.o help.o lefile.o linker.o main.o mem.o msg.o p_armpe.o p_com.o p_djgpp2.o p_elks.o p_exe.o p_lx_elf.o p_lx_exc.o p_lx_interp.o p_lx_sh.o p_mach.o p_ps1.o p_sys.o p_tmt.o p_tos.o p_unix.o p_vmlinx.o p_vmlinz.o p_w16ne.o p_w32pe.o p_w64pep.o p_wcle.o packer.o packer_c.o packer_f.o packhead.o packmast.o pefile.o s_djgpp2.o s_object.o s_vcsa.o s_win32.o snprintf.o stdcxx.o ui.o util.o work.o -lucl -lz
ld: library not found for -lucl
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [upx.out] Error 1

Any ideas what this is?

I have installed ucl v1.03 with brew and exported the variable:
export UPX_UCLDIR=/usr/local/Cellar/ucl/1.03

make --version
GNU Make 3.81
This program built for i386-apple-darwin11.3.0
c++ --version
Apple LLVM version 8.0.0 (clang-800.0.42.1)
Target: x86_64-apple-darwin16.1.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
@jreiser
Contributor
jreiser commented Dec 7, 2016

The final command line should contain the substring

-L$(UPX_UCLDIR)/src/.libs/  -lucl  # expands to -L/usr/local/Cellar/ucl/1.03/src/.libs/  -lucl

so check the line in the Makefile

LIBS += $(addprefix -L,$(dir $(wildcard $(UPX_UCLDIR)/libucl$(libext) $(UPX_UCLDIR)/src/.libs/libucl$(libext))))
@arctelix
arctelix commented Dec 7, 2016 edited

That was it, the brew install looks like this:
/usr/local/Cellar/ucl/1.03/lib/libucl.a

So I changed the line as follows:

LIBS += $(addprefix -L,$(dir $(wildcard $(UPX_UCLDIR)/libucl$(libext) $(UPX_UCLDIR)/lib/libucl$(libext))))

And the build completed. Thanks!

@ilovezfs

It seems this may have been closed prematurely. The pull request to Homebrew for the Sierra compatible version (3.92) is failing as described here: Homebrew/homebrew-core#7956 (comment)

UPX 3.92 fixes support for building/running on Sierra. The tests are currently not running: compressing executables is broken (every result returns NotCompressibleException), but decompressing executables works.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment