application crashes when attaching Frida 1.4.0 #5

Closed
guillaume-uH57J9 opened this Issue May 13, 2014 · 11 comments

Projects

None yet

2 participants

@guillaume-uH57J9

Applications are crashing when Frida 1.4.0 is attaching. I have no idea why.

This is observed with various application. The most famous of them is 7-Zip, and it's open-source, so I this reports will use 7-Zip as example for the crash.

Software: 7-Zip 9.20
OS: Windows 1 SP1 64-bit

Reproducing:

  • Start 7-Zip GUI (executable 7zfm.exe)
  • Run command frida-trace.exe -i "*add*" 7zfm.exe

Expected result: frida-trace attach to 7zfm.exe and log calls.
Actual result: frida-trace attach 7zfm.exe and it crash immediately.

Generated Handlers : http://wikisend.com/download/366112/__handlers__.7z (link valid 7 days).

Console output:

 frida-trace.exe -i "*add*" 7zfm.exe

 Attaching...
 Resolving functions...
 Uploading data...
 svn_ra_svn__get_addresses: Loaded handler at XXX\svn_ra_svn__get_addresses.js"
 apr_procattr_addrspace_set: Loaded handler at XXX
 ___unguarded_readlc_active_add_func: Loaded handler at XXX
 ..... LOG CUT HERE .....
 _unloaddll: Loaded handler at XXX
 _FXp_addh: Loaded handler at XXX
 Ready!
 Started tracing 94 functions. Press ENTER to stop.
 Target process terminated.
 Stopping...

7-Zip crash signature:

 Problem signature:
 Problem Event Name: APPCRASH
 Application Name: 7zFM.exe
 Application Version: 9.20.0.0
 Application Timestamp: 4ce55007
 Fault Module Name: frida-agent-64.dll
 Fault Module Version: 0.0.0.0
 Fault Module Timestamp: 53658c45
 Exception Code: 40000015
 Exception Offset: 00000000000e1aaa
 OS Version: 6.1.7601.2.1.0.256.4
 Locale ID: 1033
 Additional Information 1: 6c4c
 Additional Information 2: 6c4ce84510697985dd5daef2daf436b9
 Additional Information 3: da9b
 Additional Information 4: da9beea80fcb265cc2444484cad4e572

Crash callstack:

 ...
 kernel32.dll!BasepReportFault() + 0x1f bytes
 kernel32.dll!UnhandledExceptionFilter() + 0x1fc bytes
 frida-agent-64.dll!frida_agent_main() + 0xe3f1e bytes
 [Frames below may be incorrect and/or missing, no symbols loaded for frida-agent-64.dll]
 frida-agent-64.dll!frida_agent_main() + 0xde9ca bytes
 frida-agent-64.dll!frida_agent_main() + 0x4e37d bytes
 frida-agent-64.dll!frida_agent_main() + 0x940a9 bytes
 frida-agent-64.dll!frida_agent_main() + 0x93dd7 bytes
 frida-agent-64.dll!frida_agent_main() + 0x93e4f bytes
 frida-agent-64.dll!frida_agent_main() + 0x86ae3 bytes
 frida-agent-64.dll!frida_agent_main() + 0x80aac bytes
 frida-agent-64.dll!frida_agent_main() + 0x8f468 bytes
 frida-agent-64.dll!frida_agent_main() + 0x24ce4b bytes
 frida-agent-64.dll!frida_agent_main() + 0x2120fd bytes
 frida-agent-64.dll!frida_agent_main() + 0x2119f5 bytes
 00000000f8a06352()
 00000000f8a44ca1()
 0000000900000000()
 00000000eb38c469()
 0000000003f9f9e8()
 00000000f8a9775e()
 00000000f8a062a1()
 0000000003f9f990()
 00000000f8a062a1()
 0000000003f9f9a0()
 0000000003f9fa38()
 00000000f8a97641()
 000003ff92f8f5a1()
 00000000eb41ed91()
 00000000eb41ecb9()
 00000000eb372b09()
 000003ff92f8f5a1()
 00000000eb3fa961()
 00000000eb3fa961()
 00000000eb3f9d99()
 0000000003f9fa70()
 00000000f8a972f5()
 000003ff92f8c069()
 00000000eb3fa961()
 00000000eb3fa961()
 000003ff92fadeb9()
 000003ff92f6b031()
 0000000003f9faa8()
 00000000f8a44fa6()
 000003ff92f8c069()
 000003ff92fadeb9()
 00000000f8a44ee1()
 0000000800000000() 
@oleavr
Contributor
oleavr commented May 13, 2014

Interesting.

To narrow it down, does it work without crashing if you:

  • Trace 0 functions (by omitting -i altogether)
  • Trace 1 Windows API function (say CreateFileW)

If those two work fine, it is likely one (or more) functions that contain an instruction sequence that Frida isn't able to modify correctly. Frida relocates one or more instructions at the beginning of each function in order to write a JMP instruction there, and in this process it rewrites position-dependent instructions for their new location in memory.

The way I usually narrow this down to the offending function(s) is to locally modify https://github.com/frida/frida-python/blob/5568a08b0b4892820d98957118479305b7790db0/src/frida/tracer.py#L95 (if using -i) so it just returns a subset of the functions, and then I just binary-search my way until I'm left with the one function that's not being handled correctly.

Thanks!

@guillaume-uH57J9

No crash if tracing nothing, or if tracing CreateFileW.

So I'll try a binary search to narrow it down.

Another method would be attaching to methods one by one with a delay (say 1
sec). Is that possible in theory or would it be useless trying to do that ?

Guillaume

@guillaume-uH57J9

I located one specific function call that trigger a crash upon Frida
attaching.

Module: libapr_rsvn.dll ("Apache Portable Runtime Library" version 1.5.0)
Function: apr_pollset_add

This DDL is shipped with TortoiseSVN 1.8.6, which use a DLL for integration
of SVN in windows explorer. Since 7-zip integrate a file explorer I guess
it automatically loads it.

Guillaume

@oleavr
Contributor
oleavr commented May 14, 2014

Great work!

What I usually do once I find the offending function is:
frida-repl <pid>
And in that REPL I run:
Memory.readByteArray(Module.findExportByName('libapr_rsvn.dll', 'apr_pollset_add'), 16)
Equipped with the raw instructions I then feed them into capstone. Because Frida's Interceptor currently needs to overwrite the 5 first bytes (that could be optimized later), the unsupported instruction must be one of the instructions encompassed by the 5 first bytes. Then I have a look at the X86Relocator tests and notice that is indeed not an instruction supported for relocation. I then add a failing test and dive into the implementation.

It's a bit tedious, which is why it would be awesome to integrate capstone into frida-repl. :)

Cheers

@oleavr
Contributor
oleavr commented May 14, 2014

Btw, attaching to methods one by one with a delay is doable, but so far in my adventures the binary-search approach has been quite effective and not so time-consuming that I felt like building something smarter. Something a bit more automated would be great though.

@oleavr
Contributor
oleavr commented May 14, 2014

Had a look at this suspect function:

>>> Memory.readByteArray(Module.findExportByName('libapr_tsvn.dll', 'apr_pollset_add'), 16)
0000  48 8b 41 50 48 ff 60 08 cc cc cc cc cc cc cc cc   H.APH.`.........

Which disassembles to:

mov rax, qword ptr [rcx + 0x50]
jmp qword ptr [rax + 8]

Do you get the same result for your version?

@oleavr
Contributor
oleavr commented May 14, 2014

Bingo! It looks like we're hitting this assertion:
https://github.com/frida/frida-gum/blob/0a88662e13c0e036dc851ee95bd59fbb2d0ae979/gum/arch-x86/gumx86relocator.c#L413

That shouldn't be too hard to fix. I'll have a look at it.

@oleavr oleavr added a commit to frida/frida-gum that closed this issue May 14, 2014
@oleavr oleavr Fix X86Relocator's handling of indirect jmp
Was very tempted to port X86Relocator to capstone (the plan is to
gradually remove the remaining dependencies on udis86). Unfortunately it
is a big undertaking to do this part, as it is used by the Stalker, which
means we need to plug in a memory allocator that only uses pre-allocated
memory. Sigh.

Fixes frida/frida-core#5.
caa9332
@oleavr
Contributor
oleavr commented May 14, 2014

Should now be fixed as of frida/frida-gum@caa9332.

@oleavr
Contributor
oleavr commented May 14, 2014

Released 1.4.2 with the fix. Hopefully it should be working now.

Cheers!

@guillaume-uH57J9

Great :) I tested and can confirm Frida 1.4.2 fixes this one.

Unfortunately I didn't have time to push the investigation further. But
your previous explanation on instruction debugging are interesting and will
be useful next time I get crashes.

Guillaume

@oleavr
Contributor
oleavr commented May 15, 2014

Glad to hear. Thanks for testing! √

@oleavr oleavr added a commit that referenced this issue Oct 4, 2016
@oleavr oleavr Fix heap corruption caused by use-after-free
Seems like `entry` getting cleaned up results in a weak pointer being
updated on an object that we just released. This seems like a bug in
libgee, so this workaround will be reverted once we have a proper fix
in libgee.

Details:

==73504==ERROR: AddressSanitizer: heap-use-after-free on address 0x6040000218f0 at pc 0x0001088ba31b bp 0x7fff577285b0 sp 0x7fff577285a8
WRITE of size 8 at 0x6040000218f0 thread T0
    #0 0x1088ba31a in g_nullify_pointer gutils.c:2051
    #1 0x108812656 in weak_refs_notify gobject.c:2638
    #2 0x10885865c in g_data_set_internal gdataset.c:407
    #3 0x10880d1fb in g_object_unref gobject.c:3148
    #4 0x108549333 in frida_darwin_host_session_on_uninjected darwin-host-session.c:4176

0x6040000218f0 is located 32 bytes inside of 40-byte region [0x6040000218d0,0x6040000218f8)
freed by thread T0 here:
    #0 0x1108a7e29 in wrap_free (libclang_rt.asan_osx_dynamic.dylib+0x4ae29)
    #1 0x108659586 in gee_hash_map_unset_helper hashmap.c:1692
    #2 0x108657554 in gee_hash_map_real_unset hashmap.c:1520
    #3 0x108549326 in frida_darwin_host_session_on_uninjected darwin-host-session.c:4175

previously allocated by thread T0 here:
    #0 0x1108a7c60 in wrap_malloc (libclang_rt.asan_osx_dynamic.dylib+0x4ac60)
    #1 0x108880688 in g_malloc gmem.c:95
    #2 0x10889ea55 in g_slice_alloc gslice.c:1012
    #3 0x10889f0be in g_slice_alloc0 gslice.c:1038
    #4 0x108658c47 in gee_hash_map_node_new hashmap.c:2084
    #5 0x108657297 in gee_hash_map_real_set hashmap.c:1494
    #6 0x10855011e in frida_darwin_host_session_real_perform_attach_to_co darwin-host-session.c:4046
4bef5e6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment