Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add events for Windows APC, NtContinue, callback, and cbret #241

Closed
derekbruening opened this issue Nov 27, 2014 · 5 comments
Closed

add events for Windows APC, NtContinue, callback, and cbret #241

derekbruening opened this issue Nov 27, 2014 · 5 comments

Comments

@derekbruening
Copy link
Contributor

From derek.br...@gmail.com on December 11, 2009 21:39:16

This was PR 200410

NtContinue and NtCallbackReturn can be watched for via the
syscall events, and int 0x2b via the bb event. The Ki entries
can be monitored via the bb event as well. So this would be a
purely convenience set of events. The callback start and stop
events can be important when keeping thread-shared state, because
often some part of the state should be shared across callbacks
while other parts should be separated.

Original issue: http://code.google.com/p/dynamorio/issues/detail?id=241

@derekbruening
Copy link
Contributor Author

From bruen...@google.com on August 28, 2011 14:01:59

drmgr's "callback-local storage" (CLS) from issue #402 solves the state sharing
problem

@derekbruening
Copy link
Contributor Author

I'm adding a "kernel transfer" event which includes these Ki events, syscalls that change context, the signal delivery from #1693, and sigreturn.

There are many complications. I will try to summarize them:

  • Some event types only have an OS context (sigcontext or CONTEXT) and it is not worth copying it into a dr_mcontext_t if the user doesn't care about anything but the pc. So I decided to pass in the prior pc (if known), the new pc, and the new xsp (used in DrM and other shadow mem tools), and have the client call dr_get_mcontext() if it really wants more. I added new "os_cxt_ptr_t" types and support for this.

  • There don't seem to be many uses cases for changing the context: for signals we already have DR_SIGNAL_REDIRECT; changing Ki dispatchers seems unlikely to happen; we can't do it for cbret; for Windows syscalls there are complications with the app only setting some of the ContextFlags. But it's not hard to have minimal support for it so I'm putting it in.

  • Continuing the ContextFlags issue: on query it's there as well, xbp in CONTEXT_CONTROL yet in DR_MC_INTEGER. I documented the issue: up to clients. They should pass both CONTROL and INTEGER if they want xbp, and it is rare to see an app select only one or the other.

  • I decided to raise this event on dr_redirect_execution() and DR_SIGNAL_REDIRECT too.

  • I added logic to pass the real Ki pc and not the hook-displaced pc.

  • I don't think it's possible to pass the real xsi and not the syscall return address for a cbret.

  • It is not easy to get a full source context so I'm only giving the pc, when it's known.

@derekbruening
Copy link
Contributor Author

These and #1693 can be used to simplify Dr. Memory, which today looks for the Ki entry points and monitors syscalls to get this information for updating shadow state.

@derekbruening
Copy link
Contributor Author

Dr. Memory's use case needs the source xsp for sigreturn handling. But if we provide a full source
context for syscalls, doesn't it seem weird to then require dr_get_mcontext
for the full target context?

Breaking down by event type:

  • sig deliver cache: src sc_orig untranslated, tgt sc
  • sig deliver dispatch: src dc->mc, tgt dc->mc
  • sigreturn: src dc->mc, tgt sc
  • APC dispatcher: src CONTEXT, tgt state->mc
  • callback dispatcher: no src, tgt state->mc
  • exception dispatcher: src CONTEXT, tgt state->mc
  • raise dispatcher: no src, tgt state->mc
  • NtContinue: src dc->mc, tgt CONTEXT
  • NtSetContextThread: src dc->mc, tgt CONTEXT
  • NtCallbackReturn: src dc->mc, tgt dc->mc BUT some regs not there like xsi
  • int 0x2b: src dc->mc, tgt dc->mc BUT some regs not there like xsi
  • dr_redirect_execution: no src (if from fault; have src for clean call), tgt dmc
  • DR_SIGNAL_REDIRECT: src dc->mc, tgt sc

I guess the biggest perf worry should be timer signals and Windows
callbacks. Timer signals should always be from dispatch, but we need to
copy mc to a dmc before setting up for the handler, and then copy the
handler's mc to a dmc: so two large copies if we have to include simd
state.

We could only include CONTROL and INTEGER for the src: simd should be
identical in every case to the tgt. So for efficiency, we'd pass in a
source dr_mcontext_t CONTROL+INTEGER, and just the target pc+xsp, and
require dr_get_mcontext() for the target??

@derekbruening
Copy link
Contributor Author

Getting the source context for some of these end up requiring extra work but it's worth it in the end as some use cases need it. For dr_redirect_execution we can call dr_get_mcontext() if we take one extra step and point at the exception context in cur_mc so it will work for both clean calls and exception events.

derekbruening added a commit that referenced this issue Nov 29, 2017
Adds a new client API event type: a kernel-mediated control transfer.  This
includes UNIX signal delivery and return, Windows APCs, callbacks,
exceptions, NtContinue, NtSetContextThread, and callback returns.  It is
also raised on client redirects.

The new interface passes the source context (control and integer, no
multimedia) but passes just key target fields directly (new pc, new xsp)
and does not pass in the full target register state to avoid copying costs
(particulary for multimedia).  Adds a new internal os_cxt_ptr_t type to
support pointing at a CONTEXT or sig_full_cxt_t for the new events and only
copying from that state if the client calls dr_get_mcontext().  Adds extra
logic to get the source context for dr_redirect_execution() called from the
exception event.

Includes limited support for calling dr_set_mcontext() and changing the pc
or other state, though there seem to be few use cases of this and given the
difficulty in testing it, it's not clear it's a worthwhile feature.

Adds documentation on the disparity between xbp being in CONTEXT_CONTROL
yet in DR_MC_INTEGER, cautioning clients to use both INTEGER and CONTROL
when they care about xbp.

Includes logic to pass the real Ki pc and not the hook-displaced pc for
dispatchers.

I don't think it's possible to pass the real xsi and not the syscall return
address for a cbret: we live with that.

Adds corresponding routines drmgr_register_kernel_xfer_event() and
drmgr_register_kernel_xfer_event_ex().  Re-implements drmgr's CLS API using
the new kernel xfer event (this is required for proper ordering of CLS vs
other clients using the xfer event).  Removes
DRMGR_PRIORITY_INSERT_CLS_ENTRY, DRMGR_PRIORITY_INSERT_CLS_EXIT,
DRMGR_PRIORITY_NAME_CLS_ENTRY, and DRMGR_PRIORITY_NAME_CLS_EXIT.

Adds tests to client.signal, client.flush, client.events,
client.drmgr-test, and a new test client.winxfer.  Testing of
dr_set_mcontext() on Windows was done manually: automated testing is
challenging to set up and left for future work.

Fixes #241
Fixes #1693
derekbruening added a commit that referenced this issue Nov 29, 2017
Adds a new client API event type: a kernel-mediated control transfer.  This
includes UNIX signal delivery and return, Windows APCs, callbacks,
exceptions, NtContinue, NtSetContextThread, and callback returns.  It is
also raised on client redirects.

The new interface passes the source context (control and integer, no
multimedia) but passes just key target fields directly (new pc, new xsp)
and does not pass in the full target register state to avoid copying costs
(particulary for multimedia).  Adds a new internal os_cxt_ptr_t type to
support pointing at a CONTEXT or sig_full_cxt_t for the new events and only
copying from that state if the client calls dr_get_mcontext().  Adds extra
logic to get the source context for dr_redirect_execution() called from the
exception event.

Includes limited support for calling dr_set_mcontext() and changing the pc
or other state, though there seem to be few use cases of this and given the
difficulty in testing it, it's not clear it's a worthwhile feature.

Adds documentation on the disparity between xbp being in CONTEXT_CONTROL
yet in DR_MC_INTEGER, cautioning clients to use both INTEGER and CONTROL
when they care about xbp.

Includes logic to pass the real Ki pc and not the hook-displaced pc for
dispatchers.

I don't think it's possible to pass the real xsi and not the syscall return
address for a cbret: we live with that.

Adds corresponding routines drmgr_register_kernel_xfer_event() and
drmgr_register_kernel_xfer_event_ex().  Re-implements drmgr's CLS API using
the new kernel xfer event (this is required for proper ordering of CLS vs
other clients using the xfer event).  Removes
DRMGR_PRIORITY_INSERT_CLS_ENTRY, DRMGR_PRIORITY_INSERT_CLS_EXIT,
DRMGR_PRIORITY_NAME_CLS_ENTRY, and DRMGR_PRIORITY_NAME_CLS_EXIT.

Adds tests to client.signal, client.flush, client.events,
client.drmgr-test, and a new test client.winxfer.  Testing of
dr_set_mcontext() on Windows was done manually: automated testing is
challenging to set up and left for future work.

Fixes #241
Fixes #1693
fhahn pushed a commit that referenced this issue Dec 4, 2017
Adds a new client API event type: a kernel-mediated control transfer.  This
includes UNIX signal delivery and return, Windows APCs, callbacks,
exceptions, NtContinue, NtSetContextThread, and callback returns.  It is
also raised on client redirects.

The new interface passes the source context (control and integer, no
multimedia) but passes just key target fields directly (new pc, new xsp)
and does not pass in the full target register state to avoid copying costs
(particulary for multimedia).  Adds a new internal os_cxt_ptr_t type to
support pointing at a CONTEXT or sig_full_cxt_t for the new events and only
copying from that state if the client calls dr_get_mcontext().  Adds extra
logic to get the source context for dr_redirect_execution() called from the
exception event.

Includes limited support for calling dr_set_mcontext() and changing the pc
or other state, though there seem to be few use cases of this and given the
difficulty in testing it, it's not clear it's a worthwhile feature.

Adds documentation on the disparity between xbp being in CONTEXT_CONTROL
yet in DR_MC_INTEGER, cautioning clients to use both INTEGER and CONTROL
when they care about xbp.

Includes logic to pass the real Ki pc and not the hook-displaced pc for
dispatchers.

I don't think it's possible to pass the real xsi and not the syscall return
address for a cbret: we live with that.

Adds corresponding routines drmgr_register_kernel_xfer_event() and
drmgr_register_kernel_xfer_event_ex().  Re-implements drmgr's CLS API using
the new kernel xfer event (this is required for proper ordering of CLS vs
other clients using the xfer event).  Removes
DRMGR_PRIORITY_INSERT_CLS_ENTRY, DRMGR_PRIORITY_INSERT_CLS_EXIT,
DRMGR_PRIORITY_NAME_CLS_ENTRY, and DRMGR_PRIORITY_NAME_CLS_EXIT.

Adds tests to client.signal, client.flush, client.events,
client.drmgr-test, and a new test client.winxfer.  Testing of
dr_set_mcontext() on Windows was done manually: automated testing is
challenging to set up and left for future work.

Fixes #241
Fixes #1693
derekbruening added a commit that referenced this issue Jul 5, 2023
8 of the 13 tests on Mac AArch64 labeled "OSX" fail prior to this PR.
Here we fix the following:

+ Syscall success is indicated by the carry flag just like x86 Mac
+ Handle sigreturn with its extra parameters just like x86 Mac
+ Fix signal handler parameters
+ Fix stolen register support in signal contexts
+ Use MAP_JIT and pthread_jit_write_protect_np for +rwx gencode in tests
+ Use DYLD_LIBRARY_PATH on Mac in tests

Now all 13 tests pass:
---------------------------------------------------------------------------------------
ctest -j 5 -L OSX
 1/13 Test  #13: code_api|common.fib ................................  Passed  0.59 sec
 2/13 Test #243: code_api|libutil.frontend_test .....................  Passed  0.63 sec
 3/13 Test #231: code_api|api.ir ....................................  Passed  0.67 sec
 4/13 Test   #9: code_api|linux.sigaction.native ....................  Passed  0.25 sec
 5/13 Test  #31: code_api|linux.signal0000 ..........................  Passed  0.10 sec
 6/13 Test #240: code_api|api.ir-static .............................  Passed  0.34 sec
 7/13 Test #241: code_api|api.drdecode ..............................  Passed  0.38 sec
 8/13 Test #245: code_api|api.dis-a64 ...............................  Passed  1.15 sec
 9/13 Test #264: no_code_api,no_intercept_all_signals|linux.sigaction  Passed  0.08 sec
10/13 Test  #33: code_api|linux.signal0010 ..........................  Passed  0.34 sec
11/13 Test  #35: code_api|linux.signal0100 ..........................  Passed  0.42 sec
12/13 Test  #37: code_api|linux.signal0110 ..........................  Passed  0.45 sec
13/13 Test   #7: samples_proj .......................................  Passed  1.89 sec
100% tests passed, 0 tests failed out of 13
---------------------------------------------------------------------------------------

Issue: #5383
github-merge-queue bot pushed a commit that referenced this issue Jul 7, 2023
8 of the 13 tests on Mac AArch64 labeled "OSX" fail prior to this PR.
Here we fix the following:

+ Syscall success is indicated by the carry flag just like x86 Mac
+ Handle sigreturn with its extra parameters just like x86 Mac
+ Fix signal handler parameters
+ Fix stolen register support in signal contexts
+ Use MAP_JIT and pthread_jit_write_protect_np for +rwx gencode in tests
+ Use DYLD_LIBRARY_PATH on Mac in tests

Now all 13 tests pass:
```
---------------------------------------------------------------------------------------
$ ctest -j 5 -L OSX
 1/13 Test  #13: code_api|common.fib ................................  Passed  0.59 sec
 2/13 Test #243: code_api|libutil.frontend_test .....................  Passed  0.63 sec
 3/13 Test #231: code_api|api.ir ....................................  Passed  0.67 sec
 4/13 Test   #9: code_api|linux.sigaction.native ....................  Passed  0.25 sec
 5/13 Test  #31: code_api|linux.signal0000 ..........................  Passed  0.10 sec
 6/13 Test #240: code_api|api.ir-static .............................  Passed  0.34 sec
 7/13 Test #241: code_api|api.drdecode ..............................  Passed  0.38 sec
 8/13 Test #245: code_api|api.dis-a64 ...............................  Passed  1.15 sec
 9/13 Test #264: no_code_api,no_intercept_all_signals|linux.sigaction  Passed  0.08 sec
10/13 Test  #33: code_api|linux.signal0010 ..........................  Passed  0.34 sec
11/13 Test  #35: code_api|linux.signal0100 ..........................  Passed  0.42 sec
12/13 Test  #37: code_api|linux.signal0110 ..........................  Passed  0.45 sec
13/13 Test   #7: samples_proj .......................................  Passed  1.89 sec
100% tests passed, 0 tests failed out of 13
---------------------------------------------------------------------------------------
```
Issue: #5383
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant