mixed-mode clients

[derekbruening]

From bruen...@google.com on June 22, 2012 15:34:45

if we finish issue #49 and have 64-bit DR controlling both the wow64 and 32-bit
portions of a 32-bit app on a 64-bit windows kernel we then have some
questions about how clients would operate in such a mixed-mode environment.

first, do we only support a 64-bit client that itself handles mixed-mode,
or do we support two clients, one 64-bit and one 32-bit, with each seeing
just their code stream? that seems weird to me but it seems to be what Pin
does for mixed-mode. if we went the route of 2 clients, xref issue #147 for
similar questions on how to specify 2 clients (but there for multi-arch
app sequences).

xref PR 240257 "support 32-bit clients on wow64?" but that was focused on
how to support a 32-bit client with 64-bit DR which we may decide we don't
want to do.

how do we resolve the conflict of a 32-bit app being configured for
both 64-bit DR and 32-bit DR? 64 wins I guess. should document it.

the client needs to know whether the bb or trace instrlist it's being
handed is 32-bit or 64-bit code. our IR is pretty cross-platform, so it's
more about knowing what instrumentation is legal to add. get_x86_mode()
should be sufficient since it should identify the mode of all the app
instrs, or could just query the mode of the 1st instr in the list, since
we require a whole fragment to be all one mode.

for clean calls out of 32-bit app code, should we always do a mode switch
and assume the clean call target is 64-bit? (this is of course assuming
we go the route of a 64-bit client seeing all the code.)
note that in the past I measured 100 cycles as the cost for a round-trip
mode switch.

Original issue: http://code.google.com/p/dynamorio/issues/detail?id=828

[derekbruening]

From rnk@google.com on June 22, 2012 13:46:16

If we get the 32to64 translation working well, here's a strawman proposal for how things could work. When doing mixed mode instrumentation:

• DR is x64
• Client is x64
• 32-bit code is presented as a 32-bit instrlist
• client can insert x64 code into 32-bit ilist! (drmemory needs this anyway)
• translation to x64 is transparent, similar to how app cti mangling is transparent today
• client checks the x86 mode of the first instr if it wants to know the mode

This way, if we can translate everything, we can stay in x64 mode the entire time. The client can insert whatever code it likes, x64 or x86, we'll translate and preserve the semantics.

[derekbruening]

From bruen...@google.com on June 22, 2012 13:49:34

to clarify: this is about regular DR, not an interface for issue #751 (still filing issues based on old PR's), so client cannot stick in 64-bit code.