Unicorn Preload

[ehennenfent]

Modifies the Unicorn emulator module to allow it to be used for "preloading" large binaries. It uses Unicorn to execute x64 instructions in bulk while Manticore handles IO and syscalls. State changes are aggressively written from Manticore back to Unicorn, and lazily written from Unicorn to Manticore before a syscall.

The following script demonstrates this. The user can register a plugin at startup that tells Manticore to use Unicorn to quickly execute the initialization instructions leading up to the start of main. The binary in question is multiple-styles from an old example.

from manticore.native import Manticore
from manticore.core.plugin import Plugin

address_of_main = 0x4009ae

class concretePlugin(Plugin):

    def will_start_run_callback(self, state, *_args):
        state.cpu.emulate_until(address_of_main)

m = Manticore("multiple-styles", concrete_start='coldlikeminisoda')
m.register_plugin(concretePlugin())
m.run()

Once main is reached, Manticore takes over and continues execution as normal. In the case of this example, the full solution is coldlikeminisodas, so Manticore generates two different test cases for the remaining byte. The performance improvement for this example is only marginal because the bulk of the time is taken up by the solver, and relatively little initialization is required before executing main. However, more complex binaries can see very significant speedups.

There are still some limitations, but work on those will come in future PR's. They include:

• Unicorn can't handle symbolic data, so we skip over writing it back. We should instead try to concretize it to a single value, and only continue executing in Unicorn if we can do that successfully.
• Full memory maps are often copied lazily into Unicorn, and unmaps and permissions changes are not well modeled, which can lead to Unicorn's memory maps becoming desynchronized from Manticore.
• Copying large amounts of data between Manticore and Unicorn can be dismally slow. A number of optimizations could improve this, including mapping files directly into Unicorn's memory rather than copying Manticore's memory maps.

This PR also boosts unimplemented linux system calls to warnings, which I've found to be a helpful UX change when debugging.

---

This change is 

[disconnect3d]

Is there any reason to not use isinstance here?

If we want to stick with type(map) we might want to assign it to a variable as we calculate it twice in a worst case.

[ehennenfent]

It does make sense to use type in this case because we access protected members of the FileMap that aren't guaranteed to behave the same way in a subclass. Strictly speaking, the right way to do this is probably to add unprotected bulk read and write functions to filemaps and anonmaps, but for now I'll just save the type in a variable so we won't need to calculate it twice.

[disconnect3d]

Why don't we just do mem.cpu = cpu instead of mem.setCPU(cpu)?

Also, might be good to add some explanation/comment why do we need to do it in the first place :S.

[ehennenfent]

Good point. A getter/setter would make more sense.

[disconnect3d]

Loud thinking: we could probably use https://docs.python.org/3/library/resource.html#resource-limits ?

[disconnect3d]

Since this is logging, we should rather use it's internal formatting (i.e. we don't want to format if we discard the log later on). This being said if we change it to warning it's rather okay to format it right away?

[ehennenfent]

Sample invocation using the multiple-styles example

from manticore.native import Manticore
from manticore.core.plugin import Plugin

class concretePlugin(Plugin):

    def will_start_run_callback(self, state, *_args):
        state.cpu.emulate_until(0x4009ae)

m = Manticore("multiple-styles", concrete_start='coldlikeminisoda')
m.register_plugin(concretePlugin())
m.verbosity(2)
m.run()