McSema lifts x86 and amd64 binaries to LLVM bitcode modules. McSema support both Linux and Windows binaries, and most x86 and amd64 instructions, including integer, FPU, and SSE operations.
McSema is separated into two conceptual parts: control flow recovery and instruction translation. Control flow recovery is performed using the
mcsema-disass tool, which uses IDA Pro to disassemble a binary file and produces a control flow graph. Instruction translation is performed using the
mcsema-lift tool, which converts the control flow graph into LLVM bitcode.
|Linux & macOS|
- Translates 32- and 64-bit Linux ELF and Windows PE binaries to bitcode, including executables and shared libraries for each platform.
- Supports a large subset of x86 and x86-64 instructions, including most integer, FPU, and SSE operations. Use
mcsema-lift --list-supported --arch x86to see a complete list.
- Runs on both Windows and Linux, and can translate Linux binaries on Windows and Windows binaries on Linux.
- Output bitcode is compatible with the LLVM 3.8 toolchain.
- Translated bitcode can be analyzed or recompiled as a new, working executable with functionality identical to the original.
- McSema runs on Windows and Linux and has been tested on Windows 7, 10, Ubuntu 14.04, and Ubuntu 16.04.
Why would anyone translate binaries back to bitcode?
Binary Patching And Modification. Lifting to LLVM IR lets you cleanly modify the target program. You can run obfuscation or hardening passes, add features, remove features, rewrite features, or even fix that pesky typo, grammatical error, or insane logic. When done, your new creation can be recompiled to a new binary sporting all those changes. In the Cyber Grand Challenge, we were able to use mcsema to translate challenge binaries to bitcode, insert memory safety checks, and then re-emit working binaries.
Symbolic Execution with KLEE. KLEE operates on LLVM bitcode, usually generated by providing source to the LLVM toolchain. Mcsema can lift a binary to LLVM bitcode, permitting KLEE to operate on previously unavailable targets.
Re-use existing LLVM-based tools. KLEE is not the only tool that becomes available for use on bitcode. It is possible to run LLVM optimization passes and other LLVM-based tools like libFuzzer on lifted bitcode.
Analyze the binary rather than the source. Source level analysis is great but not always possible (e.g. you don't have the source) and, even when it is available, it lacks compiler transformations, re-ordering, and optimizations. Analyzing the actual binary guarantees that you're analyzing the true executed behavior.
Write one set of analysis tools. Lifting to LLVM IR means that one set of analysis tools can work on both the source and the binary. Maintaining a single set of tools saves development time and effort, and allows for a single set of better tools.
|Clang||3.8 (3.9 if using Visual Studio 2015)|
|Python Package Index||Latest|
|Visual Studio||2013+ (Windows Only)|
Getting and building the code
Step 1: Install dependencies
sudo apt-get update sudo apt-get upgrade sudo apt-get install \ git \ cmake \ libprotoc-dev libprotobuf-dev libprotobuf-dev protobuf-compiler \ python2.7 python-pip \ llvm-3.8 clang-3.8 \ realpath sudo pip install --upgrade pip sudo pip install 'protobuf==2.6.1'
Note: If you are using IDA on 64 bit Ubuntu and your IDA install does not use the system Python, you can add the
protobuf library manually to IDA's zip of modules.
# Python module dir is generally in /usr/lib or /usr/local/lib touch /path/to/python2.7/dist-packages/google/__init__.py cd /path/to/lib/python2.7/dist-packages/ sudo zip -rv /path/to/ida-6.X/python/lib/python27.zip google/ sudo chown your_user:your_user /home/taxicat/ida-6.7/python/lib/python27.zip
Step 2: Clone and enter the repository
git clone email@example.com:trailofbits/mcsema.git --depth 1
The Linux bootstrap script supports two configuration options:
--prefix: The installation directory prefix for mcsema-lift. Defaults to the directory containing the bootstrap script.
--build: Set the build type. Defaults to
cd mcsema ./bootstrap.sh --build Release
Step 3: Build and install the code
cd build make sudo make install
Step 1: Install dependencies
Download and install Chocolatey. Then, open Powershell in administrator mode, and run the following:
choco install -y --allowemptychecksum git cmake python2 pip 7zip choco install -y microsoft-visual-cpp-build-tools --installargs "/InstallSelectableItems Win81SDK_CppBuildSKUV1;Win10SDK_VisibleV1"
Mcsema should be built with clang. Newer versions of clang for Windows automatically integrate with Visual Studio. The mcsema build scripts rely on this integration. The minimum version of clang required is Clang 3.8 (when using VS 2013) or Clang 3.9 (when using VS 2015).
cmake will not be available on the command line after being installed from Chocolatey. If you have this issue, install
cmake from the official Windows installer.
Step 2: Clone the repository
Open the Developer Command Prompt for Visual Studio, and run:
cd C:\ if not exist git mkdir git cd git git clone https://github.com/trailofbits/mcsema.git --depth 1
Step 3: Build and install the code
cd mcsema bootstrap
Try it Out
If you have a binary, you can get started with the following commands. First, you recover control flow graph information using
mcsema-disass. For now, this needs to use IDA Pro as the disassembler.
mcsema-disass --disassembler /path/to/ida/idal64 --arch amd64 --os linux --output /tmp/ls.cfg --binary /bin/ls --entrypoint main
Once you have the control flow graph information, you can lift the target binary using
mcsema-lift --arch amd64 --os linux --cfg /tmp/ls.cfg --entrypoint main --output /tmp/ls.bc
There are a few things that we can do with the lifted bitcode. The usual thing to do is to recompile it back to an executable.
clang-3.8 -o /tmp/ls_lifted generated/ELF_64_linux.S /tmp/ls.bc -lpthread -ldl -lpcre /lib/x86_64-linux-gnu/libselinux.so.1
- Common Errors and Debugging Tips
- How to implement the semantics of an instruction
- How to use mcsema: A walkthrough
- Life of an instruction
- Navigating the source code
- Using Mcsema with libFuzzer
If you are experiencing problems with McSema or just want to learn more and contribute, join the
#tool-mcsema channel of the Empire Hacking Slack. Alternatively, you can join our mailing list at firstname.lastname@example.org or email us privately at email@example.com.
How do you pronounce McSema and where did the name come from?
McSema is pronounced 'em see se ma' and is short for machine code semantics.
Why do I need IDA Pro to use McSema?
McSema's goal is binary to bitcode translation. Accurate disassembly and control flow recovery is a separate and difficult problem. IDA has already invested countless man-hours into getting disassembly right, and it only makes sense that we re-use existing work. We understand that not everyone can afford an IDA license. With the original release of McSema, we shipped our own tool recursive descent disassembler. It was never as good as IDA and it never would be. Maintaining the broken tool took away valuable development time from more important McSema work. We hope to eventually transition to more accessible control flow recovery frontends, such as Binary Ninja (we have a branch with initial Binary Ninja support). We very warmly welcome pull requests that implement new control flow recovery frontends.
I'm a student and I'd like to contribute to McSema. How can I help?
We would love to take you on as an intern to help improve McSema. We have several project ideas labelled
intern_project in the issues tracker. You are not limited to those items: if you think of a great feature you want in McSema, let us know and we will sponsor it. Simply contact us on our Slack channel or via firstname.lastname@example.org and let us know what you'd want to work on and why.