Halo 2 Decompilation (work in progress)

Halo 2 Decompilation is an independent preservation and reverse-engineering project. It studies the retail executable of Halo 2 for the original Xbox and recreates its program logic in human-readable C and C++ through independent analysis. Each recreated function is checked against the retail executable: built with the period-correct compiler, it must produce exactly the same bytes. That check is what makes the recreated code verifiable. A longer-term research goal is to study how the game's code could run on other platforms, using game data that each user supplies from their own copy.

The project is non-commercial and is not affiliated with or endorsed by Microsoft, Bungie or 343 Industries. Read LEGAL.md for the project's scope, what it does not contain, and its rules for contributions, and PROVENANCE.md for where its information comes from.

Contributors are welcome. No prior decompilation experience is needed, only patience and some C. Read How to help and CONTRIBUTING.md.

• No Halo 2 game assets: no maps, textures, models, audio or cinematics.
• No game executable (XBE) and no disc image.
• No Microsoft Xbox SDK/XDK: no compiler, libraries or headers.
• No original or leaked Halo 2 source code.
• No leaked symbols, PDBs or linker maps. Names that came from such sources through a third-party dataset have been removed from the inventory, and names in the source that matched such sources have been renamed; see PROVENANCE.md.
• No confidential or internal Microsoft, Bungie or 343 Industries material.

To build or check anything, you must supply any original game files you need from a copy of Halo 2 that you are legally entitled to use. The project does not provide game files.

The project does not distribute the Microsoft Xbox SDK/XDK or any proprietary Microsoft development tools. Contributors are responsible for ensuring that any development tools they use are obtained and used lawfully. The maintainers cannot provide, link to, or assist with obtaining proprietary SDK/XDK materials. Byte-for-byte checking currently depends on the original compiler, and no open-source toolchain can reproduce its output yet. Analysis, documentation and review do not need it.

Decompilation is under way, and 6274 retail functions now match byte for byte. The checker reports:

matched 6274 of 11318 game functions (655763 of 2784283 bytes, 23.55%)
matched 6274 of 17216 functions in scope (655763 of 3739273 bytes, 17.54%)

Matched code so far includes:

• the script engine's built-in functions;
• AI, actor behaviours and actor slot handlers;
• object damage, shields and vitality;
• network sessions, message codecs and the bitstream;
• UI screens and widgets, and the game engines;
• sound sources, looping sounds and Bink movie playback;
• the game's data arrays, object lists and hash tables;
• geometry, quaternion and matrix maths;
• input, file paths, localized strings, random numbers and game state.

Every function is checked automatically on every build, so a match stays a match. Refer to docs/PROGRESS.md, which is updated as work lands.

The retail disc build of Halo 2, as found on the retail disc:

The aim is a matching decompilation: C source that the original compiler turns back into the same bytes as the retail XBE. Matching gives every function an automatic pass or fail, and that check is what made the Halo CE decompilation dependable.

Halo 2's retail build is harder than Halo CE's. It was compiled with link-time code generation (LTCG): the compiler generates code for the whole program at once, when it is linked. So functions are inlined across source files, and internal functions get custom calling conventions. 42.5% of the functions called in the retail XBE take arguments in eax, ebx, esi or edi, which MSVC's standard conventions never do. Under LTCG, a function's bytes depend on its callers and callees, not only on its own source. We know of no earlier project that has matched such a build.

So the work runs in stages:

1. Feasibility spike (done). Rebuild retail functions with the XDK 5849 compiler in LTCG mode, and find out whether their bytes can be matched. They can: custom calling conventions, deleted arguments and inlining decisions all reproduce. The test sources are in spike/.
2. Project set-up (done). The function inventory, the build with each source file's flags, the checker and the ready queue.
3. Decompilation (in progress), from the leaf functions up.
4. Native port.

1. Install the Python dependencies: pip install -r requirements-dev.txt.

2. Point the tools at your local files. The retail XBE from your own copy goes at orig/default.xbe (or set RETAIL_XBE), and the development toolchain's xbox folder at sdk/xbox (or set XDK_DIR); see Requirements. Both folders are git-ignored and must never be committed. In Git Bash, also run export MSYS_NO_PATHCONV=1, which stops Git Bash rewriting the /-style arguments (such as /O2) that the compiler takes. To read the XBE out of a disc image of your own copy:

   python tools/xiso_extract.py "Halo 2.iso" orig default.xbe
   

3. The inventory, config/functions.csv, is already committed, so you don't need to regenerate it. Its name column holds only library functions' names, from their signatures in the SDK libraries; its object column is empty. See PROVENANCE.md.

4. Run python tools/check.py. It builds the whole game as one LTCG image, compares every decompiled function with the retail bytes, and writes each function's status (matched, near or todo) back to config/functions.csv. It exits 1 while any function differs.

5. Pick work with python tools/ready.py, following docs/DECOMPILING.md. --claims takes a saved copy of the Active claims table so the list skips ranges someone already has.

Halo 2 has about 11,300 game functions, so there is room for many people.

1. Get set up as in Build and check, with your own lawfully owned copy of Halo 2 (see Requirements).
2. Pick a source file or an address range nobody has claimed (the pinned Active claims issue lists them), and open a draft pull request saying what you are taking.
3. Decompile, run python tools/check.py, and push as functions match. docs/DECOMPILING.md explains the conventions and the compiler's quirks.

Details are in CONTRIBUTING.md, including the contribution provenance policy. Questions are welcome as issues.

• halo-symbol-atlas (CC BY 4.0): the name and object columns of config/functions.csv once came from this dataset. Because its names for this build derive from linker maps of unreleased builds, that data has been removed. See PROVENANCE.md.
• punpckhdq/halo, bnunu/halo-1 and halo-ce-universal: the Halo CE decompilation and port that this project follows.
• BirchWoodGod/halo2-decomp: a functional recovery of the same XBE. Its published analysis gave us leads: the subsystem lifecycle table, the data-array layout and some register conventions. We verified each one against the retail code ourselves; none of its code or text is used here.

This project is not affiliated with, endorsed by or sponsored by Microsoft, Bungie, 343 Industries or Activision. Halo, Halo 2 and Xbox are trademarks of Microsoft; other names and marks belong to their respective owners. See LEGAL.md.

The contributors' own work in this repository is released under CC0 1.0. Refer to LICENSE.

• CC0 applies only to material the contributors have the right to license.
• It does not place Halo 2, or any intellectual property of Microsoft, Bungie, 343 Industries, Activision or anyone else, into the public domain. Halo 2's game content and trademarks remain the property of their owners.
• The name and object columns of config/functions.csv once came from halo-symbol-atlas under CC BY 4.0 and were not covered by CC0. That data has been removed (see PROVENANCE.md).

Details are in LEGAL.md.