Restunts - The Stunts reverse engineering project

Online
  web: http://forum.stunts.hu/index.php?topic=2454.0
  irc: irc://irc.efnet.org/stunts
  svn: svn://anders-e.com/restunts/trunk/restunts (no longer active)
  irc-log: http://batman.no/stuntslog.txt (no longer active)


Repository contents:
	docs
		Various technical docs related to (re)stunts itself.

	src\execombiner
		Tool which combines the original game and video drivers into a single
		debugable executable.

	src\idc
		Script for IdaPro to generate .asm-files from an ida project.

	src\restunts
		Project directory containing disassembly, ported c and makefiles to
		produce various executables based on the (re)stunts code.

	stunts
		Stunts 1.1 the game.

	tools
		Contains binaries, libraries, headers with a full toolchain for
		building restunts on Windows. The toolchain is based on TASM, TLINK,
		Turbo Debugger, Borland C++, DOSBox, and various other tools.


Contents of src\restunts:
	src\restunts\game_mod3.idb
		The analysis database for IDAPro.

	src\restunts\asm
		Contains patched, compilable disassembly generated by the script for
		IdaPro. This code is patched to work with our ported c code.

	src\restunts\asmorig
		Contains the same files as src\restunts\asm, but unpatched and does not
		use any of the ported c code.
		
	src\restunts\c
		Contains c functions ported from the disassembly.
	
	src\restunts\dos
		Makefile to build restunts for DOS.
	
	src\restunts\repldump
		Tool based on the original game code, loads replays and dumps the game
		state contents at each frame in a file for further analysis.

		
The build environment

How to build restunts on Windows:
	1) Double click tools\mount_stunts_to_s.bat (only needed once per reboot)
	2) Start cmd.exe and enter the following commands:
		S:
		cd tools
		setpath.bat
		cd \src\restunts
		make

If everything went fine, there should be a new s:\stunts\restunts.exe which can
be run in DOSBox. Note that the drive letter S: is hardcoded many places in the
makefiles, and is also mounted inside DOSBox as a fixed point of reference.

The makefile supports the following targets:

	make restunts
		The default restunts target builds an executable based on ported C code
		and patched disassembly.
		
	make restunts-original
		Builds an executable based on unpatched disassembly with the original
		codepaths intact. Does not use any of the ported C code.

	make repldump
		Builds the replay dump tool using ported C code and patched
		disassembly.
		
	make repldump-original
		Builds the replay dump tool without ported C code.
		

The toolchain

The toolchain has evolved over the years and is now (2014) fully based on 
Borland tools. For various reasons, the build process uses both 16 and 32-bit
tools. In order to compile on modern 64 bit Windows systems, the 16-bit apps
run via DOSBox. A DOSBox window pops up multiple times during a build, and
stays open in case of build errors.

Tools used:
	- TASMX + TLINK (16 bit DOS, from Borland Turbo Assembler 4.0)
	- Borland C++ 5.2 (Win32)
	- Borland Make 5.2 (Win32, from Borland C++ 5.5, patched binary)

The 16-bit tools are:
	tasmx.exe, invoked in DOSBox by tasmbox.bat
	tlink.exe, invoked in DOSBox by tlinkbox.bat


Analysis in IDA and the development cycle

Analysis happens continously in IDA 6.1 (Win32). Using a custom script in IDA,
the entire disassembly is exported to compilable .asm and .inc files in 
src\restunts\asm. This allows using latest analysis results in the build
process. As code is ported manually to C, the custom script is updated and kept
in sync, such that it generates patched assembly code calling into the ported C
functions.

The custom script also runs a second pass, where it exports a separate set of 
.asm files to src\restunts\asmorig. These files are unpatched and produce code
that behaves 100% identically to the original game.

When doing analysis and (re)naming variables and functions, it is important to
always check the ported C code if there are any references to the previous
symbol names. All symbol references in the C code need to be kept in sync with
the analysis manually. Obviously, the linker will complain about missing
symbols in case a symbol was renamed in IDA, but not in the C files.


Debugging restunts.exe

Restunts can be debugged with Turbo Debugger inside DOSBox. The DOSBox
debugging environment is an extension of the build environment described above:
	1) Double click tools\mount_stunts_to_s.bat (only needed once per reboot)
	2) Start DOSBox and enter the following commands:
		mount S S:
		S:
		cd tools
		setpath.bat
		cd \stunts
		td restunts.exe

Turbo Debugger is preconfigured to automatically find and show the source code.
Setting breakpoints, stepping etc works. The TD configuration file is stored in
stunts\tdconfig.td.


Notes regarding make

Borland Make was chosen because the first makefile was written for the DOS
version. Ultimately the DOS version ran into memory problems with the largest 
asm-files. Too late and too lazy to find a new make utility, the Windows
version of Borland Make, included in Borland C++ 5.5, was used instead. But:

Borland Make 5.2 uses MAKEFILE.@@@ as the filename for temporary inline files.
WLINK interprets @ in a special way on the command line and will not take the 
inline file as a parameter. As such make.exe was patched to use MAKEFILE.!!!
as the temporary filename instead.


Notes regarding the linker

WLINK from the Open Watcom C/C++ compiler suite was originally chosen because
it supports detailed control on how to order segment classes in the final
executable image.

Our requirement is to put the original code and data first in the file, exactly
as in the original game, only patching up function calls and data access. 
TLINK and OPTLINK were tested in the early days, but after having difficulties
with segment ordering and subsequent crashes, the project settled on WLINK.

The problems with TLINK were revisited and fixed years later, after an urgent
need to get debug information for the restunts executable. As WLINK is not able
to use the debug information format created by BCC/TASM, it became necessary to
replace either of them. After the initial attempts of using TLINK were
successful, no further investigation were made into replacing TASM.  It should
be noted that MASM seems a promising TASM replacement candidate: the syntax is
close to TASM, and it produces CodeView debug format usable by WLINK.


Notes regarding 16-bit tasmx, tlink and DOSBox

The choice of reverting to TLINK was not easy either. Only the 16-bit version
of TLINK can produce 16-bit executables such as restunts.exe, and it can only
use object files produced by the 16-bit TASM. In order to use 16-bit tools on a
64-bit Windows host, they need to be emulated. And so DOSBox became
incorporated in the build process. Another problem is the regular 16-bit
tasm.exe exceeds the available 640k of memory when compiling the largest
restunts source files. This was overcome by using the DPMI-enabled tasmx.exe
instead.


Notes regarding the CRT (libc)

When linking with cm.lib from Borland C++ 5.1, TLINK is not able to produce an
executable. To get around this, the tlib tool was used to extract object files
from cm.lib, and now restunts links with these instead.

When the project used WLINK, it was able to link with cm.lib from Borland C++
5.1, but was not able to link correctly with cm.lib from Borland C++ 3.1.
Before upgrading to Borland C++ 5.1, the makefile would link to single obj
files from Borland 3.1's CRT. 

The linker can complain about weird missing symbols f.ex at first time use of
some compiler feature or CRT function. This can be fixed by extracting the
object file from cm.lib, and adding it to the makefile.

cm.lib is the medium model libc from Borland C++ 5.2. tlib.exe is a 16-bit
Borland utility for manipulating lib files and has to be run via DOSBox.

Using a binary file viewer with cm.lib it's possible to locate missing CRT
symbol names and work out/guess the object file name by looking at strings in
the binary data before it.

To extract the object file
	1) Double click tools\mount_stunts_to_s.bat (only needed once per reboot)
	2) Start DOSBox and initialize the restunts development environment:
		mount S S:
		S:
		cd tools
		setpath.bat
		cd \tools\lib
	3) Run tlib to extract the object file, some examples:
		tlib cm.lib * GETVECT
		tlib cm.lib * LABS
		tlib cm.lib * STRCPY
		tlib cm.lib * MEMCPY
		tlib cm.lib * FMEMCPY
		tlib cm.lib * H_LDIV
		tlib cm.lib * F_LXMUL
		tlib cm.lib * F_SCOPY
		tlib cm.lib * H_LRSH
		tlib cm.lib * H_PADD
		tlib cm.lib * H_PINA
		tlib cm.lib * H_PADA
		tlib cm.lib * N_PCMP
		tlib cm.lib * F_PCMP
		tlib cm.lib * H_LURSH
		tlib cm.lib * H_PSBP
		tlib cm.lib * H_LLSH
	4) Add new CRT object files in dos\makefile - at the end of the line
	   starting with CTARGETS =


Porting a function from ASM to C

1. Open src\idc\anders.idc in a text editor and locate the PortFuncName() 
   function. The IDA script needs to be aware of all the functions that have 
   been ported in order to generate asm that does not conflict with the ported
   code. Add a line in PortFuncName() for the function you've chosen.

2. Run anders.idc from within IDA to regenerate the asm files (or skip step 1+2
   and rename the respective extrn, public and function manually in the .inc
   and asm files)

3. Add a function stub to one of the existing .c files in src\restunts\c 
3.1 If you want to add a new .c-file to the project, you need to 
	- add a target for it in c\makefile
	- add a reference to the obj in RESTUNTS_OBJFILES in dos\makefile

4. Add an extrn for the c-function in asm\custom.inc

5. Start make to see if it compiles + links

6. Port to c!