How to disassemble Fatal Racing / Whiplash

[deevus]

Hi :)

This looks really interesting as I am trying to RE Fatal Racing/Whiplash and there were WATCOM when running strings on FATAL.EXE.

It did look promising but it crashed

FATAL.EXE_zzz_log.txt

The error isn't in the log file but here is a snippet copied from the terminal:

Building data maps for data objects:
Building initial data maps from object and modules...
Data map for object 2 has 1 entries
Extending data maps based on size information in structure...
Traceback (most recent call last):
  File "/home/deevus/projects/reverse-engineering/wcdatool/Scripts/../Wcdatool/wcdatool.py", line 241, in <module>
    retval = main()
  File "/home/deevus/projects/reverse-engineering/wcdatool/Scripts/../Wcdatool/wcdatool.py", line 210, in main
    disasm = disassemble_objects(wdump, fixrel, cmd_args.objdump_exec, outfile_template)
  File "/home/deevus/projects/reverse-engineering/wcdatool/Wcdatool/modules/main_disassembler.py", line 2290, in disassemble_objects
    insert_data_map_item(object["data map"], OrderedDict([("start", sitem["start"]), ("end", sitem["end"]), ("type", type_), ("mode", mode), ("source", "structure")]))
  File "/home/deevus/projects/reverse-engineering/wcdatool/Wcdatool/modules/main_disassembler.py", line 1372, in insert_data_map_item
    raise TypeError("ins_item[\"%s\"] must be type %s or None, not %s" % (key, key_type.__name__, type(ins_item[key]).__name__))
TypeError: ins_item["end"] must be type int or None, not NoneType

[fonic]

Thanks for reporting this. That is both a mistake within the code and an issue caused by that specific EXE.

The code in line 1371/1372 should be:

if (not isinstance(ins_item[key], key_type)):
	raise TypeError("ins_item[\"%s\"] must be type %s, not %s" % (key, key_type.__name__, type(ins_item[key]).__name__))

I'll have to investigate further why this specific EXE causes this. ins_item["end"] must never be NoneType, that's why that check is there and aborts execution.

[fonic]

Finally another EXE file with debug info, awesome!

I used FATAL.EXE, size 1129239 bytes, SHA-256 checksum 7789768c0a6cdc213eaf7b1ad8cc125531eb3fe7c213eccac6188e58f428687e for testing.

For now, as a workaround, please patch file Wcdatool/modules/main_disassembler.py at line 2291 by replacing this:

wcdatool/Wcdatool/modules/main_disassembler.py

Lines 2291 to 2292 in b4981ad

	except DataMapInsertError as dmie:
	logging.warning("%s: sitem: type: %s, start: 0x%x, end: 0x%x, length: 0x%x (%d), name: %s, label: %s" % (str(dmie), sitem["type"], sitem["start"], sitem["end"], sitem["length"], sitem["length"], sitem["name"], sitem["label"]))

with this (make sure to keep the original indentation):

except Exception as exception:
	logging.warning("[FIXME]: %s: sitem: %s" % (str(exception), str(sitem)))

This will skip the one faulty fixup item that causes the issue.

With this patch, wcdatool will continue and eventually finish processing the executable. After that, you might want to create your own object hints to refine the output further, e.g. warning

Failed to match fixup target offset 0x3b01:     3824:   01 3b                   add    DWORD PTR [ebx],edi

is caused by data within the code object (which is currently not detected automatically by wcdatool). A hint file with an entry like this fixes it:

                                 Object Hints
==============================================================================

Object 1:
1) start = 00003824H, end = 00003A10H, type = data, mode = dwords, comment = DWORDs

by instructing wcdatool to decode that specific part as data (specifically as DWORDS). Continue adding hints for similar warnings until most of them are eliminated.

Find the log of my test run and a sample hints files attached to this comment:
FATAL.EXE_zzz_log.txt
FATAL.EXE.txt

[deevus]

Thanks for the quick response! I will definitely try this out tonight.

Since part of the process was successful and the LE payload of the executable was split out, what I did do was use https://github.com/samunders-core/le_disasm to disassemble the LE payload which worked great, which you might find interesting.

I have spent a bit of time already in Ghidra documenting the code and naming functions and labels, so I have updated the labels in the le_disasm output from Ghidra by navigating to their addresses.

I'm still very new to reverse engineering, but I'm learning a lot. What can be gathered from the debug info in the EXE that might be useful to me?

I'm likely going to be using the ASM for reference only, and implementing an engine in another language.

[fonic]

Thanks for the quick response! I will definitely try this out tonight.

You should, I think it might help you a lot with the work that lies ahead!

Since part of the process was successful and the LE payload of the executable was split out, what I did do was use https://github.com/samunders-core/le_disasm to disassemble the LE payload which worked great, which you might find interesting.

I know the tool, it was initially created to help RE Syndicate Wars, I was in touch with the author a few years back.

I have spent a bit of time already in Ghidra documenting the code and naming functions and labels, so I have updated the labels in the le_disasm output from Ghidra by navigating to their addresses.

Not wanting to brag, but wcdatool does that automatically for you.

I'm still very new to reverse engineering, but I'm learning a lot. What can be gathered from the debug info in the EXE that might be useful to me?

Functions names, variable names, original source file names, mapping of sources to original source files. wcdatool makes use of all these. Here's what the unedited output looks like for FATAL.EXE:

Files:

Sample from loadtrak.c.asm:

[deevus]

😮 OMG ok yep this is an insane amount of information that will definitely help. Perhaps we can chat more via email

simon@simonhartcher.com

[fonic]

OMG ok yep this is an insane amount of information that will definitely help. Perhaps we can chat more via email

It is, isn't it :) I'll gladly drop you a line via email later today.

[fonic]

You may also want to patch the following line in main_disassembler.py to get the original sources structure:

Replace this:

wcdatool/Wcdatool/modules/main_disassembler.py

Line 2394 in b4981ad

file_name = ntpath.basename(module["name"])

With this (make sure to keep the original indentation):

file_name = module["name"].replace(":", "").replace("\\", os.path.sep)

Results:

[fonic]

Hi Simon,

FATAL.EXE was a really good find! Its structure is SO different from the Mortal Kombat executables (likely because it was originally written in C) and helps me a lot with further development.

I mapped out FATAL's code object, i.e. identified all data regions. If you like, give the current development version of wcdatool a try (hint file for FATAL.EXE with aforementioned mappings included):

wcdatool_v22.7_devel.tar.gz

[deevus]

Ohh this does look interesting.

Found this in the diff with the old version. The old version had labels for these jumps:

[fonic]

Found this in the diff with the old version. The old version had labels for these jumps

This is not right, that part should be data, not code. Are you sure you actually applied the hints file? Doesn't look like it.

If in doubt, please provide the log file.

[fonic]

For comparison, this is how that part should look if everything is applied correctly:

[deevus]

It probably went haywire with my existing wcdatool installation. I just copied your tarball over the top. I'll create a fresh installation and try again.

[deevus]

Ok I tried this again with a fresh wcdatool installation and I'm still not getting output that matches your screenshot. Note the lack of "Hint xxx" comments.

fatal.exe_zzz_log.txt

I'm using Fedora in WSL on Windows for this, which shouldn't make a difference.

[deevus]

Nevermind, I got it. My executable was lowercase fatal.exe whereas your hints file was uppercase FATAL.EXE.txt

[fonic]

Nevermind, I got it. My executable was lowercase fatal.exe whereas your hints file was uppercase FATAL.EXE.txt

Yeah, that's it. The wrapper script looks for a hints file of exactly the same name as the executable + .txt.

For the next release, I'll remove the additional scripts and rewrite the README to instruct users to use wcdatool and its command line arguments directly - I think that would help with those kind of goofs as wcdatool would have complained about a missing hints file whereas the script does not supply the corresponding command line argument in the first place if no matching hints file was found.

[fonic]

Regarding debugging (our discussion via email):
Why not use wd, the debugger that is part of the Watcom toolchain? It is fully capable of debugging DOS4G applications and it is able to make use of the executable's debug info (!). It may not be the most user-friendly application, but it gets the job done.

In the video below, I open FATAL.EXE in wd and navigate to the start of the code object by using label ___begtext-3 (code starts 3 bytes before label) and then to the start of the data object by using label _nullarea (memory inspection shows the data):

video.zip

NOTE:

• wd only works with executables that do not contain the DOS4G loader, i.e. use FATAL.EXE_split_dos4g_payload.exe of wcdatool's output
• wd needs to be run with option /tr=rsi
• for me, only wd from Watcom v1.9 seems to work while wd from Watcom v2.0 crashes

[fonic]

I just remembered what kind of setup I used to debug Mortal Kombat a few years back. OpenWatcom on Windows + DOSBox + virtual serial connection for remote debugging:

Setup guide (Google Translate is your friend):
https://www.javiergutierrezchamorro.com/depurar-aplicaciones-dos-con-watcom-c-y-openwatcom-c/

Again, only seems to work with OpenWatcom v1.9. OpenWatcom v2.0 builds seem to be broken lately.

[deevus]

After some failures I was able to get it running using OW 1.9, however it has a program error at a specific call. Were you able to get it to at least load the game menu?

It breaks somewhere inside this call

[deevus]

There is an English version of that guide you linked on the OW2 wiki here: https://github.com/open-watcom/open-watcom-v2/wiki/Debugging

[deevus]

Ok so it is trying to call chdir_ which in turn invokes int21/3bh: http://www.ctyme.com/intr/rb-2777.htm

The address where the path should be found is address 0x356698 but there is nothing there.

So it seems that there is some important code that runs in the 16bit section that loads some values into memory?

[fonic]

There is an English version of that guide you linked on the OW2 wiki here: https://github.com/open-watcom/open-watcom-v2/wiki/Debugging

Good find, thanks.

After some failures I was able to get it running using OW 1.9, however it has a program error at a specific call. Were you able to get it to at least load the game menu?

No, I was not, also crashed during my tests. I did not investigate further - that's your job :) - but here are my thoughts on that:

• Anti-debugging measures in the game's code: I fondly remember that lots of games in the 90s tried to prevent debugging (for obvious reasons: cracking, modding, reverse engineering by competitors)
• Incompatible DOS4G version: try the DOS4G version that came with the game, i.e. put FATAL.EXE_split_dos4g_stub.exe in the games's directory as DOS4GW.EXE instead of using the one from OpenWatcom's binw directory
• Issues with the DOS environment: try different versions/forks of DOSBox, try DOSEMU
• Try a different debugger (e.g. Turbo Debugger) and check if it exhibits the same behavior

[fonic]

Ok so it is trying to call chdir_ which in turn invokes int21/3bh: http://www.ctyme.com/intr/rb-2777.htm
So it seems that there is some important code that runs in the 16bit section that loads some values into memory?

IIRC, all 16-bit code is handled by DOS4GW, that's its main job. So probably an incompatible DOS4GW version? Not sure though. Could also be one of the other things I listed.

[fonic]

BTW you could also try to build OpenWatcom yourself per the instructions given in wcdatool's README - IIRC, this builds everything (i.e. Linux + DOS + Windows executables). Maybe that works better than the builds provided by the project itself.

[deevus]

I've considered it. I did try to build OW in WSL but was missing some dependencies and gave up.

[fonic]

I've considered it. I did try to build OW in WSL but was missing some dependencies and gave up.

Built it myself from the latest working commit - no change, OpenWatcom v2 sources seem to be broken.

[fonic]

I got it working with OpenWatcom's remote debug setup. Breakpoints work, I set one on `copypic_' and it breaks for the intro pictures.

Not really sure what makes it work, though. I did not change the setup. But what I did do is open DOSBox, run FATAL's SETUP.EXE, set sound+music to None, exit and save quit setup, run SERSERV.EXE /trap=rsi, start the remote debug session -> works. Reproducible. Somehow running SETUP.EXE before starting the debug session fixes things.

Weird... Got no idea what's going on there...

[Zizin13]

Hey, I'm having some trouble with my remote debugging watcom setup. I made a video showing what I have done. https://youtu.be/Fe_GHhgqUzA Do you know what I am missing? Thanks!

[fonic]

Hey, I'm having some trouble with my remote debugging watcom setup. I made a video showing what I have done. https://youtu.be/Fe_GHhgqUzA Do you know what I am missing? I can also be reached at srogers at zizin.racing. Thanks!

Hi there!

Great idea regarding the video, that makes it a lot easier to figure out what might be wrong. Also, you're in luck, I actually still have a backup of the VM with the remote debugging setup, and it's still working as it did back then.

Right of the bat, I can tell that you mixed up your .exe files:
FATAL.EXE_split_dos4g_payload.exe, size 928 KB, is the one you need to use for debugging. That one is basically the original FATAL.EXE, but with DOS4G(W) stripped out of it. I simply renamed FATAL.EXE_split_dos4g_payload.exe to FATAL2.EXE on my VM, so that's the name you'll see in the screenshots below.

As for DOS4GW.EXE, this actually needs to be, well, DOS4G(W). Mine is 260 KB in size - can't actually remember where I got it from, might have simply been copied over from some other game that shipped it separately (e.g. DOOM, Mortal Kombat). I'll simply attach it here for your convenience.

Which exact COM ports are being used should be irrelevant as long as both DOSBox and Open Watcom are configured accordingly (DOSBox gets one, Open Watcom gets the other one). That part seems fine on your system. I'll attach screenshots of my configuration below.

DOS4G(W):
dos4gw.zip

Executables:

com0com configuration:

DOSBox configuration:

Open Watcom configuration:

[fonic]

Also, in your dosbox.conf, there seems to be a whitespace character before COM4 - not sure if that'll work, mine says realport:COMx without whitespace in between.

Other than that, the COM setup seems fine to me.

[Zizin13]

Gave it a try, no luck unfortunately. https://youtu.be/5Pnj590gmfI

[fonic]

Gave it a try, no luck unfortunately. https://youtu.be/5Pnj590gmfI

Ok, so lets dig deeper. What happens in the DOSBox window when you try to start remote debugging? serserv.exe should throw messages while the debugger tries to connect.

Also, do you have the files rsi.trp and rsihelp.exp stored alongside serserv.exe? All three are required, all come from Open Watcom's binw folder.

Additionally, check Windows' device manager, section Ports (COM & LPT). If the setup is correct, com0com's virtual ports should show up there.

If all else fails, try using precompiled Open Watcom v1.9 and normal DOSBox v0.74-3 (not staging). That's what's installed within my VM. Could be an incompatibility issue related to the newer versions.

[Zizin13]

Success! Switching to DOSBox v0.74-3 did the trick. https://youtu.be/BO4V30Ij0NE I'm sure I'll have more questions later but I'm rolling now.

[fonic]

Nice! If your focus actually is Fatal Racing / Whiplash, you might want to get in touch with @deevus - I happen to know that he's still interested in continuing the reverse-engineering effort he started.

[Zizin13]

Already did! :) Got a couple other guys helping too.

[fonic]

Nice, let me know how things are progressing.

[Zizin13]

@ninjatobob got @samunders-core's unmangling algorithm adapted from "sound.c.asm" working. We can access most of the game resources now. Check it out over here: https://github.com/ninjatobob/Whiptools

[deevus]

Wow that was quick. It took me so long to get that working 99%.

[fonic]

It's kind of weird what they did there - looks like an RLE algorithm on steroids. They were likely aiming for compression and obfuscation at the same time.

[Zizin13]

Working on a track editor. Not much to it at the moment but many more features and improvements are planned. https://github.com/Zizin13/Roller

[Zizin13]

I made a video so others can get rolling with the debugger easily: https://www.youtube.com/watch?v=bG9tEZOSrQg

[fonic]

I made a video so others can get rolling with the debugger easily: https://www.youtube.com/watch?v=bG9tEZOSrQg

Very nice & helpful for other users!

"can get rolling" -> pun intended?

[Zizin13]

https://www.youtube.com/watch?v=8qC2jOHXGRo
Big step forward in the track editor. I've now got tracks rendering in 3D. @aspirations04 has been doing a lot of work on reverse engineering the data fields and most of them are figured out now too. It's just a matter of putting all the pieces together and finding out what the few remaining unknowns do.

[fonic]

Awesome, seems like you guys are making excellent progress.

[Zizin13]

I figured out how to apply textures and now have all major track surfaces rendering in the opengl context.

[Zizin13]

Made another video on the current state of the track editor. I got all the models from the game in it thanks to this disassembly tool. https://youtu.be/-uRBRUaSn3Y?feature=shared

[fonic]

Oh wow, that's quickly becoming a fully-fledged editor. I'm curious, can you already save edited tracks and play them in the original game?

[Zizin13]

Oh wow, that's quickly becoming a fully-fledged editor. I'm curious, can you already save edited tracks and play them in the original game?

Yep, that has been possible since version 0.1.0 when it was just a crappy text editor. The reason I pushed out releases so frequently is so others who are less technical could help by manipulating track values and seeing what they did in game. A way to do that on a mass scale is better than manually changing each chunk in Notepad.

[fonic]

The reason I pushed out releases so frequently is so others who are less technical could help by manipulating track values and seeing what they did in game.

Good call, I bet that helped a lot.

[Zizin13]

I've added support for exporting tracks and cars to FBX format.