Generating an elf file

[Chrs2324]

I've been using Ghidra to help try and get close to generating an elf file and I have found the entry point that I am looking for, however, I am unable to figure out what I need to do to generate the elf file. Any help would be greatly appreciated!

[Chrs2324]

I was able to generate an elf file with this bash script https://gist.github.com/JayFoxRox/5eaa45b374a44ce27143388bab1753b7
What you have to do in order to make this bash script work is you have to install n64chain in order for it to correctly generate the elf file https://github.com/tj90241/n64chain. However, I have hit a roadblock. While the entry point address needed to generate the elf file is correct, the N64Recomp program still spits out an error when feeding in the toml and elf files. The game that I am using to test this is Conker's Bad Fur Day. Hopefully this helps people get further along with figuring this out, but until now, this issue is still not resolved.

[agilly1989]

What error are you getting?

[Chrs2324]

What error are you getting?

This is the error that I am receiving when I give the program the toml file

[agilly1989]

Same here

And my (much more smarter than I) pal is also getting the same error.

[Nielk1]

I've been trying to same thing. Running N64Recomp in the debugger I see the ELF's segments array is empty, size 0, so when the .data section tries to reference segment data it fails. It appears the ELF created by these steps lacks segment data.

My experience with disassembling n64 roms is limited to that one time I made USF which was just some minor MIPS machine code hackery. I can't say if we're entirely on the wrong path here or close to the solution.

[Chrs2324]

I've been trying to same thing. Running N64Recomp in the debugger I see the ELF's segments array is empty, size 0, so when the .data section tries to reference segment data it fails. It appears the ELF created by these steps lacks segment data.

Sounds about right because I placed the elf file in Ghidra and noticed some oddities with it that didn't seem right

[Chakratos]

I also tried the process but failed just like you. Cannot wait to try Mystical ninja starring goemon out on PC, sadly the decomp project got removed from github (Author was fluvian) :(

[Mr-Wiseguy]

I don't know if ghidra-generated elf files will be compatible with this tool. The elf files I've used have all come from disassemblies, decomps, or romhacks which is the main supported use case. In the future I'll be adding a mechanism for providing a symbol list in a custom format which will let you use a ROM (provided you can generate the list) instead of an elf.

[TheHans255]

What symbols are we going to need to generate a working recomp? I'm interested in applying this to Smash 64 or Smash Remix - I don't know of any decomp projects for either of those, though I'm guessing that the existence of Smash Remix implies that a symbol table for both games can be generated.

[Jgoodwin64]

disassemblies

So, what software do you recommend? How do we use disassemblies?

[Mr-Wiseguy]

I comment it more in detail here #17, but in general it's not something I'd really be able to provide instructions for. There's a lot of foundational reverse engineering knowledge needed to set up a disassembly that would be very difficult for me to write a guide for, but in the future there may be tools to help make that easier.

[Nielk1]

@Mr-Wiseguy Am I correct to understand that the purpose of the ELF is to contain both the binary machine code, binary data, and metadata for sections and segments to allow the tool to know where the machine code content inside the ELF is vs the data segments and what virtual memory addresses to map the binary file offset to? If I am correct on that it sounds like the tool to use here is something like splat but I have almost no experience with REing n64 rom images.

I understand that REing is a unique case by case situation as I've done it with x86 machine code a few times so I'd just like to get the details of what is needed as an end result here. I'd also suggest, if I am correct on this, that you add a small bit to the README explaining that the ELF is being used to contain these bits of information and that currently having such data requires some existing reverse engineering effort to at least organize and label the sections and segments.

Personally, I'd like to try to port Battlezone: Rise of the Black Dogs to PC just because it would be interesting. I've extracted the game's assets in the past and converted them to work with the original PC version the game is based and ported from, so I have a lot of experience with this oddball of a game. Looking at its data though it appears to have a very aggressive mixing of machine code and binary data meaning manually mapping out the ranges between machine code and data, if I am correct as to the need to do so, is going to be quite a process. This is why I'm asking for clarification on what exactly is needed from the ELF before I proceed.

[Mr-Wiseguy]

That's exactly right, and splat is a great tool to use for this. If you can identify the regions of the ROM that are code and what vram address they're linked against, that's most of the way there. You'll have to also identify SDK symbols, which the https://github.com/decompals/flib tool can help with, so that they can be replaced with the native reimplementations I wrote (currently found in the Zelda64Recomp repo). Using Ghidra in conjunction with the other tools may help find code, though I don't have a lot of experience using it for N64 binaries.

[Mr-Wiseguy]

Additionally, the symbol input file mechanism that I'm currently working on should clarify exactly what info is needed by the recompiler. When that's done I'll update the Zelda64Recomp to use one that you can use as a reference, though an elf file may still be easier to provide (since most of the info in that file will have come from the elf).

[andrewCohn]

I think it's going to be up to the community to provide support for things like dumping the elf, that might fall outside the scope of this project.

With that said, ghidra and splat should be enough to get you there with minimum knowledge of the n64 architecture/ instruction set. When I get a good, consistent method going, I'll do a write up if nobody else has. I think the Nerrel video mistakenly gave people the impression this tool acted as a drag and drop replacement for emulation, which is really not what it's for, and requires a solid amount of prior experience programming beyond that of getting project64 up and running.

[TheHans255]

Could a low-level emulator also be used to generate the proper symbols? It appears that the primary purpose of the ELF file is to determine which sections of the ROM are code and which are data, so an emulator could record which cartridge load calls it ends up making while it runs, and where the MIPS instruction pointer goes, while you're playing the game.

[andrewCohn]

Could a low-level emulator also be used to generate the proper symbols?

This seems like a good idea to me. Would it require you to play the entire game s.t every instruction gets executed? Could use a n64 emulator with good plugin support to dump the instruction calls, maybe.

[TheHans255]

Could a low-level emulator also be used to generate the proper symbols?

This seems like a good idea to me. Would it require you to play the entire game s.t every instruction gets executed? Could use a n64 emulator with good plugin support to dump the instruction calls, maybe.

It probably would, which would likely mean it would take longer than Ghidra and Splat if those options are viable. Though after a good playthrough, you could probably have an algorithm highlight the portions of the ROM that never got executed or loaded, and then make educated guesses on what's left - you'd really just have to make sure that every entity in the game got used at least once.

There are also enough people who do 100% glitchless speedruns of games that you might be able to get the gameplay you need simply by recording their inputs and playing them back.

[knardhocks]

This is the current step I am stuck at.
I've tried using splat to split z64 down, and that at least gives me some information for entrypoint, but where to go from here is beyond me without further technical know-how to recompile into ELF.
I was working on trying this out on Gauntlet Legends, because that game deserves some love.