Help parsing Sega CD BRAM files?

[euan-forrester]

Hello, I'm the author of savefileconverter.com (https://github.com/euan-forrester/save-file-converter) and I'd like to add support to the site for parsing/converting Sega CD files -- especially changing from one size to another so that people can move saves between emulators or to a flash cart or MiSTer. But also extracting individual saves, and recombining individual saves into a new file.

I had a look through the code here to try and see if I could figure out the format, and after looking I haven't been very successful. Is there code you can point me to that describes the format? Or anything else, like a quick description that I could build off of? I haven't been able to find much on google either, aside from a few people asking similar questions and descriptions of the BIOS calls that devs would use to read/write save files.

Thanks very much for any pointers you're able to give me!

(Sorry if opening an issue isn't a very good way to ask a question like this: please tell me if there's a better way!)

[ekeeke]

Unfortunately, Mega CD Backup RAM format is not documented so I have no idea how 'files' are stored. This is fully handled by the provided BIOS software so, from emulator point view, we don't need to understand the file format but only emulate the hardware interface.

All I know is that when the Backup RAM is formatted by the BIOS, every bytes are cleared to 0 and a specific 64-bytes string is written at the end of the Backup RAM area. This is the same for internal and external (cartridge based) backup RAM. Most of those bytes have fixed value except a few ones that are used to encode the backup RAM size (fixed to 8KB for internal backup RAM but up to 512KB for external one).

This emulator automatically formats internal and external backup RAM files (at least in libretro, gamecube/wii and sdl frontends) so you can have a look at this 64-bytes string format here:
https://github.com/ekeeke/Genesis-Plus-GX/blob/master/sdl/sdl1/main.c#L37
and how/where backup RAM size is encoded here:
https://github.com/ekeeke/Genesis-Plus-GX/blob/master/sdl/sdl1/main.c#L824

NB: in emulator code, scd.cartridge.mask corresponds to the size of external backup RAM minus one.

[euan-forrester]

Thank you for this! It's really helpful to know that everything is handled by the BIOS so I didn't miss much when I was googling around. Also thank you for the pointers to the code here: I missed them when I had a look through, and I don't think I would have fully understood them without your explanation, so thank you!

I'll try fiddling with those bytes to see if I can resize a file, and also I see that krikzz has a "savesplitter" tool here https://krikzz.com/pub/support/mega-everdrive/pro-series/ which I'll see if I can get the code for because it may also be helpful.

Thank you again for your quick and very helpful response!

[euan-forrester]

Thank you again for your help earlier!

I just wanted to let you know that since we talked, https://github.com/superctr has posted this tool: https://github.com/superctr/buram which can parse and create BRAM files.

I've adapted his code to add this functionality to my site here: https://savefileconverter.com/#/sega-cd and if you're interested you can check out my implementation here: https://github.com/euan-forrester/save-file-converter/tree/main/frontend/src/save-formats/SegaCd

I also added a plain-language description of the format here: https://github.com/euan-forrester/save-file-converter/blob/main/frontend/src/save-formats/SegaCd/SegaCd.js#L9

One suggestion I have for this emulator would be to consider adding the ability to change the size of a BRAM file. I may be incorrect, but it appears from the experimentation that I've done that the BIOS needs to be "told" what specific size of BRAM to load, and it can't load a file of a different size. However, different emulators/flash carts/etc "tell" the BIOS different sizes, and so the files aren't always portable.

I've added the ability to resize these files to my site, but if you were also to add code to the emulator to (automatically?) resize a BRAM file then it would potentially help users use save files created elsewhere.

My implementation of resizing a BRAM file is here: https://github.com/euan-forrester/save-file-converter/blob/main/frontend/src/util/SegaCd.js#L151 . Fortunately, you don't need to be able to parse the whole file in order to resize it.

Anyway, it's just a suggestion. Mostly I wanted to let you know about superctr's work, which you might not have been aware of. Thank you again for your help earlier -- I was pretty lost before talking to you!

[ekeeke]

The correct solution in emulator is to support variable BRAM sizes, see issue #267
Currently it is hardcoded to 512KB which is the max supported size for backup RAM cart.

[euan-forrester]

Cool, yeah that sounds fair enough, and I added a couple of notes to that issue. Thank you again for all of your help!

[birdybro]

The only information I see for the file format specifically is from this technical letter:

https://drive.google.com/uc?id=1gzTH_LIHl8uvcmJGimpeM5RB72pYr4EE

The file size of each save should not exceed 256 blocks, so it stands to reason that each game is separate by that many blocks (which is consistent with other cd-based systems' external save memory, like the PSX, just a different amount of blocks used). Aside from that file size per save limitation, there didn't seem to be many other specifications, which is a typical Sega thing, where they left a lot up to the developers and did not enforce standards heavily, and didn't even push hard for official licensed status. The above document should be seen as a suggestion not an enforced policy.

There's also some minimal documentation on the backup ram in the Mega CD bios Manual:

http://techdocs.exodusemulator.com/Console/SegaMegaDrive/Documentation.html

One thing I can glean from this is that save files can either be written in data protect mode or no protect mode. In the case of data protect, it seems like it's half as efficient, probably for 1:1 redundancy, since it uses multiples of $20 bytes, whereas no protection is written in multiples of $40 bytes. That's one area where the file format is slightly different. Also to read the file it always has to have the save identifier in ASCII.

That's about all there is out there on the file format specifically.

[euan-forrester]

@birdybro Thanks for those links! That's some good information there.

There's been some new information though: as I said above, https://github.com/superctr has reverse-engineered the format and posted this tool: https://github.com/superctr/buram which can parse and create BRAM files.

I've written an implementation as well, based on his code, and made a tool available on my site: https://savefileconverter.com/#/sega-cd

There's a few other links in my note above, but one is that I wrote a plain-language explanation of the format here: https://github.com/euan-forrester/save-file-converter/blob/main/frontend/src/save-formats/SegaCd/SegaCd.js#L9

In particular, the save data isn't separated (there's [encoded] directory information at the bottom of the file), and the optional data-protection encoding does indeed double the amount of data needed but not by making a 1-to-1 copy but instead using a complex (and seemingly-inefficient) interleaving and error-correction scheme. That's why the directory information and filenames aren't visible when looking at a file in a hex editor.

Most games I tested didn't bother with encoding their data. The encoding resembles that used in reading CD data (it's called Reed-Solomon encoding) so I wonder if they had some libraries lying around and just used them.

However, there's also considerable redundancy when storing the number of free blocks and number of files, which makes me wonder if the hardware really was super error-prone.

Knowing the format has definitely raised questions as well as answering them!