Redump Sega - Mega-CD - Sega CD missing serials #839
Comments
|
Hey! Thanks for jumping into the craziness of checksum checking. Certainly could update it to use the .cue files instead, and that might help it be more consistent. It finds the entry from the game over in this code here: We could replace a lot of what's there to just force use of the CUE file if it's there. I think it was updated recently to select the first one from the .cue file, but 🤷♀️ I think just having it pick the .cue file makes more sense. |
|
Hi Rob! After a bit of butchering in the index.js (I don't know JS - or any language for that matter), I managed to output libretro DAT files with checksums for the cue files (SCD/SS). This solves the issue of games not being accounted for however no playlist gets created by RetroArch internal scan (after conversion to RDB with c_converter). I guess it can't be that easy! |
|
Great work debugging! There was some other serial debuggin that went on over here too: libretro/RetroArch#7404 Could you put up a Pull Request with your .cue changes? i'd love to check it out. |
|
Hi again Rob! I don't think you are going to like it, I really just hacked in there with very little notion of what I was actually doing... On the PSX front, I have had issues with a couple of games not being written to playlist despite being parsed in the metadat "Gunners Heaven (Japan)" and "Raiden Project (Japan)" (latest RDB from RetroArch Online Updater). But when I compiled my own DATs/RDB from the latest Redump DAT, it was the other way around (no games written to playlist, except for those two!) |
|
Disc media uses the serial number. Looking at the following URL: I can see Sonic CD is listed as 4407, but we don't have that in our DB. Would be good to build a scrapper for the serials from redump. When you load the game, does the log say it found a serial number? |
RobLoach
changed the title
|
Hi Rob! Thanks for the update! This is what RetroArch logs (RDB compiled from my own DAT file with cue checksums). However, it gets written/identified as follows in the playlist: The crc32 is that of the cue (great!) but the label is incorrect. Should be "Sonic CD (USA)" Remark 1: the online Redump database shows all version as "Sonic CD" but the Redump DAT shows differently. Based on the Track 13 checksums (cf. my first post) Remark 2: Both http://redump.org/disc/24557/ and http://redump.org/disc/29188/ share the same serial... Same issue with the two versions of "Shining Force CD (USA)" So we either:
I would be of the opinion that RetroArch favors the cue file checksum identification rather than serials but it seems to have been opposed in the past? (#423) EDIT: same issue with Sega Saturn. In the case of "Layer Section" |
|
Was able to replicate... So when RetroArch finds a cue file, it uses the first track found, and grabs its CRC. The optimal solution would be to update the scanning logic, but I think you're right about just removing the Another question. Where is Sonic CD (USA) in the database? Does the title get overriden during the build of the .dat file? I think when a duplicate is found, it overrides the entry. You're right that there are three entries from Redump with
Perhaps when there are duplicates found, it should pick the one with the least amount of characters in its title? |
|
Hello again Rob, Yes, with the current index.js routine, it looks like titles with similar Track01.bin checksums are overridden/assimilated under one single title. For example, in the case of Sonic it is "Sonic CD (USA) (RE125) (Alt)". This was the main focus of my initial post above. I originally thought that having the index.js pick up the CRC of the CUE would solve the problem (and it does in a way since all 3 versions are accounted for in the libretro-DAT if you change the script to retain the CUE checksum instead of Track01), but I know understand RetroArch has another way to identify CD-based games (serial/magic number).
This unfortunately won't work in the case other games/sets. Case in point Layer Section (in the Saturn set). See first post -- "Layer Section (Japan) (6M)" and "Layer Section (Japan) (3M)". Couldn't there be a switch in RetroArch to let the user pick up the scan method? (serial or checksums). For reference, here are the libretro-DAT files I created with CUE checksums: EDIT: readability, links to DAT files |
|
Cue files are a bad way to get crcs, mostly because the actual game is on the track where the executable is, and so are any modifications or translations. By using a cue, you're introducing false positives for any hack that targets that game. There are others SNAFUs like one of the sets for the dreamcast using the same gdis because someone in the dumping group had the brilliant idea to name track files the same, and nearly all gdis are the same there because there is no salt on the files. Of course, track files are a bad way to get to the actual index file needed to play the game shrug. These dumping formats are not preoccupied with indexability or thinking ahead. MAME and chd can't eat them all fast enough. I'm thinking of two things to 'solve' this without chd support everywhere, that wouldn't solve everything anyway, though i'm no C coder: libretro/RetroArch#8873 <- this attaches checksums to the files themselves in any posix filesystem (not windows unfortunately). This means that any filesystem in linux, macos etc would behave like a zipped file to the scanner, reading crc32 directly instead of calculating (well, after a single scan of the tool), and those values move with the files. In fact i just got finished coding the feature in the tool i linked there and using it on my collection, though now to see if retroarch takes advantage, since it's a bit obscure. It also solves the problem of 'softpatching' false positives / false negatives by attaching the checksum of the result of the softpatch, though you have to remember to rescan with rhdndat the game when you update the softpatch, so maybe 'solves' is a bit too strong. libretro/RetroArch#8672 <- this is my attempt to sketch out a 'configurable' scanner, where people could control even the types of files the scanner would consider as it travelled down a filesystem. This would be invisible to normal users, but power users could make the scanner use custom (for specialized cores) launcher files, though i'd prefer a grammar that could support crcs too, besides the name to cleanly separate the launcher file from the scanned file; so you could say 'if you find a matching crc on this dir, i want the cue file on the same dir as the launcher file'. example: top_game_dir You could even get it to scan only the 'actual game' track if you know your dump set has a name standard and divides music tracks out of the data track: |
This PR updates the scanner code to check Redump serials, when it cant identify a disc a crc is generated. The logs and playlists now contain these serials. If you wish, please clone the PR and test this issue again. Also, Rob has updated the databases to contain serials. |
It seems that the way index.js currently works at https://github.com/robloach/libretro-dats is creating issues for some games with alternate versions being misidentified/ignored.
In the case of “Sonic CD (USA)” for Sega CD, the official Redump DAT has three entries (tracklist abridged for readability) :
All versions share the same “Track 01.bin” checksums. Once parsed with the libretro-dat javascript routine, there is only one entry under “Sonic CD (USA) (RE125) (Alt)” in the libretro redump metadat:
The difference in this case only appears in Track 13:
The same is true of “Layer Section (Japan) (3M)” and “Layer Section (Japan) (6M)” in the Sega Saturn set. Except the differences appear from Track 02.bin.
Sega - Saturn Libretro MetaDAT
Could the script entries be based on the cue checksums instead to make sure all versions are accounted for?
The text was updated successfully, but these errors were encountered: