PCE CD, Sega/Mega-CD and Saturn Games will not scanned correctly

[shakkalakka]

have tried scanning some games from this Image Collections:

Sega Saturn (Redump 20160527)
NEC - PC Engine CD - TurboGrafx-CD Full Set (ReDump 20161106)
and Sega Mega-CD and Sega-CD Reference Set (ReDump 20160827)

but no playlists for that Systems was created after scanning

only my PS1 Set (ReDump 20150524) will recognized and most games (not all!!!) i try to add, appear in a playlist

CD-Image format of all images is : "cue/bin" (unzipped in folders named like images)

use Lakka-Generic.x86_64-8.0-devel-20170213022116-r25642

[markwkidd]

The scanner in lakka and in Windows has not been detecting Redump validated Sega CDs for me either.

[RobLoach]

There's a few discussions about this:

• Serial Scanning Removal for disk CRC scanning. RetroArch#3570
• (Menu) Alternative Scanning Method RetroArch#2033
• A few more....

We have a serial scanning method, which detects the serial key from an ISO file, which is faster then finding the CRC value. The problem is that once it finds that it's an ISO, it ignores the CRC detection:
https://github.com/libretro/RetroArch/blob/96c8ca5a0996badb457d145a218856a0a3f79978/tasks/task_database.c#L580-L595

We need to figure out a way to have it fallback from when the serial doesn't match.

[markwkidd]

Am I understanding this (finally)? In the past I haven't been able to quite suss out the issue.

Situation: There there are no Redump serials for these systems, only CRCs that match Redump cue/bins
1.. The scanner currently works by detecting that it is scanning a cue/bin and entering in 'serial scanning mode' (case DATABASE_TYPE_SERIAL_LOOKUP) for that file
2. The serial is not found
3. The scanner does not return back to this switch statement, so it does not perform a CRC scan on the cue or bin

Edit: I'm not sure I've got Step 1 correct.

I don't know this code yet, but there must be a way to find the end of the code path for the serial scanning process and add a function call to the CRC scanner?

[kivutar]

I fixed the magic number detection for PCE-CD. But it looks like the game serial is not in the ISO!

[meepingsnesroms]

I have this issue too, why not just use crc32 on the iso/bin for now, it will be slower but only if you update the playlist every launch.
I only scan my directory when I add a new rom, and then you just scan that file not everything.

[i30817]

Saturn cue/iso conversions have this problem too. This is troublesome because these conversions are used for hacks (to keep distribution size down by stripping error correction data), so they don't usually have crcs in dat files.

For more details see here:
libretro/RetroArch#5102

I believe the issue explains what is actually happening with the isos (i haven't really debugged). They simply get ignored in the scan because the header signature is different when a cue/bin gets turned into a iso. A bit of padding gets removed, the header starts immediately at offset zero, then it doesn't get assigned a machine.

This doesn't explain why all the other files, which aren't cue/isos, but cue/bin are not being accepted though.

[c0d3h4x0r]

I'm hitting this issue with PC Engine CD cue/bin files. RetroArch refuses to recognize them when scanning, yet I can manually launch them just fine using the Beetle/Mednafen core. This is on a GPD XD Android 4.4.4 handheld console. This is super annoying; please prioritize fixing it.

[davidrahd]

Is this fixed? I still can't detect mostly of my sega cd redump collection, some games from saturn and turbografx-cd don't show up too. All of them are redump collections except saturn which I have some darkwater rips. I'm loading them manually without problems but it would be cool to have a playlist.

[pa-august]

This problem persists. Have to use launchbox as a workaround.

[kivutar]

Let's switch to cue name matching instead of the serial?

[RobLoach]

What would we do to have that work? Is there something on the dat-side that we'd need to change?

I could update https://github.com/robloach/libretro-dats , which handles redump. Could inject the cue names automatically somewhere.

[RobLoach]

Could update libretro-dats to always select the .cue file instead........

Before

game (
	name "Metal Gear Solid (USA) (Disc 1)"
	description "Metal Gear Solid (USA) (Disc 1)"
	rom ( name "Metal Gear Solid (USA) (Disc 1).bin" size 705614112 crc 51f233cd md5 f1ff786c5ff2fc4c0701a1fee068ea95 sha1 848fcd22794526bfd093030c6d6ae6b632d87e4b )
)

After

game (
	name "Metal Gear Solid (USA) (Disc 1)"
	description "Metal Gear Solid (USA) (Disc 1)"
	rom ( name "Metal Gear Solid (USA) (Disc 1).cue" size 1501 crc sdafs23 md5 afsddfafsdfdf23f332 sha1 sdff34fref4f43 )
)

[RobLoach]

We'd also want to update libretro-build-database to reference rom.name:
https://github.com/libretro/libretro-super/blob/master/libretro-build-database.sh#L225

[markwkidd]

Are there any cases where the cue filename is going to be different than the game.name entry? I thought that for Redump the cues have filenames matching the official game title (as recorded by the redump DAT)

If I am correct there would be no need to rebuild the DATs to include the cue in the rom field.

[RobLoach]

Right now, the Redump roms reference the first .bin or .iso file they find. So thoughout the DATs, you'll see a bunch of them are referencing the .bin file instead of the cue.

Unsure which cores support .cue loading instead of the .iso though.

[RobLoach]

Submitted a PR over at libretro/libretro-super#979

I don't see a check against rom.name in task_database.c:
https://github.com/libretro/RetroArch/blob/master/tasks/task_database.c#L1101-L1133

Is that handled somewhere else?

[RobLoach]

Got Chuck Rock to scan with that update! YAAAAYYYYYYYYYYYYYYYYY

[i30817]

But crc of cue files is not unique. Or are you not touching the crc and the 'name' on the rom is just supposed to be the entry file and keep the crc to the first BIN/ISO now?

If the second, that could solve some annoyances in hack roms in cds that need audio (well, if retroarch actually used the crcs, which it doesn't). But if that's the intent, it should be documented.

[ghost]

@RobLoach i dont get how that suppose to work or help the issue. rom.name is the name of the first-track of the cd-image renamed Exile (USA) (Track 01).bin as expected and nothing happens during scan.

i already updated the database using libretro-build-database.sh

[kivutar]

Is it possible that the Chuck Rock scan is a false positive?
From my understanding this change should change nothing to scanning.

[pkos]

have tried scanning some games from this Image Collections:

Sega Saturn (Redump 20160527) NEC - PC Engine CD - TurboGrafx-CD Full Set (ReDump 20161106) and Sega Mega-CD and Sega-CD Reference Set (ReDump 20160827)

but no playlists for that Systems was created after scanning

only my PS1 Set (ReDump 20150524) will recognized and most games (not all!!!) i try to add, appear in a playlist

CD-Image format of all images is : "cue/bin" (unzipped in folders named like images)

use Lakka-Generic.x86_64-8.0-devel-20170213022116-r25642

This PR updates the scanner to detect Redump serials and Rob has updated the databases. If you wish, please clone this PR and test your issue again,