[Issue] Redump Dreamcast ROMs fail to verify after conversion to CHD format. #110
Comments
|
Thanks for the detailed steps, trying to reproduce as I write. Edit: Good news (well, sort of), I can reproduce with that particular game! Looking into it. |
|
So it looks like an issue with chdman, some dreamcast games still can't be fully reverted apparently, in the case of WWF the resulting bin is missing 352800 bytes. I don't have many Dreamcast games, and the ones I do have are all fine so I didn't catch that. Unfortunately there's no obvious difference between one that works and one that doesn't. I'll open a bug report upstream. There was also a typo on my end making rom deletion silent in check-roms, this is now fixed on the develop branch, deletion will be clearly visible now. |
|
Issue opened upstream: mamedev/mame#11903 My testing seems to indicate that chdman is removing some padding at the beginning of track 2, the size difference is exactly the number of missing If that's really the issue, and is the same for all files, I could probably repair them in oxyromon, but let's wait to hear from the mame devs before I start anything. |
|
Are the ones that test okay for you unlicensed games? I tested against a nearly full Dreamcast set and everything but the unlicensed/MIL-CD games were moved to the trash. |
|
Both sonic games, shenmue and ecco USA releases verify correctly. According to upstream, this is actually intended, chdman strips a pregap because they use the TOSEC layout which is different from the one Redump uses. Not sure what the best course of action is, I'll try to reach out to redump. |
|
New piece of information from Redump, the gap is there for accuracy and can contain actual data. I'm temporarily disabling CHD for Dreamcast, even if I could "repair" most games by padding them with a few zeroes, I can't recreate any lost data so the only foolproof solution would be for chdman to stop stripping these gaps. Pinning the issue. |
|
@TheRealGusBus If you're on x86_64 linux, would you be able to give this modified chdman a try? https://files.alucryd.xyz/chdman It works fine on WWF and other games that failed otherwise on my end. Will submit a PR upstream tomorrow if that works for you too. |
|
I can confirm that my Dreamcast ROMs are being trashed. I'd like to know what CHDMAN platforms follow TOSEC standards and are affected. oxyromon check-roms
Please select systems: Sega - Dreamcast
Processing "Sega - Dreamcast"
Processing "18 Wheeler - American Pro Trucker (USA).cue"
Processing "18 Wheeler - American Pro Trucker (USA).chd"
Extracting "18 Wheeler - American Pro Trucker (USA).chd"
Creating "/mnt/storage/Nextcloud/__groupfolders/1/Roms/Sega - Dreamcast/Trash"
Moving to "/mnt/storage/Nextcloud/__groupfolders/1/Roms/Sega - Dreamcast/Trash/18 Wheeler - American Pro Trucker (USA).chd"
Processing "AeroWings (USA).chd"
Extracting "AeroWings (USA).chd"
Moving to "/mnt/storage/Nextcloud/__groupfolders/1/Roms/Sega - Dreamcast/Trash/AeroWings (USA).chd"
Processing "AeroWings 2 - Airstrike (USA).chd"
Extracting "AeroWings 2 - Airstrike (USA).chd"
Moving to "/mnt/storage/Nextcloud/__groupfolders/1/Roms/Sega - Dreamcast/Trash/AeroWings 2 - Airstrike (USA).chd" |
|
@alucryd I did a release upgrade to the latest Ubuntu LTS, but it seems the libraries are still too old:
I can try building chdman on this system to verify with some basic instructions. |
|
Ah, sorry about that, I run Arch everywhere. I'll push the code later today so you can try building it. It's not difficult, but it might take some time as mame has a huge codebase. |
As far as I can tell, it's only in the GD-Rom code, so hopefully this only affects Dreamcast. |
|
It is fixable for the most part, but if you guys still have the BINs or can grab them from the website that records it all I'd rather spend time on the next item in my list. I will be tackling an |
|
This is my branch with the fix: https://github.com/alucryd/mame/tree/dreamcast-keep-pregap Run the following at the root: You can adjust [0] https://gitlab.archlinux.org/archlinux/packaging/packages/mame/-/blob/main/PKGBUILD?ref_type=heads |
|
The fix has been merged upstream. Not sure when their next release is due, but I'll require 0.262 preemptively in the code. |
At least for me, this is low priority since the CHD files still work in emulators, they just don't verify in oxyromon. Also, just to clarify, does this fix decompression of the CHD files so that existing CHDs will verify properly, or just newly created CHD files? |
|
The upstream fix isn't retroactive, only new CHDs will be fine. Fixing the broken files would require additional work on my end as the gaps are already lost. |
|
So I've installed libsdl2-dev and libsdl1.2-dev. qtbase5-dev was already installed. I still can't compile for some reason: |
|
On my system, these come from the Edit: According to the package search, it's part of libsdl2-dev :/ https://packages.ubuntu.com/search?searchon=contents&keywords=SDL_clipboard.h&mode=exactfilename&suite=jammy&arch=any It seems there were missing spaces in the command I gave you, sorry about that, I've updated it, please let me know how it goes. You might need to run |
|
You can also try to run |
|
Well, it's there already, so I'm more confused than I was. Edit: It seems to be compiling fine so far using the fixed command. |
|
So there's good news and bad news. The good is that the changes to chdman produce a reversible .chd file that's verified by oxyromon. The bad news is that at least the emulator Flycast doesn't launch the game now. It instead gives the error "Unsupported subtype or pre/post gap". |
|
I wonder if instead of modifying chdman to keep the gap data, it should simply pad out the data when extracting. I think that prevents the new gap data in a CHD from breaking games in existing applications, and should allow oxyromon to verify the CHD files just fine. #111 should ensure that any games with gap data don't get converted to CHD in the first place, so zero-padding converted CHD files should be a non-issue for bin/cues already managed by oxyromon. |
Isn't this just a downstream issue with the fundamental change? |
|
@TylerVigario Yeah, scratch my idea of padding it. The flycast repository already has an issue to add support and this is better for preservation purposes. |
I have a bunch of Dreamcast CHD files that don't match currently - are they toast? Probably not a huge deal but just wondering :) |
|
If you guys have time to build and test, there's a new PR upstream that should fix the fix: mamedev/mame#12087 |
Afraid so, but maybe the above PR will say otherwise. |
|
I've managed to check the type2 and type3 games listed here mamedev/mame#12087 (comment) All seem fine according to oxyromon. |
|
This seems to be fully resolved on our end |
|
I'm only up to the M's in the Dreamcast set, but I haven't seen any errors when converting the bin/cue file to CHD with the |
|
I'll update the minimum version requirement to 0.264 in the code then, thanks for all the testing guys. |
Summary
Adding a fresh ZIP file with bin/cue files to the ROM set verifies fine. However, conversion to CHD and running the
check-romscommand after results in the CHD file silently being moved to the Trash directory.Steps Taken to Reproduce
Starting off with a fresh ROM directory:
Importing single Dreamcast game:
Convert Dreamcast set to CHD:
Verify contents of directory:
Run check-roms command against Dreamcast set:
At this point:
the web server shows the game as complete:
The CHD file has been moved to the Trash:
The text was updated successfully, but these errors were encountered: