Add CHD support to PSX and PSX HW [Bounty]

[ajshell1]

This is a follow-up to a previous bounty.

All I ask is that someone adds CHD support to the Beetle/Mednafen PSX and PSX HW core.

This will have a bounty of $5. I already have all my games in .PBP format, so my PSX library is already compressed. Thus, CHD support for this core is my lowest priority. However, some testing has revealed that CHD compression is more efficient than PBP. Other people are welcome to pitch in their own funds for this bounty, and there is no doubt in my mind that other people will.

[rtissera]

Work has begun on this one.

[Zapeth]

Actually I was already working on an implementation based on how it was implemented in the beetle-saturn core and can load/boot chd files (audio tracks seem to work as well).

However I can't really thoroughly test my implementation because for some reason the core crashes in certain places for certain games for me, which is either because the msvc configuration is broken or I have something not correctly configured (because the core also crashes with the default .cue files which do work with the upstream core for me).

Anyway I've pushed the implementation to my fork if anyone wants to give it a shot, though the build scripts probably need to be changed first to include the new dependencies (I can only work with msvc here and I decided to keep those project files unchanged because it was a mess to get them working in the first place).

[Zapeth]

Alright, I got hold of a Linux system and I pushed some further changes to include the new dependencies in the build scripts.

I also got to test more games since the core crashes were not present in the gcc build and everything is working well so far.

Here is a list of games that I have tested (mainly checked if they booted and loaded up to an ingame status, nothing extensive):

• Firebugs (PAL)
• LEGO Rock Raiders (USA)
• Disney's Atlantis (German)
• Rayman (USA)
• The Italian Job (PAL) (tested libcrypt and audio support with this one)
• Riven (PAL) (tested disc swap + m3u playlist support with this one)

[rtissera]

@Zapeth I'm happy to leave you the bounty on this one, since I am working on a big libchdr refactor / improvement and this will take some time. Marking the bountysource solution as stopped so that you can take over.

[Zapeth]

@rtissera Don't worry I'm not interested in any bounties, I mainly worked on this because I wanted to use the feature myself :)

Plus the main chunk of the work wasn't done by me (the chd library and its integration in the libretro environment), so it wouldn't feel right for me to claim anything for it.

[rtissera]

@Zapeth please take it, pretty happy to see some people helping on the CHD integration ;)

[ajshell1]

@Zapeth

Smart move, using Riven to test disc swapping. I used the NTSC version of the same game to test the RetroArch's ability to swap discs as well.

I'll do some tests based on your code.

Let me know when you are satisfied with your code.

[ajshell1]

@Zapeth I compiled your fork of this core, and I seem to have trouble getting it to recognize my memory cards. That might just be me, but could you see if you have any problems saving and loading on your end?

[Zapeth]

@ajshell1 Loading seemed to work fine for me (granted I only loaded a save in one game), haven't tried saving yet.

It would surprise me if a problem like that would be caused by CD/Image loading code, but if the same thing works for you in the upstream core then I guess thats the only possible explanation.

Anyway, what game(s) did you try where it didn't work for you?

[ajshell1]

@Zapeth I tried Alunda (USA) and Wipeout 3 Special Edition (Europe).

I'm fairly certain that I compiled it incorrectly since RetroArch's history playlist describes it as "Mednafen PSX" even though I set HW_ENABLE=1 while compiling (using libretro-super).

[Zapeth]

@ajshell1 I'm not sure how libretro-super works but a simple make from the core directory compiled without issues for me on Linux (even with HW_ENABLE=1).

And I don't know what the default core settings are for the memory card, but it could be that the different core name created a new configuration and that you need to change the memory card settings to the one that you normally use (shared or separate).

As for the games you mentioned, I tried Wipeout 3 SE and save/load worked without issues for me. However using Retroarch's Vulkan driver I did encouter a couple of strange issues, one of them being that the chd_read() function returned CHDERR_DECOMPRESSION_ERROR for every game that I tried to boot after the first couple of reads (the render setting of the core didn't matter in this case).

I'm not quite sure what's up with that but considering the Vulkan driver generally behaves weird on my Linux system (might be some Intel driver issues?) and that it seems to work fine with the GL driver, I don't think its some major issue in the CHD code (assuming the root of the issue is even there).

So yeah, let me know if you can sort out those memory card issues and/or if you have any other issues, if not then I guess this is ready for a PR.

[ajshell1]

@Zapeth Make the pull request. I"m fairly certain that the memory card issue is due to stupidity on my end.

As for the bugs, I'm sure that if they aren't just on our ends, they'll be squashed by Twinaphex before he merges the pull request. And even if they aren't, they'll be easier to fix once it's in the official build.

[Zapeth]

Actually after switching to a window manager that supports Wayland (i3->sway), the Vulkan driver issues disappeared, increasing my suspicion that the issues I experienced are caused by something else (something Xorg/Xinerama related?)

However for reference purposes I'll just add here that a little further investigation revealed that LzmaDec_DecodeToBuf() in libchdr/chd.c returned SZ_ERROR_DATA, possibly because the source data got corrupted for some reason.

[inactive123]

I merged the CHD PR, so hopefully we can get people to start testing the latest nightlies shortly.

[rz5]

I just tried to compress a game with multiple bin files and plus cue sheet (Tekken 3) with chdman v146 with the command: ./chdman.exe createcd -i path/to/game.cue -o output/path/game.chd

The cue + bin files totaled in about 650MB. The output file is about 450MB.

I played a couple of rounds on Arcade mode, the soundtrack and sound effects were all being played just fine.

[Aerocatia]

I tried to convert "CTR: Crash team Racing" with chdman v188 (from Arch Linux SDLMAME Package) and the game does not boot with a bad sector (13?) error.

The game is the PAL release and the source image had an .sbi file present for subchannel data although I'm not sure if chdman does anything with those. It should get to the menu either way.

[ajshell1]

@rz5 Why did you use version 146 when the most recent release of MAME is version 189?

[ajshell1]

@Zapeth The issue I was having with my memory card no long appears with the new buildbot version. I'm satisfied for now. If you want me to, I'll close the issue and give you your money.

[Zapeth]

@rz5 the newest version of chdman is always included in every MAME release (at least for Windows builds), for other systems it might be separately available as a package or you need to compile it yourself.

@Vaporeon chdman does nothing with those (but you'll need the file to be in the same directory as the image when booting the game) and the game works fine for me on Windows.

• Does the game boot fine with the raw image file?
• Are you using Retroarch's Vulkan driver? If yes, does switching to the OpenGL driver make any difference? (don't forget to change the core setting as well if you try it)

@ajshell1 Good to hear, as for the money though, I'd prefer it if you either keep it or put it up for some other bounty if possible.

[Aerocatia]

So CTR works if I use the GL driver with the software renderer however if I use the Vulkan driver with either the software renderer or the Vulkan renderer it fails with the bad sector error.

[tony971]

Can confirm at least Spyro 2 works with chdman v189. Will convert rest of library later.

[daninthemix]

Can you CHD-ify a multi-disc game like FF7 into a single CHD? If so, do save states work properly (because they don't for multi-disc PBPs in Beetle.

[ajshell1]

@daninthemix No. One CHD can contain only a single disc. You will want to make .m3u files for multi-disc games, just like you would do with games in bin-cue format. @Zapeth says that he has already tested Riven with this method. I am going to do the same.

[Awakened0]

I converted 60 USA and Japan region games to CHD and didn't encounter any problems when testing them briefly on the Win64 software core. Multi disc games with .m3u playlists all worked fine when using the hotkeys to open the tray and switch on the fly too.

[ajshell1]

@Zapeth I'm closing this because I'm satisfied with the implementation.

If you absolutely do not want my $5, I'll try to see if I can get a refund for it. However, I would prefer if you just took the bounty money and saved me the trouble.

[daninthemix]

Sorry to bump this but I have no idea where else to ask - if I try to load an M3U file in a playlist with 3 CHDs in it, Retroarch crashes. Now admittedly I'm on RA 1.6 (not 1.67) - is that the reason?

[rz5]

@daninthemix - you could have asked on the #retroarch channel in the freenode servers or in the libretro forums (https://forums.libretro.com).

For starters, try posting the CHD filenames and the contents of the M3U file.

[daninthemix]

Playlist entry:

C:\DATA\Games\ROMs\PSX\Final Fantasy VII.m3u
Final Fantasy 7
C:\DATA\Games\RetroArch\cores\mednafen_psx_hw_libretro.dll
Beetle PSX HW
0|crc

Final Fantasy VII.m3u:

C:\DATA\Games\ROMs\PSX\multidisc\Final Fantasy VII (USA) (Disc 1).chd
C:\DATA\Games\ROMs\PSX\multidisc\Final Fantasy VII (USA) (Disc 2).chd
C:\DATA\Games\ROMs\PSX\multidisc\Final Fantasy VII (USA) (Disc 3).chd

[tony971]

I think rz5 was hinting for you to ask elsewhere, but you shouldn't include the paths to the chd files. only the filenames.

[daninthemix]

Soz, but yeah that works. Didn't realise the m3u had to be in the same folder as the CHDs.

Ta.