Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TurboGrafx-16/ PC Engine/ CoreGrafx Mini ROM dumping #112

Open
kjetil-f opened this issue Aug 17, 2023 · 69 comments
Open

TurboGrafx-16/ PC Engine/ CoreGrafx Mini ROM dumping #112

kjetil-f opened this issue Aug 17, 2023 · 69 comments

Comments

@kjetil-f
Copy link

Anyone got it to work yet?

Maybe it's possible to use a similar method as the Sega Genesis 2 Mini to achieve this?

According to this video it seems doable.

@DrAzathoth
Copy link

Supposedly, Hakchi2CE allows you to dump it like the other mini consoles it supports, and like the Genesis Mini, you’ll have to unpack M2’s archive. By coincidence, I ordered myself a CoreGrafx Mini, so I will be able to test this myself in a week or so.

@kjetil-f
Copy link
Author

kjetil-f commented Aug 17, 2023

I had my suspicions. Please report back if you are able to test it. Can't seem to find much information on the matter.

@DrAzathoth
Copy link

The CoreGrafx mini arrived today, but I have no idea how to FTP the console. Unless someone can teach me how or the Hakchi2 CE update gets released, I have no way of dumping it at the moment.

Anyone can feel free to help me out here.

@kjetil-f
Copy link
Author

Maybe the Sega Genesis 2 Mini method work. But maybe you don't need/should use the player 2 port in this case, but do as he says in the video.

@DrAzathoth
Copy link

I already tried that, but unfortunately it didn’t work.

@kjetil-f
Copy link
Author

That's a bummer. Probably have to wait for a compatible Hakchi release then (if that ever comes out).

@Levine91
Copy link

Levine91 commented Sep 5, 2023

at this point I don't think a hakchi release for the TG-16 is ever coming.
However, the pieces are clearly all there for someone to do it. Despite at least 3 different groups teasing it over the course of 3 years with nothing ever actually being released hack wise.

1.) This person dumped the exclusive games somehow.
https://archive.org/details/pcemini

2.) There is more than one video on YouTube of someone playing on a modded TG-16/PC Engine mini
https://www.youtube.com/watch?v=4kIvOiuui_E - 4 months ago, showing off various emulators
https://www.youtube.com/watch?v=o8dzJr6KwA4 - PoC of swapping games out ONE MONTH after it released in 2020.

@DrAzathoth
Copy link

I would contact KMFDManiac via email or DM, but I can’t seem to find any contact info. I guess it is about time that I comment on one of his recent videos asking for a public release of the Hakchi fork for the purpose of dumping the ROMs from the Mini.

Wow. That was quite the run-on sentence.

@Levine91
Copy link

Levine91 commented Sep 7, 2023

I would contact KMFDManiac via email or DM, but I can’t seem to find any contact info. I guess it is about time that I comment on one of his recent videos asking for a public release of the Hakchi fork for the purpose of dumping the ROMs from the Mini.

He is still pretty active on Reddit. Try there.
https://www.reddit.com/user/MDFMKanic/

@stuckinstyx
Copy link

SUCCESS!!

I was able to dump the ROMS from my PC Engine Mini using a method very similar to the Genesis 2 Mini. The key is to get it into FEL mode before running the Hakchi dumper. You can do this using a tool found in the Astro City Mini Batocera installer here:

https://github.com/acm-cfw/acm-cfw.github.io/releases

Scroll down the the ACM alpha 0.1, click assets and download acm_install20221203.zip
In the data directory of that zip folder you'll find sunxi-fel-handshake.exe
Run this and plug in the mini console into a USB port using a non-crippled USB A - Micro USB cable. Turn on the mini console and wait for the tool to tell you it is in FEL mode. This might take a few tries.

Then you can run the hakchi dumper tool linked in the Genesis Mini 2 guide
#56

It will immediately start working.

Once you have the dump, mount it using

guestmount -a /path/to/nand.bin -m /dev/sda9 --ro /path/to/mount

Note the different path from the Genesis Mini 2 guide.

The three important files (alldata.bin, alldata.psb.m, and m2engage) will be in the root folder of the mounted drive.

DO THIS AT YOUR OWN RISK.

  • Downloading and running mysterious executables from the net is always risky.
  • It is always to accidentally brick your mini console.

Notes:

  1. I couldn't get the sunxi-fel-handshake tool to work on my Win11 laptop. Might have been a power issue. It worked fine on a Win10 desktop and a Linux desktop
  2. I couldn't get the guestmount tool to work on my Win11 laptop usig WSL. The tool wanted a valid kernel image in the right place. I couldn't be bothered to troubleshoot it, I just copied the nand file over to my Linux desktop and mounted it there.
  3. The USB drivers come with the hakchi dumper tool, so you may need to install these before the FEL tool.

@kjetil-f
Copy link
Author

kjetil-f commented Jan 4, 2024

Amazing. Well done! I will test it out when time allows. So basically run sunxi-fel-handshake.exe -> plug in and turn on your mini -> wait for FEL mode -> run hakchi.exe from the Genesis 2 tutorial -> save NAND -> mount the saved NAND and extract ROMs

@DrAzathoth
Copy link

Holy crap! I applaud you, @stuckinstyx!

All that we need now is a way to uncompress the CD games for the TG16 and the Genesis 2.

@jd213
Copy link

jd213 commented Jan 4, 2024

Thanks a lot, can confirm that it works perfectly, and m2-unpacker was able to get all the roms/images! Too bad that Salamander wasn't hidden on the PCE Mini, still don't understand why it was an exclusive to the TG-16 Mini when it was never released as a TG-16 game back in the day, but Konami's gonna Konami I guess. Hopefully someone figures out how to convert the CD images as well.

The three important files (alldata.bin, alldata.psb.m, and m2engage) will be in the root folder of the mounted drive.

Glossed over this at first, took me a couple minutes to figure out that one needs to copy from /home/[user]/mnt/ instead of /home/[user]/mnt/usr/games/ like on the Genesis 2 Mini.

@DrAzathoth
Copy link

I cannot get my CG mini into FEL mode using the tool. Does this only work with the PCE mini?

@stuckinstyx
Copy link

@DrAzathoth , try a different cable. Some micro USB cables are power only.

Also, you don't need to use a second cable like the Genesis Mini 2, a single cable to the power port is fine.

Have you installed the drivers? They are included with the Hakchi dumper if you need them.

Also make sure not to use a USB hub. And try it on a desktop computer if you can. I never got it to work on my laptop. Perhaps the USB ports don't provide enough power.

@jd213
Copy link

jd213 commented Jan 5, 2024

It took me a couple tries to get it to dump, try launching both of the tools and then plugging in the mini while its switch is turned on. It also worked fine on my laptop that doesn't have a VPN installed, but not on one that does (even with all of the VPN processes ended in Process Manager).

@Levine91
Copy link

Levine91 commented Jan 5, 2024

Incredible work!
My question now is, how do we swap a game out? (Since I assume we can't just add more right now)

@DrAzathoth
Copy link

I’ve tried a wide variety of cables that I know work with other devices, but it still doesn’t work for me. I have been able to dump the Genesis Mini 1 and 2 so I am certain I have the drivers installed properly (PCE Mini is extremely similar to the Gen Mini 1’s hardware). The only feedback I receive is the message “No Allwinner devices in FEL mode detected”. The handshake tool terminates after 20 attempts.

My exact process is this:

  • Run the handshake tool
  • Within a couple seconds, plug in the CG Mini while the power is already on
  • Wait until the tool terminates and then run it a couple more times to be certain it doesn’t work

@kjetil-f
Copy link
Author

I managed to dump the nand from my CoreGrafx mini, but my Linux skills isn't exactly great. Been trying for hours now. Any concrete examples on how to mount this thing?

@jd213
Copy link

jd213 commented Jan 10, 2024

I used sudo guestmount -o uid=500,gid=users,nounix -a /mnt/c/Users/[user]/Desktop/nand.bin -m /dev/sda9 ~/mnt
Then individually extracted the files as follows:
sudo cp /home/[user]/mnt/alldata.bin /mnt/c/Users/[user]/Desktop
sudo cp /home/[user]/mnt/alldata.psb.m /mnt/c/Users/[user]/Desktop
sudo cp /home/[user]/mnt/m2engage /mnt/c/Users/[user]/Desktop

@kjetil-f
Copy link
Author

Thank you. Unfortunately I only get this error message: "libguestfs: error /usr/bin/supermin exited with error status 1"

@ghoost82
Copy link

Like for others the guestmount command failed with a libguestfs error. But the following worked for me:

sudo losetup -Prf /path/to/nand.bin
sudo mount -o ro /dev/loop0p9 /path/to/mount

@kjetil-f
Copy link
Author

Thanks. I have tried that as well, but gets "losetup: cannot find an unused loop device".

@jd213
Copy link

jd213 commented Jan 13, 2024

Hmm, I know I had similar errors when trying to mount the Genesis 2 Mini nand, but I was able to eventually mount it by googling the errors and trying various things, so hope you're able to figure it out.

@kjetil-f
Copy link
Author

Finally got it to work with my Core Grafx mini.

First I tried the same methods on another computer, but got the same errors. What worked for me in the end was:

cp /mnt/c/Users/[username]/Desktop/nand.bin .
sudo losetup -Prf nand.bin
sudo mount -o ro /dev/loop0p9 rootfs

Then I just used the regular Windows File Explorer to navigate to the "rootfs folder" and copied over the 3 required files back to Windows and used the m2-unpacker.

I couldn't find any bios files on here. Do any of you know if it's included with the system?

@DrAzathoth
Copy link

DrAzathoth commented Jan 25, 2024

@kjetil-f, I still haven’t had luck dumping my CG mini so I can’t say this with certainty, but surely the BIOSes have to be included? Holding the select button when you start a CD game on the mini console emulates running the game on an older BIOS (for example, getting the Peke easter egg instead of Rondo). Perhaps the BIOSes are hidden in the .cd archives just like how they are with the Genesis Mini 2? Yes I know the BIOSes are loose on that console, but it has been confirmed they are inside the archives as well. (Source)

@kjetil-f
Copy link
Author

Cool. How did he do that?

But I think you might be right. I took a quick peak with some of the games using a hex editor, and searched for "bios". Found a reference for it it within the games at least.

18 hits within alldata.bin.

@RealRelativeEase
Copy link

None of the commands to mount the nand file have worked for me so far. Are you using WSL2 or custom builds/actual Linux systems?

@jd213
Copy link

jd213 commented Feb 4, 2024

I used WSL2, yeah. Did you run an update in it already? I believe the Genesis Mini 2 thread shows how, as well as some other tips.

@RealRelativeEase
Copy link

I used WSL2, yeah. Did you run an update in it already? I believe the Genesis Mini 2 thread shows how, as well as some other tips.

WSL2 is version 2.0.14.0, guestfs and guestmount are also the current versions as well. I usually get an error message related to fuse:

fuse: device not found, try 'modprobe fuse' first
libguestfs: error: fuse_mount failed: /home/red/mnt, see error messages above

I tried to follow the instructions in the Genesis Mini 2 thread, but none of them worked for me.

@jd213
Copy link

jd213 commented Feb 12, 2024

It's also possible to apply romhacks to some of the PCEmini roms, so the code for the ones that work should be the same (at least for the parts that get patched). But Splatterhouse Chrome didn't work for me after patching, so I wonder what was changed for Splatterhouse.

@DrAzathoth
Copy link

DrAzathoth commented Feb 12, 2024

It's also possible to apply romhacks to some of the PCEmini roms, so the code for the ones that work should be the same (at least for the parts that get patched). But Splatterhouse Chrome didn't work for me after patching, so I wonder what was changed for Splatterhouse.

Are you sure you patched the right region ROM? According to the romhacking.net page, it needs the Japanese ROM, which is only on the PC Engine mini—not the TG16 mini or the CG mini.

Edit: The page also mentions “You have to add 512 octets (= 512 bytes / 4096 bits) at the begining of the ROM's file before patching.” Did you try that?

@jd213
Copy link

jd213 commented Feb 12, 2024

Ah, thanks. I have the PCE Mini but I missed the part about the octets, so I used Advanced_SNES_ROM_Utility to add a SNES header (which are also 512 bytes), and the patched rom worked after that (at least in Ootake).

I forget if there were other patches that didn't work, but good to know that the roms are seemingly all unaltered.

@DrAzathoth
Copy link

I have good news and bad news.

Good News: I have found a tool that extracts both the pcd and mcd formats perfectly!

Bad News: I am not allowed to distribute it.

@stuckinstyx
Copy link

stuckinstyx commented Apr 5, 2024 via email

@DrAzathoth
Copy link

I traced it to the No Intro forums, but I had to ask someone if I could have it. He told me not to distribute, so I’ll respect his wishes.

@DrAzathoth
Copy link

After looking at the tool itself, it appears to have been made by Cyanic from MMC. Possibly an overhauled version of this. Regardless, if you guys want this tool publicly released, then you’ll need to contact Cyanic, not the unrelated No-Intro forum member.

@RealRelativeEase
Copy link

Bad News: I am not allowed to distribute it.

Sigh...

What's the output format for the uncompressed disc images? And does the tool also support compression of disc images in other formats to potentially make them playable in M2's stock emulators?

@DrAzathoth
Copy link

DrAzathoth commented Apr 6, 2024

Bad News: I am not allowed to distribute it.

Sigh...

What's the output format for the uncompressed disc images? And does the tool also support compression of disc images in other formats to potentially make them playable in M2's stock emulators?

It seems like a WIP development tool used for R&D for the PCE mini hack, thus it only has extraction and debug log support. I still haven’t been able to dump the nand for my CoreGrafx mini, so the results might only be true for the Genesis Mini 2.

What it does is extract the mcd archives into an iso/cue format with uncompressed .wav files for the audio tracks. Optionally, you can choose to extract the BIOS packed with it in the archive. Curiously, the BIOSes have different checksums than the ones easily found loose. After compressing the iso/cues into a chd files, some games were recognized by RetroAchievements, so most, if not all of them, are probably clean dumps.

The only problems I encountered were both the US and Japanese versions of Ecco: The Tides of Time erroring before it extracted the last audio track. Also, the iso for the European version of Robo Aleste was unable to the mounted by me, but ran fine in an emulator.

@RealRelativeEase
Copy link

After looking at the tool itself, it appears to have been made by Cyanic from MMC. Possibly an overhauled version of this. Regardless, if you guys want this tool publicly released, then you’ll need to contact Cyanic, not the unrelated No-Intro forum member.

Does anyone have their contact info? I've found a GBATemp profile which had the same profile picture as their Nuget profile, but they haven't logged in in about four years. I'm not sure if they're maintaining an online presence at this point, so we might be out of luck.

@mjBrickman
Copy link

I believe Cyanic's GitHub is https://github.com/GMMan. I found this through using a reverse image search on their profile picture and checking to see if any of their repositories correlate to the mini modding community, which some do. They appear to be still actively using their GitHub account as well.

@RealRelativeEase
Copy link

Anyone have the means to get in touch with them and ask them about the tool? There's no DM function on Github and I'm not sure if randomly commenting on one of their repos will get their attention. Not to mention that might come off as rude.

@stuckinstyx
Copy link

Sounds like the same person who posted a description of the file format on Twitter.
https://twitter.com/GMMan_BZFlag/status/1589100542894084096
You could try DM'ing them there.

@DrAzathoth
Copy link

After looking at the tool itself, it appears to have been made by Cyanic from MMC. Possibly an overhauled version of this. Regardless, if you guys want this tool publicly released, then you’ll need to contact Cyanic, not the unrelated No-Intro forum member.

Does anyone have their contact info? I've found a GBATemp profile which had the same profile picture as their Nuget profile, but they haven't logged in in about four years. I'm not sure if they're maintaining an online presence at this point, so we might be out of luck.

There is a contact owners function on the website I gave earlier.

@RealRelativeEase
Copy link

I've contacted them, here's their response:

That's mine, and it was passed on because someone from No-Intro needed to verify whether a claimed dump from the device was genuine (it was not). For those who want to just dump games to play on other devices, I would really rather someone other than myself do some legwork. It's been four years since the device released, someone else ought to have had a crack at it already. So here's the deal. My program actually does have import capability, but what I released was a limited version and not for public distribution. If someone can demonstrate putting in the effort and making an open-source converter that can extract the data from the format (not even a proper bin/cue, just the decompressed data chunks and an Opus stream for the audio tracks), I'll open source my program with the import capability included. I just want to see someone who's actually interested in reverse engineering the format rather than only using the end result.

I don't know a single thing about programming, but if anyone has the necessary know-how, feel free to reach out to them.

@DrAzathoth
Copy link

I don't know a single thing about programming, but if anyone has the necessary know-how, feel free to reach out to them.

Maybe I'll take a crack at it despite having no experience with this sort of thing. I'll see if I can clear my schedule to work on this as well. However, I won't attempt this until I can actually dump my CoreGrafx Mini as I still can't get it to boot into FEL mode.

@RealRelativeEase
Copy link

I still can't get it to boot into FEL mode

Could you share the model number of your unit? The dumping process worked fine on the Coregrafx Mini that I bought early 2022. Maybe those were from two different production runs?

@DrAzathoth
Copy link

DrAzathoth commented Apr 10, 2024

@RealRelativeEase, mine is HTG-009.

While doing a little research, found out that there are revisions between serial numbers so I’ll include mine: 01001602
Supposedly, the 010x serial numbers are a hardware revision that fixes the USB ports. Serial numbers 009x are the old, faulty revision.

As for the model numbers:
HTG-008 PC Engine
HTG-009 CoreGrafx
HTG-010 TurboGrafx

@RealRelativeEase
Copy link

While doing a little research, found out that there are revisions between serial numbers so I’ll include mine: 01001602

Mine is 01003911. So mine is a newer model, if they're numbered in ascending order? You wrote that you've already tried different cables, so I suppose that you've also tried different USB ports and maybe even a different machine? At this point, I'm stumped.

Supposedly, the 010x serial numbers are a hardware revision that fixes the USB ports. Serial numbers 009x are the old, faulty revision.

At least that alleviates my paranoia about the faulty USB port.

@DrAzathoth
Copy link

At least that alleviates my paranoia about the faulty USB port.

Don’t take my word for it. I found a case on Reddit where someone with a 010x model had broken USB ports, so still exercise caution. This is all just a rumor.

@DrAzathoth
Copy link

I have only a single desktop PC, so I have no other devices to try it out on. I am using the same cable I use regularly with my controller and it was also used to dump the other mini consoles.

@RealRelativeEase
Copy link

Have you tried uninstalling the drivers and reinstalling the ones packaged with the Hakchi dumper? It might be an issue with the drivers on your system if the mini doesn't go into FEL mode.

@DrAzathoth
Copy link

I’ll give that a try, then.

@DrAzathoth
Copy link

Still no luck. I guess I'll have to wait until KMFDManiac releases his work or find some way to FTP the console with UART as I have heard somewhere that it is possible.

@Levine91
Copy link

Still no luck. I guess I'll have to wait until KMFDManiac releases his work or find some way to FTP the console with UART as I have heard somewhere that it is possible.

You'll be wasting your time if you or anyone else decides to just wait on KMFDManiac.
He was specifically waiting on someone else in regards to extracting/replacing stock roms, kernel flashing, etc.
All he really does is on the RetroArch side of things. Doesn't develop the tools or anything that many of us have been waiting on since Spring 2020.

@RealRelativeEase
Copy link

He did show how to extract the files required for M2 Unpacker in one of his videos, using a (to my knowledge) not yet released version of Hakchi to access the system via FTP and grab the files from there.

@jd213
Copy link

jd213 commented Apr 16, 2024

Still no luck. I guess I'll have to wait until KMFDManiac releases his work or find some way to FTP the console with UART as I have heard somewhere that it is possible.

Maybe try a Windows boot USB like WinPE or maybe a dual-boot with a fresh Windows install and see if you can get the Mini working with your PC that way. Might be some kind of software or hardware conflict preventing it from connecting, dunno.

@Zorglub51
Copy link

Hi, was anyone able to extract images or elements of the UI used in the PCE mini? I am recreating the interface for emulator front ends, I find it so nice! It would help a great deal to get the original images rather then remake them :) Thanks

@PW5190
Copy link

PW5190 commented Jun 18, 2024

I have good news and bad news.

Good News: I have found a tool that extracts both the pcd and mcd formats perfectly!

Bad News: I am not allowed to distribute it.

Now who, may I ask, is the creator of this extraction tool? And why do they forbid distribution?

@RealRelativeEase
Copy link

I have good news and bad news.
Good News: I have found a tool that extracts both the pcd and mcd formats perfectly!
Bad News: I am not allowed to distribute it.

Now who, may I ask, is the creator of this extraction tool? And why do they forbid distribution?

Their name and their response is further up in this thread. #112 (comment) They developed the tool, so it's their decision to make.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests