Opaque game window in real hardware

[JohndoGB]

Hello, first I want to say that I tested with a rom and a border that worked perfectly 1 month ago with your Super GB injector online, and today I have the same problem of "pink square" that appears after the boot screen of the Super Gb.

I tried to inject a border in a homebrew this morning with this rom (mai_nurse_v1.01.gb , download on https://lunoka.itch.io/mai-nurse) and a border created with GIMP (I've created 5 or 6 of them in the past, and they worked perfectly, so there's no need to worry about settings here.) and transformed with your Super GB border converter. When I play the game on a real SNES console, (after flashing the game on a cartridge), the central transparency doesn't appear, and the center remains pink. I tried again with a rom created with Gb injector a few weeks ago, on the same cartridge that I flashed again, and it worked, so no problem with the cartridge or the flash software!

So I thought I'd screwed up the colorimetry of the pink, and that the software wouldn't remove it because it was a different color, except that I used the basic image you provide on the site (the clone of the classic super Gb image), and the same result, the pink square remains (I forgot to specify, you can hear the sound of the background, and you can move around the menus etc. "by ear", so the game runs perfectly under the pink square).

So I tried with another rom to redo the procedure, so I re-used my files (rom + border) from 1 month ago, but same result, pink square remains in the center!
I tried a color other than pink to apply the transparency as allowed by your software, and strangely enough, it's still a pink square that appears after the Super GB boot (I tried green, blue etc. respecting the colorimetry suggested in your software, but it's still a pink square...).

A little short of ideas, I tried to tick the "custom gb palette" box, which I'd never ticked before (I've already done about 5.6 roms with your software, but that was 1 month ago), and now it works ...

In short, I just wanted to warn you. I'll also let you know that I ran the roms that left a pink square on the real SNES on an emulator that manages the Super Gb and it worked! (really incomprehensible!) but I imagine that the aim of the software is still to work on the original hardware...

I can't add the roms and borders via github because it doesn't handle this type of .gb .sgb file, but I tested with all sorts of rom + different borders and always came up with the "pink square" result. I'm posting 2 photos of the "pink square" results anyway. I know it's a shame on a flat screen, but it's all I had to hand!

Thanks again for this superb software, and thanks for reading ;)

[marcrobledo]

You can upload a zip file containing any kind of file. Please, post the ROM that is not showing the colors in real hardware, and the border files (in PNG and bin format if possible).

[xenophile127]

@JohndoGB what type of SGB are you testing on (NA NTSC, EU PAL, JP SGB2, etc.)?

Does it make a difference whether you use the "Custom GB palette"?

[JohndoGB]

@JohndoGB what type of SGB are you testing on (NA NTSC, EU PAL, JP SGB2, etc.)?

Does it make a difference whether you use the "Custom GB palette"?

I use a European Super GB on a European Snes, but I repeat the use of the same roms and borders used a month ago and which worked perfectly, no longer work today, hence the fact that I don't think the hardware is at fault.

[JohndoGB]

You can upload a zip file containing any kind of file. Please, post the ROM that is not showing the colors in real hardware, and the border files (in PNG and bin format if possible).

Thanks for your feedback, I didn't know you could uplode a zip! So I've put inside the first rom I tested, but same result on others, as well as 3 different borders.

Just in case: works on emulators, but not on "real consoles".
fichiers fonctionnent pas.zip

[xenophile127]

@JohndoGB What's changed since a month ago is a lot of timing work to get it working on as many platforms as possible. If you could, try my hack: https://www.romhacking.net/hacks/6550/

It also does not set a palette, it previously had issues very similar to what you are experiencing, and it currently (with timing tweaks) has been confirmed to work on real NA SGB, real EU (PAL) SGB, and Analogue Pocket.

The code is the assembly version of this but with tweaks.

[JohndoGB]

@JohndoGB What's changed since a month ago is a lot of timing work to get it working on as many platforms as possible. If you could, try my hack: https://www.romhacking.net/hacks/6550/

It also does not set a palette, it previously had issues very similar to what you are experiencing, and it currently (with timing tweaks) has been confirmed to work on real NA SGB, real EU (PAL) SGB, and Analogue Pocket.

The code is the assembly version of this but with tweaks.

Would you like me to try to flash your rom hack and see what's happening on my SNES? (I'd rather be sure, like any self-respecting Frenchman, my level of English is appalling!)

On the romhacking page, I can see the .zip file to download, but no rom inside?

[xenophile127]

@JohndoGB, yes, I am curious whether my rom hack works on your SNES.

It is a hack of a commercial game so for legal reasons the original rom is not included. Also, it is based off the North American English release (named Final Fantasy Adventure) instead of the European release (named Mystic Quest) which means you are not likely to own.

It might be impossible for you to test it. Sorry.

Also, I just heard someone else is debugging your rom and may have figured out the problem you are having.

[JohndoGB]

So to flash on a cartridge and test on my SNES, I need a .gb file indeed :)

As for debugging my "rom", given that it didn't work with any rom I had, if it solves the problem on just one and you have to debug each time, it won't solve the overall problem, but maybe I've misunderstood :p .

[nitro2k01]

I've looked at this and the short verdict is that it's a timing issue and that the injector needs to have a short delay before sending any data to the SNES. The more detailed report follows below:

I first tried this on bsnes-plus, and it didn't show any SGB border or anything else SGB related. The reason is that the border injector is checking for SGB using C==$14 while bsnes-plus in this case was starting the CPU with initial CPU registers. Using C==$14 for detection is a recent recommendation, and I've taken a principled stance against it because it has a number of both false positives and false negatives compared to the traditional MLT_REQ detection method. You can read my reasons here. But I just patched out that check and continued researching.

At this point, the ROM I was trying with (a patched version of Mai Nurse posted on Discord) showed a random palette on each boot, whereas it worked ok on a flashcart on real SGB hardware. At this point I was convinced that the issue I saw in bsnes was related to the issue in the report. (This indeed turned out to be true.) I ruled out some simple possibilities like relying on uninitialized memory, or sending different SGB packet data in bsnes compared to what I saw in BGB. But everything seemed ok and I was running out of ideas for what could even be wrong.

After a good night of sleep, and now with some new files from JohndoGB above, I tried again. I got the idea of inserting a delay before the ROM starts, and this worked. It seems a 1 frame delay solves the issue on real hardware, and a 4 frame delay is required on bsnes, at least with JohndoGB's files. So it seems like what's happening is that there's a race condition, where sending the data too early might cause the SGB firmware to mess up and corrupt the palettes. I have not researched exactly where in the SGB firmware this happens, but this doesn't seem too important since there's a solution.

Why does bsnes need more delay than hardware? I think it sends the initial packets (normally sent by the boot ROM) artifically, and then jumps directly to the game. The real SGB boot ROM would have a delay after each packet transfer (same as sgb_packet_transfer_wait in main.asm) which is skipped. This probably isn't a problem for most games, because they have plenty of delay anyway before they start transferring anything.

But here's a question that's maybe more important. Why does it not work properly, only sometimes? Probably because something about the injected data is causing the init code to take a different amount of time and indirectly causing a variable delay. I don't think the size of the border data is doing this because it's handled by mem_copy_sgb_4kb which copies a constant size. Maybe whether or not there's a custom palette? Not sure what else could cause it tio take a variable amount of time.

But anyway, something like waiting for 4 additional frames before starting to do anything should solve the issue completely. Or, as I would recommend, use the MLT_REQ method to detect whether it's running on SGB. This would add some extra delay through the sgb_packet_transfer_wait after sending the MLT_REQ packets which should equally solve the issue.

[JohndoGB]

Incredible! Thank you so much for your time! I didn't think the "problem" could come from the time it took to read the information.

I hope the information you've provided will help @marcrobledo (and the people who worked on it) to produce a new version of this great software!

[marcrobledo]

Thank you @nitro2k01, this information is very valuable.

I'll probably go for the MLT_REQ route since it will kill two birds with one shot.
I didn't use it because pandocs recommends the C register method. But your explanation makes a lot of sense.

@JohndoGB I'll try to prepare a beta ROM as soon a possible for you to test. If it works, then I'll adapt the injector to fix this once and for all :-) Give me a couple of days.

[JohndoGB]

I'll even give you several years !!! (and in fact I don't have to give you anything, you can do it whenever you want ;) .

[marcrobledo]

mai_nurse_sgb_test1.zip
mai_nurse_sgb_test2.zip

Can you try these in real hardware? I'll try them later today in bsnes and FPGA myself too.
I want to be sure that not only they work, but they also don't do any odd screen flashes when switching the SGB border.

I'll update the web injector once we can confirm it works in all possible hardware and emulators, as assembling the code for the JS injector is a pain in the ass.

EDIT: I made a big mistake with DMG compatibility. Do not use the test1 if you downloaded it. Try test2.

[marcrobledo]

Check out the commit f616935 for the changes I made.

[marcrobledo]

Report:

• MiSTer FPGA - no custom palette: shows a wrong palette ❌
• MiSTer FPGA - custom palette: works fine ✅
• Analogue Pocket - no custom palette: no border (doesn't seem to be detecting SGB?) ❌
• Analogue Pocket - custom palette: no border nor custom palette ❌
• bsnes/higan - no custom palette: shows a wrong palette ❌
• bsnes/higan - custom palette: works fine ✅

It now works worse than before lol Still I am curious on how the real hardware will work.

[marcrobledo]

mai_nurse_sgb_test3.zip

Added back delay before MASK_EN, fixed no palette version in MiSTer FPGA and bsnes, Analogue Pocket still fails to detect it as a SGB game for some reason:

test3 report:

• MiSTer FPGA - no custom palette: works fine ✅
• MiSTer FPGA - custom palette: works fine ✅
• Analogue Pocket - no custom palette: no border ❌
• Analogue Pocket - custom palette: no border nor custom palette ❌
• bsnes/higan - palette: works fine ✅
• bsnes/higan - custom palette: works fine ✅
• real hardware - no custom palette: needs testing
• real hardware - custom palette: needs testing

[JohndoGB]

I'm testing this later today on my SNES! I'll give you the verdict ;)

[nitro2k01]

Doesn't work on real hardware. The reason is trivial. The "warmup" delay is not optional, and must be done before sending the MLT_REQ packet. Early on after the game starts, the SGB isn't listening for SGB commands yet, so those commands get lost, and the detection fails. Not sure how bsnes/higan manages to work in this case, but you can probably generally trust the Analogue Pocket for testing these things.

[marcrobledo]

The "warmup" delay is not optional, and must be done before sending the MLT_REQ packet. Early on after the game starts, the SGB isn't listening for SGB commands yet, so those commands get lost, and the detection fails.

That's what I thought at first, but then according to this I understood the warm up was only needed before the MASK_EN command.

Not sure how bsnes/higan manages to work in this case, but you can probably generally trust the Analogue Pocket for testing these things.
I trust Analogue Pocket core. What really confuses me is that MiSTer FPGA core does show the border.

Anyway, thanks again for the information. I'll try later today to compile another version with the warm up being the first thing it is ran

[nitro2k01]

I'm knee deep into debugging this now by looking at the SGB's SNES code. But first off, those are two separate issues and this is more interesting than I thought.

The issue where MLT_REQ detection fails is, like I said above, just a matter of waiting enough time before sending the first SGB packet,or it will get ignored. I think this is counted in SNES frames because the delay needed seems to depend on PAL/NTSC.

The palette glitch is more interesting. It's somehow related to MASK_EN. Using MASK_EN to mask the screen, under some circumstances, is causing the default palette to not be written to RAM. This in turn makes the SNES load uninitialized RAM as palette data. I can see that it happens, but don't know why yet. Still investigating... Skipping the MASK_EN command solves the palette issue in all cases I've tried so far. This of course shows garbage on the screen while loading graphics over to the SNES, so it's not a solution. But it is a data point about what's going on.

[JohndoGB]

@marcrobledo : I just grabbed the 2 roms from your zip and flashed them directly onto my cartridges, and it worked perfectly! I have transparency on my SNES equipped with the Super Gb !!!

[bbbbbr]

👀 Noticed this and just subscribing in case useful info comes out of it

(for what it's worth resolved our SGB PAL issues in GBDK with a 4 frame startup delay)

[marcrobledo]

The palette glitch is more interesting. It's somehow related to MASK_EN. Using MASK_EN to mask the screen, under some circumstances, is causing the default palette to not be written to RAM. This in turn makes the SNES load uninitialized RAM as palette data. I can see that it happens, but don't know why yet. Still investigating... Skipping the MASK_EN command solves the palette issue in all cases I've tried so far. This of course shows garbage on the screen while loading graphics over to the SNES, so it's not a solution. But it is a data point about what's going on.

This is interesting. However, the latest builds seem to be working in bsnes/higan, Analogue Pocket and MiSTer, both with or without custom palettes, so I think I've got to the definitive solution. Need to confirm real hardware.

@marcrobledo : I just grabbed the 2 roms from your zip and flashed them directly onto my cartridges, and it worked perfectly! I have transparency on my SNES equipped with the Super Gb !!!

Nice, thank you! Please try the latest ones I'm attaching here:

• test5 should work but Nintendo logo will appear for a few frames.
• test6 should remove the Nintendo logo.
• test7 just removes a di call that I think it is unnecessary.

I'm crossing fingers so at least test6 works in real hardware without any issues (that's it: show the border, no opaque window and no odd colors in no_custom_palette version). This is the definitive test, if it works then I'll update the web injector today and fix it once and for all.

mai_nurse_sgb_test7.zip

[JohndoGB]

So for my part, all the versions contained in your archive from the last message run on my SNES!

mai_nurse_sgb_test5_custom_palette : ✅
mai_nurse_sgb_test5_no_palette : ✅
mai_nurse_sgb_test6_lcd_off_custom_palette : ✅
mai_nurse_sgb_test6_lcd_off_no_palette : ✅
mai_nurse_sgb_test7_lcd_off_di_custom_palette : ✅
mai_nurse_sgb_test7_lcd_off_di_no_palette : ✅

Thanks to you!

[marcrobledo]

Web injector has been updated to 1.2.1 at last! It reflects the entire latest ASM code update so it should work.

@JohndoGB I'm asking for a final test! Let me know if the web injector works (both with or without custom palette) in real hardware by flashing to your cartridges.

[nitro2k01]

I tried the ROMs attached above. They all work on a basic level. SGB detected is detected and it either shows the default palette or a custom one. However, some notes.
Test5 shows the Nintendo logo briefly.
Test6/7 no palette shows some garbage for 1-2 frames right when the mask is turned off if the switch is set to either PAL or NTSC. (Not sure which one because I haven't bothered marking the switch on the back on my SNES! :( ) This is not reproducible on the Pocket core.

Another thing I thought of which is not strictly related to this issue. With the delay before MASK_EN, the screen now goes black (from the startup animation) light (from a couple off frames of GB screen) black (from MASK_EN) light (from the game starting). It would probably be better to use mode 1 (freeze) instead of mode 2 (black color) for a better user experience with less flickering.

[JohndoGB]

@marcrobledo I tested the online version this morning on my SNES, it works perfectly (I tested 2 roms / 2 different borders, and the result is the same), bravo and thank you!

[marcrobledo]

Changed MASK_EN(2) to MASK_EN(1)...
...and crossing fingers it doesn't break anything (because this thing breaks with every small change!).

Will try it later today and see if it still works in FPGA and bsnes.

[marcrobledo]

No one has complained about real hardware after latest commit (which changed the MASK_EN parameter), so I guess it's safe.

Thank you all who helped to fix and improve compatibility once and for all! ^_^

[JohndoGB]

Thanks for your time !