MapCreator: Tilesets being overridden

[TiMMyyMMiT]

As per this post: https://forums.shiningforcecentral.com/viewtopic.php?p=847469#p847469
Map creator is not behaving as intended:

Currently having an issue with the new tilesets :
Even when I import the original map's tilesets, Map Creator adds tiles into free slots. These tiles are considered as "orphans" by mistake : they should be found in existing tilesets.

Leaving as a note for myself or others

[wizardwhosaysni]

Ok, I think map 63 contains a combination of two distinct edge cases :

• map palette 06 is the only map palette having transparency color 0 with value $000 : all other palettes have a distinct color value for index 0, in order to keep black color distinct with no transparency.
• map tileset 36 contains background tiles which use color index 8 (black color), which is not transparent : in this map palette, both colors at indexes 0 and 8 share the exact same value, most probably leading to confusion in several key actions of the toolsuite workflow.

More visually :

• First showing tileset with base map palette 0 to visualize the two distinct background colors used in this tileset :
  
• Then showing tileset with map 63's map palette 6, where both color indexes have same black color :
  

Now observing MapCreator's displayed tilesets and map from PNG :

Two observations :

• tilesets are loaded with base palette 00, but they should stick to current choice of palette coming from image import action : I'll fix it
• map image shows transparency for background plane, while it should be plain black

And now observing map 63 from Map Editor :

Same issue here : background area should have plain black background.
I'll analyze further and see where a fix is needed.

[wizardwhosaysni]

Further analysis on color confusion when several colors have the same value in a palette :

It seems that the issue comes from the way tile images are produced :
https://github.com/ShiningForceCentral/SF2GraphicsManager/blob/e09972f53cf52f5ab92745edd8a8a50b8897b63b/src/com/sfc/sf2/graphics/Tile.java#L109

Tile images are not color-indexed : they already are in simple RGB with no transparency.

In these conditions, there is a loss of information : no more index or transparency to distinguish color index 0 from any other color.

Many places will probably need to manage impact, but the starting point will be to change the way tile images are produced, by either setting transparency for color 0 from the beginning, or using indexed image type.

I'll give it a try later this week.

[wizardwhosaysni]

Struggling quite a bit with BufferedImage, apparently not able to take alpha transparency into account when searching for closest color in IndexColorModel :
https://docs.oracle.com/javase/8/docs/api/java/awt/image/BufferedImage.html#TYPE_BYTE_INDEXED

Found a workaround by changing RGB value for transparent color when required :

Now getting black background as expected for map tileset 36 : I'll confirm and manage impact where needed for MapBlockManager first, and then MapLayoutManager, MapEditor, MapCreator, BattleEditor.

[wizardwhosaysni]

Looking good for MapBlockManager, no change needed apart from a lib update.

[wizardwhosaysni]

Good results for MapLayoutManager too, but I'm noticing rendering is now much slower, most probably due to the way pixels are drawn for each tile. Optimizations will be required.

Same for MapEditor

[wizardwhosaysni]

MapEditor required a tiny change to export layout PNG with indexed colors.

But MapCreator still not able to find available tiles :

This will require a bit more work specifically on MapCreator, but nothing impossible now that we do have background area with black pixels in both tileset and PNG image.

[wizardwhosaysni]

Debug session from MapCreator helped to pinpoint rootcause :
Imported PNG map layout contains altered pixels which point to wrong color index.

I can reproduce the issue from GraphicsManager's image export, using map palette 6 and map tileset 36 : two wrong color indexes on tile 107 (first one to produce error in MapCreator for our use case on map 63).

Oddy enough, the issue doesn't happen when using map palette 0, so it looks like at some point in the image export process, "closest color" algorithm from BufferedImage fails to point to intended color when two colors are similar enough.

I'll continue analysis in this direction until I manage to make GraphicsManager export a flawless indexed color image for map palette 6 and map tileset 36.

Also confirmed : same issue on both PNG and GIF formats.

[wizardwhosaysni]

Graphics and Graphics2D's drawImage function seem to be responsible for these approximations.

There is a way to edit the BufferedImage's raster data directly :
((DataBufferByte)(currentImage.getRaster().getDataBuffer())).getData()[0] = 12;

This line does give a red pixel with map palette 6, and it might be the solution to produce tile images, as well as to produce images composed of tiles and blocks :

• Apply IndexColorModel
• And then populate the raster with dedicated functions, by applying color index directly, instead of applying a color value.

I'll be absent for a few days but I sure hope to give this a try as soon as possible !

[wizardwhosaysni]

Confirmed :
This approach works indeed, with the right colors and great performance.
Now I'll work on replacing all calls to drawImage with this alternative solution : this is probably going to be a quite important rehaul of graphics generation.
I'll take my time to ensure I don't produce any negative impact.

[wizardwhosaysni]

Quick status update

Just reached a point where our initial use case works as expected :

• MapEditor produces an indexed-color PNG image with no more error related to color approximations
• MapCreator now re-generates map 63 tilesets without wrongly identifying any orphan tile : produced tilesets are identical to original ones.

Great news, but still quite some work to do in order to properly manage all impacted features. Once again I'll take my time, and I'll probably share current work in dedicated branches first.

[Accumulus41]

Map 56 (Outside Tower) seems to be having the same problem. Uses Palette 9, Tilesets 93/95/100

Noticed issue was tiles, from tileset 93, characterizing the clouds/sky were being redrawn. After reading issues here, noticed that the blues were dithered(?).

I manipulated palette 9 to be colors that were not as similar, imported and exported map 56 with MapEditor, then processed it through the MapCreator. It came out with no orphan tiles.

So, this (I suppose) is additional confirmation that color approximation was preventing utilization of the tool. Glad I stumbled into this; it made short work of my struggle with the map.

[wizardwhosaysni]

Hey Accumulus, thank you very much for providing a complementary use case to make sure it's also fixed !

I was about to confirm this and then I noticed loading errors in my current working version of MapEditor, and then I noticed even more errors coming from other maps.
I guess my latest changes have introduced quite a few regressions so I'll fix them first and I'll let you know for your use case as soon as possible.

In any case, well done on figuring the issue by yourself with palette edits for more distinct colors !

[wizardwhosaysni]

Alright, map loading bugs actually were already fixed by Timmy so it was just a matter of updating MapBlockManager lib, nice !

Here's confirmation that the new implementations work as expected for map 56 as well :

• MapEditor produces indexed-color PNG with no color approximation
• MapCreator produces identical tilesets

[TiMMyyMMiT]

#18
I have been wondering how we can automate (or semi-automate) updating the dependent libraries (Because it is time consuming and can easily be forgotten).