Change Korean and Japanese encodings to fix file finding errors

[fdelapena]

Fixes EasyRPG/Player#649

ICU from toolchains will need to be rebuilt to make this update work properly.

After updating toolchains also closes EasyRPG/Player#746

[Ghabry]

Is that encoding not supported in the static toolchains? :(

[fdelapena]

Not tested, the icudata demo should explain codepage dependencies better. I can check it tonight.

[fdelapena]

Confirmed, it is an independent encoding, dat file needs to be updated.

[carstene1ns]

[Ghabry]

You can now embed word documents to github comments iirc. So you can embed "dat file" in the word document and upload that one :D

[fdelapena]

Here is the .dat download link, for ICU56

[Ghabry]

datfile.docx

[fdelapena]

After further testing, yet another issue. This time with middle dot conversion into UTF-8:

When trying to read some file with middle dot, we have 2 possibilities, depending on encoding:
U+00B7 · middle dot (windows-949-2000, uses backslash)
U+30FB ・ katakana middle dot (ibm-1363_P110-1997, uses Won)

Tests:
--encoding windows-949-2000 --start-map-id 11 --new-game
--encoding ibm-1363_P110-1997 --start-map-id 11 --new-game

Not sure which filename is the right on real Korean Win32 zip decoding. At least after using convmv to convert unzipped files on non-Win32, it needs windows-949-2000 to find this file properly.

[carstene1ns]

time to get a windows 98 vm ready 😀

[fdelapena]

Can't test this until EasyRPG/Player#673 is fixed.

[fdelapena]

After unpacking a zip using Korean locale on Windows, it seems to use U+00B7, so windows-949-2000 (with \ ) is the current candidate. Do not merge until checking deeper on Win32 with a full icudata.

[Ghabry]

Is this still pending fix?

[fdelapena]

Is this still pending fix?

Fixed, tested and ready.

[fdelapena]

It looks like Japanese encoding needed an update too... it does not detect filenames with ～ properly and the new encoding kills backslash to yen conversion 😭.

[fdelapena]

Ready for your ICU rebuilds. ^^