Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
Composite japanese characters cause files to be not found #1530
Experiencing a bug related to encoding problems with japanese characters with filenames in Yume2kki:
Name of the game: Yume 2kki
Save File and Log
Steps to reproduce:
Navigate to track No. 003 D in the PC music player.
In particular, the Player claims to not be able to find 夢オルゴールFC.
Interestingly enough, grepping the music folder has shown that the track does not indeed exist under that name, but exists under the name 夢オルゴールFC instead.
Looks similar right?
Hence I have further copy/pasted the correct and the wrong name into text files and ran xxd against both of them producing:
which are clearly different.
In particular the difference is in the ゴ character as shown by grepping the music folder.
In the patch Ghabry does
But this might not be right. There is NFD, NFC, NFKC, and NFKD.
In this example, the longer string is the one on the file system. For that I think we would need NFD or NKFD?
How can we know what format is used on disk? This looks like NFD/NFKD but others could be NFC/NFKC? Might we have to try them all in a loop in
changed the title
[FileFinder] Composite japanese characters cause files to be not found
Dec 6, 2018
I've implemented the fix using NFKC, and it works just fine on macOS, so I guess this isn't a problem and we don't need to guess the format.
This still needs to be verified that it works on non macOS platforms though.