Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: Japanese character ShiftJIS encoding not handled correctly in XML serialization #50

Closed
binary1230 opened this issue May 6, 2021 · 2 comments

Comments

@binary1230
Copy link
Collaborator

SNES ROM header supports encoding of Japanese characters in the game title field in ShiftJIS format. However, we're incorectly interpreting these bytes as Unicode.

Diz projects for ROMS with japanese chars in this title field will serialize the incorrect encoding to the XML, and on load, this XML will incorrectly cause Diz to think the name of the cart is different. A verification check will fail and the project will refuse to load.

I have a fix in #49 underway, along with an XML migration that will fix the issue for any affected users.

Technically we don't actually need to store the title in the Project file since we are storing the checksum bytes. However, it's nice to have one extra layer of redundancy so, let's keep it.

The fix adds a lot of extra unit testing and some extra functionality for working with rom titles, checksums, etc. I'm not sure it's comprehensive, but, it appears to work well so far.

(Originally reported by LuigisBlood in SNESLab Discord, thanks!)

@binary1230
Copy link
Collaborator Author

unit test now available showing the bug and mitigating the issue. phew. this ended up being a thing.
https://github.com/Dotsarecool/DiztinGUIsh/blob/fix_cart_title_encoding/Diz.Test/bugs/Bug050_JapaneseText.cs

@binary1230
Copy link
Collaborator Author

fixed in #49

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant