Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Windows locale problem #13107

Closed
wants to merge 7 commits into from
Closed

Fix Windows locale problem #13107

wants to merge 7 commits into from

Conversation

ken2812221
Copy link
Contributor

@ken2812221 ken2812221 commented Apr 28, 2018

Fixes #13103
This PR make internal strings of Windows always be utf8-encoded

  • Add UTF8 <-> localized string conversion function
  • Use localized string on Berkeley DB
  • Use localized string on LevelDB
  • Use localized string on fstream::open
  • Use std::fstream instead of fs::fstream

Patched the boost library to convert fs::path::string() to be UTF8 encoded. Add two functions that convert between UTF8 and Local encodes.

To those who want to test this, adding -datadir=(some non-ASCII characters) option make every file that Bitcoin Core use contain non-ASCII characters

Defect: If the user's codepage does not support the filename, it will be encoded to ?
I don't known how to solve this. Leveldb and Berkeley DB don't support wide char.

src/util.cpp Outdated
@@ -1018,3 +1018,49 @@ int ScheduleBatchPriority(void)
return 1;
#endif
}

std::string Utf8ToLocal(const std::string& utf8_string)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe move to utilstrencodings?

@laanwj
Copy link
Member

laanwj commented Apr 28, 2018

Thanks. Though I'm sad that this is needed.
To stay sane, and keep locales and encodings out of everything as much as possible:

  • Please keep all our internal state in UTF-8 everywhere. This was always the intent, so in interaction with JSONRPC we can assume all strings are UTF-8, and matches with Qt's view.
  • Only convert to local encoding when needed, so just before going to the file system layer, or when coming from the file system layer (or filenames, command-line arguments etc).

BTW, there are GUIUtil::qstringToBoostPath and GUIUtil::qstringToBoostPath to convert Qt strings from/to local paths but I don't think they are useful in this case as it doesn't happen on the interface from Qt to core code.

@ken2812221 ken2812221 changed the title Fix Windows locale problem [WIP] Fix Windows locale problem Apr 28, 2018
@ken2812221 ken2812221 changed the title [WIP] Fix Windows locale problem Fix Windows locale problem Apr 28, 2018
@ken2812221 ken2812221 closed this May 9, 2018
@bitcoin bitcoin deleted a comment from mrthuan9xpro May 9, 2018
@ken2812221 ken2812221 reopened this May 9, 2018
@ken2812221
Copy link
Contributor Author

prefer #13426

@ken2812221 ken2812221 closed this Jun 11, 2018
@ken2812221 ken2812221 deleted the win-enc branch June 11, 2018 16:27
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Sep 8, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Invalid wallet path with Chinese characters in windows
4 participants