Skip to content

Commit

Permalink
WINTERMUTE: Allow utf8ToWide() and wideToUtf8() work with ASCII strings
Browse files Browse the repository at this point in the history
This is needed for English versions of multilingual games, which use
UTF-8 strings, but we can treat them as plain ASCII, since wide and
UTF-8 strings are not yet supported in Wintermute. This allows at least
the English versions of these games to run.

This allows Reversion 2 and Shaban to start
  • Loading branch information
bluegr committed Nov 8, 2013
1 parent 7c210e0 commit 492cefa
Show file tree
Hide file tree
Showing 2 changed files with 46 additions and 2 deletions.
47 changes: 45 additions & 2 deletions engines/wintermute/utils/string_util.cpp
Expand Up @@ -48,9 +48,41 @@ bool StringUtil::compareNoCase(const AnsiString &str1, const AnsiString &str2) {
return (str1lc == str2lc);
}*/

bool StringUtil::isAscii(Common::String &str) {

This comment has been minimized.

Copy link
@wjp

wjp Nov 8, 2013

Contributor

Having a function called isAscii with side effects is not a good idea.

This comment has been minimized.

Copy link
@bluegr

bluegr Nov 8, 2013

Author Member

Indeed. Changed in 4af998b

uint strSize = str.size();
Common::String punctuation("!\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~");

for (uint32 i = 0; i < str.size(); i++) {
if (!Common::isAlnum(str[i]) && str[i] != ' ' && !punctuation.contains(str[i])) {
// Replace some UTF-8 characters with (almost) equivalent ANSII ones
if ((byte)str[i] == 0xc2 && i + 1 < str.size() && (byte)str[i + 1] == 0xa9) {
// UTF-8 copyright character, substitute with 'c'
str.deleteChar(i);
str.setChar('c', i);
strSize--;
} else {
return false;
}
}
}

return true;
}

//////////////////////////////////////////////////////////////////////////
WideString StringUtil::utf8ToWide(const Utf8String &Utf8Str) {
error("StringUtil::Utf8ToWide - WideString not supported yet");
// WORKAROUND: Since wide strings aren't supported yet, we make this function
// work at least with ASCII strings. This should cover all English versions.
Common::String asciiString = Utf8Str;
if (isAscii(asciiString)) {
// No special (UTF-8) characters found, just return the string
return asciiString;
} else {
warning("String contains special (UTF-8) characters: '%s'", Utf8Str.c_str());
}

error("StringUtil::Utf8ToWide - WideString not supported yet for UTF-8 characters");

/* size_t WideSize = Utf8Str.size();
if (sizeof(wchar_t) == 2) {
Expand Down Expand Up @@ -99,7 +131,18 @@ WideString StringUtil::utf8ToWide(const Utf8String &Utf8Str) {

//////////////////////////////////////////////////////////////////////////
Utf8String StringUtil::wideToUtf8(const WideString &WideStr) {
error("StringUtil::wideToUtf8 - Widestring not supported yet");
// WORKAROUND: Since UTF-8 strings aren't supported yet, we make this function
// work at least with ASCII strings. This should cover all English versions.
Common::String asciiString = WideStr;

This comment has been minimized.

Copy link
@wjp

wjp Nov 8, 2013

Contributor

I'm not sure what the meaning of "wide" here is, but treating it as utf8 doesn't sound right. Did you test this function?

This comment has been minimized.

Copy link
@bluegr

bluegr Nov 8, 2013

Author Member

Yes, I have. Wide strings aren't yet supported in Wintermute (WideString is a Common::String at the moment)

This comment has been minimized.

Copy link
@wjp

wjp Nov 8, 2013

Contributor

I can understand temporarily treating them as ASCII instead of "wide", but I don't get how it is desirable to have utf-8 in there. (Actually, how does utf-8 get in there? Is that ansiToWide doing something strange?)

if (isAscii(asciiString)) {
// No special (UTF-8) characters found, just return the string
return asciiString;
} else {
warning("String contains special (UTF-8) characters: '%s'", WideStr.c_str());
}

error("StringUtil::wideToUtf8 - WideString not supported yet for UTF-8 characters");

/* size_t WideSize = WideStr.length();
if (sizeof(wchar_t) == 2) {
Expand Down
1 change: 1 addition & 0 deletions engines/wintermute/utils/string_util.h
Expand Up @@ -37,6 +37,7 @@ class StringUtil {
public:
static bool compareNoCase(const AnsiString &str1, const AnsiString &str2);
//static bool compareNoCase(const WideString &str1, const WideString &str2);
static bool isAscii(Common::String &str);
static WideString utf8ToWide(const Utf8String &Utf8Str);
static Utf8String wideToUtf8(const WideString &WideStr);
static WideString ansiToWide(const AnsiString &str);
Expand Down

0 comments on commit 492cefa

Please sign in to comment.