Problem with chinese or japanese input

[0x452]

Hello, ocornut!

Thank you for imgui! I'm trying to make Japanese and Chinese input (i'm using ArialUni font with custom ranges)

static const ImWchar ranges[] =
{
	0x0020, 0xFFFF,//almost language of utf8 range
	0,
};
		
io.Fonts->AddFontFromMemoryCompressedTTF(ArialUni_compressed_data, ArialUni_compressed_size, 19.0f, NULL, &ranges[0]);

and i'm using (in ImGui_ImplDX9_WndProcHandler

if (wParam > 0 && wParam < 0x10000)
	io.AddInputCharacter((unsigned short)wParam)

But i'm getting that:

what is a problem could be?

[ocornut]

The problem is that your wParam is probably not an Unicode codepoint. Use printf to verify the values you are passing to io.AddInputCharacter(). Read windows docs about WM_CHAR, WM_UNICHAR and how application build settings affect them.

[0x452]

Thank for your answer! But we can’t use unicode setting in our project, is there any other way to convert it to unicode with WM_CHAR?

[ocornut]

Perhaps using MultiByteToWideChar() but you’ll find the answer in windows api documentation or on windows api forum, not really here.

[wbc91]

The reason why we cannot type Chinese is we disabled the Unicode. Then one Chinese character will be split into two chars. However, the ImGui uses ImWchar to accept each one of them as a wchar. I resolved it by doing so.

int input_count = 0;
char InputBuffer[2];
void ImGuiIO::AddInputCharacter(ImWchar c)
{
	const int n = ImStrlenW(InputCharacters);
	if (n + 1 < IM_ARRAYSIZE(InputCharacters))
	{
		if (c > 128)
		{
			++input_count;
			if (input_count == 1)
			{
				InputBuffer[0] = (char)c;
			}
			else if (input_count == 2)
			{
				InputBuffer[1] = (char)c;
				wchar_t pwBuffer[2];
				::MultiByteToWideChar(CP_GBK, 0, InputBuffer, -1, pwBuffer, 2);
				ImWchar outputChar = pwBuffer[0];
				InputCharacters[n] = outputChar;
				InputCharacters[n+1] = '\0';
				input_count = 0;
			}
		}
		else
		{
			InputCharacters[n] = c;
			InputCharacters[n+1] = '\0';
		}
		
	}
}

Hope this can be a solution and finish such kind of discussion.

[JX-Master]

My solution is adding WM_IME_CHAR support in WndProc function so that we don't need to change the source code of ImGui.cpp, since MultiByteToWideChar is not platform-independent.
This works for Chinese character in multi-bytes mode, hope it helps.

        case WM_IME_CHAR:
	{
		auto& io = ImGui::GetIO();
		DWORD wChar = wParam;
		if (wChar <= 127)
		{
			io.AddInputCharacter(wChar);
		}
		else
		{
			// swap lower and upper part.
			BYTE low = (BYTE)(wChar & 0x00FF);
			BYTE high = (BYTE)((wChar & 0xFF00) >> 8);
			wChar = MAKEWORD(high, low);
			wchar_t ch[6];
			MultiByteToWideChar(CP_OEMCP, 0, (LPCSTR)&wChar, 4, ch, 3);
			io.AddInputCharacter(ch[0]);
		}
		return 0;
	}

Another one-step solution will be setting your Project's character set to "Use Unicode Character Set" in your project Properties window, so we don't need to bother about multi-byte conversion and it works smoothly. I recommend this since multi-byte character set is legacy and really tough to handle with.

Just remember that when we use Unicode character set, WM_CHAR and WM_IME_CHAR behaves just the same: they receive one 16-byte Unicode character as input. So we don't even need to use WM_IME_CHAR to do that.

[wbc91]

@JX-Master Your solution is better. Awesome!

[ocornut]

Thank you @JX-Master for the precision. I have updated the FAQ to add more details because this is a common problem.

[liu-kan]

@ocornut & @JX-Master When input Japanese with MS Japanese IME under CP936, dual bytes of one Japanese char are send within one or two wParam in succession. I test it with setting Windows display language to Japanese, the result is the same, I guest in Native Japanese System is same situation. But korean IME is a totally different story. Here is some tweaks.

// in initialization code
		BYTE previous_ime_char;
		WORD lang_id;
		previous_ime_char=0;
		HKL hkl=GetKeyboardLayout(0);		
		lang_id=LOWORD(hkl);
// in HandleMsg
       case WM_IME_CHAR:
			if (lang_id==0x0412)
				break;
			auto& io = ImGui::GetIO();
			DWORD wChar = wParam;
			if (wChar <= 127)
			{
				io.AddInputCharacter(wChar);
				return 0;
			}
			else if(wChar<0xFF){
			//GBK also supports Traditional Chinese, Japanese, English, Russian and (partially) Greek.
			//When input Japanese with MS Japanese IME under CP936, dual bytes of one Japanese char are send within one or two wParam in succession.
				if (previous_ime_char==0){ 
					previous_ime_char=(BYTE)(wChar & 0x00FF);
					return 0;
				}
				else{
					BYTE b2=(BYTE)(wChar & 0x00FF);
					wChar = MAKEWORD( previous_ime_char,b2);
					previous_ime_char=0;
				}
			}
			else
			{
				// swap lower and upper part.
				BYTE low = (BYTE)(wChar & 0x00FF);
				BYTE high = (BYTE)((wChar & 0xFF00) >> 8);
				if (previous_ime_char==0)
					wChar = MAKEWORD(high, low);
				else{
					wChar = MAKEWORD(previous_ime_char, high, low);
					previous_ime_char=low;
				}
			}
			wchar_t ch[6];
			MultiByteToWideChar(CP_OEMCP, 0, (LPCSTR)&wChar, 4, ch, 3);
			io.AddInputCharacter(ch[0]);			
			return 0;
		}

Fig.1 with @JX-Master tweak, but Without this tweaks. (Char in red frame is wrong!)

Fig.2 with this tweaks, in chinese system.

Fig.3 with this tweaks, and change Windows display language to Japanese.

[sweihub]

@JX-Master Thanks for the Chinese input fix, the code should be included into official imgui demo I think.