Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Arbitrarily invoke insert mysterious words when Japanese(or not english lauguege) input #3094

Closed
zchee opened this issue Jul 24, 2015 · 22 comments
Labels
bug issues reporting wrong behavior
Milestone

Comments

@zchee
Copy link
Member

zchee commented Jul 24, 2015

Hi,

I'm troubled by arbitrarily invoking insert mysterious words when Japanese(or not English language) input on insert mode.

Tried typing Japanese Hiragana (see wikipedia) words.
After that, converting Hiragana to Kanji (see wikipedia) words.
This behavior is similar to the operation of the Omnifunc completion.

Finally, select any Kanji words and hit <Enter>.
but, Occasionally insert mysterious words on current buffer.

What happened?

For more information, see the screencast.
Screencast on Dropbox

@justinmk
Copy link
Member

I see that some of the characters are broken, but can't see the byte literals clearly in the screencast,

Can you post the literal text that you are trying to insert? That is, please post the text that you expect to insert (not the incorrect text).

@justinmk justinmk added encoding input bug issues reporting wrong behavior and removed encoding labels Jul 28, 2015
@zchee
Copy link
Member Author

zchee commented Aug 1, 2015

@justinmk I think that caused by just maybe #3092.
I will try to test again.

@justinmk
Copy link
Member

justinmk commented Aug 1, 2015

@zchee We can close this then?

@justinmk justinmk closed this as completed Aug 1, 2015
@zchee
Copy link
Member Author

zchee commented Aug 6, 2015

@justinmk
Copy insert text.
but, not UTF-8 style :(
��������

gist https://gist.github.com/zchee/63e546a9126482d57165

This problem is not fix.
Please re-open issue.

set encoding=utf-8
scriptencoding utf-8

@zchee
Copy link
Member Author

zchee commented Aug 6, 2015

@justinmk
and,

echo -n '�' | hexdump

result,

0000000 ef bf bd
0000003

@justinmk justinmk reopened this Aug 6, 2015
@justinmk justinmk added this to the todo milestone Aug 6, 2015
@zchee
Copy link
Member Author

zchee commented Aug 7, 2015

@justinmk Thanks re-open issue and add todo :)
Is there anything that I can be your help?

@justinmk
Copy link
Member

justinmk commented Aug 7, 2015

Need more info.

  • Does Hiragana have the problem also, or only Kanji?
  • Does it work in Vim?
  • What is the Unicode code-point (not the byte sequence) of the characters you are trying to insert?
  • If you insert the characters in your shell (before starting nvim) do they appear correctly?
  • Are you sure the font you are using supports the characters you want?

The characters you posted aren't displayed my web browser. Also, what is the completion menu you are using?

@zchee
Copy link
Member Author

zchee commented Sep 6, 2015

@justinmk
Sorry, late reply.
This condition does not always happen.

• Does Hiragana have the problem also, or only Kanji?

Hiragana and Kanji (and Katakata, all japanese languege).
Both issue.

• Does it work in Vim?

Original Vim is works fine.

• What is the Unicode code-point (not the byte sequence) of the characters you are trying to insert?
You can find the table of codes here: http://unicode.org/charts/ (it has Hiragana, but I don't see Kanji there)

In screencast point 1:15, I inputed ”お” is unicode u304A.
hexdump result,

0000000 e3 81 8a
0000003

• If you insert the characters in your shell (before starting nvim) do they appear correctly?

No problem.

• Are you sure the font you are using supports the characters you want?

Does this question points to the thing of the font file(otf, ttf, ttc) ?
e.g. https://github.com/adobe-fonts/source-han-code-jp
Download url: https://github.com/adobe-fonts/source-han-code-jp/archive/1.004R.zip
Be careful. This file size 52MB because it contains all Kanji words.

The characters you posted aren't displayed my web browser

I do not know why you do not see :(

Also, what is the completion menu you are using?

Does this question points to the thing of the neovim front-end?
I use YouCompleteMe.
If your question is pointing to the fact of this image, it is OS X El Capitan original IME.

untitled

FYI, IME a.k.a Input method editor wiki. https://en.wikipedia.org/wiki/Input_method

@justinmk
Copy link
Member

justinmk commented Sep 6, 2015

@zchee can you try #3246

@zchee
Copy link
Member Author

zchee commented Sep 7, 2015

@justinmk Tested #3246 ,but It has been displayed the mysterious string again... :(
but but, #3092 issue is fixed!!
Thank infomation :)

Is there anything that I can be your help?

@zchee
Copy link
Member Author

zchee commented Sep 7, 2015

@justinmk BTW, I typed "お" on patched #3246 neovim, result
screen shot 2015-09-07 at 9 59 58 am
( sorry, Û<0;76;17M is not matter )

It displayed <8a> after .
hexdump last words is 8a.
It might have been implicated something.

@zchee
Copy link
Member Author

zchee commented Nov 17, 2015

@justinmk Hi

masterr branch's neovim is this issue has been resolved!!
but, What's happened?

If you have time, I want you to tell me commit, which may be the cured reason.
Thanks.

@justinmk
Copy link
Member

justinmk commented Jan 9, 2016

Maybe related: #3168

@zchee
Copy link
Member Author

zchee commented Jan 12, 2016

@justinmk Thanks reply.
Solved.

Close.

@zchee zchee closed this as completed Jan 12, 2016
@hori-ryota
Copy link

I have also encountered a similar problem, but is this resolved?

Input
input

Result
result

I tried followings but not solved.

  • without tmux
  • without iterm2
  • nvim -u NORC
  • without google ime (with default ime)
  • without Karabiner
  • with other PC (Mac)

Is there anything I should try?

@Shougo
Copy link
Contributor

Shougo commented Jun 19, 2017

@hori-ryota You can test it on the several terminals.

@hori-ryota
Copy link

hori-ryota commented Jun 25, 2017

Buffers may be split when using IME and splited multibyte char. It seems to tk_getkeys judge keys with each buffer only.

neovim/input.c at 685ca180f7c96f77a79c78d3b26bd003f8cd834c · neovim/neovim

example,

  • buffer size is 4 (termkey_set_buffer_size(input->tk, 4);)
  • input chars ああああ
  • is e38281

ああああ will be splited to e38281e3 8281e381 82e38182

It was a report for the time being. I will continue to look it up.

@hori-ryota
Copy link

Can we reopen this issue?

@zchee
Copy link
Member Author

zchee commented Jun 26, 2017

@hori-ryota I’ll comment later.

@hori-ryota
Copy link

This trouble was avoided by the following settings.

set ttimeout
set ttimeoutlen=100

This problem seems to be similar to the problem concerning discrimination between cursor key and escape key.

Users using multibyte characters seem to have to be careful because garbled characters are made using default settings.

@Shougo
Copy link
Contributor

Shougo commented Jul 2, 2017

@hori-ryota It is the problem of ttimeout is too short?
I think it should be documented in the documentation.
And it should be in :CheckHealth check.

@hori-ryota
Copy link

Yes, if ttimeoutlen is too short or zero, tk_getkeys is called with force=true and split multibyte char.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug issues reporting wrong behavior
Projects
None yet
Development

No branches or pull requests

4 participants