-
Notifications
You must be signed in to change notification settings - Fork 730
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Display errors with certain characters #234
Comments
On Linux, this works fine for me, but on Mac OS X 10.7.3, the system does not know about this character and wcwidth() returns -1 (unprintable), so mosh does not know how many columns the character will occupy. Assuming you are using a Mac, that unfortunately is the answer. We will report this to Apple. |
Yes, I'm on OSX 10.7.3. It seems like the system does know about this character. ssh and any other application I use knows and displays it, the only application that seems not to know it is mosh. |
SSH doesn't need to know about characters; it just conveys a stream of bytes from one end to the other. Mosh has a terminal state object which is synchronized between server and client, so it needs the character metadata on both machines. What outer terminal emulator are you using; is it OS X's standard Terminal.app? And do you have any other terminal emulators in the mix, e.g. You can compile and run this C program on both machines to check if #define _XOPEN_SOURCE
#include <wchar.h>
#include <locale.h>
#include <stdio.h>
int main() {
setlocale(LC_ALL, "");
printf("%d\n", wcwidth(0x269C));
return 0;
} (I didn't test this on OS X, so it's possible it will fail to compile for some reason.) It will print a positive number iff the character is known. Make sure to run it in a Unicode locale. If you don't have one by default, you can do something like
If you get a positive number on both server and client, and yet Mosh does not work correctly, then there's a bug in Mosh and we can investigate further. (In the long run I would like to use a dedicated Unicode library, and drop our dependence on the system locale libraries, which have caused no end of trouble. See discussion on #74.) |
I think officially speaking, a Unicode app is supposed to use the "default" properties of the code point range (including width) if it doesn't know about the particular character. Unfortunately there doesn't seem to be a way to get these default properties in POSIX. A dedicated Unicode library would help with this. |
Hi Keith, Just checking. I think you can't assume wchar_t is ISO 10646. It is just an implementation defined `wide character'. If you are working with ISO 10646 inside Mosh explicitly (not wide characters), then you shouldn't use wcwidth(). In the past I once needed a compact implementation of wcwidth(), explicitly for use with ISO 10646. Markus Kuhn has an implementation that seems to work quite nicely: http://www.cl.cam.ac.uk/~mgk25/ucs/wcwidth.c Maybe it is of any use to you? Otherwise, I'm pretty sure IBM's ICU should be of use: Ed |
Hi Ed, At configure time we check for We may have to ship our own Unicode library eventually. ICU is kind of a monstrous beast though. -Keith |
Hi Keith, Thanks for the explanation! |
I just commented this on #361, but since it's about OS X it seems a bit more relevant to this ticket (although both tickets appear to be virtually the same thing): I keep periodically hitting situations where various characters don't render in Mosh, because At this point I'm thinking the only real solution to this problem is for Mosh to calculate character widths itself. Perhaps it could fall back to its own calculation if the platform-provided Or as suggested in this ticket you could just ship your own unicode library entirely. My concern is that if Mosh thinks a character has a width of 1 but the terminal emulator thinks it has a width of 2, that will presumably render incorrectly. I'm assuming that the terminal emulator agrees with |
Addendum: Apparently glibc uses Unicode 6.0 but it's |
My current thinking on Unicode issues: Mosh is a virtual terminal, split across client and server, and it This argument dictates that Mosh must have its own internal wcwidth Mosh then sends the character off to the client's terminal, where it This is in general a hard problem: Most current terminal emulators But the user can legitimately ssh into a remote host and run My current best idea for handling this is to offer the user two
One unfortunate thing here is that Unicode will continue to grow Perhaps we need to design a scheme where the client gets a character About the Markus Kuhn wcwidth implementation: It's http://osdir.com/ml/internationalization.linux/2001-01/msg00191.html Mosh is an application that uses wcwidth heavily, and can spend Separately, Google shows me a discussion on GNU libc that its wcwidth I have not looked at it as closely, or in a while, but if I remember right ICU does not directly offer a wcwidth function, and in general it's a heavyweight featureful implementation not suited to be called for individual characters as often as we do. |
This doesn't appear to be a mac-specific issue: I have this problem in |
Hi, |
Yeap, I think for a heavy terminal user, powerline is a well known package. |
I have the same problem where emojis are not rendered when connecting with |
Have the same issue as @raine : mosh does not display unicode emoji symbols(🏠 in my case), but ssh does. I tried with iTerm2 and iTerm3 Beta on OS X. |
If you don't want to bring in the beast that is ICU, you can just ship the http://www.unicode.org/Public/UCD/latest/ucd/EastAsianWidth.txt It's pretty easy to parse this and transform it into whatever form you want. |
Is there any update with a solution for this? Specially for the chars mentioned here: #234 (comment) ? Thank you! |
I've just been down the rabbit hole of this problem. There are so many places that could take responsibility for it;
In summary it just seems like a subtle problem, that can't be easily fixed in one place. So for now I'm just going to remove any special characters from my setup. |
Edit: After writing all this, I've gone over the earlier comments again and they make more sense to me now. Please disregard if all of the following is already well understood and has no fix. I've been trying to troubleshoot this over the past few days and believe I've started to make some progress in narrowing this down as far as the 🤔 emoji/utf-8 display goes (it's UTF/unicode, but I'm testing with the thinking face emoji so I'll refer to it as that here). By the way, don't try to copy the emoji from here on GitHub because they turn it into an image - instead, head to emojipedia to copy & paste into your own terminals. I don't think there's significant relevance to what (modern) terminal program is being used (gnome-terminal, macOS Terminal, iTerm2, JuiceSSH, etc - they all default quite well these days). I also don't think tmux or even irssi has anything to do with it - but to be clear, I've been testing with only plain bash and fish; no tmux, no powerline - no other user-complications to the best of my knowledge. What's working in CentOS 7, Ubuntu Server 14.04.5 LTS, Ubuntu Server 18.04 LTS, Fedora 28:
What not working in CentOS 7.5.1804 (including one non-test install; mosh 1.3.0), Ubuntu Server 14.04.5 LTS (mosh 1.3.2):
The two cases where emojis through a mosh connection does work:
Suggestion for all in this thread:
What I haven't tried:
Please let me know if this gets us any closer to where the problem might be. Edit 20180711: As per some of the other closed issues above, I only have glibc 2.17 on the server. I'm now considering a migration away from CentOS 7.5.1804 to sort this. Edit 20180808: I've just completed a migration from CentOS 7.5 (glibc 2.17) to Debian 9.5 Stable (glibc 2.24) and am satisfied with the results. Also expecting to have something like glibc 2.27 with Debian 10 next year. Those who need or wish to remain with CentOS, hopefully version 8 isn't too far away. |
I'm having a similar issue with zero width spaces (U+200B). Printing U+200B typically causes some kind of display corruption. Likely cause seems to be that my client and server disagrees about the width of this particular character (locale en_US.UTF-8): Server is Debian stable (stretch) and mosh 1.2.6, client is macOS 10.13.6 and mosh 1.3.2. |
Same issue with an OMZ theme that displays a 'gear' character if you have background processes in your shell. Works fine with a raw ssh session but not with mosh. |
@tombh regaring your "rabbit hole", I think you may be pleased to find my article, "Offering a solution for Terminal Wide Character issues" https://jeffquast.com/post/terminal_wcwidth_solution/ I have authored a demonstration CLI utility that is able to automatically detect the version of Unicode supported by the Terminal emulator, https://github.com/jquast/ucs-detect/ and a new release of python wcwidth library https://github.com/jquast/wcwidth that supports all versions of unicode by selection using the exported environment variable. |
Apologies in advance for bumping this thread, have been affected by this issue and finally was able to identify mosh as the culprit. I have annoying text alignment issues similar to @mikaabra in #361 (in a TUI email client, nonetheless!). Am not a Unicode expert by any means, so cannot begin to fathom the complexity of a fix of the root cause, but curious if there's been any other workarounds? I'm using on mosh 1.3.2 from Homebrew on OS X 11.4 "Big Sur", connecting to a Ubuntu 18.04 "Bionic Beaver" box using mosh 1.3.2 and libc6 2.27-3ubuntu1.4. Having trouble displaying the ⚾ character. I wonder if upgrading to 20.04 "Focal Fossa" would help? That seems to be using glibc 2.31-0ubuntu9.2. Although it seems like the simple passing of time hasn't done much to fix this issue :/ |
The bug also seems to exist with the "symbols for legacy computing" |
This comment was marked as duplicate.
This comment was marked as duplicate.
well i took ten hours try to find what's wrong. Then I find that the following code does not work right in msys2.
Result in msys2
Result in debian or WSL
but there is no libc in msys2.
I suggest that using another way to split string into cells would be bring a high compatibility. |
here is the pr: #1271 |
Ok so it isn't just me. I am noticing this as well and just spent forever trying to figure out what it was in the chain. I have artifacts all over the place when using mosh+tmux+iamb. I guess I can just write this off as a mosh issue? Edit: Forgot to add that I'm seeing this same behavior in blackbox (terminal) on my Fedora desktop using Gnome. ssh on its own renders just fine but when mosh connects then I get artifacts all over the place where there are emoji (tend to see this when moving around in iamb, gomuks, weechat) |
Same issue on gentoo, using UTF8 while using ohmyzsh agnoster theme with root, the thunder symbol is somehow not visible. ⚡ mosh-1.4.0 |
I use a custom irssi theme that contains the UTF-8 "Fleur de Lys" symbol (U+269C - ⚜). While this character is displayed just fine when I use ssh, it just disappears in mosh. Also, there are display errors in irssi: random characters just disappear or get swapped by other characters. This only occurs when I use my custom theme so there might be a connection.
The text was updated successfully, but these errors were encountered: