unicode \u23CE ⏎ character preceding every prompt in fish in WSL #1402

Closed
opened this Issue Jan 14, 2018 · 27 comments

Projects
3 participants

Versions

ConEmu build: 180111 x64
OS version: Windows 10 Professional x64 build 16299.192, Chinese (Simplified)
Used shell version: same as Windows

Problem description

Basically same as fish-shell/fish-shell#789, just in WSL instead of Cygwin; another related issue is cmderdev/cmder#1069. Behavior confirmed using a latest (180111) portable build fresh from the download page (a new profile is used, of course).

More precisely, here's the output starting a new task bash ~ -c fish (fish stylized with oh-my-fish so output may not match that of a clean install):

Welcome to fish, the friendly interactive shell
Type help for instructions on how to use fish
⏎
~ 


Starting bash.exe in native console (i.e. cmd.exe) will not print that annoying ⏎.

Steps to reproduce

1. Create console bash.exe
2. fish
3. Viola! an unexpected ⏎.

Actual results

Welcome to fish, the friendly interactive shell
Type help for instructions on how to use fish
⏎


Expected results

Welcome to fish, the friendly interactive shell
Type help for instructions on how to use fish


Owner

TJYSunset commented Jan 14, 2018

 Funny, the arrow still persists even after I used wslbridge.
Owner

Maximus5 commented Jan 14, 2018

 What ConEmu version do you use exactly? One on the screenshot, another in the issue title.
Owner

Maximus5 commented Jan 14, 2018

 Moreover, the question is how do you run new console?

TJYSunset commented Jan 14, 2018

Well, when I wrote the issue title, I was using 171109 - the latest version bundled with Cmder binary package, as seen hours ago (correct me if I'm wrong) - but on the second screenshot that was 180111.

If the inconsistent versions are causing difficulty troubleshooting, I'll reproduce the problem using a 180111 portable build, with a new profile. Fish, Cygwin and wslbridge (built from source) installed hours ago. I don't really remember when I set up my WSL, but the Ubuntu version is 16.04.

set "PATH=%ConEmuBaseDirShort%\wsl;%PATH%" & %ConEmuBaseDirShort%\conemu-cyg-64.exe --wsl -cur_console:pm:/mnt

Owner

Maximus5 commented Mar 14, 2018

 Fresh cmder? It's a third-party bundle of various software. I asked about ConEmu version.

TJYSunset commented Mar 14, 2018

 And by that I means I got everything old cleared (some setting went seriously wrong). It is ConEmu 180309 that I'm using.

 Test result is here. Reinstalled Ubuntu from Microsoft Store. Directly run wsl.exe Welcome to fish, the friendly interactive shell ⏎ todd@TODD-LAPTOP /m/c/U/TJYSunset>  Old self-buit wslbridge $uname -a CYGWIN_NT-10.0 TODD-LAPTOP 2.10.0(0.325/5/3) 2018-02-02 15:16 x86_64 Cygwin$ fish Welcome to fish, the friendly interactive shell ⏎ todd@TODD-LAPTOP /m/d/O/l/C/h/T/w/out>  The wslbridge came with ConEmu $./wslbridge 2 [main] wslbridge (1516) path\to\cmder\vendor\conemu-maximus5\conemu\wsl\wslbridge.exe: *** fatal error - cygheap base mismatch detected - 0x1802FD410/0x1802FF408. This problem is probably due to using incompatible versions of the cygwin DLL. Search for cygwin1.dll using the Windows Start->Find/Search facility and delete all but the most recent version. The most recent version *should* reside in x:\cygwin\bin, where 'x' is the drive on which you have installed the cygwin distribution. Rebooting is also suggested if you are unable to find another cygwin DLL. 2 [main] wslbridge (1516) path\to\cmder\vendor\conemu-maximus5\conemu\wsl\wslbridge.exe: *** fatal error - cygheap base mismatch detected - 0x1802FD410/0x1802FF408. This problem is probably due to using incompatible versions of the cygwin DLL. Search for cygwin1.dll using the Windows Start->Find/Search facility and delete all but the most recent version. The most recent version *should* reside in x:\cygwin\bin, where 'x' is the drive on which you have installed the cygwin distribution. Rebooting is also suggested if you are unable to find another cygwin DLL.  lsb_release -a output remains unchanged. Owner Maximus5 commented Mar 14, 2018  Download build 180309 Unpack it to clean folder Run ConEmu64.exe -basic -run {bash} from this folder. Neither of your tests in prev comment are correct. wsl.exe does not start connector+wslbridge I don't know how do you run "old-built wslbridge", without connector? And of course ./wslbridge is not a correct way to start it. TJYSunset commented Mar 14, 2018 Sorry for the inconvenience; I followed the exact steps given above and here is (hopefully) the right result. C:\Users\TJYSunset\Temp\ConEmuPack.180309>conemu64 -basic -run {bash}  todd@TODD-LAPTOP:/mnt/c/Users/TJYSunset$ fish
[some random custom welcome message]
⏎
⋊> /m/c/U/TJYSunset                                                     04:06:48


Meanwhile, SSHing from KiTTY inside ConEmu:

Using username "todd".
[some random custom welcome message]
⋊> ~                                                                                                     04:11:53


This is... unexpected. Maybe some more detailed version info could help?

Windows

PS> [System.Environment]::OSVersion.Version

Major  Minor  Build  Revision
-----  -----  -----  --------
10     0      17115  0


For testing purpose (and various fix for annoying wsl problems in stable release) I just grabbed the latest Windows 10 Insider from Microsoft, prior to making this post.

ConEmu

As an unnecessary confirmation, the window title reads, "Settings [reg] ConEmu 180309 [64] {Preview}".

Ubuntu

From Microsoft Store. Changelog says, "20170830.1 build of Ubuntu 16.04 LTS". Installed hours ago (and before that, I was using the old lxrun.exe distributed Ubuntu, exact version unknown).

Fish

⋊> ~ fish --version
fish, version 2.7.1


Miscellaneous settings

• Windows locale is zh_CN; The problem exists regardless of Ubuntu's locale.
• Fallback system encoding (or "ANSI") is GB2312. Problem still occurs if chcp utf-8 is added into ConEmu Startup->Environment.

That is some weird issue to track. I would love to share some diagnostic data like a memory dump if helpful.

Owner

Maximus5 commented Mar 15, 2018

 KiTTY is ChildGui, it renders output internally The issue is really weird because it is not reproduced in my environment at all. Perhaps it depens on some fish or ubuntu settings, I don't know. LogFiles are required. In the folder C:\Users\TJYSunset\Temp\ConEmuPack.180309\ run ConEmu64.exe -loadcfgfile .\ConEmu.xml  Change the contents of the {Bash::bash} task to set "PATH=%ConEmuBaseDirShort%\wsl;%PATH%" & %ConEmuBaseDirShort%\conemu-cyg-64.exe --log C:/Users/TJYSunset/Temp/ConEmuPack.180309/ --wsl -cur_console:pm:/mnt  Save settings, exit ConEmu Now in the folder C:\Users\TJYSunset\Temp\ConEmuPack.180309\ run ConEmu64.exe -log -loadcfgfile .\ConEmu.xml  If the ⏎ sign appears (as expected) than close ConEmu, zip all created *.log files (in your ConEmuPack.180309 folder) and post here link to your archive.
Owner

Maximus5 commented Mar 15, 2018

 Also fish installed via apt-get shows fish, version 2.2.0  Did you install fish manually?

TJYSunset commented Mar 15, 2018

 No, it's from the official PPA. The one from central apt repository is outdated and lacks some features like string which other modules I use rely on. (I definitely have no idea why.) logs.zip
Owner

Maximus5 commented Mar 19, 2018

 Please update to 180318 and execute from ConEmu64.exe -basic -run cmd  the following ConEmuC -CheckUnicode  I'm interested in screenshots from your default codepage and chcp 65001.

TJYSunset commented Mar 19, 2018

 Code page 936 Code page 65001 Also there's code page 54936. I don't think it's the default code page of any zh-Hans Windows distribution now, but just in case. Also I have changed the console font to an official variant of Iosevka before this. Hope this won't affect the results.
Owner

Maximus5 commented Mar 19, 2018

 Hm, results are expected. Waiting for log-files #1402 (comment)

TJYSunset commented Mar 20, 2018

 (Well the results looks identical to me...)

Owner

Maximus5 commented Mar 31, 2018

 I need screenshot with the problem and visible StatusBar with TermInfo column.

Maximus5 added a commit that referenced this issue Apr 1, 2018

 gh-1402: Add debug assertions for troubleshooting. 
 b6f2af9 

TJYSunset commented Apr 3, 2018

 ...turns out the ⏎ disappeared all the sudden, after I reinstalled WSL because I somehow broke it. What the...? Well that's kind of good news...?

Owner

Maximus5 commented Apr 3, 2018

 Don't know what happened on your side. Sadly. It may be or may be not ConEmu problem, and without reproduction we can't be sure. However nobody else joined the thread.
Owner

Maximus5 commented Apr 10, 2018

 That is definitely Microsoft conhost bug. The problem is that on CJK Windows versions conhost still behaves differently than on European versions. Especially in that case, when fish outputs the ⏎ (U+23CE) conhost advances the cursor by two cells on CJK Windows 10 (I'm trying Chinese build 10.0.16299.15), but on European version the cursor is advanced by only one cell as expected! Conhost fills two cells with single-width glyph! Yeah, it sets the COMMON_LVB_LEADING_BYTE and COMMON_LVB_TRAILING_BYTE flags in these two cells, but terminal app (fish) expects that it output only one character and things are broken. From my tests, a lot of glyphs are treated improperly as double-cell characters on CJK windows (at least all Russian and Greek letters). /cc @miniksa

Owner

Maximus5 commented Apr 10, 2018

 Actually, that happens even without ConEmu. PS. Going to create a workaround.

Maximus5 added a commit that referenced this issue Apr 10, 2018

 gh-1402: Workaround for \u23CE ⏎ character preceding every prompt in … 
…fish on CJK versions of Win10.
 9bce8f1 

Open

Leoyzen commented May 25, 2018

 We should also consider the situation that ssh from windows to a remote server which used fish rather than windows local
Owner