unicode \u23CE ⏎ character preceding every prompt in fish in WSL

[TJYSunset]

Versions

ConEmu build: 180111 x64
OS version: Windows 10 Professional x64 build 16299.192, Chinese (Simplified)
Used shell version: same as Windows

Problem description

Basically same as fish-shell/fish-shell#789, just in WSL instead of Cygwin; another related issue is cmderdev/cmder#1069. Behavior confirmed using a latest (180111) portable build fresh from the download page (a new profile is used, of course).

More precisely, here's the output starting a new task bash ~ -c fish (fish stylized with oh-my-fish so output may not match that of a clean install):

Welcome to fish, the friendly interactive shell
Type help for instructions on how to use fish
⏎
 ~ 

Starting bash.exe in native console (i.e. cmd.exe) will not print that annoying ⏎.

Steps to reproduce

1. Create console bash.exe
2. fish
3. Viola! an unexpected ⏎.

Actual results

Welcome to fish, the friendly interactive shell
Type help for instructions on how to use fish
⏎

Expected results

Welcome to fish, the friendly interactive shell
Type help for instructions on how to use fish

Additional files

For I just have deleted the downloaded portable version the screenshot was made in Cmder.

[Maximus5]

https://conemu.github.io/en/BashOnWindows.html#connector

[TJYSunset]

Funny, the arrow still persists even after I used wslbridge.

[Maximus5]

What ConEmu version do you use exactly? One on the screenshot, another in the issue title.

[Maximus5]

Moreover, the question is how do you run new console?

[TJYSunset]

Well, when I wrote the issue title, I was using 171109 - the latest version bundled with Cmder binary package, as seen hours ago (correct me if I'm wrong) - but on the second screenshot that was 180111.

If the inconsistent versions are causing difficulty troubleshooting, I'll reproduce the problem using a 180111 portable build, with a new profile. Fish, Cygwin and wslbridge (built from source) installed hours ago. I don't really remember when I set up my WSL, but the Ubuntu version is 16.04.

Running default {Bash:bash} task

set "PATH=%ConEmuBaseDirShort%\wsl;%PATH%" & %ConEmuBaseDirShort%\conemu-cyg-64.exe --wsl -cur_console:pm:/mnt

todd@TODD-LAPTOP:~$ fish
Welcome to fish, the friendly interactive shell
Type help for instructions on how to use fish
⏎
 ~ 

Running default {Bash::CygWin bash} task

set CHERE_INVOKING=1 & set "PATH=path\to\Cygwin\bin;%PATH%" & %ConEmuBaseDirShort%\conemu-cyg-64.exe -new_console:p path\to\Cygwin\bin\bash.exe --login -i -new_console:C:"path\to\Cygwin\Cygwin.ico"

TJYSunset@TODD-LAPTOP /path/to/wslbridge/out
$ ./wslbridge
todd@TODD-LAPTOP:/path/to/wslbridge/out$ fish
Welcome to fish, the friendly interactive shell
Type help for instructions on how to use fish
⏎
 /p/t/w/out   master  

[Maximus5]

Seems like all correct, except I'm using wslbridge bundled with ConEmu.
And I don't observe ⏎ character before fish prompt (clean profile, fish just installed).

[Maximus5]

• What are repro steps?
• Are you using ConEmu build 180309?
• Version of Linux: lsb_release -a?

[TJYSunset]

• Happens I got a fresh install of Cmder yesterday, and I'm still getting the dumb ⏎ on either running that old self-built wslbridge (the one came with ConEmu refuses to run, see output below) or simply running wsl.exe. For now I'm accessing wsl from KiTTY via SSH, which also provides 256-color support. As for the question, I'm getting convinced that this is a wsl-side problem - going to backup and reinstall that unholy thing, and then post the test result here. May take a while before that, I'm heading to bed right now.

$ ./wslbridge
      2 [main] wslbridge (7092) path\to\cmder\vendor\conemu-maximus5\conemu\wsl\wslbridge.exe: *** f
      2 [main] wslbridge (7092) path\to\cmder\vendor\conemu-maximus5\conemu\wsl\wslbridge.exe: *** fatal error - cygheap base mismatch detected - 0x1802FD410/0x1802FF408.
This problem is probably due to using incompatible versions of the cygwin DLL.
Search for cygwin1.dll using the Windows Start->Find/Search facility
and delete all but the most recent version.  The most recent version *should*
reside in x:\cygwin\bin, where 'x' is the drive on which you have
installed the cygwin distribution.  Rebooting is also suggested if you
are unable to find another cygwin DLL.
atal error - cygheap base mismatch detected - 0x1802FD410/0x1802FF408.
This problem is probably due to using incompatible versions of the cygwin DLL.
Search for cygwin1.dll using the Windows Start->Find/Search facility
and delete all but the most recent version.  The most recent version *should*
reside in x:\cygwin\bin, where 'x' is the drive on which you have
installed the cygwin distribution.  Rebooting is also suggested if you
are unable to find another cygwin DLL.

CygWin still gave me this after an upgrade. Only one cygwin1.dll exists in cygwin\bin.

• Yes.

No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.4 LTS
Release:        16.04
Codename:       xenial

[Maximus5]

Fresh cmder? It's a third-party bundle of various software.
I asked about ConEmu version.

[TJYSunset]

And by that I means I got everything old cleared (some setting went seriously wrong).
It is ConEmu 180309 that I'm using.

[TJYSunset]

Test result is here. Reinstalled Ubuntu from Microsoft Store.
Directly run wsl.exe

Welcome to fish, the friendly interactive shell
⏎
todd@TODD-LAPTOP /m/c/U/TJYSunset>

Old self-buit wslbridge

$ uname -a
CYGWIN_NT-10.0 TODD-LAPTOP 2.10.0(0.325/5/3) 2018-02-02 15:16 x86_64 Cygwin
$ fish
Welcome to fish, the friendly interactive shell
⏎
todd@TODD-LAPTOP /m/d/O/l/C/h/T/w/out>

The wslbridge came with ConEmu

$ ./wslbridge
      2 [main] wslbridge (1516) path\to\cmder\vendor\conemu-maximus5\conemu\wsl\wslbridge.exe: *** fatal error - cygheap base mismatch detected - 0x1802FD410/0x1802FF408.
This problem is probably due to using incompatible versions of the cygwin DLL.
Search for cygwin1.dll using the Windows Start->Find/Search facility
and delete all but the most recent version.  The most recent version *should*
reside in x:\cygwin\bin, where 'x' is the drive on which you have
installed the cygwin distribution.  Rebooting is also suggested if you
are unable to find another cygwin DLL.
      2 [main] wslbridge (1516) path\to\cmder\vendor\conemu-maximus5\conemu\wsl\wslbridge.exe: *** fatal error - cygheap base mismatch detected - 0x1802FD410/0x1802FF408.
This problem is probably due to using incompatible versions of the cygwin DLL.
Search for cygwin1.dll using the Windows Start->Find/Search facility
and delete all but the most recent version.  The most recent version *should*
reside in x:\cygwin\bin, where 'x' is the drive on which you have
installed the cygwin distribution.  Rebooting is also suggested if you
are unable to find another cygwin DLL.

lsb_release -a output remains unchanged.

[Maximus5]

1. Download build 180309
2. Unpack it to clean folder
3. Run ConEmu64.exe -basic -run {bash} from this folder.

Neither of your tests in prev comment are correct.

• wsl.exe does not start connector+wslbridge
• I don't know how do you run "old-built wslbridge", without connector?
• And of course ./wslbridge is not a correct way to start it.

[TJYSunset]

Sorry for the inconvenience; I followed the exact steps given above and here is (hopefully) the right result.

C:\Users\TJYSunset\Temp\ConEmuPack.180309>conemu64 -basic -run {bash}

todd@TODD-LAPTOP:/mnt/c/Users/TJYSunset$ fish
[some random custom welcome message]
⏎
⋊> /m/c/U/TJYSunset                                                     04:06:48

Meanwhile, SSHing from KiTTY inside ConEmu:

Using username "todd".
[some random custom welcome message]
⋊> ~                                                                                                     04:11:53

This is... unexpected. Maybe some more detailed version info could help?

Windows

PS> [System.Environment]::OSVersion.Version

Major  Minor  Build  Revision
-----  -----  -----  --------
10     0      17115  0

For testing purpose (and various fix for annoying wsl problems in stable release) I just grabbed the latest Windows 10 Insider from Microsoft, prior to making this post.

ConEmu

As an unnecessary confirmation, the window title reads, "Settings [reg] ConEmu 180309 [64] {Preview}".

Ubuntu

From Microsoft Store. Changelog says, "20170830.1 build of Ubuntu 16.04 LTS". Installed hours ago (and before that, I was using the old lxrun.exe distributed Ubuntu, exact version unknown).

Fish

⋊> ~ fish --version
fish, version 2.7.1

Miscellaneous settings

• Windows locale is zh_CN; The problem exists regardless of Ubuntu's locale.
• Fallback system encoding (or "ANSI") is GB2312. Problem still occurs if chcp utf-8 is added into ConEmu Startup->Environment.

That is some weird issue to track. I would love to share some diagnostic data like a memory dump if helpful.

[Maximus5]

KiTTY is ChildGui, it renders output internally

The issue is really weird because it is not reproduced in my environment at all. Perhaps it depens on some fish or ubuntu settings, I don't know.

LogFiles are required.

• In the folder C:\Users\TJYSunset\Temp\ConEmuPack.180309\ run

  ConEmu64.exe -loadcfgfile .\ConEmu.xml
  

• Change the contents of the {Bash::bash} task to

  set "PATH=%ConEmuBaseDirShort%\wsl;%PATH%" & %ConEmuBaseDirShort%\conemu-cyg-64.exe --log C:/Users/TJYSunset/Temp/ConEmuPack.180309/ --wsl -cur_console:pm:/mnt
  

• Save settings, exit ConEmu

• Now in the folder C:\Users\TJYSunset\Temp\ConEmuPack.180309\ run

  ConEmu64.exe -log -loadcfgfile .\ConEmu.xml
  

• If the ⏎ sign appears (as expected) than close ConEmu, zip all created *.log files (in your ConEmuPack.180309 folder) and post here link to your archive.

[Maximus5]

Also fish installed via apt-get shows

fish, version 2.2.0

Did you install fish manually?

[TJYSunset]

No, it's from the official PPA. The one from central apt repository is outdated and lacks some features like string which other modules I use rely on. (I definitely have no idea why.)
logs.zip

[Maximus5]

Please update to 180318 and execute from

ConEmu64.exe -basic -run cmd

the following

ConEmuC -CheckUnicode

I'm interested in screenshots from your default codepage and chcp 65001.

[TJYSunset]

• Code page 936
  
• Code page 65001
  
• Also there's code page 54936. I don't think it's the default code page of any zh-Hans Windows distribution now, but just in case.
  

Also I have changed the console font to an official variant of Iosevka before this. Hope this won't affect the results.

[Maximus5]

Hm, results are expected.

Waiting for log-files
#1402 (comment)

[TJYSunset]

936.zip
65001.zip
54936.zip

(Well the results looks identical to me...)

[Maximus5]

I need screenshot with the problem and visible StatusBar with TermInfo column.

[TJYSunset]

...turns out the ⏎ disappeared all the sudden, after I reinstalled WSL because I somehow broke it. What the...?
Well that's kind of good news...?

[Maximus5]

Don't know what happened on your side. Sadly. It may be or may be not ConEmu problem, and without reproduction we can't be sure.
However nobody else joined the thread.

[Maximus5]

That is definitely Microsoft conhost bug.

The problem is that on CJK Windows versions conhost still behaves differently than on European versions.
Especially in that case, when fish outputs the ⏎ (U+23CE) conhost advances the cursor by two cells on CJK Windows 10 (I'm trying Chinese build 10.0.16299.15), but on European version the cursor is advanced by only one cell as expected!

Conhost fills two cells with single-width glyph! Yeah, it sets the COMMON_LVB_LEADING_BYTE and COMMON_LVB_TRAILING_BYTE flags in these two cells, but terminal app (fish) expects that it output only one character and things are broken.

From my tests, a lot of glyphs are treated improperly as double-cell characters on CJK windows (at least all Russian and Greek letters).

/cc @miniksa

[Maximus5]

Actually, that happens even without ConEmu.

PS. Going to create a workaround.

[Leoyzen]

We should also consider the situation that ssh from windows to a remote server which used fish rather than windows local

[Maximus5]

@Leoyzen What is your comment about?

[copyliu]

it seems that workaround doesn't fully resolve the issue.

tap tab key, cursor position will go back in CJK windows