New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unicode \u23CE ⏎ character preceding every prompt in fish in WSL #1402

Closed
TJYSunset opened this Issue Jan 14, 2018 · 27 comments

Comments

Projects
3 participants
@TJYSunset

TJYSunset commented Jan 14, 2018

Versions

ConEmu build: 180111 x64
OS version: Windows 10 Professional x64 build 16299.192, Chinese (Simplified)
Used shell version: same as Windows

Problem description

Basically same as fish-shell/fish-shell#789, just in WSL instead of Cygwin; another related issue is cmderdev/cmder#1069. Behavior confirmed using a latest (180111) portable build fresh from the download page (a new profile is used, of course).

More precisely, here's the output starting a new task bash ~ -c fish (fish stylized with oh-my-fish so output may not match that of a clean install):

Welcome to fish, the friendly interactive shell
Type help for instructions on how to use fish
⏎
 ~ 

Starting bash.exe in native console (i.e. cmd.exe) will not print that annoying .

Steps to reproduce

  1. Create console bash.exe
  2. fish
  3. Viola! an unexpected .

Actual results

Welcome to fish, the friendly interactive shell
Type help for instructions on how to use fish
⏎

Expected results

Welcome to fish, the friendly interactive shell
Type help for instructions on how to use fish

Additional files

screenshot
For I just have deleted the downloaded portable version the screenshot was made in Cmder.

@Maximus5

This comment has been minimized.

@Maximus5 Maximus5 added the other-wsl label Jan 14, 2018

@TJYSunset

This comment has been minimized.

TJYSunset commented Jan 14, 2018

Funny, the arrow still persists even after I used wslbridge.

screenshot

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Jan 14, 2018

What ConEmu version do you use exactly? One on the screenshot, another in the issue title.

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Jan 14, 2018

Moreover, the question is how do you run new console?

@TJYSunset

This comment has been minimized.

TJYSunset commented Jan 14, 2018

Well, when I wrote the issue title, I was using 171109 - the latest version bundled with Cmder binary package, as seen hours ago (correct me if I'm wrong) - but on the second screenshot that was 180111.

If the inconsistent versions are causing difficulty troubleshooting, I'll reproduce the problem using a 180111 portable build, with a new profile. Fish, Cygwin and wslbridge (built from source) installed hours ago. I don't really remember when I set up my WSL, but the Ubuntu version is 16.04.

Running default {Bash:bash} task

set "PATH=%ConEmuBaseDirShort%\wsl;%PATH%" & %ConEmuBaseDirShort%\conemu-cyg-64.exe --wsl -cur_console:pm:/mnt

todd@TODD-LAPTOP:~$ fish
Welcome to fish, the friendly interactive shell
Type help for instructions on how to use fish
⏎
 ~ 

Running default {Bash::CygWin bash} task

set CHERE_INVOKING=1 & set "PATH=path\to\Cygwin\bin;%PATH%" & %ConEmuBaseDirShort%\conemu-cyg-64.exe -new_console:p path\to\Cygwin\bin\bash.exe --login -i -new_console:C:"path\to\Cygwin\Cygwin.ico"

TJYSunset@TODD-LAPTOP /path/to/wslbridge/out
$ ./wslbridge
todd@TODD-LAPTOP:/path/to/wslbridge/out$ fish
Welcome to fish, the friendly interactive shell
Type help for instructions on how to use fish
⏎
 /p/t/w/out   master  
@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Jan 14, 2018

Seems like all correct, except I'm using wslbridge bundled with ConEmu.
And I don't observe ⏎ character before fish prompt (clean profile, fish just installed).

@Maximus5 Maximus5 added this to To Do in ANSI via automation Jan 15, 2018

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Mar 14, 2018

  • What are repro steps?
  • Are you using ConEmu build 180309?
  • Version of Linux: lsb_release -a?
@TJYSunset

This comment has been minimized.

TJYSunset commented Mar 14, 2018

  • Happens I got a fresh install of Cmder yesterday, and I'm still getting the dumb ⏎ on either running that old self-built wslbridge (the one came with ConEmu refuses to run, see output below) or simply running wsl.exe. For now I'm accessing wsl from KiTTY via SSH, which also provides 256-color support. As for the question, I'm getting convinced that this is a wsl-side problem - going to backup and reinstall that unholy thing, and then post the test result here. May take a while before that, I'm heading to bed right now.
$ ./wslbridge
      2 [main] wslbridge (7092) path\to\cmder\vendor\conemu-maximus5\conemu\wsl\wslbridge.exe: *** f
      2 [main] wslbridge (7092) path\to\cmder\vendor\conemu-maximus5\conemu\wsl\wslbridge.exe: *** fatal error - cygheap base mismatch detected - 0x1802FD410/0x1802FF408.
This problem is probably due to using incompatible versions of the cygwin DLL.
Search for cygwin1.dll using the Windows Start->Find/Search facility
and delete all but the most recent version.  The most recent version *should*
reside in x:\cygwin\bin, where 'x' is the drive on which you have
installed the cygwin distribution.  Rebooting is also suggested if you
are unable to find another cygwin DLL.
atal error - cygheap base mismatch detected - 0x1802FD410/0x1802FF408.
This problem is probably due to using incompatible versions of the cygwin DLL.
Search for cygwin1.dll using the Windows Start->Find/Search facility
and delete all but the most recent version.  The most recent version *should*
reside in x:\cygwin\bin, where 'x' is the drive on which you have
installed the cygwin distribution.  Rebooting is also suggested if you
are unable to find another cygwin DLL.

CygWin still gave me this after an upgrade. Only one cygwin1.dll exists in cygwin\bin.

  • Yes.
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.4 LTS
Release:        16.04
Codename:       xenial
@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Mar 14, 2018

Fresh cmder? It's a third-party bundle of various software.
I asked about ConEmu version.

@TJYSunset

This comment has been minimized.

TJYSunset commented Mar 14, 2018

And by that I means I got everything old cleared (some setting went seriously wrong).
It is ConEmu 180309 that I'm using.

@TJYSunset

This comment has been minimized.

TJYSunset commented Mar 14, 2018

Test result is here. Reinstalled Ubuntu from Microsoft Store.
Directly run wsl.exe

Welcome to fish, the friendly interactive shell
⏎
todd@TODD-LAPTOP /m/c/U/TJYSunset>

Old self-buit wslbridge

$ uname -a
CYGWIN_NT-10.0 TODD-LAPTOP 2.10.0(0.325/5/3) 2018-02-02 15:16 x86_64 Cygwin
$ fish
Welcome to fish, the friendly interactive shell
⏎
todd@TODD-LAPTOP /m/d/O/l/C/h/T/w/out>

The wslbridge came with ConEmu

$ ./wslbridge
      2 [main] wslbridge (1516) path\to\cmder\vendor\conemu-maximus5\conemu\wsl\wslbridge.exe: *** fatal error - cygheap base mismatch detected - 0x1802FD410/0x1802FF408.
This problem is probably due to using incompatible versions of the cygwin DLL.
Search for cygwin1.dll using the Windows Start->Find/Search facility
and delete all but the most recent version.  The most recent version *should*
reside in x:\cygwin\bin, where 'x' is the drive on which you have
installed the cygwin distribution.  Rebooting is also suggested if you
are unable to find another cygwin DLL.
      2 [main] wslbridge (1516) path\to\cmder\vendor\conemu-maximus5\conemu\wsl\wslbridge.exe: *** fatal error - cygheap base mismatch detected - 0x1802FD410/0x1802FF408.
This problem is probably due to using incompatible versions of the cygwin DLL.
Search for cygwin1.dll using the Windows Start->Find/Search facility
and delete all but the most recent version.  The most recent version *should*
reside in x:\cygwin\bin, where 'x' is the drive on which you have
installed the cygwin distribution.  Rebooting is also suggested if you
are unable to find another cygwin DLL.

lsb_release -a output remains unchanged.

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Mar 14, 2018

  1. Download build 180309
  2. Unpack it to clean folder
  3. Run ConEmu64.exe -basic -run {bash} from this folder.

Neither of your tests in prev comment are correct.

  • wsl.exe does not start connector+wslbridge
  • I don't know how do you run "old-built wslbridge", without connector?
  • And of course ./wslbridge is not a correct way to start it.
@TJYSunset

This comment has been minimized.

TJYSunset commented Mar 14, 2018

Sorry for the inconvenience; I followed the exact steps given above and here is (hopefully) the right result.

C:\Users\TJYSunset\Temp\ConEmuPack.180309>conemu64 -basic -run {bash}
todd@TODD-LAPTOP:/mnt/c/Users/TJYSunset$ fish
[some random custom welcome message]
⏎
⋊> /m/c/U/TJYSunset                                                     04:06:48

Meanwhile, SSHing from KiTTY inside ConEmu:

Using username "todd".
[some random custom welcome message]
⋊> ~                                                                                                     04:11:53

This is... unexpected. Maybe some more detailed version info could help?

Windows

PS> [System.Environment]::OSVersion.Version

Major  Minor  Build  Revision
-----  -----  -----  --------
10     0      17115  0

For testing purpose (and various fix for annoying wsl problems in stable release) I just grabbed the latest Windows 10 Insider from Microsoft, prior to making this post.

ConEmu

As an unnecessary confirmation, the window title reads, "Settings [reg] ConEmu 180309 [64] {Preview}".

Ubuntu

From Microsoft Store. Changelog says, "20170830.1 build of Ubuntu 16.04 LTS". Installed hours ago (and before that, I was using the old lxrun.exe distributed Ubuntu, exact version unknown).

Fish

⋊> ~ fish --version
fish, version 2.7.1

Miscellaneous settings

  • Windows locale is zh_CN; The problem exists regardless of Ubuntu's locale.
  • Fallback system encoding (or "ANSI") is GB2312. Problem still occurs if chcp utf-8 is added into ConEmu Startup->Environment.

That is some weird issue to track. I would love to share some diagnostic data like a memory dump if helpful.

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Mar 15, 2018

KiTTY is ChildGui, it renders output internally

The issue is really weird because it is not reproduced in my environment at all. Perhaps it depens on some fish or ubuntu settings, I don't know.

LogFiles are required.

  • In the folder C:\Users\TJYSunset\Temp\ConEmuPack.180309\ run

    ConEmu64.exe -loadcfgfile .\ConEmu.xml
    
  • Change the contents of the {Bash::bash} task to

    set "PATH=%ConEmuBaseDirShort%\wsl;%PATH%" & %ConEmuBaseDirShort%\conemu-cyg-64.exe --log C:/Users/TJYSunset/Temp/ConEmuPack.180309/ --wsl -cur_console:pm:/mnt
    
  • Save settings, exit ConEmu

  • Now in the folder C:\Users\TJYSunset\Temp\ConEmuPack.180309\ run

    ConEmu64.exe -log -loadcfgfile .\ConEmu.xml
    
  • If the sign appears (as expected) than close ConEmu, zip all created *.log files (in your ConEmuPack.180309 folder) and post here link to your archive.

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Mar 15, 2018

Also fish installed via apt-get shows

fish, version 2.2.0

Did you install fish manually?

@TJYSunset

This comment has been minimized.

TJYSunset commented Mar 15, 2018

No, it's from the official PPA. The one from central apt repository is outdated and lacks some features like string which other modules I use rely on. (I definitely have no idea why.)
logs.zip

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Mar 19, 2018

Please update to 180318 and execute from

ConEmu64.exe -basic -run cmd

the following

ConEmuC -CheckUnicode

I'm interested in screenshots from your default codepage and chcp 65001.

@TJYSunset

This comment has been minimized.

TJYSunset commented Mar 19, 2018

  • Code page 936
    936
  • Code page 65001
    65001
  • Also there's code page 54936. I don't think it's the default code page of any zh-Hans Windows distribution now, but just in case.
    54936

Also I have changed the console font to an official variant of Iosevka before this. Hope this won't affect the results.

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Mar 19, 2018

Hm, results are expected.

Waiting for log-files
#1402 (comment)

@TJYSunset

This comment has been minimized.

TJYSunset commented Mar 20, 2018

936.zip
65001.zip
54936.zip

(Well the results looks identical to me...)

@Maximus5 Maximus5 moved this from To Do to In progress in ANSI Mar 24, 2018

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Mar 31, 2018

I need screenshot with the problem and visible StatusBar with TermInfo column.

@TJYSunset

This comment has been minimized.

TJYSunset commented Apr 3, 2018

...turns out the disappeared all the sudden, after I reinstalled WSL because I somehow broke it. What the...?
Well that's kind of good news...?

@TJYSunset TJYSunset closed this Apr 3, 2018

ANSI automation moved this from In progress to Done Apr 3, 2018

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Apr 3, 2018

Don't know what happened on your side. Sadly. It may be or may be not ConEmu problem, and without reproduction we can't be sure.
However nobody else joined the thread.

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Apr 10, 2018

That is definitely Microsoft conhost bug.

The problem is that on CJK Windows versions conhost still behaves differently than on European versions.
Especially in that case, when fish outputs the (U+23CE) conhost advances the cursor by two cells on CJK Windows 10 (I'm trying Chinese build 10.0.16299.15), but on European version the cursor is advanced by only one cell as expected!

Conhost fills two cells with single-width glyph! Yeah, it sets the COMMON_LVB_LEADING_BYTE and COMMON_LVB_TRAILING_BYTE flags in these two cells, but terminal app (fish) expects that it output only one character and things are broken.

From my tests, a lot of glyphs are treated improperly as double-cell characters on CJK windows (at least all Russian and Greek letters).

/cc @miniksa

@Maximus5 Maximus5 reopened this Apr 10, 2018

ANSI automation moved this from Done to In progress Apr 10, 2018

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented Apr 10, 2018

Actually, that happens even without ConEmu.

2018-04-10_13-05-55

PS. Going to create a workaround.

Maximus5 added a commit that referenced this issue Apr 10, 2018

@Maximus5 Maximus5 moved this from In progress to Ready for Testing in ANSI Apr 10, 2018

@Maximus5 Maximus5 closed this Apr 12, 2018

ANSI automation moved this from Ready for Testing to Done Apr 12, 2018

@Leoyzen

This comment has been minimized.

Leoyzen commented May 25, 2018

We should also consider the situation that ssh from windows to a remote server which used fish rather than windows local

@Maximus5

This comment has been minimized.

Owner

Maximus5 commented May 29, 2018

@Leoyzen What is your comment about?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment