Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Terminal emulator gives blank screen when executing HTOP or SCREEN #7660

Closed
1 of 2 tasks
AriESQ opened this issue Nov 9, 2021 · 58 comments
Closed
1 of 2 tasks

Terminal emulator gives blank screen when executing HTOP or SCREEN #7660

AriESQ opened this issue Nov 9, 2021 · 58 comments

Comments

@AriESQ
Copy link

AriESQ commented Nov 9, 2021

Version

Microsoft 10 Enterprise LTSC 1809 [Version 10.0.17763.2268]

WSL Version

  • WSL 2
  • WSL 1

Kernel Version

Linux 4.4.0-17763-Microsoft x86_64

Distro Version

Ubuntu 18.04 LTS

Other Software

openssh.exe
cmd.exe
ssh
htop
screen
ncurses

Repro Steps

  1. Open an WSL instance, which drops you to a functioning command prompt. Execute HTOP, screen goes blank. Press Q to exit HTOP, and you will see a portion of HTOP at the top of your screen, confirming it was running.

  2. ssh into a remote linux instance from the WSL environment. Execute SCREEN or HTOP, you will receive a blank screen, but keyboard input is still active.

  3. open CMD, run openssh.exe to ssh into a linux machine. Execute Screen or HTOP, you will receive a blank screen, but keyboard input is still active.

  4. While attempting to attach strace logs, I ran man strace and found that even man is broken, giving a blank screen. type man strace and you will have a blank screen, exit with q and you will have some scrollbck of the man page.

Because this behavior is shared between WSL and CMD.exe in terminal mode, I believe that the problem may lie in the underlying windows terminal emulation.

KB5006744 appears to be the breaking update. If I uninstall this update the problem goes away, if I re-install it the problem comes back.

I suspect the problem is that when the remote system or PTY grabs control of the terminal, such as by issuing a window resize, ASCII control code, or the like the window terminal emulator is misinterpreting it and painting a black screen.

I have troubleshooted by removing .bashrc and .profile custom PS1 prompt configurations that might be transmitting ANSI color codes. I have switched users to a bare root /bin/sh shell. I have tested on different local and remote linux installations.

The terminal in vs-code is not affected by this issue, and third-party terminal emulator ConEmu is also not affected by this flaw.

I may have traced things to default pager less. Or, less has the same problems experienced by htop and screen. Overriding the default pager via man -P more allows man to execute. So less is affected but more is not. top is unaffected but htop is.

I have attached strace but nothing jumps out at me as the culprit.

wsl.exe --status produces no output, however the binary has a modified date of 2020-03-10. I believe I have an older version of WSL1, perhaps installed via github? I mention this extensively because my version of windows is not supported by Windows Terminal.

Expected Behavior

The terminal application should not go to a blank screen when executing programs such as HTOP (Ncurses) or SCREEN (spawns a PTY).

Actual Behavior

When executing commands such as HTOP or Screen, the terminal emulator paints a black screen.

Diagnostic Logs

No response

@StnVsl
Copy link

StnVsl commented Nov 10, 2021

Same issues with vim and mc on Windows Server 2019 Standard 1809 (build 17763.2300) after KB5007206 update.

@josuenadal
Copy link

josuenadal commented Nov 14, 2021

This is happening to me when i try to run nano, vim or a Text-based User Interface (TUI) update screen on my debian machine through SSH on the Windows CMD or PowerShell (not WSL, just CMD & PowerShell).

Where normally i would see the Nano TUI or the VIM TUI i am seeing a blank screen with a cursor in the top corner that can recieve input, but does not show any change. There is nothing upon the screen until i CTRL+Z or CTRL+C my way out of the "open" application. Then i see that it was apparently open and some traces of the TUI are on the top of the screen as well as some of my input, scattered.

This is on Windows 10 LTSC, Version 1809, Build 17763.2300.

@ligouras
Copy link

ligouras commented Nov 15, 2021

I'm experiencing the exact same behavior as Atlas667. I am too using Windows 10 LTSC 1809, WSL1 + Ubuntu 20.04 (focal). This seems to have happened after a windows update reboot. The last two updates that applied were KB5007206 and KB5007298.

I also cannot use vim or htop when connected to a remote server through SSH. Though I'm invoking vim on the remote server, the terminal running on WSL still draws a blank screen.

@sanikolov
Copy link

sanikolov commented Nov 15, 2021

+1
the writeup by ligouras matches my diagnosis exactly.
I am seeing this on server 2019 dev essentials.

@acanakoglu
Copy link

+1
I am using Windows 10 LTSC 1809, WSL1 + Ubuntu 18.04 LTS.

I removed the updates KB5007206 and then KB5006744, the problem is resolved.

@bouvetcj
Copy link

+1
Windows server 2019 version 1809, WSL1 + Debian 10

@imanabu
Copy link

imanabu commented Nov 22, 2021

Also confirms that this happens on Windows Server 2019 with Ubuntu 20. Looks like some form of termcap issue to me, but I do not know enough (or time to figure that out yet) to diagnose this one. This does happen on vim and also emacs I am able to do this on WSL2 on Windows 11.

@Primozz
Copy link

Primozz commented Nov 23, 2021

Same on Windows 10 LTSC 1809, WSL1, Debian 9.
Like imanabu I was also able to trace the issue down to termcap, but don't know enough to go further.
Before finding this issue, I worked around the problem by setting TERM to one of the other available terminals from ls /lib/terminfo/*.
export TERM=vt220 works for me. Not an alt terminal though, but still better than blank.

@giofx
Copy link

giofx commented Nov 23, 2021

Hi, I got the same issue last week on Windows 10 enterprise LTSC 1809 17763.2183, WSL 1 and ubuntu 18.04 LTS.
As suggested above, I removed KB5007206 (I had no KB5006744), and after reboot nano/vim started working as before

@NikWP
Copy link

NikWP commented Nov 23, 2021

Same on Windows 10 LTSC 1809, WSL1, Debian 9. Like imanabu I was also able to trace the issue down to termcap, but don't know enough to go further. Before finding this issue, I worked around the problem by setting TERM to one of the other available terminals from ls /lib/terminfo/*. TERM=vt220 works for me. Not an alt terminal though, but still better than blank.

Thank you!
export TERM=xterm-color works for me!

@foobird13
Copy link

I'm observing this issue on Server 2019 (multiple machines) running Ubuntu 18.04 under WSL1. In addition to the blank screen issue, as another person mentioned, if I had tried editing a file I would see portions of the file contents displayed in the console after exiting the app.

Exporting a different TERM does work to resolve the issue of a blank screen in vi, nano, etc., but I do still see some artifacts. For instance, after exiting nano, the console screen would be empty (i.e. none of my history showing) except the prompt at the bottom. So the issue seems to be affecting the buffer as well.

@stemabo
Copy link

stemabo commented Nov 29, 2021

+1 cannot use nano, vim or htop (on WindowsServer 2019)
Terminal appears to be fine first, but starting nano or vim results in a blank black screen.
Need to then hit CTRL+X or CTRL+C to get my prompt back, this also displays some zombie-nano content: the content of the file I was trying to open in nano + a grey "GNU nano 4.8" header - but at the bottom I am back at the command prompt (smiliar with htop).
reboot does not resolve the issue - but export TERM=xterm-color is a workaround for now
(might be related to KB5008602 and/or KB5006368 but I am unable to roll these back in order to find out)

@ehm-wehb
Copy link

Same issue here, just recently lost htop, vim, and nano in WSL. Recently installed KB5006368.

export TERM=xterm-color worked for me.

Server 2019 - 1809, Ubuntu 20.04

@moodmosaic
Copy link

FYI, export TERM=xterm-color seems to be working also for me. It'd be good to know why KB5x update broke it.

@moodmosaic
Copy link

FYI, export TERM=xterm-color seems to be working also for me. It'd be good to know why KB5x update broke it.

FWIW, this also fixes clear where otherwise you'd have to do clear -x.

@AriESQ
Copy link
Author

AriESQ commented Dec 12, 2021

export TERM=xterm-color is not a complete fix. That replaces the blank screen issue with improper display of color codes. such as with the ncurses interface of the package nmon and some functionality with arrow keys displaying control characters instead of being interpreted.

Microsoft, you have broken my ability to use terminal for weeks. Behind an opaque update process where I can't even identify the breaking patch.

Do you understand what a cardinal sin it is to break the terminal, for someone that wants to productively use their system?!

@epistlewool
Copy link

I am having the same problem since KB5007206 which I am unable to roll back. I noticed that the terminal in vscode works as expected when running in WSL mode. So that is an alternative workaround.

@HontoNoRoger
Copy link

I had the same issue, and as already pointed out by @AriESQ xterm-color causes problems with arrow keys.

For me it was issues when using a previous command and changing it. The terminal replaced the input with the pre-existing output rather than inserting it.
When executing the command however, the input seemed to have been inserted properly, so it was just a display issue.

Setting export TERM=Eterm-color (Thanks @Primozz for showing the other available terminals) actually helped me with this issue as well, so I'm issue free now, as far as I can tell.

@craigloewen-msft
Copy link
Member

Hi folks, we're taking a look into this issue. I'll post updates here as soon as they're available.

@AriESQ
Copy link
Author

AriESQ commented Dec 15, 2021

Thank you @craigloewen-msft There is a lot of information in my initial post. I'd like to point out that this is reproducible in the WSL default shell. As well as when using a CMD window and SSH.exe into a remote host. To me that says there is a shared operating-system level component involved.

Also, from all the people in this thread, we have a pretty good idea which KB updates were involved. I was personally able to narrow it down by uninstalling/re-installing. However, as other's have mentioned the KB update is no longer available to be uninstalled. I think this has something to do with how MS rolls out "preview" updates, but it's complicated and I don't personally understand it. I would have otherwise worked to diff my system before/after the update to narrow down what changes were made.

TY

@JackMerlin
Copy link

JackMerlin commented Dec 17, 2021

Hello, @craigloewen-msft

I can confirm this problem. We are a small business with 10 computers running LTSC 2019. We received a report on this problem after the November update. an employee reported that files cannot be edited after logging into our unix-like servers using the SSH client in CMD, and both vim and nano commands will return a blank screen. Our IT contacted Microsoft Enterprise Support and they did not provide any useful solutions. we are currently using third-party software (PuTTY) to replace the affected system components.

At present, our system has been updated to the December update (KB5008218), and it can be confirmed that the current problem still exists.

Please solve this problem as soon as possible, because it has caused many problems for us, thank you.

Update:

The export TERM=xterm-color command does't work for our unix-like servers, and the blank screen is still returned under vim. but nano will return Error opening terminal: xterm-color.

However, using export TERM=linux [1] allows us to use nano, and vim still cannot work in the built-in SSH client on Windows 10.

[1] https://superuser.com/questions/1172222/issues-editing-files-with-nano-in-bash-windows-10

@AriESQ
Copy link
Author

AriESQ commented Dec 17, 2021

@JackMerlin Thank you for providing additional detail, it is noteworthy that you are experiencing this in CMD.exe and SSH.exe; as I have. Maybe people are experiencing this with the WSL terminal, so it seems reasonable that Microsoft updated a component that is shared between those terminals.

My sense is tha vim and nano seize control of the terminal window. And perhaps operate differently than an ncurses program that draws special characters to the screen. Would you be able to confirm you are experiencing the problem with an ncurses program such as htop?

I think we have tee'ed this up with plenty of info for Microsoft to resolve, many thanks to everyone supplying info.

@JackMerlin
Copy link

@JackMerlin Thank you for providing additional detail, it is noteworthy that you are experiencing this in CMD.exe and SSH.exe; as I have. Maybe people are experiencing this with the WSL terminal, so it seems reasonable that Microsoft updated a component that is shared between those terminals.

Yes, we did not use WSL, we did not even use Powershell, just cmd.exe and the built-in openssh client (OpenSSH_for_Windows_7.7p1, LibreSSL 2.6.5). We have already ruled out the cause of the problem caused by our servers in contact with Microsoft technical experts. We can reproduce the problem after installing LTSC 2019 on a virtual machine and updating to the latest cumulative update. So we insist that it is the Windows update that caused the problem.

This is a screenshot of the blank screen returned by vim and nano, hope it helps.
blank_screen_returned_by_vim_and_nano

My sense is tha vim and nano seize control of the terminal window. And perhaps operate differently than an ncurses program that draws special characters to the screen. Would you be able to confirm you are experiencing the problem with an ncurses program such as htop?

Sorry, our system uses top instead of htop. In my case, top can work normally.

I think we have tee'ed this up with plenty of info for Microsoft to resolve, many thanks to everyone supplying info.

Thank you, I think this issues is more useful than some technical experts, because they did a lot of diagnosis, but did not provide any useful solutions.

I am here to reply to the hope that Microsoft will pay attention to this issue and hope that it can be fixed in the next update.

Thank you.

@mihalycsaba
Copy link

I think I have this issue, every TUI program stopped working on one server I administer, like 2 months ago, but I have the same issue using putty too. Also I can't connect with winscp to that server. ctrl+c and ctrl+z doesn't do anything,

@mihalycsaba
Copy link

Forgot to mention, that other people use that server too, but they don't have issues. So this is why I suspect, that it was a windows update that broke something on my machine.

@colhountech
Copy link

colhountech commented Jan 6, 2022

+1
Windows Server 2019 Standard - 10.0.17763 Build 17763

  • Fresh WSL 1 Install
  • Ubuntu 20.04

@AriESQ
Copy link
Author

AriESQ commented Jan 6, 2022

Here is some documentation on Windows Console: https://docs.microsoft.com/en-us/windows/console/

I'm realizing that looking at process trees, conhost.exe is the common link between this issue presenting under cmd.exe and wsl.exe.

@craigloewen-msft any movement on this? We have identified in-thread the updates that introduced breaking changes. I wish I could give a diff of what files changed, but I don't understand how "preview" updates work, and KB5006744 is no longer available for me to uninstall from my system.

My terminal, a fundamental, basic thing to how I work, has been broken for months now. This is leaving a really bad taste in my mouth about using WSL going forward.

@JackMerlin
Copy link

@craigloewen-msft We need some updates and progress, is there a release schedule out there, will we see a fix in the February cumulative update?

@craigloewen-msft
Copy link
Member

We've found the cause of this bug, identified a fix, and are working on servicing that fix out to Windows! I will be sure to post updates to this thread when they're available.

@datvv
Copy link

datvv commented Feb 13, 2022

I just installed Windows 11, Now I can't use HTOP and edit files on my server via VIM editor. What should I do now?

@mihalycsaba
Copy link

Hey guys, I wrote here earlier, I had a problem with a server a while ago, last week some of the client win10 machines started to have problems too. The TUI app stopped loading.

Had another symptom, winscp couldn't connect. With this I managed to figure out it was a problem with the MTU size, the ISP changed something on their end(couldn't ping the server ip), that broke the mtu size setting.

Fixed it with reducing the mtu size of the ethernet interface on the server.

@datvv
Copy link

datvv commented Feb 27, 2022

this issue is fixed after I upgraded to use wsl2.
https://pureinfotech.com/install-windows-subsystem-linux-2-windows-10/

@greenaussie
Copy link

greenaussie commented Mar 3, 2022

I have this issue with neither of the hot fixes mentioned above installed. Server 2019 Datacentre / 1809 / 17763.2565 (on AWS WorkSpaces).

Looking forward to the fix!

@sietse
Copy link

sietse commented Mar 8, 2022

@AriESQ , thank you for your overview of how well different $TERM values work for you. What did eterm-color do better than xterm-color for you? I ask because I had the opposite experience, described below.

For me, xterm-color worked better:

  • (WSL sends as Right-arrow as <Esc>[D)
  • When TERM=eterm-color was in the environment, some programs failed to interpret arrow keys.
    • in htop the Function and arrow keys did not work;
    • in lazydocker, if I hit the right-arrow key it was treated as 'Escape' and the program quit.
    • in the Fish shell and in IPython, on the other hand, the arrow keys worked fine.
  • With TERM=xterm-color all programs correctly interpreted the arrow key.

(This was with WSL 2, Ubuntu-20.04, WSL last updated on 2021-11-12, Kernel version: 5.10.60.1.)

@AriESQ
Copy link
Author

AriESQ commented Mar 8, 2022

@sietse Are you having the blank screen issue that is the topic of this ticket? xterm-color which is the default does not seem to be working for people and eterm-color is a workaround.

@sietse
Copy link

sietse commented Mar 10, 2022

@AriESQ apologies, I don't have the blank screen issue. Your overview comment was helpful to me, and I wanted to offer further information as thanks; but I was so intent on replying to your comment that I forgot to consider this is an issue thread. I'm sorry for the noise.

@AriESQ
Copy link
Author

AriESQ commented Mar 16, 2022

127 days (4 months and 7 days) since this was reported.

Microsoft broke the Console in an opaque way and still hasn't fixed it. This is embarrassing.

@moodmosaic
Copy link

It really is.

@mikepietruszka
Copy link

Count me in as another person having this issue :(

@craigloewen-msft
Copy link
Member

Hi all, we have released a fix for this as part of

KB5011551. Right now this is available for seekers, but next month it will be part of the patch Tuesday patch.

Thank you for filing this! I'll close this issue once it's pushed as part of a regular update.

@AriESQ
Copy link
Author

AriESQ commented Mar 23, 2022

I don't see the fix listed in the release notes, so it looks like we may never know the root cause or the solution. Very disappointing compared to open source tools where there is transparency.

I haven't tested this yet, but looking at the changed files, my guess is that the fix is contained in :

Windows 10 version 1809 LCU x64-based				
File name	File version	Date	Time	File size
conhost.exe	10.0.17763.2746	7-Mar-22	23:39	826,880

@sietse
Copy link

sietse commented Mar 25, 2022

@craigloewen-msft , good on you! I look forward to trying it out, the patch will probably reach me next month, if I remember I'll post a little update here.

Out of curiosity, what sort of problem caused this behaviour? Was it indeed some terminal control sequences getting misinterpreted by the WSL console / terminal emulator?

Cheers!

@AriESQ
Copy link
Author

AriESQ commented Mar 25, 2022

Via the Microsoft Terminal project; it looks like MS recently open-sourced the traditional ConHost terminal. 😲https://github.com/microsoft/terminal/tree/main/src/host

I did a quick look there (time bound by the appearance and patch of the bug.) And I didn't see anything obvious as a root cause of this present bug, in my casual search.

@JackMerlin
Copy link

@craigloewen-msft thank you, I can confirm that this does fix the issue and we will be deploying to all affected PCs within the next month.

Thanks also to this issue and to everyone who reported the issue to Microsoft, your efforts made this fix available.

@Gyross
Copy link

Gyross commented Mar 30, 2022

@AriESQ @sietse Have a look at this: microsoft/terminal#12329

My hunch is that it's the alternate screen buffer sequence not being handled properly: the console view changes correctly, but not the write destination, but I'm 100% sure it's a conhost issue. I didn't check that classic API screen buffers still worked, but if they didn't it would probably break a lot of native windows tools as well.

@AriESQ
Copy link
Author

AriESQ commented Mar 30, 2022

@Gyross Thank you, I always suspected this was an issue in microsoft/terminal But because their minimum requirement is Windows 10 version 18362.0 or higher, I expected a ticket there for my problem would be closed immediately. I suspected MS Terminal, because who else would be modifying this ancient feature of Windows?

@AriESQ
Copy link
Author

AriESQ commented Mar 30, 2022

I have installed Cumulative Update Preview for Windows 10 Version 1809 for x64-based Systems (KB5011551) on the system listed in this original ticket. The screen blanking issue appears to be solved. This is with the default $TERM = xterm-256color. I have tested the WSL console, as well as CMD.exe and openssh.exe.

However, @craigloewen-msft this issue may not be fixed in it's entirely. I run a 2560x1440 monitor. And I use the "window snapping" feature (i.e. pressing Win+arrow key, or dragging window to the edge of monitor.) I can use the WSL terminal window in two configurations, "half-screen snapped" and "full-screen maximized".

Executing a full-screen (ncurses?) command in half-screen mode, and then moving to maximized, the bottom line of text stops displaying (possibly only when the Properties > Layout >Wrap text output on resize checkbox is un-checked. Unchecked appears to be a non-default option.)

This behavior is inconsistent between when $TERM is set to xterm-256color, eterm-256-color, and eterm. Eterm which was the previous temporary fix does not appear to be affected by this issue of the bottom line not being displayed

This solution is working for my purposes, I suppose. I don't have the time to do any more free troubleshooting for Microsoft. I am unsure at the correctness of the solution. I just think of all the effort that people have taken to ensure the integrity of POSIX terminals and I am disappointed. Microsoft in their own codebases are known for rigorous backwards compatability.

Anyways Craig, thank you for (mostly) running this to ground and being responsive to the bug. Whether you want to close this bug is outside of my discretion. Cheers. 👍

@craigloewen-msft
Copy link
Member

@AriESQ thanks for giving the write up here that this issue might not be totally fixed. I'll follow up with the Windows console team to see if the behaviour you mentioned is a bug. I'll keep this issue open for now!

@AriESQ
Copy link
Author

AriESQ commented Apr 15, 2022

@craigloewen-msft I don't expect ya'll to try to replicate my exact bug report. My thinking was that you could run this behavior by the patch author, and see if it rings a bell like "oh yeah, I could see an off-by-one error during window resizing."

Am I being overly pedantic with this bug fix? Maybe. Is there some industrial control system somewhere integrating Macro Recorder to programmatically interact with a WSL shell window... Possibly? That's the only reason I am raising the potential incorrectness of the provided solution. Or some other hacked-together, yet vital usage...

I'm about ready for this ticket to be closed. I would have put more effort into reviewing this solution, but the opaque Microsoft update process has turned me off. If I had the original breaking patch, and the updated patch; I could know what to test. This is sort of frustrating, because it seems that this info exists in MS-Terminal and the newly open-sourced ConHost repo. 🤷‍♂️

@craigloewen-msft
Copy link
Member

Since the original issue (The screen going blank when executing HTOP or SCREEN) is fixed, I'll be closing out this issue. @AriESQ for the extra issues you mentioned I'd recommend reaching out to the Terminal team by filing an issue at https://github.com/microsoft/terminal (As you've already identified). Thank you all!

@AriESQ
Copy link
Author

AriESQ commented May 10, 2022

Is anyone experiencing an issue where the mouse does not interact with the WSL terminal screen? I can not select text. Also when I paste into the terminal with shift-ins I do not get content I get control characters: 2~

Powershell and CMD.exe work with mouse selection as normal.

If no-one else is experiencing this, I just don't have the energy to track down Microsoft's broken stuff anymore and I won't be following up.

@craigloewen-msft Were there any changes recently that might have touched this functionality?

I think my most recent updates are
KB5013868
KB501394

Same config as in original post.

@foobird13
Copy link

I do see the control characters when pressing Shift-INS (same with Ctrl-INS). Although, that's not a key-combo I've ever used before, I just stick with Ctrl-Shift-C/V for copying/pasting (as long as the option is enabled in the console settings).

I don't have any issue with mouse interaction. Selecting text works just fine for me, as well as pasting with right-click.

Last update was KB5012647 (May updates haven't been installed yet).

@AriESQ
Copy link
Author

AriESQ commented May 11, 2022

Thank you @foobird13 If I don't see acknowledgement of this issue, I'll assume it is something unique to my setup that is broken. (Always a possibility).

@StnVsl
Copy link

StnVsl commented May 14, 2022

ctrl+v and shift+ins don't work in WSL terminal, but ctrl+shift+v does if enabled in settings.
Mouse selection, right-click copy/paste and ctrl-c/ctrl-ins work as usual.
KB5013868/KB5013941 didn't change these things on Server 2019.

@djpbessems
Copy link

Since the original issue (The screen going blank when executing HTOP or SCREEN) is fixed, I'll be closing out this issue. @AriESQ for the extra issues you mentioned I'd recommend reaching out to the Terminal team by filing an issue at https://github.com/microsoft/terminal (As you've already identified). Thank you all!

What?? You're closing this when it's still an issue for running openssh within powershell and possibly many other combinations?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests