ConPTY Passthrough mode

[be5invis]

Modern apps won’t read the hidden character grid and do everything in VT. So why not an API/console mode to tell Console Host to completely throw away that?

[DHowett-MSFT]

@zadjii-msft was looking into this. The main issue, if I recall correctly, is that we need to trash the entire buffer when something enters or exits “passthrough” mode. It’s also only truly applicable when there is a connected pseudoconsole session.

[DHowett-MSFT]

And the reason one console might enter and exit passthrough mode multiple times is that you may run coolNewThing.exe from CMD, and perhaps it might launch a further process that needs legacy support.

Additional concerns: if you have a tree of four processes, each of which wants passthrough to be different, should the ones that are doing ReadConsoleOutput be able to read the buffers from the other legacy/non-passthrough ones?

It’s complicated when you get into compatibility discussions. 😄

[zadjii-msft]

Yea, I tried getting this working for like a day last year, but it's something I've wanted to work on for a while.

As Dustin mentioned, there'd be real weirdness moving between passthrough mode and non-passthrough mode. However, I think it might still be something good to investigate.

[zadjii-msft]

From #1985:

I've discussed this a couple times on different threads before, but I think I never made a real issue for it.

The idea of conpty passthrough mode is that a commandline client application
that knows it's only going to use VT sequences (and not the API) to modify
the console could set a special mode, ENABLE_PASSTHROUGH_MODE. If conpty is
active, the console would then stop rendering itself over conpty, and anything
that was written to the console would go straight to the terminal.

• This would work especially well for something like wsl, where it's only ever going to be talking VT.
• It would not work for cmd.exe, because of cmd.exe's heavy reliance on the API.
• Windows Powershell again needs the API pretty heavily, but maybe Powershell
  Core, which is cross-platform, could make use of it.

This has some rough edges that need to be sorted out.

• What happens when the app exits, and passthrough mode is turned off? The
  terminal and conpty's buffers would not be equivalent anymore!
  • I've been toying with the idea of having conpty both pass the sequences
    through, and also process them itself, so that the terminal and conpty stay
    in sync.
• What happens when someone tries to call Console API's in passthrough mode?
  • I'm thinking we just cause them to fail. All save for Get/SetConsoleMode and reading input / writing output.
  • We'll also probably need to be able to read resize events.

This needs a real spec written, but it probably needs prototyping done before that.

[oising]

We need a SIGWINCH asynchronous signal for resizing somehow.

[mintty]

This is an essential feature for opening MS text-mode software (esp. WSL) to 3rd-party environments.
The current approach seems to tighly connect the Windows terminal implementation with ConPTY, however that limits applications unnecessarily and makes them highly dependent on Windows terminal progress (which may still take years, honestly).

It would not work for cmd.exe, because of cmd.exe's heavy reliance on the API.

It should be a combined mode: Whenever a console-API-based application is run, e.g. as started from a pure terminal-based application, the console API calls should be transformed into terminal escape sequences. Note as there are not so many features in the console API, this is a much easier approach than the reverse mapping, trying to squeeze terminal features through the conhost bottleneck.

[therealkenc]

Just realized #2035 is this ask framed differently.

however that limits applications unnecessarily and makes them highly dependent on Windows terminal progress

Not just progress. Behavior. This guy is writing a Tek4010 emulator. He is going to need a PTY if say his code were ever ported to native win32. And conhost sure as heck doesn't know what to do with the bytestream coming down that PTY. Never will. And need not care.

We need a SIGWINCH asynchronous signal for resizing somehow.

That too. Doesn't have to be a signal, mind, if there is some religious/philosophical reasons against. But if not it needs to be a separate (third rail) HANDLE on which I can WaitForMultipleObjects(), because no one said there is a ReadFile() byte coming, ever.

[be5invis]

@mintty
There are many absurd console API usages, like reading the screen. That's really a legacy that ... IBM PC has a video card while PDPs aren't.
One idea is that ConPTY can have a "sync" stuff to read back the screen from a console application if some console API applications want to something strange. Otherwise the console host could simply convert console API calls into VT sequence, and forward that to the terminal app.

[therealkenc]

There are many absurd console API usages, like reading the screen.

Great. Invent a new CSI sequence for that. Send the PCHAR_INFO lpBuffer back as I dunno a base64 gzip. The feature is not supported by Mintty. Yet. No biggie. Not even unusual. Any 6502/Z80 assembler programmer working 1981 was free to add their own crazy vendor-specific sequence to their company's terminal if they were bored enough on a weekend too.

[mintty]

There are many absurd console API usages, like reading the screen.

Great. Invent a new CSI sequence for that.

I assume this concern is more about the other direction of such adaptation, i.e. how would you serve a Windows console program that wants to use that "absurd" feature? You could run a second, hidden console in parallel, to maintain backwards compatibility.

[DHowett-MSFT]

run a second, hidden console in parallel

The people who hate how ConPTY is implemented today will absolutely hate how it's implemented if we do that. 😁

[mintty]

They wouldn't even notice in pure pass-through applications. It might be necessary to solve an otherwise unresolvable dilemma.

[therealkenc]

run a second, hidden console in parallel

This solution makes no sense because there is no window in sshd.exe nor someoldprogram.exe. Both those programs are text only applications that wouldn't know a Consolas Font from a hole in the ground.

To the point of #2035, neither sshd.exe nor someoldprogram.exe are terminal emulators. Gnome Terminal is a terminal emulator. And gnome-terminal is the only thing that can give the correct answer as to the contents its screen buffer; ie what ReadConsoleOutput() should return. Similarly, the only thing that knows the contents of (VS Code) xterm.js's screen buffer is xterm.js. ConPTY doesn't have a clue what xterm.js internal screen buffer contains. At best it can only guess by scraping the data passing by.

I assume this concern is more about the other direction of such adaptation,

Yes. And as a practical matter I wouldn't expect the ESC[?GIVEMEBUFFERASIBM feature being added to vteterminal anytime soon. Unless someone can point out the killer app that calls ReadConsoleOutput().

[a hidden window] might be necessary to solve an otherwise unresolvable dilemma.

I think we are closer than that. If conhost wants to read-only scrape the data going by for the sole purpose of keeping a wild-ass-guess as what the actual terminal emulator's buffer looks like, sure, I can live with that. ReadConsoleOutput() is a red herring. If conhost (call it a urxvtd analogy) wants to maintain a shadow buffer, it can knock itself out.

...So long as it is quiet about it. There is no reason for a pass-through "mode", which is how this issue was framed. There is never a reason for ConPTY to inject a VT sequence into a WriteFile() on a PTY handle -- ever. Daniel over on the VS Code team can't possibly care; because he has his own terminal emulator. PTYs don't care about VT100 sequences. Never heard of them.

[Then we need SIGWINCH (or equivalent thereof). Which isn't related to VT100 sequences or screen buffers either.]

[DHowett-MSFT]

@therealkenc,

Like it or not, conhost is the API server that, regardless of whether it presents a window, makes all existing Win32 console applications work. It must continue to make those applications work, because organizations really hate love when Microsoft rolls through with a new standard and tells everybody to drop what they're doing and throw out thousands of lines of code.

I think you're looking to turn this request, and this project, into something it's not. You may be attempting to turn CreatePseudoConsole into something it's not, too. Things work the way they work because we need to maintain compatibility with the thirty years of applications written since the VGA text mode buffer became the "official" design inspiration for how consoles should work in DOS and Windows.

ConPTY exists to--narrowly-stated--allow an application that understands a number of sequences as specified by an xterm-256color terminfo to host a windows console application; to wit: an application that would otherwise run in conhost should be able to run in a "terminal emulator" of sufficient compatibility. It's not intended to support a TEK4010 application (those are not windows console subsystem applications), and it's not likely to want to support a TEK4010 terminal emulator that is expecting to receive a bytestream from a TEK4010 application. That guy will probably end up doing what everybody ELSE who doesn't want to write a windows console subsystem application does: use pipes, because they don't have the same compatibility requirements (neé guarantees) as the windows pseudoconsole infrastructure.

Through that lens, a "passthrough" mode is required. A console application by default, and this cannot be changed for compatibility reasons, starts up in a mode where it just has full access to all of the stupid Win32 console APIs that no terminal emulator developer wants to countenance. Nobody should be able to read back the contents of a terminal buffer, local or remote, that they wrote to. Nobody should be able to write into the offscreen section of the buffer, because there's no guarantee anywhere else that it actually exists. But, they do. Developers use this. Applications expect this, because they were written as windows console subsystem applications. A passthrough mode--mode!--is the only way we can offer an application a way to say "I promise I won't use the old ways" while still being a windows console subsystem application. That's the first step we can make towards ConPTY being the dumb pipe you want it to be.

If you'd like to debate whether the Windows Console was the right choice, or was well-designed, I'm happy to have you do it--but not here.

---

Two asides.

1. If you're looking for a better SIGWINCH, follow #281.
2. On machines where isatty(3) is a libc-provided fixture, you also have openpty(3). When somebody spawns a child application with a stdin/stdout hooked up to file descriptors they get back from openpty(3), isatty(3) suddenly returns 1. That application will, more often than not, decide that what that means is that it can send back VT100 escape sequences. That's what it means to most applications to be "on a tty." Sure, it's a terrible abstraction and a poor design and applications should be smarter than this, but they're not. The Windows Pseudoconsole fits right in, here, with the understanding that "I've allocated a PTY, which means I want VT".

[therealkenc]

I'm not intending to criticize the design or implementation -- at all. That was not the intent. I am trying (more slowly than intended) to find a solution to open-for-a-year issue WSL#3279. Of which Biswa96 has a very good start.

[zadjii-msft]

woah this thread got pretty out of hand over the weekend.

The stars aligned Friday, and I actually got a chance to play around with implementing a passthrough mode for conpty. I'm pretty happy with how it works so far, so I think it needs a spec and some polish, and maybe we can ship it one day. Here's the approach I've been taking:

• I introduced a new ENABLE_PASSTHROUGH_MODE to the SetConsoleMode flags.
• When a commandline application enables passthrough mode, and the console is currently attached to a terminal (it's in conpty mode), any text they write to the console is written straight to the terminal as well, with no munging by conhost.
• In passthrough mode, conpty stops "rendering" any changes to its buffer.
• If a commandline application knows it's not going to be doing anything with the console API, it can safely set passthrough mode to be able to talk directly to the terminal.
• [NOT DONE YET] If a commandline app in passthrough mode tries to call any API's, we'll either:
  • Convert the effects of that API call to VT, and pass that through to the terminal. We'll do this for API's where this is reasonable - SetConsoleTextAttribute is a good example.
  • For some APIs where the terminal doesn't really matter, we'll just keep doing what we're currently doing (case in point GetConsoleProcessList).
  • For APIs that don't make sense for conhost to be able to respond in passthrough mode, or we can't create a VT sequence to perform the requested operation, we'll return E_UNSUPPORTED_API (or a real error) indicating that API isn't supported in passthrough mode. Case in point: ReadConsoleOutput*, ScrollConsoleScreenBuffer, SetConsoleDisplayMode, etc.
    • Since the client app was updated to add support for passthrough mode, it should be able to be updated to handle this error case as well.
• conhost also processes the strings that a commandline app emitted. This is to try and keep conpty's state in sync. This is important for the following.
• When a client app exits passthrough mode, there's a chance that the terminal is in a torn state from the conpty. Case in point, if cmd.exe launched wsl.exe, and wsl enabled passthrough, did some stuff, then wsl exited, and cmd restored the console back to non-passthrough mode (as it needs to use the API). When passthrough mode is exited, conpty will redraw its screen, to re-sync the terminal to what conpty believes the buffer looks like.
  • For things like just launching wsl.exe or ssh.exe directly, or for mintty, this won't be as important, as the client will always be in passthrough mode, and the conpty will never "exit" passthrough.

This provides a way where we can be sure that apps that weren't updated for running in a pty will still have access to the entire console API, but apps that want to live in the new world can say "I promise I know what I'm doing", and run even smoother in conpty. I believe there's going to be an incredibly small intersection of apps that want to use VT and also call things like ReadConsoleOutput. Most apps are wither going to be console-like, using the API heavily, and might not be as likely to be updated for such a mode. However, for *nix-like apps that aren't going to be using the API so much and are primarily speak VT, this is an excellent option. This flag creates a clear distinction between the two when running in a pty.

[davidhewitt]

Thanks to everyone working on this! I've been hoping to see this for some time, so very excited to see progress!

Actually the passthrough mode could be assumed by default and ConPTY can be lazily enabled.

I just wanted to weigh in that passthrough-by-default seems highly desirable to me.

If I'm reading right, passthrough will result in faster and better rendering (e.g. resolves #405) more or less universally. The only downside is that it would break compatibility with some console APIs (which cross-platform VT-based apps will not need).

So it seems to me that a good balance would be:

• By default apps run in passthrough. If an app uses an incompatible console API then switch to conpty mode.
• Apps could ensure better performance / rendering by calling SetConsoleMode to explicitly enable passthrough mode. In this state, calling unsupported APIs will result in an unsupported error.
• Apps could also call SetConsoleMode to explicitly disable passthrough mode. They might want to do this on startup so they can safely use incompatible APIs without causing a mode switch & redraw during program operation.

Also some interesting questions which just struck me:

• what happens if a child process which inherits the parent's console attempts to enable passthrough mode, but the parent depends on incompatible APIs?
• if a child process disables/enables passthrough mode and then exits, should the previous mode be restored once the child process exits?

[davidhewitt]

A further thought: will the hosting terminal emulators be able to query if the pseudoconsole is in passthrough mode? This might be needed, for example when the window resizes:

• If it is in passthrough, then the terminal emulator should perform text reflow when the window resizes
• If it is not in passthrough, then ConPTY will perfom text reflow (so the terminal emulator should not)

[christianparpart]

I think if the terminal emulators request a ConPTY and state whether or not to to get pass through mode, then the terminal emulator already knows what to do in such circumstances.

…

On Mon, 9 Dec 2019, 06:51 David Hewitt ***@***.***> wrote: A further thought: will the hosting terminal emulators be able to query if the pseudoconsole is in passthrough mode? This might be needed, for example when the window resizes: - If it is in passthrough, then the terminal emulator should perform text reflow when the window resizes - If it is not in passthrough, then ConPTY will perfom text reflow (so the terminal emulator should not) — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1173?email_source=notifications&email_token=AAAN3O2FBNHB5LIVP5O22FDQXXMHZA5CNFSM4HWHDMHKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGH5QDA#issuecomment-563075084>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAN3OYWW4ENVN5F3DALG4TQXXMHZANCNFSM4HWHDMHA> .

[davidhewitt]

I think if the terminal emulators request a ConPTY and state whether or not
to to get pass through mode, then the terminal emulator already knows what
to do in such circumstances.

Sure; however it sounded to me like for the solution proposed here the commandline app would request passthrough mode, rather than the terminal emulator. So the terminal emulator would have no knowledge of whether passthrough mode is enabled.

[christianparpart]

IMHO it doesn't make sense for the app (running inside a terminal emulator) to request passthrough mode but the terminal emulator, the one to interpret the VT sequences, must see the VT sequences as equivalent as possible to what the app produced. ?The problem with ConPTY is, that it is going to reimplement every single VT sequence out there, and it lags behind. Hence, terminal emulator writers where reporting bugs (or missing features), and the best way to work around that is to "just pass all VT sequences through \"as-is\"", eliminating a whole range of issues within ConPTY, and thus, giving the terminal emulator the chance to actually see what the apps produce, and render that. The app inside any terminal emulator, while it should know when it is going to be resized, should not care about what terminal emulator of man-in-the-middle (conpty) it is running in. Do you agree? Am Mo., 9. Dez. 2019 um 08:51 Uhr schrieb David Hewitt < notifications@github.com>:

…

I think if the terminal emulators request a ConPTY and state whether or not to to get pass through mode, then the terminal emulator already knows what to do in such circumstances. Sure; however it sounded to me like for the solution proposed here the commandline app would request passthrough mode, rather than the terminal emulator. So the terminal emulator would have no knowledge of whether passthrough mode is enabled. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1173?email_source=notifications&email_token=AAAN3OYOEQS5Q5RM5ORTTG3QXX2G7A5CNFSM4HWHDMHKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGIF25I#issuecomment-563109237>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAN3O4XF6KLNQELYXBXMJLQXX2G7ANCNFSM4HWHDMHA> .

[DHowett-MSFT]

@christianparpart sometimes, there are no VT sequences coming out of an application. The Windows Console APIs are not implemented using VT sequences.
The application must request passthrough mode to tell the console that it will not use one of the APIs that cannot be translated into VT sequences.

The terminal cannot set passthrough mode because it cannot force legacy applications to become compatible with VT sequences.

This is why ConPTY exists.

[ExE-Boss]

For a terminal, whether conhost is in ConPTY or passthrough mode should be irrelevant, because the terminal application will just receive VT sequences through the PTY pipe.

[davidhewitt]

For a terminal, whether conhost is in ConPTY or passthrough mode should be irrelevant, because the terminal application will just receive VT sequences through the PTY pipe.

That's my point; it's not irrelevant in certain edge cases. For example when the window resizes: at the moment ConPTY reflows its buffer and sends the terminal emulator instructions to redraw everything. In passthrough mode this won't happen, so the terminal emulator will have to do reflow itself.

[davidhewitt]

Idea: Perhaps an OSC escape sequence could be sent by conhost to inform the terminal emulator that conhost is entering / exiting passthrough mode?

[mintty]

The terminal cannot set passthrough mode because it cannot force legacy applications to become compatible with VT sequences.

No, the use case is a terminal running a specific backend, particularly WSL, so it knows the client will use VT mode and passthrough is needed to give it full functionality.

... when the window resizes: at the moment ConPTY reflows its buffer and sends the terminal emulator instructions to redraw everything. In passthrough mode this won't happen, so the terminal emulator will have to do reflow itself.

The terminal would "normally" notify the pty about changed terminal size. The pty would then send a SIGWINCH signal to the client application. ConPTY should handle this signal forwarding somehow. Then the client application can redraw itself.

[davidhewitt]

No, the use case is a terminal running a specific backend, particularly WSL, so it knows the client will use VT mode and passthrough is needed to give it full functionality.

I disagree. Knowledge that passthrough is needed for full functionality would be better written once in the client app (e.g. wsl.exe) than in every terminal emulator which wants to host WSL.

The terminal would "normally" notify the pty about changed terminal size. The pty would then send a SIGWINCH signal to the client application. ConPTY should handle this signal forwarding somehow. Then the client application can redraw itself.

Agreed (I think it might already send this signal?). But I'm not talking about SIGWINCH here. I'm talking about the terminal emulator's own text layout & redraw, such as wrapping overflowing lines. At the moment ConPTY does line wrapping on its internal buffer and sends VT output to the emulator on how to draw the wrapped text. In passthrough mode the emulator will get unwrapped text and need to wrap it itself.