Windows support

[melund]

I guess this is another feature request for prompt-toolkit, but it would be fantastic if the VT100 backend could be forced on windows. Having a terminal multiplexor in windows could be great.

Here is an example of pymux running in the windows terminal (conhost). It works great and really fast. It is not really Windows since it runs in the 'Windows subsystem for linux'. So it is the linux versions of pymux, xonsh and PTK that are running. But the windows console handles the VT100 output quite nicely.

[jonathanslenders]

This is interesting, and definitely useful!

The difficult thing to make this work outside of the Linux subsystem is probably that there's no PTY available, and we need to emulate a native console. It's also on my wish list, but probably rather complex.

[melund]

Ahh. Yes. I see the issue there. I think I read somewhere that the WSL team had such a task on their backlog.

But still having having PTK support full VT100 on Windows would be nice, at least for the 24bit color. I will try to figure if there is a way to detect if the console supports the new features.

[sejktmcsdlmfhsc]

That would be great, if possible! I'd love have xonsh / pymux on my windows machine @work, too.
@melund what kind of linux subsystem did you get it to run? is it cygwin mingw or msys?

[melund]

It is not msys or cygwin at all. As I understand they are Unix stuff built for windows.

I was running the 'Windows subsystem for Linux', which is a new feature in windows 10. It allows you to run the real user-mode binaries in a ubuntu subsystem on windows. I am no expert so you must read up on it yourself.
https://msdn.microsoft.com/en-us/commandline/wsl/about

Basically, it is is the real linux version of python, prompt_toolkit, xonsh and pymux I was using. However, it was the Windows console that rendered it. It worked well showing that a windows version is also possible.

[dragon788]

If you run this under msys2 on Windows there is a winpty helper that allows you to run applications that need a real pty that behaves like a Linux one. It is typically called for single commands where the application can't be extended to handle Windows' process/pipe handling, but it might be a workaround for those OUTSIDE of the WSL mentioned above.

[prabirshrestha]

Given that vim8 and neovim now have terminal seems like this could be possible using winpty.

Here is an equivalent issue in tmux tmux/tmux#1029. But there are other things besides just pty that needs to be ported to windows.

[prabirshrestha]

Seems like someone has already written a python bindings for winpty at https://github.com/spyder-ide/pywinpty

[jonathanslenders]

Hi @prabirshrestha and @melund, I'm working on this, but there are still a couple of things to be done.

If you have some time available, feel free to test the prompt-toolkit-2.0 branch from this repository, install ptterm (https://github.com/jonathanslenders/ptterm ), and install winpty.

pymux standalone should work, but there are several other things I need in order to get this server/client architecture working. Right now, I'm struggling with overlapped I/O working with Windows pipes. So, if someone has a good example of a simple client and server script that uses Windows Pipes and overlapped I/O, that would be great. :-)

[melund]

Quite impressive. I think @astronouth7303 may have some good input for this. He has mucked quite a bit with this for https://github.com/xonsh/slug

[prabirshrestha]

@jonathanslenders This is awesome. Can't wait to use replace ConEmu with pymux.

Few things I noticed.

1. Would be good if pywin32 was automatically installed as dependency. I had to manually type pip install pywin32 to make it work.
2. Changing prefix seems to crash. This is currently the biggest blocker for me even if client/server doesn't work.

Here is the config

set-option prefix C-a
unbind C-b
bind C-a send-prefix

Here is the log

C:\Users\prshrest>pymux standalone
Traceback (most recent call last):
  File "C:\Program Files\Python36\Scripts\pymux-script.py", line 11, in <module>
    load_entry_point('pymux==0.13', 'console_scripts', 'pymux')()
  File "c:\program files\python36\lib\site-packages\pymux\entry_points\run_pymux.py", line 90, in run
    mux.run_standalone(color_depth=color_depth)
  File "c:\program files\python36\lib\site-packages\pymux\main.py", line 643, in run_standalone
    connection=None)
  File "c:\program files\python36\lib\site-packages\pymux\main.py", line 652, in add_client
    color_depth=color_depth)
  File "c:\program files\python36\lib\site-packages\pymux\main.py", line 91, in __init__
    self.app = self._create_app()
  File "c:\program files\python36\lib\site-packages\pymux\main.py", line 189, in _create_app
    pymux.startup()
  File "c:\program files\python36\lib\site-packages\pymux\main.py", line 346, in startup
    call_command_handler('source-file', self, [self.source_file])
  File "c:\program files\python36\lib\site-packages\pymux\commands\commands.py", line 92, in call_command_handler
    handler(pymux, arguments)
  File "c:\program files\python36\lib\site-packages\pymux\commands\commands.py", line 148, in command_wrapper
    func(pymux, received_options)
  File "c:\program files\python36\lib\site-packages\pymux\commands\commands.py", line 560, in source_file
    handle_command(pymux, line)
  File "c:\program files\python36\lib\site-packages\pymux\commands\commands.py", line 72, in handle_command
    call_command_handler(parts[0], pymux, parts[1:])
  File "c:\program files\python36\lib\site-packages\pymux\commands\commands.py", line 92, in call_command_handler
    handler(pymux, arguments)
  File "c:\program files\python36\lib\site-packages\pymux\commands\commands.py", line 148, in command_wrapper
    func(pymux, received_options)
  File "c:\program files\python36\lib\site-packages\pymux\commands\commands.py", line 577, in set_option
    option.set_value(pymux, value)
  File "c:\program files\python36\lib\site-packages\pymux\options.py", line 126, in set_value
    pymux.key_bindings_manager.prefix = keys
  File "c:\program files\python36\lib\site-packages\pymux\key_bindings.py", line 82, in prefix
    self._load_prefix_binding()
  File "c:\program files\python36\lib\site-packages\pymux\key_bindings.py", line 58, in _load_prefix_binding
    self.key_bindings.remove_binding(self._prefix_binding)
AttributeError: '_MergedKeyBindings' object has no attribute 'remove_binding'

[jonathanslenders]

Thanks for pointing out. That's a bug, which is unrelated to Windows. I can fix that. I'll add pywin32 as well (I didn't realize I used an external package for that, actually.)

edit: I'll probably rewrite it using ctypes.windll.kernel32 and drop pywin32. This is how I did the win32 calls in prompt_toolkit, and that work fine.

[prabirshrestha]

@jonathanslenders any updates? Would like to give it a try again.

[AstraLuma]

xonsh/slug doesn't do overlapped IO, it spawns a thread for every piece of plumbing.

[jonathanslenders]

Hi all, I managed to create a pure Python win32 pipe server/client using overlapped I/O. Hopefully, I have some time next weekend to complete the integration.

[jonathanslenders]

Hi all, Windows support is improving.

The client/server model using Windows pipes and overlapped I/O seems to work fine. I can have two clients connected to the same server.
There are still multiple bugs:

• Detaching needs to be fixed (shouldn't be too hard).
• The Windows console Window seems to crash regularly. Very often after detaching the pymux client. Weird is that running "echo hello" runs fine after pymux is detached, but "ls" crashes the cmd window. I don't understand this part.

Spawning the server in the background still needs to be done (using the DETACHED flag for Popen).
Right now, run pymux start-server in one window and pymux attach -S \\.\pipe\pymux.sock.jonathan.42 in another window. (The pipe name is still hardcoded too.)

@prabirshrestha: the unbind bug has been fixed.

If you seriously want to try it on Windows, I suggest to use pymux standalone for now, which doesn't use the client/server model.

Finally, I'm using yawinpty for the Python Winpty bindings. However this repository disappeared very recently... https://github.com/PSoWin/yawinpty

Also worth noting that Microsoft is working hard on improving the Windows Console lately. I think there's a good chance that we don't need this winpty anymore soon.

Cheers,
Jonathan

[prabirshrestha]

Great improvements.

Here are some bugs I noticed when playing around.

1. If I run in admin mode I see some random text - 0;Administrator:
   

2. If I resize with panes, redraw is a bit slow. Takes almost 3-5sec for redraw.

3. Panes render correctly running standalone, if attached it renders weird characters.

Standalone:

Attached to existing server:

4. Mouse scroll should scroll the buffer. Useful for reading build logs which usually spans multiple pages. This one is important.
   Seems like mouse doesn't general work. Other nice to have would be to allow changing window/pane focus based of mouse. Resizing pane by mouse.

Might be good to set the default pipe name to be \\.\pipe\pymux\default? So pymux would auto create the pipe if not exist

Since mostly only devs would use pymux I'm totally ok only supporting the latest windows without winpty. Not sure about other devs.