Add Windows support for raw_display #448

mhils · 2021-01-04T14:16:09Z

Checklist

I've ensured that similar functionality has not already been implemented
I've ensured that similar functionality has not earlier been proposed and declined
I've branched off the master or python-dual-support branch
I've merged fresh upstream into my branch recently
I've ran tox successfully in local environment (yes, but I do not have 2.7, 3.5, and pypy at hand)
I've included docstrings and/or documentation and/or examples for my code (if this is a new feature)

Description:

Inspired by @asmith-kepler's fantastic work in #447, I set out to implement Windows support for raw_display. Turns out that this is all working pretty seamlessly with the latest Windows console improvements. 🎉

Main changes:

Use socket.socketpair() instead of os.pipe() on all OSes as Windows doesn't support non-socket file descriptors in select(). This probably has a tiny amount of overhead, but works across all platforms.
Use ctypes to interface with the Windows API, which works without any additional dependencies.
Use a dedicated thread to read input from the Windows console and pipe it into another socketpair, which is then processed by raw_display. This is slightly ugly, but works across all events loops.

coveralls · 2021-01-04T14:23:54Z

Coverage decreased (-0.8%) to 77.084% when pulling f4fd669 on mhils:win32-rawdisplay into 4c739b6 on urwid:master.

penguinolog · 2023-03-31T07:42:30Z

urwid/raw_display.py

            self._old_termios_settings = termios.tcgetattr(fd)
            tty.setcbreak(fd)

-        self.signal_init()
+        if IS_WINDOWS:
+            hOut = win32.GetStdHandle(win32.STD_OUTPUT_HANDLE)


will be hood if such long branches will be in own methods like:

if IS_WINDOWS: self.windows_signal_init() else: self.signal_init()

penguinolog · 2023-03-31T07:43:16Z

urwid/raw_display.py

@@ -274,6 +310,14 @@ def _stop(self):
        if self._old_signal_keys:
            self.tty_signal_keys(*(self._old_signal_keys + (fd,)))

+        if IS_WINDOWS:
+            hOut = win32.GetStdHandle(win32.STD_OUTPUT_HANDLE)


also here windows_stop will be good method

penguinolog · 2023-03-31T07:43:46Z

urwid/raw_display.py

@@ -395,6 +439,10 @@ def unhook_event_loop(self, event_loop):
        """
        Remove any hooks added by hook_event_loop.
        """
+        if self._input_thread is not None:
+            self._input_thread.should_exit = True
+            self._input_thread = None


no wait / join?

penguinolog · 2023-03-31T07:45:10Z

urwid/raw_display.py

-                buf = fcntl.ioctl(self._term_output_file.fileno(),
-                                termios.TIOCGWINSZ, ' '*4)
-                y, x = struct.unpack('hh', buf)
+                if IS_WINDOWS:


better to make dedicated methods for both branches

penguinolog

please split logic to reduce platform specific if/else branches size

Restore PR urwid#448 with pick of urwid#447 for `tty_signal_keys`

Restore PR urwid#448 with pick of urwid#447 for `tty_signal_keys` * Vterm is fully not supported under windows * Do not import ZMQEventLoop globally due to python crash * rework test for event loops to be windows compatible: on windows `select` do not support pipe Skip tests: * TwistedEventLoop is not supported under windows * ZMQEventLoop is not supported under windows

Restore PR urwid#448 with pick of urwid#447 for `tty_signal_keys` * Vterm is fully not supported under windows * Do not import ZMQEventLoop on windows due to python crash * rework test for event loops to be windows compatible: on windows `select` do not support pipe Skip tests: * ZMQEventLoop is not supported under windows

Restore PR urwid#448 with pick of urwid#447 for `tty_signal_keys` * Vterm is fully not supported under windows * Do not import ZMQEventLoop on windows due to python crash * rework test for event loops to be windows compatible: on windows `select` do not support pipe Skip tests: * ZMQEventLoop is not supported under windows * `AsyncioEventLoop`, `TornadoEventLoop` and `TwistedEventLoop`: * Issues with asyncio loop (changes between python version, select support) * Twisted and tornado tests show race conditions

* Vterm is fully not supported under windows * Do not import ZMQEventLoop on windows due to python crash * rework test for event loops to be windows compatible: on windows `select` do not support pipe * on windows11 console encoding is detected as cp1252, BTW it's validated manually that cms uses UTF-8 * fix 24 bit colors terminal Skip tests: * ZMQEventLoop is not supported under windows * `AsyncioEventLoop`, `TornadoEventLoop` and `TwistedEventLoop`: * Issues with asyncio loop (changes between python version, select support) * Twisted and tornado tests show race conditions

* Restore PR #448 with pick of #447 for `tty_signal_keys` * Vterm is fully not supported under windows * Do not import ZMQEventLoop on windows due to python crash * rework test for event loops to be windows compatible: on windows `select` do not support pipe * on windows11 console encoding is detected as cp1252, BTW it's validated manually that cms uses UTF-8 * fix 24 bit colors terminal Skip tests: * ZMQEventLoop is not supported under windows * `AsyncioEventLoop`, `TornadoEventLoop` and `TwistedEventLoop`: * Issues with asyncio loop (changes between python version, select support) * Twisted and tornado tests show race conditions * Make shared base class for raw display * Reduce amount of copy-paste * Base class is abstract (too huge difference in concrete implementations) * use `selectors.DefaultSelector` instead of `select.select` Automatically select OS specific optimal route by python stdlib.

penguinolog · 2023-12-08T11:31:51Z

Restored updated, tested and merged

mhils · 2023-12-08T18:24:11Z

I totally missed the comments in March, my apologies. Thanks for taking care of things, this is super duper fantastic! 😃 🍰 🚀

mhils force-pushed the win32-rawdisplay branch from 1563e39 to 2f15904 Compare January 4, 2021 17:40

mhils added a commit to mhils/mitmproxy that referenced this pull request Jan 4, 2021

vendor urwid/urwid#448

168bccf

mhils mentioned this pull request Jan 4, 2021

mitmproxy.exe 🎉🥳 mitmproxy/mitmproxy#4384

Merged

2 tasks

raw_display: add Windows support

f4fd669

mhils force-pushed the win32-rawdisplay branch from 2f15904 to f4fd669 Compare January 5, 2021 06:46

mhils mentioned this pull request Jan 5, 2022

Implement Windows Driver Textualize/textual#14

Closed

penguinolog reviewed Mar 31, 2023

View reviewed changes

penguinolog requested changes Mar 31, 2023

View reviewed changes

penguinolog added the Feature Feature request/implementation label May 1, 2023

daviewales mentioned this pull request Jul 31, 2023

Cross-platform inducer/pudb#158

Open

penguinolog added the Waiting for response label Sep 22, 2023

penguinolog added the Windows Windows support label Oct 2, 2023

penguinolog added a commit to penguinolog/urwid that referenced this pull request Nov 27, 2023

Implement basic win32 support in RawDisplay

0cf9161

Restore PR urwid#448 with pick of urwid#447 for `tty_signal_keys`

penguinolog added a commit to penguinolog/urwid that referenced this pull request Nov 27, 2023

Implement basic win32 support in RawDisplay

cd83f7a

Restore PR urwid#448 with pick of urwid#447 for `tty_signal_keys`

penguinolog mentioned this pull request Nov 27, 2023

Implement basic win32 support in RawDisplay #681

Closed

6 tasks

penguinolog added a commit to penguinolog/urwid that referenced this pull request Nov 27, 2023

Implement basic win32 support in RawDisplay

0a7b4d9

Restore PR urwid#448 with pick of urwid#447 for `tty_signal_keys`

penguinolog closed this Dec 8, 2023

penguinolog mentioned this pull request Dec 9, 2023

Fix windows encoding detection: encoding depends on python build options #693

Merged

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Windows support for raw_display #448

Add Windows support for raw_display #448

mhils commented Jan 4, 2021

coveralls commented Jan 4, 2021 •

edited

Loading

penguinolog Mar 31, 2023

penguinolog Mar 31, 2023

penguinolog Mar 31, 2023

penguinolog Mar 31, 2023

penguinolog left a comment

penguinolog commented Dec 8, 2023

mhils commented Dec 8, 2023

Add Windows support for raw_display #448

Add Windows support for raw_display #448

Conversation

mhils commented Jan 4, 2021

Checklist

Description:

coveralls commented Jan 4, 2021 • edited Loading

penguinolog Mar 31, 2023

Choose a reason for hiding this comment

penguinolog Mar 31, 2023

Choose a reason for hiding this comment

penguinolog Mar 31, 2023

Choose a reason for hiding this comment

penguinolog Mar 31, 2023

Choose a reason for hiding this comment

penguinolog left a comment

Choose a reason for hiding this comment

penguinolog commented Dec 8, 2023

mhils commented Dec 8, 2023

coveralls commented Jan 4, 2021 •

edited

Loading