enlighten for Windows

[toshiro92]

Hi !

This will probably be more an information than an issue: On Windows, the module does not working well. I've tried the following default below code:

import time
import enlighten

manager = enlighten.get_manager()
ticks = manager.counter(total=100, desc='Ticks', unit='ticks')
tocks = manager.counter(total=20, desc='Tocks', unit='tocks')

for num in range(100):
    time.sleep(0.1)  # Simulate work
    print(num)
    ticks.update()
    if not num % 5:
        tocks.update()

manager.stop()

And it looks like it does not working on Windows:

I also had the following error message:

C:....\lib\site-packages\blessed\terminal.py:32:
UserWarning: One or more of the modules: 'termios', 'fcntl', and 'tty' are not found on your platform 'win32'. The following methods of Terminal are dummy/no-op unless a deriving class overrides them: setraw, cbreak, kbhit, height, width
warnings.warn(_MSG_NOSUPPORT)

I'm not sure if termios, fcntl and tty are mandatory, however termios is only available on Linux...

[avylove]

This is already noted in the FAQ. I have some proof of concept code, but I haven't prioritized the work because there hasn't been a lot of demand. So Windows support is coming, but I'm not sure when.

[avylove]

@toshiro92 I've added support for Windows 10 in the windows10 branch. Please take a look and see if it meets your needs.

I'm looking at other options for earlier versions because there doesn't seem to be a Windows system call for setting the scroll window on the console. If there is one, I could probably put something together, but right now it looks like earlier versions of Windows will need to run on an alternate console. Maybe something like ansicon.

The version in this branch will also raise a NotImplementedError on import if running on an older version of Windows. I'm not sure that's ideal. What do you think?

[avylove]

It wasn't much more work to get it to work with ANSICON, so now that works. ANSICON just has to be enabled for the command window before Enlighten is imported.

[toshiro92]

Hi @avylove, I've just tried the branch on Win10, and it worked well :)
For the ANSICON, maybe you can add it on the README file for Windows 10 users ?

Anyway thank you for this :-)

[monkeyonagazeboo]

Hi this util looks really great.
Unfortunately on my windows machine after

pbar = enlighten.Counter(total=100, desc='Basic', unit='ticks')
i'm getting following error:
"OSError: [WinError 6] The handle is invalid. "

Do you have any clue what may be the problem?
Thank you

[avylove]

@monkeyonagazeboo not off the top of my head, but I have a few questions.

1. What version of Windows are you using?
2. Are you using the windows10 branch?
3. Do you get the same error if you do?

   manager = enlighten.get_manager
   pbar = manager.counter(total=100, desc='Basic', unit='ticks')

[monkeyonagazeboo]

@monkeyonagazeboo not off the top of my head, but I have a few questions.

1. What version of Windows are you using?
2. Are you using the windows10 branch?
3. Do you get the same error if you do?

   manager = enlighten.get_manager
   pbar = manager.counter(total=100, desc='Basic', unit='ticks')

Hi, thx for quick respose.
I'm using win 10 64bit version. I checked out the windows branch & python setup.py install 'ed it.
FYI this is the full output:

>>> import enlighten
>>> pbar = enlighten.Counter(total=embsnp.shape[0]//step, desc='Basic', unit='ticks')
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\enlighten-1.2.0-py3.6.egg\enlighten\counter.py", line 31, in __init__
    counter_class=self.__class__, set_scroll=False)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\enlighten-1.2.0-py3.6.egg\enlighten\_manager.py", line 78, in __init__
    self.term = Terminal(stream=self.stream)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\enlighten-1.2.0-py3.6.egg\enlighten\_terminal.py", line 41, in __init__
    super(Terminal, self).__init__(*args, **kwargs)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\enlighten-1.2.0-py3.6.egg\enlighten\_win_terminal.py", line 142, in __init__
    enable_vt_mode(self.stream_fh)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\enlighten-1.2.0-py3.6.egg\enlighten\_win_terminal.py", line 108, in enable_vt_mode
    KERNEL32.GetConsoleMode(filehandle, ctypes.byref(current_mode))
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\enlighten-1.2.0-py3.6.egg\enlighten\_win_terminal.py", line 70, in _check_bool
    raise ctypes.WinError(ctypes.get_last_error())
OSError: [WinError 6] The handle is invalid.
>>> manager = enlighten.get_manager
>>> pbar = manager.counter(total=100, desc='Basic', unit='ticks')
Traceback (most recent call last):
  File "<input>", line 2, in <module>
AttributeError: 'function' object has no attribute 'counter'

[avylove]

That was my fault, I left off the parenthesis. Should be:

manager = enlighten.get_manager()
pbar = manager.counter(total=100, desc='Basic', unit='ticks')

Are you running this in a normal console or under visual studio? Basically it looks like it's looking up the file handle for stdout and that file handle is invalid.

Try the above and also try

import sys
enlighten.Counter(total=100, desc='Basic', unit='ticks', stream=sys.__stdout__)

[monkeyonagazeboo]

That was my fault, I left off the parenthesis. Should be:

manager = enlighten.get_manager()
pbar = manager.counter(total=100, desc='Basic', unit='ticks')

Are you running this in a normal console or under visual studio? Basically it looks like it's looking up the file handle for stdout and that file handle is invalid.

Try the above and also try

import sys
enlighten.Counter(total=100, desc='Basic', unit='ticks', stream=sys.__stdout__)

1. And mine :o. I was hasty. I figured that later, but the result is the same error (invalid handle).

import sys
enlighten.Counter(total=100, desc='Basic', unit='ticks', stream=sys.__stdout__)
Traceback (most recent call last):
  File "<input>", line 2, in <module>
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\enlighten-1.2.0-py3.6.egg\enlighten\counter.py", line 31, in __init__
    counter_class=self.__class__, set_scroll=False)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\enlighten-1.2.0-py3.6.egg\enlighten\_manager.py", line 78, in __init__
    self.term = Terminal(stream=self.stream)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\enlighten-1.2.0-py3.6.egg\enlighten\_terminal.py", line 41, in __init__
    super(Terminal, self).__init__(*args, **kwargs)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\enlighten-1.2.0-py3.6.egg\enlighten\_win_terminal.py", line 142, in __init__
    enable_vt_mode(self.stream_fh)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\enlighten-1.2.0-py3.6.egg\enlighten\_win_terminal.py", line 108, in enable_vt_mode
    KERNEL32.GetConsoleMode(filehandle, ctypes.byref(current_mode))
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\enlighten-1.2.0-py3.6.egg\enlighten\_win_terminal.py", line 70, in _check_bool
    raise ctypes.WinError(ctypes.get_last_error())
OSError: [WinError 6] The handle is invalid.

[avylove]

Are you running this in a normal console or under visual studio?

Does this behave the same?

import sys
enlighten.Counter(total=100, desc='Basic', unit='ticks', stream=sys.__stdout__)

[monkeyonagazeboo]

Are you running this in a normal console or under visual studio?

Does this behave the same?

import sys
enlighten.Counter(total=100, desc='Basic', unit='ticks', stream=sys.__stdout__)

Hi

I tried pycharm(my default) console Debug&Relese and also vs2017 python interactive window both resulting in the Error.
But, when I run the python.exe from powershell & test it the Couter works fine!
So the problem is in the environment.

M

[avylove]

My guess is both of these environments are using their own consoles and not using a standard Windows console. I'll have to see if there is a good way to detect that.

[avylove]

In the VS and PyCharm consoles, is TERM listed in os.environ?

[monkeyonagazeboo]

Thank you ... appreciated.

[monkeyonagazeboo]

pycharm console & debug console : NO
VS interactive win :

>>> os.environ['TERM']
'dumb'

[avylove]

Ok, Looks like the branch works fine in the Pycharm and VSCode terminals. It also works fine in the standard Python REPL and the VSCode REPL. But it doesn't work in the Visual Studio REPL or the PYCharm REPL. Both of these do not expose the real STDOUT.

In Visual Studio, stdout is a ptvsd.repl._ReplOutput object. When a file handle is requested, it returns a pipe. It may be possible to detect this is a _ReplOutput object and try using sys.stdout.out_old, but in my initial tests that didn't seem to work.

In PYCharm, it launches pydev\pydevconsole.py for the repl. I believe that works as a client/server pair. The code is a bit complicated and I'm not even sure it will support the terminal codes Enlighten requires.

So since the console terminals all seem to work and the Standard and VSCode REPLs work, I'm going to say these are corner cases without apparent solutions, so I'm not going to fix them for now and maybe someone who is invested in them can send a pull request later on.

[monkeyonagazeboo]

Understood.
Thing is that when I write code in pycharm it is convenient to 'single-key' run/debug in the pycharm console. The workaround would be to catch the exception and leave the code in my project in case someone runs it from terminal in the future.
Otherwise super job. I use logging for every project, therefore unable to have progress bars of any sort until now. Thank you.

[avylove]

@toshiro92 I ended up creating the ansicon package so for older versions of Windows it will "just work". I think I have things cleaned up with the windows branch so unless something comes up in the next day or two, I'll probably release it.

@monkeyonagazeboo If you can get the info needed to support the terminal, I'll happily add it. The FAQ for the next release will have information on what is needed. Here's that entry.

Can you add support for _______ terminal?

We are happy to add support for as many terminals as we can.
However, not all terminals can be supported. There a few requirements.

1. The terminal must be detectable programmatically

   We need to be able to identify the terminal in some reasonable way
   and differentiate it from other terminals. This could be through environment variables,
   the platform module, or some other method.

2. A subset of terminal codes must be supported

   While these codes may vary among terminals, the capability must be
   provided and activated by printing a terminal sequence.
   The required codes are listed below.

   • move / CUP - Cursor Position
   • hide_cursor / DECTCEM - Text Cursor Enable Mode
   • show_cursor / DECTCEM - Text Cursor Enable Mode
   • csr / DECSTBM - Set Top and Bottom Margins
   • clear_eos / ED - Erase in Display
   • clear_eol / EL - Erase in Line
   • feed / CUD - Cursor Down (Or scroll with linefeed)

3. Terminal dimensions must be detectable

   The height and width of the terminal must be available to the running process.

[avylove]

Version 1.3.0 has been released to Github and Pypi with Windows support. I'm going to go ahead and close this. If you run into any problems, please submit a new issue.