Extra character in watch colour mode on Ubuntu 17.10

[Rizhiy]

When I use command watch --color -n1.0 gpustat --color I get a lot of extra ^: https://imgur.com/a/A9Fxc

This problem doesn't occur without watch.
I'm on Ubuntu 17.10 with wayland.

[wookayin]

Which version of watch are you using? Or, what can be an output of watch gpustat --color (without --color option on the watch)? What is the value of $TERM?

[emberquill]

I have the same issue. This is what I get with `watch gpustat --color

I'm also on Ubuntu 17.10, using the watch utility provided by procps-ng 3.3.12

Edit: I took a look at it and as far as I can tell, the issue has something to do with term.normal. Every time it's used in the output, it has an extra \x0f character at the end.

watch --color "gpustat --color | sed 's/\x0f//g'" works as a workaround. I don't know much about the blessings package so I can't really help beyond that.

[Stonesjtu]

I'll look into the bug. Working on setting a test environment on docker.

[Stonesjtu]

I tested as the following and found that the new watch may not escape characters as my terminal.
Because my test computer has no GPUs, test_gpustat.py is used here

 # set force_color=True for gpustat --color
python test_gpustat.py > fc.log
cat fc.log # displays as normal
watch --color -n 1 cat fc.log # many ^ tails

There's a great PR implementing watch/top like monitoring function, which may solve the problems caused by watch.

The original output log file is attached fc.log on Ubuntu 17.10.

[wookayin]

For other environments such as macOS (my local), I can't reproduce the issue with the same version of watch procps-ng 3.3.12. (it could be watch 3.3.12 for LINUX only has the bug) I suspect printed the escape sequence for resetting color might be different.

The attachment is my output fc.log (in macOS), generated using your instructions. We can see the resetting sequence is ^[[0;10m, e.g. ^[[34mGeForce GTX TITAN 0^[[0;10m, while @emberquill's output reads ^[34mGeForce GTX 1070^[m^.

@Stonesjtu Can you please upload your fc.log file for comparison?

If so, I suspect blessings (ncurses) is in charge of this. Can you verify this?

>>> from blessings import Terminal
>>> term = Terminal(kind='linux', force_styling=True)
>>> term.normal
'\x1b[0;10m'

[Stonesjtu]

The output on mac renders as normal without extra ^ in Ubuntu 17.10. And yes the term.normal differs unexpectedly.

In [1]: from blessings import Terminal

In [2]: term = Terminal(kind='linux', force_styling=True)

In [3]: term.normal
Out[3]: '\x1b[m\x0f'

And on my PC (Ubuntu 16.04), the term.normal returns correct sequence.

In [3]: term.normal
Out[3]: u'\x1b[0;10m'

[wookayin]

Right. It is from curses.tigetstr('sgr0'), which can be \x1b[0;10m or \x1b[m\x0f or \x1b(B\x1b[m, depending on TERM. The constants are defined in terminfo databases, which might differ in Ubuntu 17.10. Technically speaking, all sequences are correct to reset attributes (if TERM matches).

A problem is that watch is not smart enough to correctly handle all these ANSI or VT-100 escape sequences. Only workaround we can do is just strip those characters that are unknown to watch, just for compatibility.

One easiest way is simply truncate additional control characters (\x0f, or (Bx1b[m), but let me think more if there's more elegant way.

[wookayin]

I've come up with a good workaround. Can you please try it out by installing the latest one from master branch?

Still, we would need a test case and travis setup with Ubuntu 17.10.

[Rizhiy]

I no longer have this issue when using the latest version of the master branch.

[wookayin]

Great, thanks! But let me keep this issue open for a while until I add a test/travis setup :)

[mjmikulski]

Hi!

First of all, thank you for a great tool!

I have the same issue in version 0.4.1 in Ubuntu 16.04. As I understand, this issue was fixed before 0.4.1, am I wrong?

Have a look on the screenshot, a lot of dashes:

OS: Ubuntu 16.04.4 LTS

# gpustat --version
gpustat 0.4.1
# watch --version
watch from procps-ng 3.3.10

Intriguingly, on another, very similar machine, there is no dashes. The only difference is the ubuntu version: 16.04.3 LTS.

Any comments?

[Stonesjtu]

it's time to merge the builtin top watch support to workaround strange bugs caused by watch.

[wookayin]

I have reverted 8d3f917, because with the xterm-color term the color becomes weird in watch. To have the original issue not happening again, I forced the use of (correct) escape sequence.

@mjmikulski @Stonesjtu Would you be available to verify it again on your environments? (with master branch, 132e99a)

[hak8or]

Due to this being reverted, is there consideration for a workaround to add a flag which does not show the first line? That way, using a simple while loop would work well enough. I created an issue (#43) for this.

Edit: My suggestion is already in mainline. For anyone else seeing this from google, if there isn't a new release then get a mainline version of this via pip install git+https://github.com/wookayin/gpustat.git@master.

[wookayin]

This is now released as 0.5.0. Sorry for making it a bit late.