New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect colours on Windows #260

Closed
Aankhen opened this Issue Sep 2, 2018 · 17 comments

Comments

Projects
None yet
3 participants
@Aankhen
Contributor

Aankhen commented Sep 2, 2018

(copied from my comment in #107)

I installed bat on a recent Windows 10 Pro x64 build today and tried it in Cmder, the Cmder installation of ConEmu, PowerShell, and the regular Command Prompt. The first two gave me this:

White-on-grey text everywhere except the filename

PowerShell gave me this (presumably just different default terminal colours; please excuse the crude black censor bar):

White-on-pink text everywhere except the filename

Command Prompt gave me this, like the first two:

White-on-grey text everywhere except the filename

bat looks really exciting and I’m eager to use it, so I hope this is just a misconfiguration on my end! FWIW, I get the same results without a BAT_THEME and with a BAT_THEME of Monokai Extended Bright.

@sharkdp sharkdp added the windows label Sep 3, 2018

@sharkdp

This comment has been minimized.

Owner

sharkdp commented Sep 3, 2018

Thank you for the bug report!

This does not look like the same error as #252 because you actually get the styling (grid, line numbers, etc.) as well as some colors (although incorrect ones).

Something seems wrong with ANSI code handling on your terminals. Can you try to use --color=never to see if that produces a correct (colorless) output? Could you also try to switch off the pager (--paging=never) to make sure that nothing else is interfering?

@sharkdp sharkdp added the help wanted label Sep 3, 2018

@Aankhen

This comment has been minimized.

Contributor

Aankhen commented Sep 4, 2018

Sure thing. Here’s --color=never:

Correct uncoloured output

And here’s --paging=never:

Correct coloured output without paging

Looks like that’s indeed where the issue is, so I did some digging…

The less being used was from Git Windows. I also have the one from UnxUtils, so I temporarily removed the Git one. Now I get ‘invalid charset name’, because, as it turns out, the UnxUtils version of less only supports a handful of encodings. I can get it work in general by unsetting LESSCHARSET, but bat explicitly sets it:

p.args(&args).env("LESSCHARSET", "UTF-8");

I tried removing it, but apparently this less won’t handle colour codes, and nor will a newer version I found:

Visible escape codes in output

FWIW, Git’s version in Git Bash gives the single-colour result from my original post, just as it does everywhere else, so that wouldn’t appear to be the problem.

At any rate, it looks like this is an issue with pagers on Windows, not with bat. Sorry for the noise.

@sharkdp

This comment has been minimized.

Owner

sharkdp commented Sep 4, 2018

Ok, thank you for the detailed analysis!

Maybe we should think about disabling automatic paging by default. Users could opt in by setting BAT_PAGER=less or maybe by setting BAT_PAGING=auto.

@Aankhen

This comment has been minimized.

Contributor

Aankhen commented Sep 4, 2018

Maybe, although I really wish I could figure out a way to have my paging and eat it too. Alternatively, paging could be opt-in only on Windows.

@sharkdp

This comment has been minimized.

Owner

sharkdp commented Sep 5, 2018

If you find a pager that can handle ANSI codes, you can just set the BAT_PAGER environment variable (see README).

I can get it work in general by unsetting LESSCHARSET, but bat explicitly sets it:

You can use a separate script (as also suggested in the README) to set specific environment variables like LESSCHARSET and then call less from the script. bat will only set this environment variable if $BAT_PAGER == less or when no other pager is specified.

@Aankhen

This comment has been minimized.

Contributor

Aankhen commented Sep 5, 2018

I tried it without LESSCHARSET; that’s when I discovered that the three less binaries I tried couldn’t handle the ANSI codes. I haven’t yet found a pager for Windows that can, unfortunately. I’l keep looking, though.

@nickcox

This comment has been minimized.

nickcox commented Sep 5, 2018

Colours work for me after installing the version found on Chocolatey.

PS> Get-Command less

CommandType     Name                                               Version    Source
-----------     ----                                               -------    ------
Application     less.exe                                           0.0.0.0    C:\Program Files\Git\usr\bin\less.exe

PS> less --version
less 529 (PCRE regular expressions)
Copyright (C) 1984-2017  Mark Nudelman

less comes with NO WARRANTY, to the extent permitted by law.
For information about the terms of redistribution,
see the file named README in the less distribution.
Homepage: http://www.greenwoodsoftware.com/less

PS> bat .\readme.md
# no colours...
PS> choco install less
PS> Get-Command less

CommandType     Name                                               Version    Source
-----------     ----                                               -------    ------
Application     less.exe                                           0.8.1.0    C:\ProgramData\chocolatey\bin\less.exe

PS> less --version
less 529 (Spencer V8 regular expressions)
Copyright (C) 1984-2017  Mark Nudelman

less comes with NO WARRANTY, to the extent permitted by law.
For information about the terms of redistribution,
see the file named README in the less distribution.
Homepage: http://www.greenwoodsoftware.com/less

PS> bat .\readme.md
# all the colours
@Aankhen

This comment has been minimized.

Contributor

Aankhen commented Sep 6, 2018

Strange… using less 5.30 from the same source as Chocolatey or less 5.29 from Chocolatey itself in PowerShell, Command Prompt, or ConEmu, I see garbled codes instead of colours. I must have something misconfigured.

@sharkdp

This comment has been minimized.

Owner

sharkdp commented Sep 6, 2018

Make sure that you use the -R option of less (either by wrapping it in a script or by setting the environment variable LESS="-R")

@nickcox

This comment has been minimized.

nickcox commented Sep 6, 2018

FWIW, I didn't have to set any options or environment variables to get this to work. (I guess bat does that for me automatically?) Just:

bat ./readme.md

image

I do have HKEY_CURRENT_USER\Console\VirtualTerminalLevel=1. Not sure if that's important for conhost.exe?

@Aankhen

This comment has been minimized.

Contributor

Aankhen commented Sep 6, 2018

Yup, I’ve got that set too. What version of Windows are you running? I’m on Windows 10 Pro x64.

EDIT:
I’ve found some interesting things. An answer on Stack Overflow pointed out that you can set the LESS environment variable to include -r to always allow colours in less output. This led me down an interesting rabbit hole.

  1. Without any changes, I get the white-on-grey text from my very first screenshot.

  2. Then I set BAT_PAGER to the downloaded less 5.30. I get garbled ANSI codes like in my last screenshot. No change so far.

  3. I set LESS=-Xr. Now I see something almost correct:

    Terminal with correct colouring of text but encoding errors instead of borders in UI

    This was so close!

  4. I looked for something else I could run to test colourized output. Fortunately, another answer in that SO thread mentioned jq -C, so I gave that a whirl. It works fine, but it doesn’t involve any borders:

    Correct colourized output from jq

  5. On an impulse, I tried bat again. Lo and behold:

    bat with correct output

I can reliably reproduce this: if I set the environment variables and run bat, I get the right colours but the borders are messed up. If I run jq -C once, then run bat any number of times, I get the right output. Maybe jq -C prints something that makes the terminal behave itself.

@sharkdp

This comment has been minimized.

Owner

sharkdp commented Sep 6, 2018

bat should set the right flags (you need -R, not -r) if either

  1. PAGER and BAT_PAGER are unset or
  2. BAT_PAGER is unset and PAGER == "less"
  3. BAT_PAGER == "less"
@Aankhen

This comment has been minimized.

Contributor

Aankhen commented Sep 6, 2018

Ah, I see. I have to specify the full path to less since it’s not the default one, so in my case, BAT_PAGER != "less". I just tried swapping the order so that the default one is correct, and it solved everything—no environment variables required! I do still see the issue with encoding if I set BAT_PAGER to the full path and set LESS to include either -r or -R, if that’s something you want to look into, but it seems like my issue is solved. Thank you very much for following up on this.

EDIT:
It just needed LESSCHARSET to be set to utf-8, which bat does if the executable is less or ends with /less, which is not the case with an absolute path or a filename with extension on Windows.

@sharkdp

This comment has been minimized.

Owner

sharkdp commented Sep 7, 2018

@Aankhen So with your PR merged, what's the full story with using less on Windows?

Is there something we should add to the README (Troubleshooting?) or will it work out of the box with your fix (and less installed)?

@Aankhen

This comment has been minimized.

Contributor

Aankhen commented Sep 7, 2018

Git’s less (which I believe is MSYS’s less) won’t be able to handle colours—you’ll get the white-on-grey from my first screenshot—and I can’t find a way to make it work, so maybe the README should mention that. Other than that, it should indeed work out of the box! We could mention that if the user doesn’t have a pager installed, they can get a working less either through Chocolatey or from the author directly at http://www.greenwoodsoftware.com/less/.

@sharkdp sharkdp added documentation and removed help wanted labels Sep 7, 2018

@sharkdp

This comment has been minimized.

Owner

sharkdp commented Sep 7, 2018

Thanks! Let's add a "Using a pager on Windows" section (or similar) to the Troubleshooting section with both of your points.

@Aankhen

This comment has been minimized.

Contributor

Aankhen commented Sep 8, 2018

Done, #295.

@sharkdp sharkdp closed this Sep 12, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment