Vim theme should avoid 256-color version

[jan-warchol]

Selenized theme for vim is meant to be used with exact selenized color palette (this requires either truecolor-enabled terminal or configuring ANSI color palette of one's terminal). 256-color version is only an inferior approximation.

Unfortunately, when a terminal advertises itself as supporting 256 colors (e.g. when TERM=xterm-256color), Vim will try to use 256-color selenized. It is possible to work around this by setting t_Co=16, but

• it's an ugly hack, theme files shouldn't do things like that
• it doesn't work in neovim anyway

So, the goal is to find a way of making selenized use either true color capabilities of the terminal or ANSI color codes, avoiding 256-color version unless explicitly overridden with some option.

[jan-warchol]

I have tried an approach based on what lifepillar did for solarized8 (see lifepillar/vim-solarized8#71 and lifepillar/vim-solarized8@de124fa) but the results were crazy: most of the colors worked except for bright yellow, which was still using 256-color palette.

[jan-warchol]

See also discussion in #59 starting from this comment.

[jahschwa]

I think I figured out the bright yellow bit after a lot of struggling and digging while attempting to switch to neovim from vim. Namely: neither of them preserve all the builtin color aliases (which selenized currently uses in its highlight definitions) between different t_Co values. If we look here:
https://github.com/vim/vim/blob/8291e91c6b10e0cdeb2f29c8f1a0aad6d5b5c684/src/highlight.c#L521-L567

We end up with (Bright variations are capitalized):

             |          8 |         16 |         88 |        256 | Same
-------------+------------+------------+------------+------------+-----
       Black |    black 0 |    black 0 |    black 0 |    black 0 |  YES
    DarkBlue |     blue 4 |      red 1 |     blue 4 |     blue 4 |  !16
   DarkGreen |    green 2 |    green 2 |    green 2 |    green 2 |  YES
    DarkCyan |     cyan 6 |   yellow 3 |     cyan 6 |     cyan 6 |  !16
-------------+------------+------------+------------+------------+-----
     DarkRed |      red 1 |     blue 4 |      red 1 |      red 1 |  !16
 DarkMagenta |  magenta 5 |  magenta 5 |  magenta 5 |  magenta 5 |  YES
       Brown |   yellow 3 |     cyan 6 |         32 |        130 |
  DarkYellow |   yellow 3 |     cyan 6 |         72 |   yellow 3 |
-------------+------------+------------+------------+------------+-----
        Gray |    white 7 |    white 7 |         84 |        248 |
        Grey |    white 7 |    white 7 |         84 |        248 |
   LightGray |    white 7 |    white 7 |    white 7 |    white 7 |  YES
   LightGrey |    white 7 |    white 7 |    white 7 |    white 7 |  YES
-------------+------------+------------+------------+------------+-----
    DarkGray |    BLACK 8 |    BLACK 8 |         82 |        242 |
    DarkGrey |    BLACK 8 |    BLACK 8 |         82 |        242 |
        Blue |    BLUE 12 |      RED 9 |    BLUE 12 |    BLUE 12 |
   LightBlue |    BLUE 12 |      RED 9 |         43 |         81 |
-------------+------------+------------+------------+------------+-----
       Green |   GREEN 10 |   GREEN 10 |   GREEN 10 |   GREEN 10 |  YES
  LightGreen |   GREEN 10 |   GREEN 10 |         61 |        121 |
        Cyan |    CYAN 14 |  YELLOW 11 |    CYAN 14 |    CYAN 14 |  !16
   LightCyan |    CYAN 14 |  YELLOW 11 |         63 |        159 |
-------------+------------+------------+------------+------------+-----
         Red |      RED 9 |    BLUE 12 |      RED 9 |      RED 9 |  !16
    LightRed |      RED 9 |    BLUE 12 |         74 |        224 |
     Magenta | MAGENTA 13 | MAGENTA 13 | MAGENTA 13 | MAGENTA 13 |  YES
LightMagenta | MAGENTA 13 | MAGENTA 13 |         75 |        225 |
-------------+------------+------------+------------+------------+-----
      Yellow |  YELLOW 11 |    CYAN 14 |  YELLOW 11 |  YELLOW 11 |  !16
 LightYellow |  YELLOW 11 |    CYAN 14 |         78 |        229 |
       White |   WHITE 15 |   WHITE 15 |   WHITE 15 |   WHITE 15 |  YES
        NONE |         -1 |         -1 |         -1 |         -1 |  YES
-------------+------------+------------+------------+------------+-----

To be perfectly honest I have no idea what's up with 16 compared to the other maps... it appears to have swapped blue/red and yellow/cyan??? Is that like a compatibility thing or something? I don't get it.

But anyway, if vim thinks you are running with 256 colors, then LightYellow is going to map to 229. This is doubly devious in neovim, where the value of t_Co isn't available during init as implied in the description of this issue.

So it seems like, if we want consistent behavior with terminal color palette overrides, we should never use any of the Light_ aliases... except LightGray? So maybe it would be better to just hard-code the integers? I wonder if this is more a bug in vim-colortemplate than actually selenized itself...

IDK, I've been staring at this for too many hours. Hopefully gonna come back and attempt some sort of PR next week.

[jahschwa]

This is not directly relevant to the issue; feel free not to read this!

If anyone's curious, I figured out why the 16 color mapping is like that... at various points windows terminals were BGR rather than RGB:
microsoft/terminal#571 (comment)

So color_numbers_16 contains the weird non-ANSI BGR mapping. Then vim has an if statement to try and detect if the terminal actually is ANSI and uses the color_numbers_8 mapping instead:
https://github.com/vim/vim/blob/f6272551bdae3f265b6948a4155b079c37fe110f/src/highlight.c#L609-L625

Whereas neovim just always forces the ANSI (RGB) mapping:
https://github.com/neovim/neovim/blob/08986bb5972d5289b5ae581eee20c56387cf5ddd/src/nvim/highlight_group.c#L860-L861