xterm-direct is not carried over from its bash shell into gvim's `term`

[egberts]

Updated: Used this repo, commit 7b293c7 master, tag: v8.2.0539

Using xterm-direct (16777216 colors), I am attempting to add support for more concise coloring (at RGB scale), but it doesn't quite take on the gvim yet.

Looks like t_Co variable works only within a console-based vim, and not in a GUI-based gvim.

VIM xterm-direct (console-mode)

To see the total color available in vim, I executed:

TERM=xterm-direct vim --clean

Then typed in the following vimscript:

:verbose set term? t_Co? termguicolors?
  term=xterm-direct
  t_Co=16777216
notermguicolors

GVIM xterm-direct (GUI-mode)

But t_Co returns an empty value under GUI-based gvim, when I executed:

TERM=xterm-direct gvim --clean

Then typed in the following vimscript:

:verbose set term? t_Co? termguicolors?
  term=builtin_gui
  t_Co=
notermguicolors

Describe the solution you'd like

Updated: Perhaps add GUI support to term to correctly reflect xterm-direct.
Perhaps add support to t_Co variable name while in gvim GUI-mode?

Describe alternatives you've considered

Perhaps, there is a better vimscript keyword alternative to the terminal resolution of 2, 8, 16, and 256 that vim supports (that I am not aware of).

[egberts]

Ooops, and I'm running

• Vim 8.1, on
• updated Debian 11,
• FreeDesktop Xorg X11 1.6.7-1,
• LXDE Display Manager 0.5.3-2.1,

uname -a
Linux arca 4.19.0-8-amd64 #1 SMP Debian 4.19.98-1 (2020-01-26) x86_64 GNU/Linux

VIM - Vi IMproved 8.1 (2018 May 18, compiled Jun 15 2019 16:41:15)
Included patches: 1-875, 878, 884, 948, 1046, 1365-1368, 1382, 1401
Modified by team+vim@tracker.debian.org
Compiled by team+vim@tracker.debian.org
Huge version with GTK3 GUI.  Features included (+) or not (-):
+acl               +extra_search      +mouse_netterm     +tag_old_static
+arabic            +farsi             +mouse_sgr         -tag_any_white
+autocmd           +file_in_path      -mouse_sysmouse    +tcl
+autochdir         +find_in_path      +mouse_urxvt       +termguicolors
-autoservername    +float             +mouse_xterm       +terminal
+balloon_eval      +folding           +multi_byte        +terminfo
+balloon_eval_term -footer            +multi_lang        +termresponse
+browse            +fork()            -mzscheme          +textobjects
++builtin_terms    +gettext           +netbeans_intg     +textprop
+byte_offset       -hangul_input      +num64             +timers
+channel           +iconv             +packages          +title
+cindent           +insert_expand     +path_extra        +toolbar
+clientserver      +job               +perl              +user_commands
+clipboard         +jumplist          +persistent_undo   +vartabs
+cmdline_compl     +keymap            +postscript        +vertsplit
+cmdline_hist      +lambda            +printer           +virtualedit
+cmdline_info      +langmap           +profile           +visual
+comments          +libcall           -python            +visualextra
+conceal           +linebreak         +python3           +viminfo
+cryptv            +lispindent        +quickfix          +vreplace
+cscope            +listcmds          +reltime           +wildignore
+cursorbind        +localmap          +rightleft         +wildmenu
+cursorshape       +lua               +ruby              +windows
+dialog_con_gui    +menu              +scrollbind        +writebackup
+diff              +mksession         +signs             +X11
+digraphs          +modify_fname      +smartindent       -xfontset
+dnd               +mouse             +startuptime       +xim
-ebcdic            +mouseshape        +statusline        +xpm
+emacs_tags        +mouse_dec         -sun_workshop      +xsmp_interact
+eval              +mouse_gpm         +syntax            +xterm_clipboard
+ex_extra          -mouse_jsbterm     +tag_binary        -xterm_save
   system vimrc file: "$VIM/vimrc"
     user vimrc file: "$HOME/.vimrc"
 2nd user vimrc file: "~/.vim/vimrc"
      user exrc file: "$HOME/.exrc"
  system gvimrc file: "$VIM/gvimrc"
    user gvimrc file: "$HOME/.gvimrc"
2nd user gvimrc file: "~/.vim/gvimrc"
       defaults file: "$VIMRUNTIME/defaults.vim"
    system menu file: "$VIMRUNTIME/menu.vim"
  fall-back for $VIM: "/usr/share/vim"
Compilation: gcc -c -I. -Iproto -DHAVE_CONFIG_H -DFEAT_GUI_GTK  -pthread -I/usr/include/gtk-3.0 -I/usr/include/at-spi2-atk/2.0 -I/usr/include/at-spi-2.0 -I/usr/include/dbus-1.0 -I/usr/lib/x86_64-linux-gnu/dbus-1.0/include -I/usr/include/gtk-3.0 -I/usr/include/gio-unix-2.0 -I/usr/include/cairo -I/usr/include/libdrm -I/usr/include/pango-1.0 -I/usr/include/harfbuzz -I/usr/include/pango-1.0 -I/usr/include/fribidi -I/usr/include/atk-1.0 -I/usr/include/cairo -I/usr/include/pixman-1 -I/usr/include/freetype2 -I/usr/include/libpng16 -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/libmount -I/usr/include/blkid -I/usr/include/uuid -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -Wdate-time  -g -O2 -fdebug-prefix-map=/build/vim-4Pursk/vim-8.1.0875=. -fstack-protector-strong -Wformat -Werror=format-security -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1       
Linking: gcc   -L. -Wl,-z,relro -Wl,-z,now -fstack-protector -rdynamic -Wl,-export-dynamic -Wl,-E  -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -o vim   -lgtk-3 -lgdk-3 -lpangocairo-1.0 -lpango-1.0 -latk-1.0 -lcairo-gobject -lcairo -lgdk_pixbuf-2.0 -lgio-2.0 -lgobject-2.0 -lglib-2.0 -lSM -lICE -lXpm -lXt -lX11 -lXdmcp -lSM -lICE  -lm -ltinfo -lnsl  -lselinux  -lacl -lattr -lgpm -ldl  -L/usr/lib -llua5.2 -Wl,-E  -fstack-protector-strong -L/usr/local/lib  -L/usr/lib/x86_64-linux-gnu/perl/5.28/CORE -lperl -ldl -lm -lpthread -lcrypt  -L/usr/lib/python3.7/config-3.7m-x86_64-linux-gnu -lpython3.7m -lcrypt -lpthread -ldl -lutil -lm -L/usr/lib/x86_64-linux-gnu -ltcl8.6 -ldl -lz -lpthread -lm -lruby-2.5 -lpthread -lgmp -ldl -lcrypt -lm  

[tonymec]

Gvim doesn't use t_Co: once it has started the GUI (and, usually, forked from any terminal it might have been launched from) it uses X11 or Windows or… and that means 16777216 colors even though t_Co isn't set. $TERM is irrelevant in gvim, it interfaces with the keyboard and display without going through a terminal emulator.

I'm not sure what happens if you use the :terminal command in a gvim compiled with +terminal, as my usual Big gvim is compiled with -terminal. But can you repeat your test with (g)Vim 8.2.518 or later? That patchlevel had a fix to pass the proper number of colours with the $TERM of the :terminal.

[egberts]

Will retest with 8.2.518 or master.

Update: Built it out of this repo, commit 7b293c7 tag: v8.2.0539
there is no gvim binary. Perusing Google for gvim build.

Update2: Google takes me back to #1334, @tonymec.
Built it using:

./configure \
    --with-features=huge \
   --enable-gui=gnome2 \
   --enable-rubyinterp=yes \
   --enable-pythoninterp=yes

Update3: vim -g gives same result.

SAME RESULT

Latest Vim master repo gave the virtually identical result to the original posting above except I substituted gvim with vim -g.

Vim info:

VIM - Vi IMproved 8.2 (2019 Dec 12, compiled Apr  9 2020 19:17:37)
Included patches: 1-539
Compiled by steve@arca
Huge version with GTK2 GUI.  Features included (+) or not (-):
+acl               -farsi             -mouse_sysmouse    -tag_old_static
+arabic            +file_in_path      +mouse_urxvt       -tag_any_white
+autocmd           +find_in_path      +mouse_xterm       -tcl
+autochdir         +float             +multi_byte        +termguicolors
-autoservername    +folding           +multi_lang        +terminal
+balloon_eval      -footer            -mzscheme          +terminfo
+balloon_eval_term +fork()            +netbeans_intg     +termresponse
+browse            +gettext           +num64             +textobjects
++builtin_terms    -hangul_input      +packages          +textprop
+byte_offset       +iconv             +path_extra        +timers
+channel           +insert_expand     -perl              +title
+cindent           +job               +persistent_undo   +toolbar
+clientserver      +jumplist          +popupwin          +user_commands
+clipboard         +keymap            +postscript        +vartabs
+cmdline_compl     +lambda            +printer           +vertsplit
+cmdline_hist      +langmap           +profile           +virtualedit
+cmdline_info      +libcall           -python            +visual
+comments          +linebreak         -python3           +visualextra
+conceal           +lispindent        +quickfix          +viminfo
+cryptv            +listcmds          +reltime           +vreplace
+cscope            +localmap          +rightleft         +wildignore
+cursorbind        -lua               -ruby              +wildmenu
+cursorshape       +menu              +scrollbind        +windows
+dialog_con_gui    +mksession         +signs             +writebackup
+diff              +modify_fname      +smartindent       +X11
+digraphs          +mouse             -sound             -xfontset
+dnd               +mouseshape        +spell             +xim
-ebcdic            +mouse_dec         +startuptime       -xpm
+emacs_tags        -mouse_gpm         +statusline        +xsmp_interact
+eval              -mouse_jsbterm     -sun_workshop      +xterm_clipboard
+ex_extra          +mouse_netterm     +syntax            -xterm_save
+extra_search      +mouse_sgr         +tag_binary        
   system vimrc file: "$VIM/vimrc"
     user vimrc file: "$HOME/.vimrc"
 2nd user vimrc file: "~/.vim/vimrc"
      user exrc file: "$HOME/.exrc"
  system gvimrc file: "$VIM/gvimrc"
    user gvimrc file: "$HOME/.gvimrc"
2nd user gvimrc file: "~/.vim/gvimrc"
       defaults file: "$VIMRUNTIME/defaults.vim"
    system menu file: "$VIMRUNTIME/menu.vim"
  fall-back for $VIM: "/usr/local/share/vim"
Compilation: gcc -c -I. -Iproto -DHAVE_CONFIG_H -DFEAT_GUI_GTK  -pthread -I/usr/include/gtk-2.0 -I/usr/lib/x86_64-linux-gnu/gtk-2.0/include -I/usr/include/gio-unix-2.0 -I/usr/include/cairo -I/usr/include/pango-1.0 -I/usr/include/atk-1.0 -I/usr/include/cairo -I/usr/include/pixman-1 -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/libmount -I/usr/include/blkid -I/usr/include/pango-1.0 -I/usr/include/harfbuzz -I/usr/include/pango-1.0 -I/usr/include/fribidi -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/usr/include/uuid -I/usr/include/freetype2 -I/usr/include/libpng16   -g -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1       
Linking: gcc   -L/usr/local/lib -Wl,--as-needed -o vim   -lgtk-x11-2.0 -lgdk-x11-2.0 -lpangocairo-1.0 -latk-1.0 -lcairo -lgdk_pixbuf-2.0 -lgio-2.0 -lpangoft2-1.0 -lpango-1.0 -lgobject-2.0 -lglib-2.0 -lfontconfig -lfreetype -lSM -lICE -lXt -lX11 -lXdmcp -lSM -lICE  -lm -ltinfo -lelf -lnsl  -lselinux -ldl  

[tonymec]

The following (untested) will work in an edit window, I think, if the $TERM variable is correctly set before startup:

function getcolors()
        if has('gui_running')
                return 16777216
        else
                return &t_Co + 0
        endif
endfunction

It won't tell you how many colors (8, 16, 88 or 256) you can use in a :terminal started from gvim. The "+ 0" operation is to make sure that it always returns a Number ('t_Co' has type String).

P.S. The following expression is possibly faster, it gives the same result, but I find it less human-readable:

(has('gui_running')?16777216:&t_Co + 0)

[egberts]

Just a clarification (after this issue got closed): I'm still showing term variable as builtin_gui.

Is this going to be the expected (normal) thing that gvim (or vim -g) term variable will be reporting (thereon) as builtin_gui, REGARDLESS of whether the terminal console is using TERM=xterm-direct, TERM=xterm-256color, or TERM=xterm? Just builtin_gui?

I am showing the following:

TERM=xterm-direct src/vim -g --clean
TERM=xterm-256color src/vim -g --clean
TERM=xterm-color src/vim -g --clean
TERM=xterm src/vim -g --clean

all as reporting:

:verbose echo &term ", " &t_Co
  builtin_gui ,
:!echo $TERM
  dumb

Noticed that t_Co is still blank? And its term is dumb?

This is against the master repo:

VIM - Vi IMproved 8.2 (2019 Dec 12, compiled Apr 10 2020 09:45:15)
Included patches: 1-539

Probably need to REOPEN this issue because the suggested script would be non-operative based on my findings.

[tonymec]

Yes, AFAIK gvim has always had 'term' set to builtin_gui. This is set during GUI startup (see :help gui-init) and cannot be changed (I just tried :set term=xterm, and got E530: Cannot change term in GUI: term=xterm). I don't expect this ever to change. BTW the helptag for E530 is where the 'term' option is described.

The +terminal feature is new stuff, and IIUC within its terminal $TERM is set to some variety of xterm depending on the number of colors supported. I think this latter property has now reached maturity but I could be wrong about that.

P.S. has('gui_running') is nonzero when Vim is invoked with -g or as gvim, gview, gvimdiff, etc.; or when the :gui ex-command has been called. In a Console Vim (which hasn't been told it will start the GUI) it is zero. That's why I test it in the snippet given in a previous post.

[tonymec]

P.P.S. In a :shell or :! command started from gvim, $TERM used to be set to dumb as that command will be run in a very limited terminal which can go only forward, just like teletype consoles used to; and your experiments seem to indicate that this hasn't changed. In a Vim compiled with +terminal you can use the :terminal command, and IIUC that terminal is a more powerful one.

[egberts]

That is correct with regard to :! echo $TERM still reporting as dumb. It's surprising there.

On other hand, :terminal echo $TERM reports linux. And thank you very much for the detail.

And my background-detecting bash script also does the trick in nearly 100% reliably determine whether the current terminal's background is darker than its foreground: term-background.bash

At your suggestion, :terminal command shows:

:terminal

$ bash ~/term-background.bash
Dark background from xterm control

So we are good here.

On a side note (not related to this issue):

I tried passing a command to :terminal, notably bash

:terminal /bin/bash
$ bash ~/term-background.bash
Dark background from xterm control

So far so good... But this terminal command failed:

:terminal ~/term-background.bash
executing job failed: No such file or directory

And this failed:

:terminal /bin/bash ~/term-background.bash
/bin/bash ~/term-background.bash: No such file or directory

And lastly, there is no way to find out the background color of this particular vim command:

:!bash term-background.bash
xterm/vte Esc-] OSC has empty string
Can't decide

shell returned 1

Which is what we discussed on this thread; this is an expected result. Exclamation mark (as a bash shell) makes for crappy environment with regard to coloring/highlighting. We use :terminal.

Now, if I could figure out how to pass an argument to :terminal commands that would not failed as my above examples have shown.

[tonymec]

I'm not sure… Out of my head, I can think of two possibilities, but only you can check them:

• Is your bash script executable? (if not, chmod -c a+rx ~/term-background.bash will set it and tell you what it changed.)
• Is the env program in /usr/bin/ on your system? (On mine it is. In a bash shell, type -a env will show you all known env commands, even internal to the shell.)

[egberts]

File permission is ok.

$ ls -lat term-background.bash 
-rwxr-xr-x 1 XXXX XXXX 6631 Apr 10 18:40 term-background.bash

And env is what it is.

$ head term-background.bash 
#!/usr/bin/env bash
# ^^^^^^^^^^^^ Use env rather than assume we've got /bin/bash.
# This works better on OSX where /bin/bash is brain dead.

#   Copyright (C) 2019, Rocky Bernstein <rocky@gnu.org>
#   This program is free software; you can redistribute it and/or
#   modify it under the terms of the GNU General Public License as
#   published by the Free Software Foundation; either version 2, or
#   (at your option) any later version.
#
$ type -a env
env is /usr/bin/env

Checking type -a env in each type of terminal environment...

• gnome-terminal - env is /usr/bin/env
• xterm - env is /usr/bin/env
• vim --clean and :!type -a env is the same
• vim -g --clean and !type -a env, same result
• vim -g --clean and :terminal, then type -a env, same result

Argument works normally for other ELF-based executable commands:

• vim -g --clean and :terminal ls /
• vim -g --clean and :terminal echo "This works"

But the following...

• vim -g --clean and :terminal type -a env, reported executing job failed: No such file or directory
• vim -g --clean and :terminal ls /, reported

I think I figured this out.... something to do with tilde ~.

• :terminal /home/XXXX/bin/term-background.bash works
• :terminal ~/bin/term-background.bash fails
• :terminal $HOME/term-background.bash
  Yep, it's the tilde and $HOME, both failed.

I take it that my environment variables did not get copied over to the :terminal.

Checking.... :terminal set | grep HOME, failed. Can't use Unix pipe here. Noted.

:terminal set, executing job failed: No such file or directory.. Must be in a shell, Noted.

:terminal bash set, `bash: set: No such file or directory.'. Mmmmm, got the shell, cannot pass console-based commands as runtime argument, Noted.

:terminal, and THEN set, that works. Let's look for $HOME inside vim -g...

:terminal, set | grep HOME, that works HOME=/home/XXXX.

So that brief moment in time during initial :terminal inline argument command processing, there is no $HOME nor tilde (~) available.

Vim documentation of terminal-unix states no $HOME yet. So, I'm guessing this is to be expected.

Conclusion

No, you cannot run bash script directly from :terminal, especially IF $HOME or tilde symbol is programmatically referenced inside the bash script.

This kinda sucks, no?

(Making this comment, an issue.).

[tonymec]

I know! Your script is not in the $PATH. Either move it to ~/bin/, or else add a symlink as follows:

pushd ~/bin
ln -sv ../term-background.bash
popd