termopen() ignores on_stderr channel and bumps everything in stdout

[saccarosium]

Problem

I was making a PR for a code runner plugin. The main objective was to implement error handling via quickfix or diagnostics.

For running the commands we use the termopen() function, to make possible to handle running/testing even TUI or something that requires input from the user. Reading the docs I saw that termopen supports the same arguments as jobstart, so I tryied to implement something but I kept getting the errors only from stdout and all the coloured output had the color escape sequences still on it. Comparing to jobstart is another level of user experience.

Steps to reproduce

assuming this minimal C file:

// /tmp/nvim_bug_report/test.c
int main() {
    int a = 5;
    int a = 3;
}

testing functions with following code:

-- /tmp/nvim_bug_report/lua/test.lua
local function debug(_, data, event)
    if data then
        print("from " .. event)
        vim.print(data)
    end
end

local cmd_table = {"gcc",  "--std=c11", "-Wall", "/tmp/nvim_bug_report/test.c"}
local cmd_string = "gcc --std=c11 -Wall /tmp/nvim_bug_report/test.c"

local args = {
    stderr_buffered = true,
    stdout_buffered = true,
    on_stderr = debug,
    on_stdout = debug,
    on_exit = debug,
}

-- Test varius functions
print("From termopen with table input")
vim.cmd.new()
vim.fn.termopen(cmd_table, args)
-- print("From termopen with string input")
-- vim.cmd.new()
-- vim.fn.termopen(cmd_string, args)

-- print("From jobstart with table input")
-- vim.fn.jobstart(cmd_table, args)
-- print("From jobstart with string input")
-- vim.fn.jobstart(cmd_string, args)

Steps

• nvim --clean /tmp/nvim_bug_report/lua/test.lua
• :so
• :messages
  at the bottom should be a report of how the cmd went.

Expected behavior

The behaviour I expect is the same I have with jobstart.

Termopen

From termopen with table input
from stdout                                                                                                                                                                         
{ "\27[1m/tmp/nvim_bug_report/test.c:3:9: \27[0m\27[0;1;31merror: \27[0m\27[1mredefinition of 'a'\27[0m\r", "    int a = 3;\r", "\27[0;1;32m        ^\r", "\27[0m\27[1m/tmp/nvim_bug
_report/test.c:2:9: \27[0m\27[0;1;30mnote: \27[0mprevious definition is here\27[0m\r", "    int a = 5;\r", "\27[0;1;32m        ^\r", "\27[0m1 error generated.\r", "" }             
from exit                                                                                                                                                                           
1           

From termopen with string input                                                                                                                                                     
from stdout                                                                                                                                                                         
{ "\27[1m/tmp/nvim_bug_report/test.c:3:9: \27[0m\27[0;1;31merror: \27[0m\27[1mredefinition of 'a'\27[0m\r", "    int a = 3;\r", "\27[0;1;32m        ^\r", "\27[0m\27[1m/tmp/nvim_bug
_report/test.c:2:9: \27[0m\27[0;1;30mnote: \27[0mprevious definition is here\27[0m\r", "    int a = 5;\r", "\27[0;1;32m        ^\r", "\27[0m1 error generated.\r", "" }             
from exit                                                                                                                                                                           
1

JobStart

From jobstart with table input                                                                                                                                                      
from stdout                                                                                                                                                                         
{ "" }                                                                                                                                                                              
from stderr                                                                                                                                                                         
{ "/tmp/nvim_bug_report/test.c:3:9: error: redefinition of 'a'", "    int a = 3;", "        ^", "/tmp/nvim_bug_report/test.c:2:9: note: previous definition is here", "    int a = 5
;", "        ^", "1 error generated.", "" }                                                                                                                                         
from exit                                                                                                                                                                           
1                

From jobstart with string input                                                                                                                                                     
from stdout                                                                                                                                                                         
{ "" }                                                                                                                                                                              
from stderr                                                                                                                                                                         
{ "/tmp/nvim_bug_report/test.c:3:9: error: redefinition of 'a'", "    int a = 3;", "        ^", "/tmp/nvim_bug_report/test.c:2:9: note: previous definition is here", "    int a = 5
;", "        ^", "1 error generated.", "" }                                                                                                                                         
from exit                                                                                                                                                                           
1         

Neovim version (nvim -v)

v0.9.0 Release

Vim (not Nvim) behaves the same?

no, doesn't seem to have the same function

Operating system/version

macOS 13.3.1

Terminal name/version

kitty 0.28.1

$TERM environment variable

xterm-kitty

Installation

homebrew

[zeertzjq]

It's mentioned in docs that with pty argument of jobstart(), on_stderr is ignored.

		  pty:	      (boolean) Connect the job to a new pseudo
			      terminal, and its streams to the master file
			      descriptor. `on_stdout` receives all output,
			      `on_stderr` is ignored. |terminal-start|

And termopen() behaves like this.

[saccarosium]

Oh I've missed that! Thanks. As for the escape sequences there is a similar explanation?

[bfredl]

Support of this is planned although the API is gonna look a bit different:

nvim_open_term() will be augmented with the ability of open a real pty and make the slave fd available somewhere, then jobstart() and uv.spawn() should both be allowed to take this fd and replace any subset of the three stdio pipes with the pty slave.

[justinmk]

• PTY madness #23747