Question: the arrival order of uv.spawn on_exit and data == nil in read_start callback on Windows 10?

[linrongbin16]

Hi,

I'm using vim.loop.spawn (e.g., the luv library) in Neovim v0.9.2 to develop a plugin, which supports Windows.

I use the spawn API in this way:

    local function println(line)
        io.write(line .. '\n')
    end

    --- @param data_buffer string
    --- @param fn_line_processor fun(line:string?):nil
    local function consume(data_buffer, fn_line_processor)
        local i = 1
        while i <= #data_buffer do
            local newline_pos = shell_helpers.string_find(data_buffer, "\n", i)
            if not newline_pos then
                break
            end
            local line = data_buffer:sub(i, newline_pos)
            fn_line_processor(line)
            i = newline_pos + 1
        end
        return i
    end

    local cmds = {'fd', '.', '-cnever', '-tf', '-i', '-u'}
    local out_pipe = vim.loop.new_pipe()
    local err_pipe = vim.loop.new_pipe()
    
    local data_buffer = nil

    local function on_exit(code)
        out_pipe:close()
        err_pipe:close()
        vim.loop.stop()
    end

    local process_handler, process_id = vim.loop.spawn(cmds[1], {
        args = { unpack(cmds, 2) },
        stdio = { nil, out_pipe, err_pipe },
    }, function(code, signal)
        out_pipe:read_stop()
        err_pipe:read_stop()
        out_pipe:shutdown()
        err_pipe:shutdown()
        on_exit(code)
    end)

    local function on_output(err, data)
        if err then
            on_exit(1)
            return
        end

        if not data then
            if data_buffer then
                -- foreach the data_buffer and find every line
                local i = consume(data_buffer, println)
                if i <= #data_buffer then
                    local line = data_buffer:sub(i, #data_buffer)
                    println(line)
                    data_buffer = nil
                end
            end
            on_exit(0)
            return
        end

        -- append data to data_buffer
        data_buffer = data_buffer and (data_buffer .. data) or data
        -- foreach the data_buffer and find every line
        local i = consume(data_buffer, println)
        -- truncate the printed lines if found any
        data_buffer = i <= #data_buffer and data_buffer:sub(i, #data_buffer)
            or nil
    end

    local function on_error(err, data)
        -- if err then
        --     on_exit(1)
        --     return
        -- end
        -- if not data then
        --     on_exit(0)
        --     return
        -- end
    end

    out_pipe:read_start(on_output)
    err_pipe:read_start(on_error)
    vim.loop.run() -- start uv loop here, it will block the main thread until the command ends

This is a sample code which is simple but mostly tells the basic structure of the logic.

But in this screen recording, you can see the print data is not consist.
You can see that, during two fd searchings, the lines count is different: first is 1550, second is 1388.

2_5.C__WINDOWS_system32_cmd.exe.2023-10-03.07-10-25.mp4

I guess the reason is that the timing of on_exit callback in spawn 3rd parameter comes, there's still some data not read into buffer.

But is there an arrival order?

[linrongbin16]

I add some log, and it looks like that when fd command exit, there maybe still some data not read to buffer, but I called the out_pipe:read_stop() function in on_exit function, which could stop reading.

So maybe I should just close the process handle in spawn's on_exit (because it's just being invoked when the process exit).

Invoke read_stop and close out_pipe when data == nil in read_start callback (e.g. on_output), this will ensure all data is read into buffer.

Let me try this solution when I am back to keyboard.

[linrongbin16]

I finally make it working correctly! see this: linrongbin16/fzfx.nvim#227