add support for virtual text or padding in textprop

[prabirshrestha]

I would like vim to support virtual text. Neovim does support one but only at the end of the line. Here is an example of how it looks like in neovim. Notice the diagnostics error that is displayed at the end by vim-lsp.

This also seems like quite a popular request based on the 49 likes on the poll. #3573 (comment)

I'm primarily interested in implementing codelens support for language server but there are tons of other use cases. Notice "Run" and "Debug".

Another usage would be to show types that was implicitly discovered. Notice how I only type let result = a + b; but vscode with lsp automatically adds u32 i.e. let result: u32 = a + b;

Few ways it could be implemented.

1. Use the api same as neovim's virtual text - [RFC] support "virtual text" annotations (currently after EOL only) neovim/neovim#8180.

call nvim_buf_set_virtual_text(bufnr, namespace_id, linenr, ["more text", "Comment"]], {})

2. Extend textprops to add padding.

call prop_type_add('lspcodelens', { 'highlight': 'Comment' })
call prop_add(11, 12, { length:3, type: 'lspcodelens', 'padding': [0,0,0,5] })

Then we can get the coordinates of the textprop padding using a new api and use popup to render text on the padding space.

1 would be easier to implement as well as use from a plugin author but 2 would be very powerful allowing us to support neovim's end of line virtual text or even above and below.

Not sure about perf for 2 but another option would be to have something similar to sign_define() api and support for inserting a virtual line and another api for virtual text at EOL similar to neovim. This might be performant but will not support inline types but is still better than having nothing. If textprop apis is better and powerful it might be better to incrementally support this as in we can only support padding top first.

The reason I suggested textprop api with popup window support is that we might in the future want to focus inside this virtual text and perform actions such as Run Tests or Debug Tests. For now I wanted it to do the simplest task that is give me whitespace so I can use popup to render text. When popup supports focusing we can then interact with it.

Another example of where virtual text can be used is to detect the node_modules bundle file size. ref: https://devtips.theanubhav.com/posts/cost-of-import

VS Code calls this decorations and is similar to vim's textprop. You can read more about it on their official website including lot of other extensions that takes advantage of it.

VS Code use after and before instead of padding to accomplish this.

const decorationType = vscode.window.createTextEditorDecorationType({});
const decorationOptions: vscode.DecorationOptions = {
	range: new vscode.Range(0, 0, 0, 0),
	renderOptions: {
		after: {
			contentText: 'blah',
                        width: '200px',
                        height: '50px'
                       // full options can be found here. https://code.visualstudio.com/api/references/vscode-api#ThemableDecorationAttachmentRenderOptions
		},
	},
};
editor.setDecorations(decorationType, [decorationOptions]);

[lacygoill]

I think you could get something similar using :h popup-textprop-pos:

vim9script

def AddVirtualText()
    var lines =<< trim END
        fn main() {
            println!("Hello, world!");
        }

        fn add(a: u32, b: u32) -> u32 {
            a + b
        }

        fn sub(a: u32, b: u32) -> u32 {
            a - b
        }
    END
    setline(1, lines)

    for lnum in [5, 9]
        prop_type_add('markendofline' .. lnum, {})
        prop_add(lnum, col([lnum, '$']), {
            type: 'markendofline' .. lnum,
            })
        var text = {
            5: 'function is never used: `add`^@`#[warn(dead_code)]` on by default',
            9: 'function is never used: `sub`',
            }[lnum]
        popup_create(text, {
            textprop: 'markendofline' .. lnum,
            highlight: 'Comment',
            line: -1,
            col: 2,
            zindex: 49,
            })
    endfor
enddef
AddVirtualText()

But I guess that's not exactly what you want. For example, if you add a split to the right of a window displaying virtual text, I assume you don't want the virtual text to overflow on the latter.

[prabirshrestha]

Wasn't aware of popup-textprop-pos. Seems like that will fix one of the asks I had earlier.

function! g:TestVirtualText() abort
  call setline(1, ['hello', 'world'])
  call cursor(2, 1)
  call prop_type_add('codelens', {})
  call prop_add(2, 1, { 'type': 'codelens' })
  call popup_create("Run", {
    \   'textprop': 'codelens',
    \   'higlight': 'Comment',
    \   'line': -2,
    \   'col': 1,
    \   'zindex': 49,
    \ })
endfunction
call g:TestVirtualText()

This doesn't seem to quite work. Notice how Run is overlapping hello.

.

I would want it to be something like. Notice how it is 3 lines but when I navigate down and up it should only go to hello and world.

hello
Run
world

What I would need is adding padding top in prop_type_add.

call prop_type_add('codelens', { 'padding': [1, 0, 0, 0] })

Another thing I noticed is how do you close the popup? Once the text have been deleted I still see the popup window.

If I use dd it does close the popup correctly.

[lacygoill]

I would want it to be something like. Notice how it is 3 lines but when I navigate down and up it should only go to hello and world.

OK, I understand better now. You want the ability to add a virtual line between 2 lines of text and use it to display some arbitrary information. I don't think that's currently possible; at least I don't know how to do that. As you said, maybe a new padding property in a prop_*() function would help.

Another thing I noticed is how do you close the popup? Once the text have been deleted I still see the popup window.

You need to give a non-zero length to the text property.

Suppose your text looks like this:

aaa bbb ccc

And you want your trailing virtual text to be closed when bbb is deleted. Then, you need to:

• apply your text property on bbb
• position your popup next to the text property
• add some left padding to the text in the popup
• make the padding transparent so that the real text below the popup remains visible

For the last bullet point, see :h popup-mask.

As an example:

vim9script

def AddVirtualText()
    var lines =<< trim END
        fn main() {
            println!("Hello, world!");
        }

        fn add(a: u32, b: u32) -> u32 {
            a + b
        }

        fn sub(a: u32, b: u32) -> u32 {
            a - b
        }
    END
    setline(1, lines)

    for [lnum, col, length, virtual_text]
        in [[5, 4, 3, 'function is never used: `add`^@`#[warn(dead_code)]` on by default'],
            [9, 4, 3, 'function is never used: `sub`']]
        prop_type_add('virtualText' .. lnum, {})
        prop_add(lnum, col, {
            type: 'virtualText' .. lnum,
            length: length,
            })
        var offset = 2
        var left_padding = col([lnum, '$']) - col + length + offset
        popup_create(virtual_text, {
            textprop: 'virtualText' .. lnum,
            highlight: 'Comment',
            line: -1,
            col: offset,
            zindex: 50 - 1,
            wrap: false,
            mask: [[1, left_padding, 1, 1]],
            padding: [0, 0, 0, left_padding],
            pos: 'topright',
            })
    endfor
enddef
AddVirtualText()

Tested on 8.2.2230, Ubuntu 16.04:

[prabirshrestha]

@lacygoill That looks great.

I noticed the length is hard-coded to 3, so if you change add to multiplication popup will be on top.

[lacygoill]

I noticed the length is hard-coded to 3, so if you change add to multiplication popup will be on top.

Yes, the code had issues. This one seems better:

vim9
var lines: list<string> =<< trim END
    I met a traveller from an antique land,
    Who said—“Two vast and trunkless legs of stone
    Stand in the desert. . . . Near them, on the sand,
    Half sunk a shattered visage lies, whose frown,
    And wrinkled lip, and sneer of cold command,
    Tell that its sculptor well those passions read
    Which yet survive, stamped on these lifeless things,
    The hand that mocked them, and the heart that fed;
    And on the pedestal, these words appear:
    My name is Ozymandias, King of Kings;
    Look on my Works, ye Mighty, and despair!
    Nothing beside remains. Round the decay
    Of that colossal Wreck, boundless and bare
    The lone and level sands stretch far away.”
END
setline(1, lines)

var lnum2popup_id: dict<dict<number>> = {}

def AddVirtualText(props: dict<any>): number
    var lnum: number = props.lnum
    var col: number = props.col
    var length: number = props.length
    var text: string = props.text
    var highlight_text: string = has_key(props, 'highlight_text')
        ? props.highlight_text
        : 'Normal'
    var highlight_virtualtext: string = has_key(props, 'highlight_virtualtext')
        ? props.highlight_virtualtext
        : 'Normal'
    if has_key(props, 'highlight_virtualtext')
        highlight_virtualtext = props.highlight_virtualtext
    endif

    var buf: number = bufnr('%')
    if prop_type_list({bufnr: buf})->index('virtualText' .. lnum) == -1
        prop_type_add('virtualText' .. lnum, {bufnr: buf, highlight: highlight_text})
    endif
    if has_key(lnum2popup_id, buf) && has_key(lnum2popup_id[buf], lnum)
        popup_close(lnum2popup_id[buf][lnum])
    endif
    prop_add(lnum, col, {
        type: 'virtualText' .. lnum,
        length: length,
        bufnr: buf,
        })
    var left_padding: number = col([lnum, '$'])
    var popup_id: number = popup_create(text, {
        textprop: 'virtualText' .. lnum,
        highlight: highlight_virtualtext,
        line: -1,
        zindex: 50 - 1,
        wrap: false,
        mask: [[1, left_padding, 1, 1]],
        padding: [0, 0, 0, left_padding],
        pos: 'topright',
        })
    if !has_key(lnum2popup_id, buf)
        extend(lnum2popup_id, {[buf->string()]: {}})
    endif
    extend(lnum2popup_id[buf], {[lnum->string()]: popup_id})
    return popup_id
enddef

var ozymandias_pos: list<number> = searchpos('Ozymandias')
AddVirtualText({
    lnum: ozymandias_pos[0],
    col: ozymandias_pos[1],
    length: strchars('Ozymandias'),
    text: 'Greek name for Ramesses II, pharaoh of Egypt',
    highlight_text: 'Search',
    highlight_virtualtext: 'MoreMsg',
    })

---

But it still has some issue. If we insert text on the line where there is virtual text, some real text at the end of the line gets hidden by the popup.

That's because left_padding needs to be updated. We would need to re-invoke the function every time we insert or remove text on a line where there is virtual text. We could try an autocmd:

au TextChanged,TextChangedI,TextChangedP * AddVirtualText({
    \ lnum: ozymandias_pos[0],
    \ col: ozymandias_pos[1],
    \ length: strchars('Ozymandias'),
    \ text: 'Greek name for Ramesses II, pharaoh of Egypt',
    \ highlight_text: 'Search',
    \ highlight_virtualtext: 'MoreMsg',
    \ })

But this one is wrong; because ozymandias_pos might also need to be updated.
Maybe AddVirtualText() should be called as a callback via listener_add(). When I have more time, I'll see if I can improve the code.

[lacygoill]

I think that the issue could be fixed when we insert something before the text to which a popup is attached. For this, we would need to use the col key when invoking popup_create(). According to the help, the latter is relative to the text property:

vim/runtime/doc/popup.txt

Lines 607 to 608 in e2edc2e

	When using "textprop" the number is relative to the
	text property and can be negative.

Unfortunately, it seems to be ignored (contrary to line which works as expected). It could be a Vim bug. I'll make more tests; if it is a bug, I'll report it.

But even then, it wouldn't fix the issue when we insert something after the text to which a popup is attached.

---

Right now, the textprop key which we can use when invoking popup_create() controls two things:

• it determines where the popup should be created
• it determines when it should be visible
  (i.e. when the text to which the property is attached is deleted, the popup is automatically closed; when it's restored, the popup is automatically re-opened)

I think what you would need is the ability to use 2 different text properties:

• the first one would determine where the popup should be created
• the second one would determine when it should be visible

We could then tell Vim that we want the popup to be positioned at the end of the line (by adding a text property there), and only stay visible while a different text on the line has not been deleted (by adding another text property on the latter).

It wouldn't fulfill your other requirement, which is the ability to display virtual lines (BTW, someone else is interested in the same feature), but it would already be very helpful.

[prabirshrestha]

Besides when and where I would also need to specify padding so I know the popup is not not overriding other text.

Ideally it would be better if vim supports it by default similar to how there are already bunch of helpers for different style of popup.

This would encourage more people to use the feature instead of having a custom one.

https://github.com/prabirshrestha/vim-lsp/blob/51a566c87824b4fe214d5ff63d2271254f22d461/autoload/lsp/internal/diagnostics/virtual_text.vim#L125-L134

In neovim it is as simple as calling one function. My guess is that it was easier to implement and hence virtual text in neovim only works for end of line.

  call nvim_buf_set_virtual_text(a:bufnr, s:namespace_id, l:line,
            \ [[g:lsp_diagnostics_virtual_text_prefix . l:item['message'], l:hl_name]], {})

Related: neovim/neovim#13661

My thought was something like this.

call prop_add(l:start_line, l:start_col, {
                    \ 'end_lnum': l:end_line,
                    \ 'end_col': l:end_col,
                    \ 'bufnr': a:bufnr,
                    \ 'type': 'virtualtext',
                    \ 'padding_eol': 10
                    \ })

padding_eol would give support for neovim's virtual text support. One open question would be if the padding needs to be in the start line or end line. padding_endline_eol padding_startline_eol.

padding_endcol would give support for type hints for implicit known types. This would always pad right of endcol.
padding_startline would give support for code lens support i.e. Run | Debug. This would always pad top of startline.

[bfredl]

we (neovim) plan to support virtual text anywhere eventually but it is quite a bit trickier than just adding it at the end, basically for the same reason that conceal is tricky (including things like visual cursor pos and mouse clicks).

[brammool]

we (neovim) plan to support virtual text anywhere eventually but it is quite a bit trickier than just adding it at the end, basically for the same reason that conceal is tricky (including things like visual cursor pos and mouse clicks).

Yeah, I was also thinking that it will be a lot of work to do right. If text is to be inserted in a way it moves the "real text", then all computations for position need to be adjusted. Perhaps looking how filler lines are inserted in diff mode gives an idea. I think having a marker with a hover-over text will be good enough in many cases, and that is what text properties offer.

…

-- hundred-and-one symptoms of being an internet addict: 178. You look for an icon to double-click to open your bedroom window. /// Bram Moolenaar -- Bram@Moolenaar.net -- http://www.Moolenaar.net \\\ /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\ \\\ an exciting new programming language -- http://www.Zimbu.org /// \\\ help me help AIDS victims -- http://ICCF-Holland.org ///

[lacygoill]

Ideally it would be better if vim supports it by default similar to how there are already bunch of helpers for different style of popup.

This would encourage more people to use the feature instead of having a custom one.

Oh yes, I completely agree. Even if we can currently achieve something with builtin functions, a new helper function would make sense for this use case, similar to popup_atcursor(), popup_dialog(), popup_menu(), and popup_notification().

---

Unfortunately, it seems to be ignored (contrary to line which works as expected). It could be a Vim bug. I'll make more tests; if it is a bug, I'll report it.

I was wrong, it's not ignored. But the code made it look like so, because it wrongly used pos: 'topright', which changed how col was interpreted. Once it's removed, the popups are correctly moved when we insert text before a text property. But not when we insert text afterward. For that, I think we need a listener callback to update the padding and mask options of the popups.

Here is an updated version of the previous code:

vim9

var lnum2popup_id: dict<dict<number>> = {}

def AddVirtualText(props: dict<any>): number
    var lnum: number = props.lnum
    var col: number = props.col
    var length: number = props.length
    var text: string = props.text
    var highlight_text: string = has_key(props, 'highlight_text')
        ? props.highlight_text
        : 'Normal'
    var highlight_virtualtext: string = has_key(props, 'highlight_virtualtext')
        ? props.highlight_virtualtext
        : 'Normal'
    if has_key(props, 'highlight_virtualtext')
        highlight_virtualtext = props.highlight_virtualtext
    endif

    var buf: number = bufnr('%')

    if !lnum2popup_id->has_key(buf)
        listener_add(UpdatePadding, buf)
    endif

    if prop_type_list({bufnr: buf})->index('virtualText' .. lnum) == -1
        prop_type_add('virtualText' .. lnum, {
            bufnr: buf,
            highlight: highlight_text
            })
    endif
    if has_key(lnum2popup_id, buf) && has_key(lnum2popup_id[buf], lnum)
        popup_close(lnum2popup_id[buf][lnum])
    endif
    prop_add(lnum, col, {
        type: 'virtualText' .. lnum,
        length: length,
        bufnr: buf,
        })

    var left_padding: number = col([lnum, '$']) - length - col + 1

    var popup_id: number = popup_create(text, {
        line: -1,
        padding: [0, 0, 0, left_padding],
        mask: [[1, left_padding, 1, 1]],
        textprop: 'virtualText' .. lnum,
        highlight: highlight_virtualtext,
        fixed: true,
        wrap: false,
        zindex: 50 - 1,
        })
    if !has_key(lnum2popup_id, buf)
        extend(lnum2popup_id, {[buf->string()]: {}})
    endif
    extend(lnum2popup_id[buf], {[lnum->string()]: popup_id})

    return popup_id
enddef

def UpdatePadding(
    buffer: number,
    start: number,
    ...l: any
    )
    if start > line('$')
        return
    endif
    var prop_list: list<dict<any>> = start->prop_list()
    var i: number = prop_list->match('virtualText')
    if i == -1
        return
    endif
    var textprop: dict<any> = prop_list[i]
    var left_padding: number = col([start, '$']) - textprop.length - textprop.col + 1

    if !has_key(lnum2popup_id, buffer) || !has_key(lnum2popup_id[buffer], start)
        return
    endif
    var popup_id: number = lnum2popup_id[buffer][start]
    popup_setoptions(popup_id, {
        padding: [0, 0, 0, left_padding],
        mask: [[1, left_padding, 1, 1]]
        })
enddef

var lines: list<string> =<< trim END
    I met a traveller from an antique land,
    Who said—“Two vast and trunkless legs of stone
    Stand in the desert. . . . Near them, on the sand,
    Half sunk a shattered visage lies, whose frown,
    And wrinkled lip, and sneer of cold command,
    Tell that its sculptor well those passions read
    Which yet survive, stamped on these lifeless things,
    The hand that mocked them, and the heart that fed;
    And on the pedestal, these words appear:
    My name is Ozymandias, King of Kings;
    Look on my Works, ye Mighty, and despair!
    Nothing beside remains. Round the decay
    Of that colossal Wreck, boundless and bare
    The lone and level sands stretch far away.”
END
setline(1, lines)

var shattered_pos: list<number> = searchpos('shattered', 'n')
AddVirtualText({
    lnum: shattered_pos[0],
    col: shattered_pos[1],
    length: strchars('shattered'),
    text: 'broken into many pieces',
    highlight_text: 'Search',
    highlight_virtualtext: 'MoreMsg',
    })

var sneer_pos: list<number> = searchpos('sneer', 'n')
AddVirtualText({
    lnum: sneer_pos[0],
    col: sneer_pos[1],
    length: strchars('sneer'),
    text: 'a contemptuous or mocking smile, remark, or tone',
    highlight_text: 'Search',
    highlight_virtualtext: 'MoreMsg',
    })

var ozymandias_pos: list<number> = searchpos('Ozymandias', 'n')
AddVirtualText({
    lnum: ozymandias_pos[0],
    col: ozymandias_pos[1],
    length: strchars('Ozymandias'),
    text: 'Greek name for Ramesses II, pharaoh of Egypt',
    highlight_text: 'Search',
    highlight_virtualtext: 'MoreMsg',
    })

var wreck_pos: list<number> = searchpos('Wreck', 'n')
AddVirtualText({
    lnum: wreck_pos[0],
    col: wreck_pos[1],
    length: strchars('Wreck'),
    text: 'something that has been badly damaged or destroyed',
    highlight_text: 'Search',
    highlight_virtualtext: 'MoreMsg',
    })

---

If I find other issues, I'll try to fix them in this gist.

[vim-ml]

Hi,

On Thu, Jan 21, 2021 at 1:50 AM lacygoill ***@***.***> wrote: Ideally it would be better if vim supports it by default similar to how there are already bunch of helpers for different style of popup. This would encourage more people to use the feature instead of having a custom one. Oh yes, I completely agree. Even if we can currently achieve something with builtin functions, a new helper function would make sense for this use case, similar to popup_atcursor() <https://vimhelp.org/popup.txt.html#popup_atcursor()>, popup_dialog() <https://vimhelp.org/popup.txt.html#popup_dialog()>, popup_menu() <https://vimhelp.org/popup.txt.html#popup_menu()>, and popup_notification() <https://vimhelp.org/popup.txt.html#popup_notification()>.

Agree. It makes sense to add a helper function for the virtual text. But how will it handle the listener callback (shown below) that updates the left padding of the popup? If this is not internally managed, then every plugin needs to add support for this. Thanks, Yegappan

…

------------------------------ Unfortunately, it seems to be ignored (contrary to line which works as expected). It could be a Vim bug. I'll make more tests; if it is a bug, I'll report it. I was wrong, it's not ignored. But the code made it look like so, because it wrongly used pos: 'topright', which changed how col was interpreted. Once it's removed, the popups are correctly moved when we insert text before a text property. But not when we insert text afterward. For that, I think we need a listener callback to update the padding and mask options of the popups. Here is an updated version: vim9 var lines: list<string> =<< trim END I met a traveller from an antique land, Who said—“Two vast and trunkless legs of stone Stand in the desert. . . . Near them, on the sand, Half sunk a shattered visage lies, whose frown, And wrinkled lip, and sneer of cold command, Tell that its sculptor well those passions read Which yet survive, stamped on these lifeless things, The hand that mocked them, and the heart that fed; And on the pedestal, these words appear: My name is Ozymandias, King of Kings; Look on my Works, ye Mighty, and despair! Nothing beside remains. Round the decay Of that colossal Wreck, boundless and bare The lone and level sands stretch far away.” END setline(1, lines) var lnum2popup_id: dict<dict<number>> = {} def AddVirtualText(props: dict<any>): number var lnum: number = props.lnum var col: number = props.col var length: number = props.length var text: string = props.text var highlight_text: string = has_key(props, 'highlight_text') ? props.highlight_text : 'Normal' var highlight_virtualtext: string = has_key(props, 'highlight_virtualtext') ? props.highlight_virtualtext : 'Normal' if has_key(props, 'highlight_virtualtext') highlight_virtualtext = props.highlight_virtualtext endif var buf: number = bufnr('%') # We need to make sure that only 1 listener is added per buffer. # Otherwise, there's the risk that when you insert text after a text # property, *all* popups are moved. if !lnum2popup_id->has_key(buf) listener_add(UpdatePadding, buf) endif if prop_type_list({bufnr: buf})->index('virtualText' .. lnum) == -1 prop_type_add('virtualText' .. lnum, { bufnr: buf, highlight: highlight_text }) endif if has_key(lnum2popup_id, buf) && has_key(lnum2popup_id[buf], lnum) popup_close(lnum2popup_id[buf][lnum]) endif prop_add(lnum, col, { type: 'virtualText' .. lnum, length: length, bufnr: buf, }) var left_padding: number = col([lnum, '$']) - length - col + 1 var popup_id: number = popup_create(text, { line: -1, padding: [0, 0, 0, left_padding], mask: [[1, left_padding, 1, 1]], textprop: 'virtualText' .. lnum, highlight: highlight_virtualtext, fixed: true, wrap: false, zindex: 50 - 1, }) if !has_key(lnum2popup_id, buf) extend(lnum2popup_id, {[buf->string()]: {}}) endif extend(lnum2popup_id[buf], {[lnum->string()]: popup_id}) return popup_id enddef def UpdatePadding( buffer: number, start: number, ...l: any ) if start > line('$') return endif var prop_list: list<dict<any>> = start->prop_list() var i: number = prop_list->match('virtualText') if i == -1 return endif var textprop: dict<any> = prop_list[i] var left_padding: number = col([start, '$']) - textprop.length - textprop.col + 1 if !has_key(lnum2popup_id, buffer) || !has_key(lnum2popup_id[buffer], start) return endif var popup_id: number = lnum2popup_id[buffer][start] popup_setoptions(popup_id, { padding: [0, 0, 0, left_padding], mask: [[1, left_padding, 1, 1]] }) enddef var shattered_pos: list<number> = searchpos('shattered', 'n') AddVirtualText({ lnum: shattered_pos[0], col: shattered_pos[1], length: strchars('shattered'), text: 'broken into many pieces', highlight_text: 'Search', highlight_virtualtext: 'MoreMsg', }) var sneer_pos: list<number> = searchpos('sneer', 'n') AddVirtualText({ lnum: sneer_pos[0], col: sneer_pos[1], length: strchars('sneer'), text: 'a contemptuous or mocking smile, remark, or tone', highlight_text: 'Search', highlight_virtualtext: 'MoreMsg', }) var ozymandias_pos: list<number> = searchpos('Ozymandias', 'n') AddVirtualText({ lnum: ozymandias_pos[0], col: ozymandias_pos[1], length: strchars('Ozymandias'), text: 'Greek name for Ramesses II, pharaoh of Egypt', highlight_text: 'Search', highlight_virtualtext: 'MoreMsg', }) var wreck_pos: list<number> = searchpos('Wreck', 'n') AddVirtualText({ lnum: wreck_pos[0], col: wreck_pos[1], length: strchars('Wreck'), text: 'something that has been badly damaged or destroyed', highlight_text: 'Search', highlight_virtualtext: 'MoreMsg', }) [image: gif] <https://user-images.githubusercontent.com/8505073/105333437-03cfa180-5bd6-11eb-9de0-d7d6de2ff3c3.gif> ------------------------------ If I find other issues, I'll try to fix them in this gist <https://gist.github.com/lacygoill/09da3dddea83e83bc15e6d9a9044bc95>. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#7553 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACY5DGDAGJWY4LXHV7BIIE3S272FDANCNFSM4VJQ2U6Q> . -- -- You received this message from the "vim_dev" maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups "vim_dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to ***@***.*** To view this discussion on the web visit https://groups.google.com/d/msgid/vim_dev/vim/vim/issues/7553/764514812%40github.com <https://groups.google.com/d/msgid/vim_dev/vim/vim/issues/7553/764514812%40github.com?utm_medium=email&utm_source=footer> .

[poetaman]

@lacygoill @brammool Is virtual-text support going to be added in Vim or is it decided to remain as a separate plugin? https://github.com/lacygoill/vim-virtual-text

[jclsn]

Vim really needs this feature. Tim Pope is writing a Github Copilot plugin and it would be so awesome if this worked in Vim as well and not just Neovim

https://github.com/github/copilot.vim

[CoelacanthusHex]

As mentioned in several of the above duplicate issues, this feature is very helpful for language development with automatic type inference like rust and haskell, mainly used to annotate the variables and return and parameter type of functions.
The inlay hint has now been promoted as part of the LSP standard by rust-analyzer, which means that the full implementation of LSP functionality requires the support of this feature.
As many other IDEs do, neovim has provided this functionality for rust users for a long time.
As a vim user, I want this feature to be part of vim and make vim better.

[brammool]

There are references to "this feature", but I'm not sure what that is about. Please be specific.

As suggested above (more than a year ago) using a text property and attaching a popup window already comes very close.
See the help for popup-textprop-pos.
It seems the only thing needed is to position the popup not right next to the text property, but after the end of the line.
That works fine if there is enough white space, but when there isn't it gets tricky.
Does the text in the popup get truncated when there is not enough space?
When it is truncated (perhaps to just a couple of characters or a placeholder) should it show all the text on hover?
What does Neovim do when there is not enough space?
Also, does that popup overlap another window? Or perhaps only when the window is the current window?

All these questions need to be answered to be able to implement this.
I don't think a hack that ignores these questions is acceptable, later changes will then cause plugins to break.

[CoelacanthusHex]

There are references to "this feature", but I'm not sure what that is about. Please be specific.

add some virttext right next to text property or at the end of the text, The former can be used to show the type of a variable, and the latter can be used to show what type the function at the end of the line will return. The former looks as described in #9155 and the latter looks as described in this issue begin.
This is how a friend of mine wrote rust using this feature of neovim (he uses a post-version of this feature).

As suggested above (more than a year ago) using a text property and attaching a popup window already comes very close.
See the help for popup-textprop-pos.
It seems the only thing needed is to position the popup not right next to the text property, but after the end of the line.

Whether implementing an API based on an existing API using viml, or implementing an API using C, in any case, we need a direct API instead of doing all the work ourselves.

That works fine if there is enough white space, but when there isn't it gets tricky.
Does the text in the popup get truncated when there is not enough space?
When it is truncated (perhaps to just a couple of characters or a placeholder) should it show all the text on hover?
What does Neovim do when there is not enough space?
Also, does that popup overlap another window? Or perhaps only when the window is the current window?

These issues are very necessary to discuss, and we should list them here in detail and directly to discuss and solve them together. Only when a problem is pointed out can it be solved.
And according to my personal opinion, I prefer to maintain the same behavior as normal text, that is, if normal text is to be truncated, then virttext is also truncated, and if normal text is folded, then virttext is also folded.

All these questions need to be answered to be able to implement this. I don't think a hack that ignores these questions is acceptable, later changes will then cause plugins to break.

I'm not trying to rush, I'm just stating the necessity of this thing: a lot of things are ready to rely on such a feature.

[Avimitin]

Does the text in the popup get truncated when there is not enough space?

The inlay hint in neovim work like trailing comment text. When the line is too long and user set the wrap option, the inlay hint will still attach at the end of the code.

It just works like the developer write a comment for the type at the end, but it is done by the LSP and the plugin.

[91khr]

@CoelacanthusHex I think maybe virttext can be treated in the same way as normal text, because most times it's used to insert some hints or to provide some information, in a more... fusing way than a popup window, i.e. appear as if they are normal text. As the name 'virtual text' indicates, they should be able to be inserted directly like normal text does, overlapping nothing, and have its own place in the text buffer.

If virttexts are rendered in a much different way than normal text, virttext's advancement that they can mix more naturally with normal text would lose, and if so, it should be enough to add some features to popup windows, e.g. utilities to simplify the control to some subtle behaviors like truncating, and some more features to allow it to be wrapped like virttext in nvim.

So I think most render facilities in Vim should treat virttext basically the same way as normal text, e.g. wrap them in the same way how normal text is wrapped, and virttext should be able to be inserted directly into the highlighting stream.

An already-existed example is the parameter name hint in IDEA:

Here, the definition: is the parameter name hint; when it's hard to determine the meaning of the argument, like in this screenshot, or in a more evil function invocation like CreateWindow(800, 600, CENTER, CENTER, "Hello"), the hint is useful, and apparently the hints should be inserted before the arguments, otherwise, they would make no sense.

As another example --- I don't know if there's any editors/IDEs that implement this --- a Rust language client may render the source code as:

fn f(&'_ self) -> T {
    let v : Vec<{unknown}> = vec![];
    // ...
}

where the lifetime annotation '_ and the type annotation : Vec<{unknown}> are both virttext, and they can be highlighted just like normal Rust code, except that the {unknown} marker may be processed specially.

And back to virttext itself, there are some more questions and possibilities. Virttexts may be able to be concealed in insert mode, and there may be a new option for concealing that allows virttext be concealed only if insert cursor gets very close to them (the definition of 'close' may also be able to be controlled through some options), so that users can view the hint without intentionally moving to another line or leaving insert mode, and won't get confused when editing text close to the virttext. And maybe users can be allowed to interact directly with virttexts, e.g. edit them, or they can be anchors indicating a location where a jump-to command would take the user to.

So the discussion above may be the reason why popup windows aren't enough --- they're not a part of the current buffer by design, and behave differently from normal text; and there's need for something similar to normal buffer text.

[jclsn]

There are references to "this feature", but I'm not sure what that is about. Please be specific.

Tbh I can't really say. I just read that this feature was missing on the page of the Github Copilot plugin, which is written by Tim Pope, who has been a long-time Vim plugin developer. If he says that only Neovim is supported, I believe him. I am currently still waiting for my Copilot license and could probably describe it better, when I see it. This issue just sounds very much like it. I didn't want to create a duplicate issue.

Ah the others already described it.

[chrisbra]

perhaps @tpope wants to comment?

[tpope]

The reference in question is to virtual lines. The defining characteristic is that text isn't obscured: Displaying a virtual line after line 4 causes line 5, 6, etc to shift down.

[brammool]

Tim Pope wrote:

The reference in question is to virtual _lines_. The defining characteristic is that text isn't obscured: Displaying a virtual line after line 4 causes line 5, 6, etc to shift down.

As I mentioned before, inserting screen lines without actual text is a lot of work to get right. When adding gaps for diff mode it took quite a while to find all places where a change was needed. I also hesitate to show text that isn't actually in the file in a way that it looks like it's part of the file. It should be obvious to the user that this is not an actual text line from the file. I do understand the concern that adding extra text without obscuring the file text is desired. As I mentioned, adding some text after the line, be it a popup or virtual text, has problems when there is not enough space. Lots of corner cases to take care of. Nevertheless, for quite a few purposes, such as displaying a variable type, adding a screenline is not the solution.

…

-- hundred-and-one symptoms of being an internet addict: 18. Your wife drapes a blond wig over your monitor to remind you of what she looks like. /// Bram Moolenaar -- ***@***.*** -- http://www.Moolenaar.net \\\ /// \\\ \\\ sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ /// \\\ help me help AIDS victims -- http://ICCF-Holland.org ///

[CoelacanthusHex]

So the discussion above may be the reason why popup windows aren't enough --- they're not a part of the current buffer by design, and behave differently from normal text; and there's need for something similar to normal buffer text.

You are right, most of the time we want to treat virttext as text. But I don't think it's a good idea to let users edit virttext directly, what we need is a tool that shows the information implicit in the text itself, obviously, it should be derived using some kind of automated tool, such as linter and LSP, instead of us modifying it manually. I'd like it to be displayed as normal text (of course, in most cases differentiated by special styles (like darker)), such as truncation and line wrapping that "look"; but when processing text, like move or search and text replace, I want it as if it doesn't exist. The former is why it is called text, and the latter is why it is virt.

I also hesitate to show text that isn't actually in the file in a way
that it looks like it's part of the file. It should be obvious to the
user that this is not an actual text line from the file.

In fact here, vim just provides a mechanism. Users should know what effects the plugins they install or the code they write will do, right? In other words, when the user encounters an indistinguishable problem, he should go to the plugin to change, which is the plugin's bug, not vim itself.
If we had to consider vim's responsibility in this matter, then we probably shouldn't have added popup either, which would also allow for an indistinguishable "virt" text.
The people who make the knives are not accomplices of the murderer.

[CoelacanthusHex]

And, looking at the whole feature request, depending on the requirements, this feature can be divided into two parts, the short message used by linter, and the LSP and git-blame (which generally avoid triggering line breaks) and copilot, which will insert a large paragraph of text and will face a lot of line breaks and truncation options, as well as serious performance problems.
So maybe we can split this feature instead of implementing the full feature at once?

[91khr]

As I mentioned before, inserting screen lines without actual text is a
lot of work to get right. When adding gaps for diff mode it took quite
a while to find all places where a change was needed.

In this issue there are two kinds of virttext discussing: one for inlay hints like type/parameter name hint or git blame that would not require extra logic lines, which may cause the physical line to overflow that can already be handled well by current wrapping mechanic, and they are expected to behave in a similar way as normal buffer text; one for features such as codelens and copilot, which need extra screen lines and require gaps between buffer lines, like diff mode.

At least there may be less performance concerns for the prior one. And even if so, those who decide to use the mechanic should be ready to afford the cost; as long as there is little performance lose when the feature is not used, I think the cost may be acceptable.

As I mentioned, adding some text after the line,
be it a popup or virtual text, has problems when there is not enough
space. Lots of corner cases to take care of. Nevertheless, for quite a
few purposes, such as displaying a variable type, adding a screenline is
not the solution.

This is true, but those who calls for virttext that adds extra lines are not focusing on these purposes, but on copilot and codelens; 'inline' virttext is enough for hints, holes in proof assistants, etc., and as illustrated above, 'inline' virttext can be rendered in the same way normal text is rendered.

I also hesitate to show text that isn't actually in the file in a way
that it looks like it's part of the file. It should be obvious to the
user that this is not an actual text line from the file.

New highlight groups can be added to highlight virttext differently from normal text in the buffer. With proper color scheme, virttext can be obviously different to the user.

Personally, I think it's OK to allow plugins to change virttext's highlight, even if this would confuse users, because after all, plugins have the same privilege as the user, and if a plugin do want to do harm to users, confusing users with virttext may be the last thing to concern.

[CoelacanthusHex]

At least there may be less performance concerns for the prior one. And even if so, those who decide to use the mechanic should be ready to afford the cost; as long as there is little performance lose when the feature is not used, I think the cost may be acceptable.

From what I've seen in the neovim community and my friends who use neovim, few people complain about performance issues with simple inline virttext.

Personally, I think it's OK to allow plugins to change virttext's highlight, even if this would confuse users, because after all, plugins have the same privilege as the user, and if a plugin do want to do harm to users, confusing users with virttext may be the last thing to concern.

If a plugin deliberately confuses users, it should be reported to their code host (eg GitHub), rather than complaining that vim provides this avenue. It's important to know that tools are not sinful, it is the people who use them to do bad things that are sinful.

[91khr]

From what I've seen in the neovim community and my friends who use neovim, few people complain about performance issues with simple inline virttext.

But compatibility with old machines may still be a goal of vim, while nvim seems to give up this. After all, VSCode, which uses a browser to render its editor and provides much more features adding decorations and controls to the editor, runs well on most modern machine. Regretfully, I have no knowledge to vim's rendering, thus I can't tell if the feature, even if not used, would add much overhead.

If a plugin deliberately confuses users, it should be reported to their code host (eg GitHub), rather than complaining that vim provides this avenue. It's important to know that tools are not sinful, it is the people who use them to do bad things that are sinful.

Here I mean a plugin can even remove everything from user's home directory silently, and if even this is allowed, denying plugins from the possibility of providing better experience for security reason may be more unreasonable; for example, type hints can be highlighted like normal types with a darker background to be more readable while still remaining obvious enough that it's not a part of actual buffer text.

[bfredl]

But compatibility with old machines may still be a goal of vim, while nvim seems to give up this.
After all, VSCode, which uses a browser to render its editor
and provides much more features adding decorations and controls to the editor,
runs well on most modern machine.
Regretfully, I have no knowledge to vim's rendering,
thus I can't tell if the feature, even if not used, would add much overhead.

The neovim implementation makes sure, that when virtual lines are not used at all, there is no extra overhead.

[brammool]

> So the discussion above may be the reason why popup windows aren't > enough --- they're not a part of the current buffer by design, and > behave differently from normal text; and there's need for something > similar to normal buffer text. You are right, most of the time we want to treat virttext as text. But I don't think it's a good idea to let users edit virttext directly, what we need is a tool that shows the information **implicit** in the text itself, obviously, it should be derived using some kind of automated tool, such as linter and LSP, instead of us modifying it manually. I'd like it to be displayed as normal text (of course, in most cases differentiated by special styles (like darker)), such as truncation and line wrapping that "look"; but when processing text, like move or search and text replace, I want it as if it doesn't exist. The former is why it is called text, and the latter is why it is virt. > I also hesitate to show text that isn't actually in the file in a way > that it looks like it's part of the file. It should be obvious to the > user that this is not an actual text line from the file. In fact here, vim just provides a mechanism. Users should know what effects the plugins they install or the code they write will do, right? In other words, when the user encounters an indistinguishable problem, he should go to the plugin to change, which is the plugin's bug, not vim itself. If we had to consider vim's responsibility in this matter, then we probably shouldn't have added `popup` either, which would also allow for an indistinguishable "virt" text. The people who make the knives are not accomplices of the murderer.

There is a difference between providing a feature that, when used normally, will not cause trouble, but can be abused, and a feature that is an invitation to use in the wrong way. When a popup looks too much like regular text, this is usually uncovered quickly (e.g. the cursor can't go into it). If we make something that looks and behaves like normal text but isn't (won't be stored in the file) then I can predict lots of mistakes being made by the user. Best would be if we could use another font for text that isn't in the file, but in a terminal that isn't really possible. Adding text without shifting the file text is not too difficult (which is why neoVim has a simple virtual text implementation). As soon as file text is moved then lots of things need to be aware of this and it's very difficult to get right. Check how many patches were needed to make conceal mode work properly (and it still can't move the line break, because that is too complicated, and mouse positioning is still wrong). So let's first try to do this with popups, since they have already been implemented and have many ways to control them. Perhaps they need some additional setting. If that is not enough, the idea to insert an empty screenline, to make room to put annotations on, is already complicated, but still a lot simpler than shifting text horizontally.

…

-- hundred-and-one symptoms of being an internet addict: 20. When looking at a pageful of someone else's links, you notice all of them are already highlighted in purple. /// Bram Moolenaar -- ***@***.*** -- http://www.Moolenaar.net \\\ /// \\\ \\\ sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ /// \\\ help me help AIDS victims -- http://ICCF-Holland.org ///

[brammool]

> But compatibility with old machines may still be a goal of vim, > while nvim seems to give up this.

I do have the impression that neoVim users then to have "state of the art" hardware, while people just using whatever they have been using for many years tend to use Vim. Also, it's still possible to use Vim over a slow remote connection, thus minimizing the number of bytes used for redrawing does matter to Vim. E.g, using a terminal insert-line or delete-line command instead of redrawing all following lines.

After all, VSCode, which uses a browser to render its editor and provides much more features adding decorations and controls to the editor, runs well on most modern machine. Regretfully, I have no knowledge to vim's rendering, thus I can't tell if the feature, even if not used, would add much overhead. The neovim implementation makes sure, that when virtual lines are not used at all, there is no extra overhead.

I don't think drawing virtual text would have much influence on performance. Syntax highlighting is the main performance killer when it comes to screen updating. Looking through old notes for concealing and how complex it is to make the mouse work directly (try clicking in help text after concealed characters), I thought of a solution that would cost memory (which is cheap these days) but make the computations much faster and simpler. Once Vim9 is done I might have a go at that.

…

-- hundred-and-one symptoms of being an internet addict: 22. You've already visited all the links at Yahoo and you're halfway through Lycos. /// Bram Moolenaar -- ***@***.*** -- http://www.Moolenaar.net \\\ /// \\\ \\\ sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ /// \\\ help me help AIDS victims -- http://ICCF-Holland.org ///

[jclsn]

Once Vim9 is done I might have a go at that.

I am looking forward to it! The Gitlab Copilot plugin will give Vim code completion features of modern IDEs like CLion. I really don't think that this will work with popups or without shifting text: https://www.youtube.com/watch?v=lAYSPU2swAg

[lukepass]

I second this request, as @jclsn wrote Copilot plugin (which is very quick and useful) only works with Neovim because of the missing ghost text feature:

Thanks.

[brammool]

The problem with mouse positioning after concealed text should be fixed now.
Work on virtual text started, but it will take a long time before it's fully working.

[jclsn]

@brammool Cool, thank you :)

[brammool]

first version of virtual text, using text properties, has been implemented. What I tried works, but it's likely some problems will be uncovered when actually using it. Anyway, this issue can be closed. If something doesn't work well, please create a specific issue for that.

[jclsn]

Awesome!

@tpope Go for it!

[lukepass]

Thanks a lot @brammool! Do you think that the Copilot plugin needs to be adjusted by @tpope or it will work automatically? Thanks again to Tim Pope too.

EDIT: I think that the plugin will require changes because it checks for Neovim:

https://github.com/github/copilot.vim/blob/release/autoload/copilot.vim#L8

[jclsn]

@lukepass I doubt that that the API is the same as in Neovim. So copilot won't work out of the box.

[lukepass]

It seems that now Copilot supports Vim's virtual text. Many thanks to @tpope and @brammool for being so active in the Vim community!

github/copilot.vim@da286d8

https://github.com/github/copilot.vim/blob/release/autoload/copilot.vim#L9

[jclsn]

Too bad my trial period just expired yesterday XD I am not willing to pay for it yet

…

Am 26.08.2022 um 11:48 schrieb lukepass ***@***.***>:  It seems that now Copilot supports Vim's virtual text. Many thanks to @tpope and @brammool for being so active in the Vim community! ***@***.*** https://github.com/github/copilot.vim/blob/release/autoload/copilot.vim#L9 — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.