Provide an async api to work with other plugin like asyncomplete.vim

[kepbod]

I want to make tmux-complete.vim work with asyncomplete.vim, but there is no document about it. Thanks to @prabirshrestha, he mentioned that it would be better for tmux-complete.vim to have an an async api so that tmux-complete.vim could work asynchronously. So I am wondering wether you could offer an async api in tmux-complete.vim as mentioned here. Thanks a lot.

[wellle]

Hey, thanks for the interest. I'll see what I can do 👍

[prabirshrestha]

Here is my take on this. (this is me writing without actually trying it and by glancing at tmux-complete.vim source, so there could be bugs here)

Here is what the asyncomplete source would look like. You might want to have another parameter to detect if there is an error or not.

function s:completor(opt, ctx)
  call tmuxcomplete#get_async_completions({ candidates, startcol -> asyncomplete#complete(a:opt['name'], a:ctx, startcol, candidates) })
endfunction

call asyncomplete#register_source({
    \ 'name': 'tmuxcomplete',
    \ 'whitelist': [*],
    \ 'completor': function('s:completor'),
    \ })

Here is what the async completions for tmux complete would look like. I'm using async.vim to normalize jobs between vim8 and neovim. If you want to avoid external plugin dependency you can also embed async.vim in your plugin https://github.com/prabirshrestha/async.vim#embedding

function! tmuxcomplete#get_async_completions(cb) abort
    let l:cmd = tmuxcomplete#getcommand('', 'words')
    let l:opt = {
        \ 'callback': a:cb,
        \ 'buffer': '',
        \ 'startcol': tmuxcomplete#complete(1, '')
        \ }
    let l:jobid = async#jobs#start({
        \ 'cmd': l:cmd,
        \ 'on_stdout': function('s:on_stdout', [l:opt]),
        \ 'on_exit': function('s:on_exit', [l:opt]),
        \ })
    " if l:jobid is <= 0 failed to start job
endfunction

function! s:on_stdout(opt, id, data, event) abort 
    let l:opt['buffer'] = l:opt['buffer'] . join(a:data, "\n")
endfunction

function! s:on_exit(opt, id, exitCode, event) abort
    " handle bad exit code
    call a:opt['callback'](split(a:buffer, "\n"), l:opt['startcol'])
endfunction

There are other optimizations you could do. If you know your command always splits candidates based on "\n" you could use words as array instead of buffer as string so you don't have to join and split.

[wellle]

@prabirshrestha: Wow, thanks for stepping in! I did look at some of your plugins and documentation on Tuesday to see how this would be done. I have to say they all looked really good and I think I found the pieces I would have needed.

I will try to look into this over the weekend, thank you very much 👍

[wellle]

@prabirshrestha: Thanks again! I took your suggestion and made some changes and got it working. Please have a look at #77. 🙏

Btw, one thing I noticed in its current stage. In a buffer, when I insert a single w as a new word, after a short time it now suggest wellle as a completion (taken from another tmux window). If now I add e it still suggest wellle, as we is still a prefix. On the other hand, if I quickly type we as a new word, it does not offer wellle as completion.

I suspect that asyncomplete triggers autocompletion on the first character I enter. If I enter the next one before the first completion suggestions come in, that job gets aborted, but it seems like no new job for the new prefix is triggered. I tried similar things with the gocode completion and couldn't manage to reproduce the issue there. Maybe they are just quick enough with their suggestions, or is there any sort of option I might be missing? Let me know if you'd want me to open an issue about this in your asyncomplete.vim repo. I just want to check here quickly in case I'm missing something simple.

[prabirshrestha]

@wellle What you experienced is actually the current feature of asyncomplete.vim and not purely a bug. It is a known issue that I would like to solve in the near future.

I suspect that asyncomplete triggers autocompletion on the first character I enter.

Yes.

If I enter the next one before the first completion suggestions come in, that job gets aborted, but it seems like no new job for the new prefix is triggered

Partially correct.
If you enter the second one before the suggestions come, the job actually doesn't get aborted. When you call asyncomplete#complete one of the parameters you pass is context. Internally asyncomplete uses asyncomplete#context_changed(ctx) to know if the context (cursors changed). If it does change it ignores the candidates. Hence you didn't see the list updated.
You can force refresh anytime if you have a mapping set. imap <c-space> <Plug>(asyncomplete_force_refresh).
Before asyncomplete used to auto force refresh but I noticed that with some sources that are too slow it becomes very chatty so I disabled this by default. You can use let g:asyncomplete_force_refresh_on_context_changed = 1 If you like to auto refresh by default. Seems like I need to allow this value to be changed per source (prabirshrestha/asyncomplete.vim#38).

seems like no new job for the new prefix is triggered

when you call asyncomplete#complete it actually caches the result based on your source name and the startcol. If you have server.sta| and at | you force refresh you need to set you startcol at where . is and not where | is. That means you need to provide all completions for server. and not server.sta which is startServer, stopServer and restartServer. asyncomplete would then automatically use the prefix to filter out candidates. This makes more sense when there is fuzzy search implemented (currently not supported but in PR prabirshrestha/asyncomplete.vim#16). I can type sS and it would filter out restartServer.

If you really do want to call your completor function on every prefix change it is possible too. asyncomplete#complete(name, ctx, startcol, candidates, 1). Just make sure to pass 1 which is an optional parameter defaulting to 0. It tells asyncomplete that the candidates is incomplete so make sure to call completor again if the user types a new character. This is usueful if you are making an asyncomplete source for github user search. github.com/p. It is imposible to return all the results for users starting with p so you just return partial results. Then when the user types pr you again search and tell the result is incomplete. This way you can create a powerful autocomplete for github users, projects, npm packages and so on. Unlike other autocomplete plugins which waits for all sources to return, asyncomplete shows the results as the results come in making it purly async as well as fast.

If you are interested in reading more about the algorithm you can read more of it at roxma/nvim-completion-manager#30 (comment)

[wellle]

@prabirshrestha: Thanks for all that information, that's very helpful!

I tried the suggested imap and that does indeed work to trigger completion again in that case. It works, but it doesn't feel great to have to do that.

When I tested I did actually have g:asyncomplete_force_refresh_on_context_changed set, so that doesn't actually seem to work in this case. It's the same when I add the , 1 parameter to asyncomplete#complete. I tested a bit and here's what I found (with either of these changes made):

1. If I type w and wait for the completions, then on every subsequent character my completor gets called again.
2. But: If I type we quickly enough so the completions (which were triggered on w) didn't show up at all (the case I called "aborted"), then on subsequent characters my completor does not get triggered again.

I did however find a hack that seems to make it work: If in my completor I add this line before actually starting the async job (04d2aa7):

call asyncomplete#complete(a:opt['name'], a:ctx, l:startcol, [''], 1)

Then, even in the case where I type very quickly, the results show up in the end. So I guess by sending a first completion (empty, doesn't match anything) I open the door to send more completions later, even if the context changed in between. Seeing that this works, I wonder if asyncomplete couldn't take care of this itself? I mean it could add this first empty completion on its own (as incomplete set) to make this case work. But I imagine there must be a cleaner way to set up the internal structure than this hack.

[kepbod]

You guys are so nice. Thanks a lot. : )

[prabirshrestha]

@wellle Seems like could be a bug in asyncomplete. The source code is quite small so feel free to have a look at it there.

All async vim autocomplete plugins are currently based of timer hacks, so it could also be an issue with timing. What if you set the g:asyncomplete_completion_delay to a smaller value like 30. Currently it defaults to 100. Do you still have the issue? Side note: I have also sent a PR to vim to make autocomplete authors work without timer hacks at vim/vim#2372. Discussions at vim/vim#1691.

call asyncomplete#complete(a:opt['name'], a:ctx, l:startcol, [''], 1) actually seems like a good hack although it would be good if asyncomplete auto handled this. But this one is a bit tricky since it would be difficult for asyncomplete to know if it can use the cached results or not primarily because the prefix could be something totally different for a slow sources. This also means every character typed will now start calling your completor function until it is set to 0.

Assume server.s| triggers the completer function i.e. the prefix is server. and the startcol is 7 (assuming start index is 1). If the source is slow, it could return after typing server.star| at 11th index. But if it returns startcol as 7 and the prefix is the same, instead of throwing away the result asyncomplete could be smart at reusing it. We would need to change the following function.

function! asyncomplete#context_changed(ctx) abort
    " return (b:changedtick!=a:ctx['changedtick']) || (getcurpos()!=a:ctx['curpos'])
    " Note: changedtick is triggered when `<c-x><c-u>` is pressed due to vim's
    " bug, use curpos as workaround
    return getcurpos() != a:ctx['curpos']
endfunction

[wellle]

@prabirshrestha: Thanks! I just tried let g:asyncomplete_completion_delay = 1 and even then I can easily type quick enough to not have autocomplete show up. The hack seems to work so far so I guess I'll just leave that in for now. I understand your concerns about changing this in a general way, but maybe you want to mention this in your docs somewhere so other "slow" completions might benefit from it for the time being. I wish you good luck for your PR, would be great if we could all benefit from less hacky autocompletion mechanisms. Thanks for all your work on that 👏