Extend API to support snippets from completion managers and LSP clients

[andreyorst]

This is not an issue, but a feature wote post.

TL;DR

Extend UltiSnips API to support snippet definition from completion engines, and extend syntax to support LSP style snippets from LSP clients.

Preface

Modern completions managers, can gather info based on semantic code analisis, that comes from Language Server Protocol (LSP), and various Lint Engines, or by the plugin itself. Some are containing it's own lint realisation, some integrate well with other plugins. That means, that plugins can define snippets inside them, based on user input, and file contents. Easiest example can be parameter expansion. For example, let's assume that you have some function with several parameters defined in your file:

void function_with_parameters(const string& name, const string& pet_name)
{
    // use name and pet_name somehow
    // ...
}

int user()
{
    // very important code, that lacks that function with those names
    // I wonder, what name should go first, Stieve, or Fluffy?..
}

And you want to use it somewhere in user function, but you defined it long time ago, and don't remember how exactly names should go. You can search for it, or check tags in tagbar, or you can use snippet, generated for you by semantic completion engine, or by LSP. So you type:
fwpTab
and your lightning fast fuzzy matcher completion engine completes it like so:

int user()
{
    // ...
    function_with_parameters(|name, pet_name)
}

Where | is your cursor position, and name among with pet_name are just tabstops ${1:name}, ${2:pet_name} supported by Ultisnips, or other plugin. Therefore you just replace placeholders by jumping through, and finish by adding ; at the end, or execute some member function, that again will have semantic generated snippet. Super convenient. Here gif to demonstrate how Neovim Completion Manager (NCM) does such cool stuff:

There are may be other situations where you wouldn't write a snippet, because it might not be used again like never, but you don't hesitate when someone generate it to you simultaniously while you're typing.

So LSP does that.

Other plugins do it too.

UltiSnips is great engine.

...

Lets combine best of both worlds?

LSP

From LSP github page:

The Language Server Protocol (LSP) defines the protocol used between an editor or IDE and a language server that provides language features like auto complete, go to definition, find all references etc.

Since there are bunch of LSP plugins, that may expose this snippet feature, I think that Ultisnips should support it. LSP documentation is complete, and considered stable. Lots of servers apper each month, there are lots on their pages already, and it's not complete list, as far as I know.

Here is the documentation to LSP Snippets: https://github.com/Microsoft/language-server-protocol/blob/master/snippetSyntax.md

By the quick lookup it seems that Ultisnips already meet the requirements in most places. Exceptions might be choise feature.

Language Client Neovim (LCN) already supports LSP snippets as I can see:

(that's parameter expansion done by LCN + UltiSnips + Deoplete bundle. Without UltiSnips completed result doesn't contain placeholders)
But seems that deoplete can't handle it well.

Standard interface for plugins

Also standard interface for using snippets generated by plugins was proposed in #888. Other plugins like Deoppet planning to support LSP snippets, so this will become common feature in the future. Other editors are using it too. Completion managers already play nicely with snippet managers, most of them are do it with their own facilities, so no further maintaince really needed to support new completion engines, if standard interface wil be defined. It will be plugin writer task to support Ultisnips, Deoppet, and others, but with unifyed interface it will be lot easier. As LSP already defines some rules to snippets, I think they may be applied to Ultisnips, and it is already mostly compatible.

While I know that this is a lot of work, and may lead to rework plugin entierly, but this is new feature, that can boost productivicy, and Snippet engine was desined with boosting work in mind, so I think it is path of further development, that should ve accepted.

[SirVer]

I am confused. This is already supported: https://github.com/SirVer/ultisnips/blob/master/doc/UltiSnips.txt#L454

Or am I missing something?

[andreyorst]

Hmm, I've thought, that UltiSnips#Anon is used to define snippets from config file, like this abbreviation

inoremap <silent> $$ $$<C-R>=UltiSnips#Anon(':latex:\`$1\`', '$$')<cr>

Maybe I misunderstood this part of documentation.

So it could be used by other plugins to talk with Ultisnips? For example if LSP wants to add a snippet in current scope, because user's input matches some function definition for example, it can give it for ultisnips by executing

UltiSnips#Anon('foo_name(${1:par1}, ${2:par2})', foo_name)

And it will appear in the list of completions, like other snippets do?

I think that by implementing an interface, I meant that LSP might want to define all of it's possible snippets somewhere (on fileopen for example), so Ultisnips could to look through them too, but using it's default mechanism. Like searching for snippets in the file UltiSnips/ft.snippets, but seaching for them in RAM, or temporary file, created by LSP. So all the snippets will have same semantics, and behaviour. This will give us groving list of snippets, that will contain LSP, or other plugins snippets.

Edit:
I think that it is done right in NCM plugin (maybe you should try it, for better understanding of the mechanics). It creates a snippet wich is listed alongside with other snippets, and user can choose to trigger it, or not. Can't imagine how to do it with UltiSnips#Anon, because it expands a snippet, but not defines it in snippet list. It's kinda different behavior.

ultisnips/doc/UltiSnips.txt

Lines 456 to 458 in 8ec3a61

	The second function is UltiSnips#Anon(value, ...). It expands an anonymous
	snippet. Anonymous snippets are defined on the spot, expanded and immediately
	discarded again. Anonymous snippets are not added to the global list of

Or maybe I still don't understand documentation the right way.

[SirVer]

There are two ways external plugins can add snippets to UltiSnips.

1. UltiSnips#Anon can be used to define and immediately trigger a snippet. I think this is the common case for LSP - once the completion plugin identifies which snippet the user wants from the ones LSP provides, it uses the Anon function to send the snippet definition to UltiSnips and it will immediately expand the snippet.
   
   As for choice, this can be nicely simulated using the complete functionality as vim_snippets uses it:
   
   

2. The second option is using python and adding a new SnippetSource. That way you can dynamically add and remove snippets or generate them on the fly as you like.

I think for this use case, 1 is the right choice and everything seems to be in place to support it already. Only completion plugins need to be written to call into UltiSnips, I doubt we need to support more on our end.

If there is need to understand and parse the LSP snippet format, that would be fairly easy to do, but before we do this work, we should have somebody interested in this functionality in their completion plugin.

[andreyorst]

I wonder what @Shougo or @roxma may think about this. As an authors of completion plugins they may know mechanics better and maybe propose another variant. Shougo also doing his own snippet plugin, so maybe he alredy did something in this direction, perhaps all snippet plugins should use mostly similar ways of implementing this.

Roxma isn't active on github for almost half of a year, so I don't expect his apperiance here in near future, but since he uses ultisnips for parameter expansion in his NCM plugin, decigions that may be posted here may be interesting for him in the future, if he will continue working on NCM, ofcourse.

I'm kinda liking how it is done in NCM, despite that the completion that NCM provides is quite messed up. But as it generates useful function snippets, i'm missing such feature in deoplete.

[andreyorst]

@SirVer I still don't understand how can UltiSnips#Anon be used, to communicate with UltiSnips.
I read documentation once again, and did slightly differently. Instad of making a

inoremap $$ $$=UltiSnips#Anon(':latex:`$1`', '$$')

I tried to call it directly and nothing happened. I've tried to call it while my cursor is at $$ trigger, and yet, nothing happaned once again. As stated in documentation:

It expands an anonymous snippet. Anonymous snippets are defined on the spot, expanded and immediately discarded again.

Wich isn't quite look like a way to communicate. Well plugin may tell Ultisnips to expand a snippet, but then the user can't decide if hi want to expand or not. If completion manager want's just to define snippet, and let user choose to expand it or not, this function isn't handy at all.

---

As I can't use UltiSnips an my GPD Pocket, and phone, I've decided to develop a plugin to use on those devices. As part of a plugin I've created a function, that lets define a snippet, wich can be used later, but not store it at hard drive. Take a look:

Here I add a snippet by hands, but this process can be automated if completion plugin will know about such function. Later these snippets can be used, until Vim is exited, because snippets are stored inside scriprs variable. That leads us to situation when completion managers and oter plugins, like linter and language servers are adding snippets and user deciding to expand it or not.

[butterflyfish]

i'm looking forward somebody address problem.