Integrate multi-file context

[nenkoru]

I was wondering that it would be nice-to-have multi-file context by integration of usage of local vector storage.
For instance I would like to have a multi-file context for current completion.
Consider this case:

def get_time(): return 101

import utils
print(utils.get_t

I would like the plugin to complete the rest of the function call for me.
For now we are only getting the context before and after a cursor.
But what if we could get a search within a vector store and embed those pieces within a context? Particularly in before window.

[nenkoru]

I believe to make it less dependent on a specific vector db or a piece of software that could provide those chunks of code to inject into the context window, we could make a unified REST API interface. So that you just need to implement and run a local(or a remote) web-server which would provide a simple /search functionality for the plugin.

[McPatate]

I was thinking about how making the context of the prompt more relevant to the actual piece of code being worked on.
My initial idea was to look into working with treesitter to fetch the different functions being called in the current scope of the cursor position. I think having a vector db filled with the different symbols and pieces of code may be simpler to use and while not being perfectly relevant might bring in additional context based on proximity that would not be present with a treesitter approach.
Here's a good resource on how the Copilot plugin internals work - yet another approach to building a good prompt.

[nenkoru]

Alright, I have tried replicating the same example of a prompt provided in that article - and it worked.

https://gist.github.com/nenkoru/fc6f0d517655a4cded9ac321611fa9fa

Try to autocomplete print statement. Like remove the invocation of a function within print statement and see what happens.
Also you could try asking for a list of functions in utils.timezone module. Tried with WizardCoder 1.0. FIM enabled.

[nenkoru]

Now what's left is to find the way of how to obtain those additional snippets, as well managing the maximum context of a model that is being used.

[McPatate]

Now what's left is to find the way of how to obtain those additional snippets

The hard part 😄

as well managing the maximum context of a model that is being used

I think for that at the moment we can rely on the config you added and let users set the correct values.

We're going to have to think about setting values in the before or after part when fim.enabled == true. Maybe instead of having two configuration fields before/after length, we can have only one context_length parameter and devise the appropriate strategy to create the prompt.

[McPatate]

It also works nicely with starcoder*

[McPatate]

I would like the plugin to complete the rest of the function call for me.

Building a hfcc-cmp was on my todo-list, I haven't looked into how copilot-cmp or cmp-tabnine are built yet, so I don't know if the vector db approach is overkill, meaning there may be a simpler solution.

[McPatate]

Alright here it is : https://github.com/huggingface/ccserver

I've still yet to decide what method of the LSP protocol we'll be using for completions. As I said above, we can leverage vim.lsp.rpc.request() to send requests to the LSP in hfcc. Lots to do!

[nenkoru]

Alright, was quite busy this week.
So what's the current status and what I could take to implement?

[McPatate]

No probs! Gave me some time to start hacking :)

I'm currently trying to be +/- "iso-feature", almost done, you can check #32 & huggingface/llm-ls#2 to see where I'm at.

Once that's done, I was planning to start experimenting how to build a good prompt. So either looking at the vectordb route or other things. I'd suggest waiting for me to finish at least that first layer before starting coding on your side, I have some cleaning up to do 😄

[McPatate]

Some of the things I'd like to explore, in no particular order:

• building a good prompt
• request caching
• improving request latency
• run a model locally (thinking of using https://github.com/huggingface/candle)
• improve multiline / single line completions

[McPatate]

@nenkoru merged, let's get started!