Recently modified and open files

[ilan-schemoul]

Cursor and SuperMaven, I believe, send list of recent modification (what files modified, what was modified) to improve autosuggestion.
So I go in one file and add setCurrentDate() {} then go to another file it immediately knows I want to add setCurrentDate there and offers that. I think supermaven has some implementation here ?
Is this something you'd like to add to VectorCode ? Or is this out of scope ?
Because I'd like to add such a things to minuet AI but unsure how to proceed and if VectorCode is the right place to handle this ?

[Davidyz]

Hi, imo there are 2 ways that you can approach this:

1. Add a prompt component that contains the edit history. The LLM would receive a list of diffs;
2. When using the async cache API, you can write your own function for the query_cb option of the register_buffer function so that the Vectorcode search query contains recent edits. This should help getting some files that are relevant to recent edits. The LLM would receive a list of files related to recent edits.

Both methods requires obtaining recent edits, but I think only the second one will be in the scope of this project. I will take a look at it.

[Davidyz]

I've made a simpler (probably less powerful too) version for the 2nd approach I mentioned. I'm closing this issue for now because this is not directly related to the core functionality of VectorCode and can be easily configured by users. A more complicated, per-buffer diff system like the one used by supermaven would require more effort. At the moment, it won't be at the top of my priorities. I'll close this issue for now, but feel free to leave some feedback/suggestions on this subject if there are any updates!

[ilan-schemoul]

Thanks a lot

[ilan-schemoul]

For the record I tried to make a simple POC (spoiler alert: it was a shitshow) first without vectorcode here's my finding:

1. took me many hours to just get last diff. It seems to simple but well straight up using register(".") gives a ton of garbage value <80>bs whatever. You could use Davidyz implementation (I kinda forgot to look at it), he captures the output of changes. I personnally did InsertEnter I save the text from the buffer, InsertLeave I do a diff of the two texts to get the diff (couldn't find simple performant algo, one could use vim.diff but the format is a bit much for LLM maybe).
2. I tried passing the diff to codestral
   Content that follows [USER_EDIT] is a modification made by the user recently. Make use of them if necessary. [USER EDIT] def foo():
   return 4
   Content that follows [USER_EDIT] is a modification made by the user recently. Make use of them if necessary. Perform fill in the middle completion in the language typescript based
   on the following content. [USER_EDIT] ar.recipientSix();
   [PREFIX] ....
   Results are garbage non sense for some reasons it breaks the LLM.
   VectorCode uses something similar for context tho ?

Moral of the story: fiddling with LLMs seemed harder than expected, when they're not happy they just spit out nonsense and I don't see a way to debug them as they're weird black box. Also nvim sucks for getting last insertion txt (or maybe there's a way to filter the stupid <80>bs characters idk)

[Davidyz]

From my personal experience, smaller LLMs tend to perform badly when there is extra context in the prompt (that is, context that they're not trained on or supplied in a format that they're not trained on). For example, qwen2.5-coder only works well with repo-context when using the specific separators. I'd suggest trying to use this with larger models as they usually understand the context better, and may understand delimiters that they're not specifically trained on.

[ilan-schemoul]

Welll codestral is 22B. 22B seems already a lot for just autocompletion. Aren't bigger models slow for autocompletions (what I read on the internet).

[Davidyz]

Welll codestral is 22B. 22B seems already a lot for just autocompletion. Aren't bigger models slow for autocompletions (what I read on the internet).

True, but that kinda depends on how you use it. For example, I prefer to manually trigger AI completion when I feel like I need it. In this case the speed difference between a 22b and a 600b model doesn't matter that much because I'll just be waiting anyways.

[ilan-schemoul]

Oh I feel like that's a non typical use and I think it partly explains the need for your plugin, which is kinda heavy for typical use case of autocompletion (in terms of tokens, CPU/GPU clocks etc.). If it works for you great obviously. I believe most people, including me currently, use autocompletions for "dumb" stuff (boilerplate, init, obvious error handling, logging, repetitive test, repetitive test set up/teardown and so on). For thinks requiring thinking AI, I believe stuff like chat or inline actions (such as provided by codecompanion and many others) are used.
At least it's the workflow I believe is the most common and intend to follow. I really like automatizing quickly all the stuff I do not want to write (and it speeds me up a lil, like nothing massive but it's something).

[milanglacier]

maybe this can help if you want to get the recent edit history.

local changes = vim.fn.execute "changes"

However you will not get a structured data representation, you will only get a string as printed changes.

I do agree that getting the recent edit history is cumbersome in vim.

[ilan-schemoul]

Output is weird tho, full of --invalid-- things etc
It kinda sucks and anyway doesn't play well with any LLM that's not a big chat one but I only use small fast FIM