VSCode plugin

[mithandir]

Hi,I tried using the VSCode plugin using the supplied configuration, but the plugin throws the following error:

[INFO] [auth] [2022-08-03T06:47:46.784Z] Invalid copilot token: missing token: 403
[ERROR] [default] [2022-08-03T06:47:46.787Z] GitHub Copilot could not connect to server. Extension activation failed: "User not authorized"

Do we need a GitHub Copilot subscription to get a working token?

[moyix]

It appears so unfortunately! It is probably possible to bypass this by changing the code of the extension, but that's going a bit beyond what I'm comfortable with.

You may want to check out this extension developed by some of my colleagues, which also talks to the OpenAI API, and so should be easy to use with FauxPilot:

https://github.com/synoet/Pincer/tree/main/extension

However, it needs a bit of updating now that the InlineCompletions API has been finalized (it was originally written for the VSCode beta). You'll probably also want to tear out the telemetry that we added for our user study.

[Xelef2000]

I think this vs code extension could be modified easily for fauxpilot.

[Venthe]

After some modifications to the VSCode plugin; I've managed to bypass the token and have a SINGLE autocompletion. And I couldn't manage to repeat that ever since :(

[Venthe]

I should have a barebones extension (code, I dunno how to upload it to marketplace) in a few hours, just teaching myself how to do so

[Venthe]

https://github.com/Venthe/vscode-fauxpilot

This is a (very) basic implementation; as I've just hacked around. It is NOT published, but it does work :)

@moyix if I would wish to publish it and work on it; can I use the name Fauxpilot? Or would you rather prefer for me not to use that name?

e: https://github.com/Venthe/vscode-fauxpilot/releases/tag/0.0.1 here you have the release. It is not tested (as I did this in dev mode only). I will spend some time on it tomorrow, gotta go back to work!

e2: Just confirmed working from VSIX, have fun!

e3: v 0.0.2 released. Including expanded prompt, enable/disable, delay and fixed annoying bug with completion not being shown

[moyix]

Sure, feel free to use the name!

[LukyBruce]

Can we update this plugin in the readme, the vscode for github copilot just does not work.

[moyix]

Do you mean it doesn't work without having a valid token for Copilot? Or is something going wrong after that too?

[xinlnix]

Do you mean it doesn't work without having a valid token for Copilot? Or is something going wrong after that too?

Yes, I also have trouble to use the Copilot vscode plugin.

[linonetwo]

@Venthe what does max token and suggestion delay mean in your plugin? These are not very clear in the Readme.

And default max token is 4, is pretty weird, I think code snippet need at least 200 token.

[moyix]

Do you mean it doesn't work without having a valid token for Copilot? Or is something going wrong after that too?

Yes, I also have trouble to use the Copilot vscode plugin.

I will try to find some time to test again with the most recent Copilot. Last time I checked there were issues with the tokenizers between Copilot and CodeGen being slightly different, so Copilot sometimes makes requests that are too long for CodeGen – I think this can be fixed by replacing the vocab.bpe and tokenizer.json files in the VSCode plugin directory with the CodeGen versions found here: https://github.com/moyix/fauxpilot/tree/main/copilot_proxy/cgtok/openai_format

I have not tested this though, just very short on time now that school is back in session :(

[petronny]

[INFO] [auth] [2022-08-03T06:47:46.784Z] Invalid copilot token: missing token: 403
[ERROR] [default] [2022-08-03T06:47:46.787Z] GitHub Copilot could not connect to server. Extension activation failed: "User not authorized"

I can help with this since this error also appears in the official vim plugin.

The plugin uses https://api.github.com/copilot_internal/v2/token to get a token.
So we can hack the address to http://localhost:5000/copilot_internal/v2/token to bypass this step.

It should return a json string with token, expires_at and refresh_in. But I don't know the format of expires_at since I have no subscription to copilot.

Updated: I've tried out the minimal output of https://api.github.com/copilot_internal/v2/token. See #72 .

[fdegier]

The next release of the GitLab VS code extension will have support for FauxPilot. Welcoming contributions 😄

Extension:
https://gitlab.com/gitlab-org/gitlab-vscode-extension

Merge Request:
https://gitlab.com/gitlab-org/gitlab-vscode-extension/-/merge_requests/648

[Venthe]

@Venthe what does max token and suggestion delay mean in your plugin? These are not very clear in the Readme.

And default max token is 4, is pretty weird, I think code snippet need at least 200 token.

Hey, sorry for missing this comment. To be completely honest, I do not remember why I've set it this way. This is / was a rough implementation only; just to have something working.

Delay: This sets the time for the debounce-like mechanism
Max token: I do not know, but see https://help.openai.com/en/articles/4936856-what-are-tokens-and-how-to-count-them . As for the value 4, it seems low indeed; I'll update the defaults

[shadowwider]

After testing several extensions I ended up going with github copilot. It seems more faster,more features and smarter ，like set different stopwords in different situations.
And I find a way to use it without subscribe， you can try. Like moyix said we must to rewrite raw extensions code for skip the boring authentication process.

1. set github copilot extension config

"github.copilot.advanced": {
        "model": "fastertransformer",
        "debug.overrideEngine": "codegen",
        "debug.testOverrideProxyUrl": "http://192.168.1.161:5000",
        "debug.overrideProxyUrl": "http://192.168.1.161:5000"    
    },

2. Use the attached js file to override copilot raw file on ~\.vscode\extensions\github.copilot-1.54.7077\dist\extension.js（you can do it by youself, the file I provided only for version 1.54.7077 ）
3. Set copilot to Ignore updates or you will lost the Hack code

Another thing is I must change this to yield f'{json.dumps(completion)}. Seems stream fomart are different.

[pai4451]

Hi @shadowwider, where is the attached js? I also want to use GitHub copilot extension with fauxpilot server.

[shadowwider]

Hi @shadowwider, where is the attached js? I also want to use GitHub copilot extension with fauxpilot server.

https://github.com/shadowwider/brook/releases/download/v1/extension.js
you can try this

[dewacandra4]

I encounter this error while using the copilot plugin [ERROR] [streamChoices] Error parsing JSON stream data.
Is there anything that I missing?

[thakkarparth007]

The Copilot extension needs well formatted streaming response, but the copilot_proxy code seems to return badly formatted data. Here's an example of the streamed response:

data: data: {"id": "cmpl-6TMRfgynXT3wpdrNOVZgS4is5DadC", "model": "codegen", "object": "text_completion", "created": 1671816259, "choices": [{"text": ",\n        )\n        return self._s", "index": 0, "finish_reason": "length", "logprobs": null}], "usage": {"completion_tokens": 10, "prompt_tokens": 1, "total_tokens": 11}}
data: 
data: 

data: data: [DONE]
data: 
data: 

As you can see, there's data: data: {..actual_json_stuff..}, whereas it should just be data: {..actual_json_stuff..}. I'm unable to determine what's adding the first "data:". The second "data:" is coming from code in utils/code_gen.py. Removing the data: prefix there seems to fix this issue -- I do see inline suggestions.

However, I think we need to make some more changes on the backend (preferrably in the fastertransformer backend, otherwise at least in the proxy) for better integration with the Copilot extension (if that's desired).

For instance, currently the "stop tokens" logic is not correct ( #85 ), because of which the client gets an overly long response. In my case, 500 tokens were being returned, including logprobs. The client then just trims the result with appropriate stop tokens. This results in wasted compute and poor latency. Ideally, the stopword logic would be correctly handled in the fastertransformer backend itself, however that's not happening.

[brandonvessel]

Hello all. The comments above have been very helpful in setting up the Copilot extension. I managed to get it to work with my instance and figured I would combine the steps I used (this is for Windows. Linux installation is similar, just different locations):

1. Go through the standard Fauxpilot server setup. If the docker containers are running and are query-able with curl, move ahead. I used @Frederisk's fauxpilot-windows fork for the launch.ps1 and setup.ps1 scripts. More or less, it's the same. (source: https://github.com/Frederisk/fauxpilot-windows)

2. Install GitHub Copilot into VSCode (we will defang some of this later).

3. Add the GitHub copilot overrides to the settings.json file. These steps are outlined for this in https://github.com/fauxpilot/fauxpilot/blob/main/documentation/client.md (Copilot Plugin).

4. @petronny has some steps for getting VIM to work with Fauxpilot (VIM support with official copilot.vim #72). We will use these steps to fix the VSCode version. Find and replace the following strings in the "%userprofile%.vscode\extensions\github.copilot-#.##.####\dist\extension.js" file:

• https://api.github.com/copilot_internal -> http://127.0.0.1:5000/copilot_internal
• https://copilot-proxy.githubusercontent.com -> http://127.0.0.1:5000

This is essentially a manual override for the GitHub token retrieval to use our local instance instead.

5. Not all telemetry is removed from the extension. Removal of all the telemetry may violate the TOS for Copilot. Do so at your own discretion. It will, however, not send your code to Microsoft.

6. Make sure your extensions are not set to automatically update or the extension.js changes will be reverted when GitHub Copilot updates.

7. Copilot should behave as usual, but is now using Fauxpilot's docker setup for token generation instead. There are occasional issues with token amount as mentioned above, but these are the steps I used to get it at least running.

[hsooya]

@Venthe First of all thanks for making the plugin! It worked for me. I was looking at the source code for your plugin. I was wondering how much of the plugin you have adapted from Open AI Co-pilot plugin. The reason I was asking this is merely coz I want to make sure that there is no way that the extension can contact the OpenAI servers due to security policy of being able to use it at work.

For example, even when we set the server address to the local codegen one using config in the OpenAI co-pilot plugin ( that is used for recommendation generation) , there were traces of code responsible for token handling that were contacting Github.

I'm gonna try to read more about VSCode plugins in the meantime. Again, thanks for your plugin.

[Venthe]

@Venthe First of all thanks for making the plugin! It worked for me. I was looking at the source code for your plugin. I was wondering how much of the plugin you have adapted from Open AI Co-pilot plugin. The reason I was asking this is merely coz I want to make sure that there is no way that the extension can contact the OpenAI servers due to security policy of being able to use it at work.

For example, even when we set the server address to the local codegen one using config in the OpenAI co-pilot plugin ( that is used for recommendation generation) , there were traces of code responsible for token handling that were contacting Github.

I'm gonna try to read more about VSCode plugins in the meantime. Again, thanks for your plugin.

Not at all, happy to provide.

It was - and is - basically a hack done in few hours (and my first plugin for vscode); and the only parts that I've been "using" from the other repo was the actual API call. At the time there was no option to use fauxpilot in Code without severe tinkering with original copilot plugin (basically removing the token handling and more as far as I remember).

At this point the plug-in for me is done as it is working fine; I'm only improving it as per feature requests though considering that Gitlab has made their own version of fauxpilot client, i believe that mine is quite obsolete at this point

[fdegier]

Hi @Venthe

At this point the plug-in for me is done as it is working fine; I'm only improving it as per feature requests though considering that Gitlab has made their own version of fauxpilot client, i believe that mine is quite obsolete at this point

GitLab actually (very unfortunately) dropped support for FauxPilot https://gitlab.com/gitlab-org/gitlab-vscode-extension/-/merge_requests/697

It might be worth adopting some of the mechanisms from that plugin such as the debounce mechanism, yours doesn't actually work as intended. I think it would be a good idea to adopt your plugin into the FauxPilot organization since you have the plugin already published on the marketplace. WDYT?

But to answer @mithandir's question, ideally the plugin for FauxPilot should adopt the interface from OpenAI in the bare bones setup so we are 100% sure no data is send to OpenAI (now or in the future).

[thakkarparth007]

Sourcegraph recently open-sourced their client Cody (https://github.com/sourcegraph/sourcegraph/blob/main/client/cody/src/completions/index.ts). They're still building it, but that's also a good opportunity to contribute early enough so that there's support for FauxPilot.

[Venthe]

@fdegier

GitLab actually (very unfortunately) dropped support for FauxPilot https://gitlab.com/gitlab-org/gitlab-vscode-extension/-/merge_requests/697

Good to know, thanks

It might be worth adopting some of the mechanisms from that plugin such as the debounce mechanism, yours doesn't actually work as intended.

Hahah, yeah, it does not. I've cobbled a dirty solution because the only library that I've worked with that had debounce was heavy, and I needed it to "at least" work. I knew that it will catch up to me:)

I think it would be a good idea to adopt your plugin into the FauxPilot organization since you have the plugin already published on the marketplace. WDYT?

No problem with that; I can keep it in my namespace or move it to org; but this is not ultimately question for me I believe. My only "requirement" is for it to stay MIT or similar license.

(...) ideally the plugin for FauxPilot should adopt the interface from OpenAI

Please bear in mind; that I have zero understanding about the API - that's why I've specifically noted that it was a quick job, the plug-in. I'm happy to contribute, but I'll need at least some guidance in that regard.

[AdithyaRamanChippa]

hello @brandonvessel I am not able to find this occurrence https://api.github.com/copilot_internal in my extension.js file

[brandonvessel]

@AdithyaRamanChippa I have observed this change as well with the recent changes to the extension. This combability was originally confirmed by me on version v1.77.9225 of the GitHub Copilot extension.

[tolsadus]

After testing several extensions I ended up going with github copilot. It seems more faster,more features and smarter ，like set different stopwords in different situations. And I find a way to use it without subscribe， you can try. Like moyix said we must to rewrite raw extensions code for skip the boring authentication process.

1. set github copilot extension config

"github.copilot.advanced": {
        "model": "fastertransformer",
        "debug.overrideEngine": "codegen",
        "debug.testOverrideProxyUrl": "http://192.168.1.161:5000",
        "debug.overrideProxyUrl": "http://192.168.1.161:5000"    
    },

2. Use the attached js file to override copilot raw file on ~\.vscode\extensions\github.copilot-1.54.7077\dist\extension.js（you can do it by youself, the file I provided only for version 1.54.7077 ）
3. Set copilot to Ignore updates or you will lost the Hack code

Another thing is I must change this to yield f'{json.dumps(completion)}. Seems stream fomart are different.

What are the changes you made in the extension.js, do you have a diff maybe? I'll like to implement this for 1.143.0

[shadowwider]

After testing several extensions I ended up going with github copilot. It seems more faster,more features and smarter ，like set different stopwords in different situations. And I find a way to use it without subscribe， you can try. Like moyix said we must to rewrite raw extensions code for skip the boring authentication process.

1. set github copilot extension config

"github.copilot.advanced": {
        "model": "fastertransformer",
        "debug.overrideEngine": "codegen",
        "debug.testOverrideProxyUrl": "http://192.168.1.161:5000",
        "debug.overrideProxyUrl": "http://192.168.1.161:5000"    
    },

2. Use the attached js file to override copilot raw file on ~\.vscode\extensions\github.copilot-1.54.7077\dist\extension.js（you can do it by youself, the file I provided only for version 1.54.7077 ）
3. Set copilot to Ignore updates or you will lost the Hack code

Another thing is I must change this to yield f'{json.dumps(completion)}. Seems stream fomart are different.

What are the changes you made in the extension.js, do you have a diff maybe? I'll like to implement this for 1.143.0

I'm sorry, I haven't had a new version for a long time and I've forgotten about it now. Maybe you could try running a git diff between my file and the corresponding official version's source file to see what changes were made; there should be very few. It's just that I removed a few parts for verifying accounts, which doesn't affect the basic logic