Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce exclusion patterns for files that shouldn't be sent over the network #1049

Closed
Tracked by #921 ...
beyang opened this issue Sep 13, 2023 · 5 comments · Fixed by #1382
Closed
Tracked by #921 ...

Introduce exclusion patterns for files that shouldn't be sent over the network #1049

beyang opened this issue Sep 13, 2023 · 5 comments · Fixed by #1382
Assignees
Labels
chat/commands Chat and Commands clients/vscode enhancement New feature or request

Comments

@beyang
Copy link
Member

beyang commented Sep 13, 2023

Cody should exclude .env files and other files that may contain secrets from being sent over the network.

One potential solution here is to exclude files that aren't part of the Git repository.

Customer requests:

@jdorfman
Copy link
Member

@abeatrix
Copy link
Contributor

abeatrix commented Oct 18, 2023

10/18

RFC create, pending for review : https://docs.google.com/document/d/15XJnQ-qa-Y3SM2eltuKfuESmgYYfx7JhEdi2447YF_0/edit?usp=sharing

11/3

Haven't gotten to follow up on this issue since I started working on #1377, but I've discussed with Danny regarding the WIP PR draft: #1549 on Slack and made some updates based on his review. The PR is in a working state but Danny has suggested a better way to handle the ignore file, tbd.

Slack thread: https://sourcegraph.slack.com/archives/C05AGQYD528/p1699006480263549

11/5: Danny is adding supports for nested workspaces in : #1634

@beyang
Copy link
Member Author

beyang commented Dec 11, 2023

@DanTup @abeatrix what's the latest status on this? Still to be done?

@DanTup
Copy link
Contributor

DanTup commented Dec 11, 2023

I think Beatrix was/is working on supporting multi-folder workspaces but I don't know if it's complete yet. The current PR is at #1382 but looks like it has a bunch of merge conflicts now that will need resolving.

@abeatrix
Copy link
Contributor

The work was done in the linked PR, including working with milti root repo and nest repo, and was waiting for Cody strat team to confirm but then became stale because it was implemented before simple chat was introduced and doesn't work with bfg yet. We will have to pick it back up after GA as we discussed in one of our product meetings @kalanchan

abeatrix added a commit that referenced this issue Dec 29, 2023
CLOSE #1049

For details, please see [RFC 852 File Exclusion Patterns for
Cody](https://docs.google.com/document/d/15XJnQ-qa-Y3SM2eltuKfuESmgYYfx7JhEdi2447YF_0/edit?usp=sharing)

feat: filter context from ignored files

> IMPORTANT: Currently behind the `cody.internal.unstable` config and
must be enabled to try the feature. When this is out of internal
experimental stage, this should be enabled by default.

- Add `isCodyIgnoreFile` utility to check if file is defined in
.cody/.ignore (at workspace level)
- Update Interaction to filter context messages from ignored files when
first created
- Update context gathering in completions to skip ignored files
- Add check in editor to skip ignored files
- ignore all `.env` files / path by default

All clients should provide file content from the .cody/.ignore file at
start up, and update the ignoreList when the file is changed.

## Note

This PR includes update to the VS Code clients so that the ignore list
is set at Editor start up, and whenever the .cody/.ignore file is
updated, the ignore list will be updated accordingly.

This does not make any changes to the current embedding steps or BFG,
which will be a feature to be implemented in the future as listed by Dom
in the discussion below.

- [x] PRFAQ to agree on the `.cody/.ignore`name

## Test plan

<!-- Required. See
https://docs.sourcegraph.com/dev/background-information/testing_principles.
-->

Prep: 

<img width="1000" alt="image"
src="https://github.com/sourcegraph/cody/assets/68532117/703840e0-042e-431e-b90e-149a254d1d83">


- Build Cody from this branch, start in VS Code debug mode
- Turn on the Unstable Experimental Features from the Cody Settings
- Open the `cody` repository 
- Create a .codyignore file
- Add the following to the .codyignore file
```
vscode/**
utils.ts
```

### Chat

Ask cody about fixup in Chat View: "what is fixup?" 

You should see Cody did not read any files from the `vscode` directory
to answer your question:

<img width="2001" alt="image"
src="https://github.com/sourcegraph/cody/assets/68532117/b53493e7-f382-4325-8c47-5b2229340aac">

### Autocomplete 

Go to any of the file inside the `vscode` directory, you should not get
any autocomplete suggestions.

<img width="1010" alt="image"
src="https://github.com/sourcegraph/cody/assets/68532117/aabc747f-3734-43bb-b3b2-f17e8bac3cdc">

### Command

Go to lib/shared/src/utils.ts file, and then select some code, right
click and select `Explain Code`:

![image](https://github.com/sourcegraph/cody/assets/68532117/37f2626e-4aeb-4fe7-af56-1bc6ba16ef8a)

You should see cody explain a random code (hallucinating because we
didn't provide any code as context) that is not your selected code.

<img width="2011" alt="image"
src="https://github.com/sourcegraph/cody/assets/68532117/735d3a17-587b-46e2-8c26-1f1e9762eb7a">

You can check the output channel to confirm no context was sent to the
LLM:

<img width="2006" alt="image"
src="https://github.com/sourcegraph/cody/assets/68532117/a8bede49-f161-455c-98e6-d852b95f3870">


## Demo


https://github.com/sourcegraph/cody/assets/68532117/9ea47b4c-311b-40b4-954b-db6d91769a21


### Update

Here is a loom video of codyignore working in Simple chat panel:

https://www.loom.com/share/83fe3e0b0728449385d130d45c317cfa

---------

Co-authored-by: Philipp Spiess <hello@philippspiess.com>
Co-authored-by: Danny Tuppeny <danny@tuppeny.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
chat/commands Chat and Commands clients/vscode enhancement New feature or request
Projects
None yet
4 participants