Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Assistant Context Area for Panel #9705

Open
1 task done
rgbkrk opened this issue Mar 22, 2024 · 1 comment
Open
1 task done

Assistant Context Area for Panel #9705

rgbkrk opened this issue Mar 22, 2024 · 1 comment
Labels
ai Improvement related to Assistant, Copilot, or other AI features assistant AI feedback for Assistant editor Feedback for code editing, formatting, editor iterations, etc enhancement [core label]

Comments

@rgbkrk
Copy link
Member

rgbkrk commented Mar 22, 2024

Check for existing issues

  • Completed

Describe the feature

image

In order to be productive with Zed's assistant panel, you have to copy paste code and error context over. Often, a decent context injection prompt looks like this:

System:

File crates/gpui/src/geometry.rs:

//! The GPUI geometry module is a collection of types and traits that
//! can be used to describe common units, concepts, and the relationships
//! between them.

use core::fmt::Debug;
use derive_more::{Add, AddAssign, Div, DivAssign, Mul, Neg, Sub, SubAssign}
... rest of copy/pasted code...

Error:

   Compiling gpui v0.1.0 (/Users/kylekelley/code/src/github.com/zed-industries/zed/crates/gpui)
error[E0599]: the method `step_by` exists for struct `Range<Pixels>`, but its trait bounds were not satisfied
    --> crates/gpui/src/geometry.rs:2954:46
     |
2011 | pub struct Pixels(pub f32);
     | ----------------- doesn't satisfy `geometry::Pixels: std::iter::Step`
...
2954 |         for y in (Pixels(1.0)..Pixels(10.0)).step_by(Pixels(2.0)) {
     |                                              ^^^^^^^ method cannot be called on `Range<Pixels>` due to unsatisfied trait bounds
     |
    ::: /Users/kylekelley/.rustup/toolchains/1.76-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/ops/range.rs:80:1
     |
80   | pub struct Range<Idx> {
     | --------------------- doesn't satisfy `std::ops::Range<geometry::Pixels>: Iterator`
     |
     = note: the following trait bounds were not satisfied:
             `geometry::Pixels: std::iter::Step`
             which is required by `std::ops::Range<geometry::Pixels>: Iterator`
             `std::ops::Range<geometry::Pixels>: Iterator`
             which is required by `&mut std::ops::Range<geometry::Pixels>: Iterator`
note: the trait `std::iter::Step` must be implemented
    --> /Users/kylekelley/.rustup/toolchains/1.76-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/iter/range.rs:26:1
     |
26   | pub trait Step: Clone + PartialOrd + Sized {
     | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

You:

I'm trying to add the step_by method to Pixels so I can iterate over a range of pixels. My main goal is being able to write code like this:

for y in (start_y..end_y).step_by(square_size) {
    // paint a quad
}

That'll give me good results to pair with the model for the first message. As I converse with the assistant all that old context sticks around unless I scroll back to delete it. Sometimes that takes up too much of the token limit and other times it can confuse the assistant to have all this extra context around.

Proposal

Introduce context into the assistant panel.

image

The context view shows to the user what the current context is that will be sent to the model, including:

  1. File Context
  2. Project Diagnostics
  3. Deep Code Context (tree sitter queries, general search against the code base, other tools the model can call on)

image

The user would have the option of disabling sending any amount of context to the model.

On the model's side, it'll see this as a collection of system messages or tools available for it to call. Importantly, previous contexts don't have to be sent to the model -- only the most recent up to date version.

As you refactor your code, make changes, etc., the context sent should be up to date and wouldn't include old context (unless necessary).

If applicable, add mockups / screenshots to help present your vision of the feature

image

image

@rgbkrk rgbkrk added enhancement [core label] triage Maintainer needs to classify the issue admin read Pending admin review labels Mar 22, 2024
@Moshyfawn Moshyfawn added editor Feedback for code editing, formatting, editor iterations, etc ai Improvement related to Assistant, Copilot, or other AI features assistant AI feedback for Assistant and removed triage Maintainer needs to classify the issue labels Mar 22, 2024
@JosephTLyons JosephTLyons removed the admin read Pending admin review label Mar 26, 2024
rgbkrk added a commit that referenced this issue Mar 29, 2024
Introducing the Active File Context portion of #9705. When someone is in
the assistant panel it now includes the active file as a system message
on send while showing them a nice little display in the lower right:


![image](https://github.com/zed-industries/zed/assets/836375/9abc56e0-e8f2-45ee-9e7e-b83b28b483ea)

For this iteration, I'd love to see the following before we land this:

* [x] Toggle-able context - user should be able to disable sending this
context
* [x] Show nothing if there is no context coming in
* [x] Update token count as we change items
* [x] Listen for a more finely scoped event for when the active item
changes
* [x] Create a global for pulling a file icon based on a path. Zed's
main way to do this is nested within project panel's `FileAssociation`s.
* [x] Get the code fence name for a Language for the system prompt
* [x] Update the token count when the buffer content changes

I'm seeing this PR as the foundation for providing other kinds of
context -- diagnostic summaries, failing tests, additional files, etc.

Release Notes:

- Added file context to assistant chat panel
([#9705](#9705)).

<img width="1558" alt="image"
src="https://github.com/zed-industries/zed/assets/836375/86eb7e50-3e28-4754-9c3f-895be588616d">

---------

Co-authored-by: Conrad Irwin <conrad@zed.dev>
Co-authored-by: Nathan <nathan@zed.dev>
Co-authored-by: Antonio Scandurra <me@as-cii.com>
Co-authored-by: Mikayla Maki <mikayla@zed.dev>
mrnugget pushed a commit to ciiqr/zed that referenced this issue Apr 3, 2024
Introducing the Active File Context portion of zed-industries#9705. When someone is in
the assistant panel it now includes the active file as a system message
on send while showing them a nice little display in the lower right:


![image](https://github.com/zed-industries/zed/assets/836375/9abc56e0-e8f2-45ee-9e7e-b83b28b483ea)

For this iteration, I'd love to see the following before we land this:

* [x] Toggle-able context - user should be able to disable sending this
context
* [x] Show nothing if there is no context coming in
* [x] Update token count as we change items
* [x] Listen for a more finely scoped event for when the active item
changes
* [x] Create a global for pulling a file icon based on a path. Zed's
main way to do this is nested within project panel's `FileAssociation`s.
* [x] Get the code fence name for a Language for the system prompt
* [x] Update the token count when the buffer content changes

I'm seeing this PR as the foundation for providing other kinds of
context -- diagnostic summaries, failing tests, additional files, etc.

Release Notes:

- Added file context to assistant chat panel
([zed-industries#9705](zed-industries#9705)).

<img width="1558" alt="image"
src="https://github.com/zed-industries/zed/assets/836375/86eb7e50-3e28-4754-9c3f-895be588616d">

---------

Co-authored-by: Conrad Irwin <conrad@zed.dev>
Co-authored-by: Nathan <nathan@zed.dev>
Co-authored-by: Antonio Scandurra <me@as-cii.com>
Co-authored-by: Mikayla Maki <mikayla@zed.dev>
@golebiewsky
Copy link

Why is it enabled by default and how to change it's behaviour so I have to explicitly allow this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ai Improvement related to Assistant, Copilot, or other AI features assistant AI feedback for Assistant editor Feedback for code editing, formatting, editor iterations, etc enhancement [core label]
Projects
None yet
Development

No branches or pull requests

4 participants