diff --git a/com.microsoft.copilot.eclipse.swtbot.test/test-plans/add-context/add-context.md b/com.microsoft.copilot.eclipse.swtbot.test/test-plans/add-context/add-context.md new file mode 100644 index 00000000..05d214ca --- /dev/null +++ b/com.microsoft.copilot.eclipse.swtbot.test/test-plans/add-context/add-context.md @@ -0,0 +1,120 @@ +# Chat: Add Context (Attach Files) + +## Overview + +Tests the **Add Context** button in the Chat view, which lets users attach +workspace files to a chat prompt. Attached files provide additional context to +Copilot so it can give more relevant answers. The button is an icon-only flat +button (tooltip: **Add Context...**) in the action bar's file reference area. + +After attaching, files appear as **chips** below the input area showing the +file name and a close (×) button. Users can click a chip to open the file, or +click × to remove it. + +Entry points exercised: +- **Ctrl+Alt+I** (or status bar → **Open Chat**), then click the Add Context + button in the action bar. + +--- + +## Prerequisites + +- Eclipse IDE with the GitHub Copilot for Eclipse plugin installed and + activated. +- A signed-in Copilot account. +- A Java project is open in the workspace with at least two files. + +--- + +## 1. Attach files, verify chips, remove, and send with context + +### TC-001: Attach files via the Add Context button, verify file chips, remove a file, and send a prompt with attached context + +**Type:** `Happy Path` +**Priority:** `P0` + +#### Preconditions + +- The Chat view is open. +- The workspace contains at least two Java files. + +#### Steps + +1. Locate the **Add Context** button in the action bar (icon-only button with + tooltip **Add Context...**). +2. Verify the button appears as a flat icon without a visible rectangular + border. +3. Click the **Add Context** button. +4. Verify a file picker dialog opens (title: **Search attachments**). +5. Select two files from the workspace and click **OK**. +6. Verify the dialog closes and two **file chips** appear below the chat input + area, each showing the file name and a close (×) button. +7. Click the × button on one of the file chips. +8. Verify that chip is removed and only one file chip remains. +9. Click on the remaining file chip's file name. +10. Verify the corresponding file opens (or is revealed) in the editor. +11. Type a prompt (e.g. `explain this file`) in the chat input and click + **Send**. +12. Wait for the Copilot turn to complete. +13. Verify the response references or uses content from the attached file. + +#### Expected Result + +- The Add Context button opens the file picker dialog. +- Selected files appear as removable chips below the input area. +- Clicking × removes the chip. +- Clicking the file name opens the file in the editor. +- Sending a prompt with an attached file includes that file as context in the + conversation. + +#### 📸 Key Screenshots + +- [ ] **Add Context button** — flat icon button in the action bar, no border. +- [ ] **File picker dialog** — dialog showing workspace files to select. +- [ ] **File chips** — two file chips visible below the chat input area. +- [ ] **After removing one chip** — only one chip remaining. +- [ ] **Response with context** — Copilot turn that references the attached + file content. + +--- + +## 2. Currently active file is shown automatically + +### TC-002: The currently open file appears as a reference in the action bar + +**Type:** `Happy Path` +**Priority:** `P0` + +#### Preconditions + +- The Chat view is open. +- A Java file is open and active in the editor. + +#### Steps + +1. Open a Java file in the editor (click on it in the Package Explorer or + switch to an already-open tab). +2. Switch focus to the Chat view. +3. Observe the file reference area in the action bar. + +#### Expected Result + +- The currently active editor file is automatically shown as a reference in + the action bar (displaying the file name). +- Switching to a different file in the editor updates the displayed reference. + +#### 📸 Key Screenshots + +- [ ] **Current file reference** — the action bar showing the active file name + as a context reference. + +--- + +## Notes on failure modes + +- File picker dialog does not open → the Add Context button's selection + listener may not be wired; check the Eclipse error log. +- File chips do not appear after selection → the ReferencedFileService may not + have received the files; verify the file selection result is non-empty. +- Clicking × does not remove the chip → the close button's mouse listener + may not be attached; check the ReferencedFile widget. diff --git a/com.microsoft.copilot.eclipse.swtbot.test/test-plans/model-picker/model-picker.md b/com.microsoft.copilot.eclipse.swtbot.test/test-plans/model-picker/model-picker.md new file mode 100644 index 00000000..1d18ab35 --- /dev/null +++ b/com.microsoft.copilot.eclipse.swtbot.test/test-plans/model-picker/model-picker.md @@ -0,0 +1,177 @@ +# Chat: Model Picker + +## Overview + +Tests the **model picker** dropdown in the Chat view action bar. The model +picker lets users choose which AI model powers their chat conversations. It +displays available models grouped by tier (Standard, Premium, Custom), shows +per-model details on hover, and includes a **Manage Models...** shortcut for +users with BYOK enabled. + +Entry points exercised: +- **Ctrl+Alt+I** (or status bar → **Open Chat**), then interact with the model + picker button in the action bar. + +--- + +## Prerequisites + +- Eclipse IDE with the GitHub Copilot for Eclipse plugin installed and + activated. +- A signed-in Copilot account. Without a valid account the model list cannot + be fetched and the picker will show no items. + +--- + +## 1. Model picker loads, opens, selects, and routes a message + +### TC-001: Open model picker, browse grouped models, switch model, and verify a response uses it + +**Type:** `Happy Path` +**Priority:** `P0` + +#### Preconditions + +- The Eclipse workbench is open. +- The user is signed in to Copilot. +- No previous Chat view is open. + +#### Steps + +1. Open the **Copilot Chat** view via the keyboard shortcut **Ctrl+Alt+I** + (Windows/Linux) or **Ctrl+Cmd+I** (macOS), or click the Copilot status bar + icon and select **Open Chat**. +2. Wait for the Chat view to fully load (the input area is editable, the model + picker shows a resolved model name instead of "Loading..."). +3. Verify the model picker button on the right side of the action bar displays + a model name (e.g. `Claude Sonnet 4.6`) with a dropdown arrow. +4. Hover the mouse over the model picker button and verify a tooltip appears + (e.g. `Pick Model`). +5. Click the model picker button to open the dropdown popup. +6. Verify the dropdown lists models grouped under labelled headers (e.g. + **Standard Models**, **Premium Models**), with each model showing a cost + multiplier on the right (e.g. `0x`, `1x`, `3x`). +7. Verify the currently selected model is marked with a **checkmark** (✓) + in the list. +8. Hover over a model item in the dropdown and verify a **detail card** + appears showing the model's **family** (e.g. `Family: claude-sonnet-4.5`), + **cost** (e.g. `Cost: 1x premium`), and **category tag** (e.g. + `Versatile`). +9. Click a model different from the currently selected one. +10. Verify the dropdown closes and the model picker button label updates to + show the newly selected model name. +11. Type a prompt (e.g. `hello`) in the chat input and click **Send**. +12. Wait for the Copilot turn to complete (the model-info-label footer + appears). +13. Verify the model info label at the bottom of the completed turn matches + the model selected in step 9. + +#### Expected Result + +- The model picker loads and shows the default model. +- The dropdown opens with grouped models and cost multipliers; the selected + model has a checkmark. +- Hovering a model shows a detail card with family, cost, and category. +- Selecting a different model updates the picker label immediately. +- The response is served by the newly selected model, confirmed by the turn + footer. + +#### 📸 Key Screenshots + +- [ ] **Model picker loaded** — action bar showing the resolved model name. +- [ ] **Dropdown open** — popup showing grouped model list with the active + model marked with a checkmark. +- [ ] **Model hover card** — detail card showing family, cost, and category + tag for a model item. +- [ ] **Model switched** — picker button showing the new model name. +- [ ] **Response with new model** — completed Copilot turn whose footer shows + the new model name. + +--- + +## 2. Chat mode interaction + +### TC-002: Switching chat mode updates the model picker + +**Type:** `Happy Path` +**Priority:** `P1` + +#### Preconditions + +- The Chat view is open. +- Both **Ask** and **Agent** modes are available in the mode picker. + +#### Steps + +1. Note the model currently shown in the model picker. +2. Switch the chat mode from **Agent** to **Ask** (or vice versa) using the + mode picker on the left side of the action bar. +3. Observe the model picker button — it should still show a valid model name + (which may differ from the previous one if the old model is not available + in the new mode). +4. Open the model picker dropdown and verify the model list has loaded (it may + contain different models than before). +5. Close the dropdown (click outside or press Escape). + +#### Expected Result + +- The model picker updates automatically when the chat mode changes. +- No crash, blank picker, or stale model name occurs. + +#### 📸 Key Screenshots + +- [ ] **After mode switch** — action bar showing the updated mode label and the + model picker with a valid model name. + +--- + +## 3. BYOK custom models and Manage Models shortcut + +### TC-003: Custom models appear in the picker and "Manage Models..." opens the preference page + +**Type:** `Happy Path` +**Priority:** `P1` + +#### Preconditions + +- The user is signed in with an account that has BYOK enabled. +- At least one custom model has been added and enabled via **Window → + Preferences → GitHub Copilot → Model Management**. + +#### Steps + +1. Open the model picker dropdown. +2. Verify a **Custom Models** group is present at the bottom of the list, + containing the configured custom model(s). +3. Select a custom model and verify the picker label updates. +4. Open the model picker dropdown again. +5. Click the **Manage Models...** item at the bottom of the dropdown. +6. Verify the **Preferences** dialog opens directly on the **Model Management** + page. +7. Close the Preferences dialog. + +#### Expected Result + +- Custom models are listed under a dedicated **Custom Models** group. +- Selecting a custom model works the same as selecting a built-in model. +- **Manage Models...** navigates directly to the BYOK preference page. + +#### 📸 Key Screenshots + +- [ ] **Custom Models group** — dropdown showing the Custom Models section with + the user's configured models. +- [ ] **Manage Models... action** — Preferences dialog opened to the Model + Management page after clicking the shortcut. + +--- + +## Notes on failure modes + +- Picker button stays blank / never shows a model name → the model list fetch + failed; check that the account is signed in and network access is available. +- Dropdown is empty → auth is valid but the server returned no models; try + signing out and back in to refresh the token. +- Selected model is not reflected in the completed turn's footer → the model + switch event did not propagate; re-select the model and resend. +- Custom Models group does not appear → BYOK is disabled by org policy, or no + custom model has been added; verify in Model Management preferences. diff --git a/com.microsoft.copilot.eclipse.swtbot.test/test-plans/nes/nes.md b/com.microsoft.copilot.eclipse.swtbot.test/test-plans/nes/nes.md new file mode 100644 index 00000000..dc4bc348 --- /dev/null +++ b/com.microsoft.copilot.eclipse.swtbot.test/test-plans/nes/nes.md @@ -0,0 +1,311 @@ +# Next Edit Suggestions (NES) + +## Overview + +Tests the **Next Edit Suggestions** feature, which proactively shows AI-powered +code changes in the editor as the user types. When Copilot detects that a +recent edit implies a follow-up change elsewhere in the file, it highlights the +affected range with a background color, shows a diff popup with the proposed +change in green ghost text, and renders a Copilot icon in the editor gutter. + +Users can: +- **Tab** to accept the suggestion (or jump to it if off-screen). +- **Esc** to dismiss the suggestion. +- **Click the gutter icon** to open an Accept / Reject action menu. +- **Hover the gutter icon** to see a tooltip with keyboard shortcuts. + +--- + +## Prerequisites + +- Eclipse IDE with the GitHub Copilot for Eclipse plugin installed and + activated. +- A signed-in Copilot account. +- **Enable Next Edit Suggestions** is checked in **Window → Preferences → + GitHub Copilot → Completions** (this is the default). +- A Java project is open in the workspace with at least one Java source file. + +--- + +## 1. Trigger a suggestion, inspect it, and accept via Tab + +### TC-001: NES appears after related edits, shows diff and gutter icon, and is accepted with Tab + +**Type:** `Happy Path` +**Priority:** `P0` + +#### Preconditions + +- A Java file with multiple references to the same variable or method is open. +- NES is enabled in Preferences. + +#### Steps + +1. In the editor, make a series of related edits (e.g. rename a variable on + one line, then move the cursor to another line where the same variable is + used — or add a parameter to a method declaration, then navigate to a call + site of that method). +2. After each edit, pause briefly (~1–3 seconds) and observe the editor. +3. Once a NES suggestion appears, verify: + - A **background highlight** is shown on the text range that would be + changed. + - A **diff popup** appears next to the highlighted range showing the + proposed replacement in **green ghost text**. + - A **Copilot icon** appears in the editor gutter on the suggestion line. +4. Hover the mouse over the Copilot gutter icon and verify a tooltip appears + showing keyboard shortcuts: **Tab: Accept suggestion** and + **Esc: Dismiss suggestion**. +5. Press **Tab** on the keyboard. +6. Verify the suggestion is applied: the highlighted text is replaced with the + proposed change, the diff popup disappears, the gutter icon disappears, and + the file is modified (unsaved asterisk in the tab title). + +#### Expected Result + +- NES suggestion appears with highlight, diff popup, and gutter icon. +- Gutter icon tooltip shows Tab / Esc hints. +- Pressing Tab applies the change and clears all NES decorations. + +#### 📸 Key Screenshots + +- [ ] **NES suggestion visible** — editor showing the highlighted range, the + diff popup with green ghost text, and the Copilot gutter icon. +- [ ] **Gutter icon tooltip** — tooltip showing "Tab: Accept / Esc: Dismiss". +- [ ] **After Tab accept** — editor showing the applied change with no + remaining NES decorations. + +--- + +## 2. Reject via Esc and use the gutter icon action menu + +### TC-002: Dismiss a suggestion with Esc, trigger another, and use gutter icon Accept / Reject + +**Type:** `Happy Path` +**Priority:** `P0` + +#### Preconditions + +- A Java file is open in the editor. +- NES is enabled. + +#### Steps + +1. Make a series of related edits to trigger a NES suggestion (same approach + as TC-001 steps 1–2). +2. Once the suggestion appears, press **Esc**. +3. Verify the diff popup, background highlight, and gutter icon all disappear. +4. Verify the file content is unchanged — no text was inserted or deleted. +5. Make another series of related edits to trigger a new suggestion. +6. Once the new suggestion appears, **click the Copilot icon** in the gutter. +7. Verify an action menu appears with two options: **Accept** and **Reject**. +8. Click **Reject** in the action menu. +9. Verify the suggestion is dismissed (same as pressing Esc: all decorations + disappear, file unchanged). +10. Trigger yet another suggestion. +11. Click the gutter icon again and this time click **Accept**. +12. Verify the suggestion is applied (same result as pressing Tab in TC-001). + +#### Expected Result + +- Esc dismisses without modifying the file. +- Gutter icon click opens an Accept / Reject menu. +- Reject via menu dismisses the suggestion. +- Accept via menu applies the suggestion. + +#### 📸 Key Screenshots + +- [ ] **After Esc dismiss** — editor with no NES decorations, file unchanged. +- [ ] **Gutter icon menu** — the Accept / Reject action menu next to the + gutter icon. +- [ ] **After menu Accept** — editor showing the applied change. + +--- + +## 3. Out-of-viewport navigation + +### TC-003: Tab jumps to an off-screen suggestion, then a second Tab accepts it + +**Type:** `Happy Path` +**Priority:** `P1` + +#### Preconditions + +- A long Java file (200+ lines) is open. +- NES is enabled. + +#### Steps + +1. Make a series of related edits to trigger a NES suggestion (same approach + as TC-001). +2. Once the suggestion is visible (highlight, diff popup, gutter icon), + **scroll away** from the suggestion line so it is no longer in the viewport. +3. Verify a **notification bar** (pill) appears at the bottom of the editor + with a message like **"Press Tab to jump to Next Edit Suggestion"**. +4. Press **Tab**. +5. Verify the editor scrolls back to reveal the suggestion line, the diff + popup and gutter icon become visible again, and the notification bar + disappears. +6. Verify the suggestion is **not** accepted yet — the highlight and diff + popup are still showing. +7. Press **Tab** again. +8. Verify the suggestion is now accepted and all NES decorations disappear. + +#### Expected Result + +- First Tab: jumps to the suggestion without accepting it. +- Second Tab: accepts the suggestion. + +#### 📸 Key Screenshots + +- [ ] **Bottom bar visible** — notification pill at the bottom of the editor. +- [ ] **After first Tab** — editor scrolled to show the suggestion with diff + popup and gutter icon. +- [ ] **After second Tab** — suggestion accepted, all decorations cleared. + +--- + +## 4. Suggestion auto-dismiss on further edits + +### TC-004: Editing the file while a suggestion is active causes it to disappear + +**Type:** `Happy Path` +**Priority:** `P1` + +#### Preconditions + +- A Java file is open in the editor. +- NES is enabled. + +#### Steps + +1. Trigger a NES suggestion via a series of related edits (same approach as + TC-001). +2. Once the suggestion is visible (highlight, diff popup, gutter icon), do + **not** accept or reject it. +3. Instead, type additional text anywhere in the file (e.g. add a comment on a + different line, or directly edit the text in the suggested range). +4. Observe whether the NES suggestion disappears immediately after the edit. + +#### Expected Result + +- The NES suggestion (highlight, diff popup, gutter icon) disappears as soon + as the file content is modified by the user. +- The newly typed text is applied normally — no conflict with the old + suggestion. + +#### 📸 Key Screenshots + +- [ ] **Suggestion visible before edit** — NES decorations present. +- [ ] **After typing** — all NES decorations gone, user's new edit applied. + +--- + +## 5. Enable / disable toggle + +### TC-005: Disabling and re-enabling NES in Preferences + +**Type:** `Happy Path` +**Priority:** `P1` + +#### Preconditions + +- NES is currently enabled. +- A Java file is open. + +#### Steps + +1. Open **Window → Preferences → GitHub Copilot → Completions**. +2. Uncheck **Enable Next Edit Suggestions**. +3. Click **Apply and Close**. +4. Make a series of related edits in the Java file (same approach as TC-001). +5. Verify no NES suggestion appears (no highlight, no diff popup, no gutter + icon). +6. Open **Window → Preferences → GitHub Copilot → Completions** again. +7. Check **Enable Next Edit Suggestions**. +8. Click **Apply and Close**. +9. Make a series of related edits in the Java file again. +10. Verify a NES suggestion appears. + +#### Expected Result + +- Disabling: no NES decorations appear after edits. +- Re-enabling: NES suggestions resume without restarting Eclipse. +- Inline tab-completions are unaffected by either toggle. + +#### 📸 Key Screenshots + +- [ ] **NES disabled** — Preferences page with the checkbox unchecked. +- [ ] **No suggestion after edit** — editor with no NES decorations after + editing with NES disabled. +- [ ] **NES re-enabled** — suggestion appears again after re-enabling. + +--- + +## 6. No interference with other editor decorations + +### TC-006: No red rectangular border from Copilot appears on SonarQube issue locations + +**Type:** `Edge Case` +**Priority:** `P1` + +#### Preconditions + +- Both **GitHub Copilot for Eclipse** and **SonarQube for Eclipse** are + installed. +- NES is enabled. +- A Java project is open with a file that contains a SonarQube issue. For + example, a file like: + ```java + import java.util.List; + + public class TestSonar { + boolean isEmpty(List l) { + return l == null || l.size() == 0; + } + } + ``` + where `l.size() == 0` triggers a SonarQube rule. +- SonarQube analysis has completed and its markers are visible in the editor. + +#### Steps + +1. Open the Java file that has SonarQube issue markers. +2. Do **not** make any edits — just observe the editor. +3. Check whether a **red rectangular border** (BOX style) with the label + `"Text to be Deleted"` appears around the SonarQube issue location. +4. Hover over the SonarQube-annotated area and verify the tooltip shows + SonarQube's own message — not `"Text to be Deleted"`. +5. If a SonarQube quick fix is available, apply it. +6. Verify the red border does not persist after the quick fix is applied. +7. Open a different Java file that has compiler warnings but no SonarQube + issues, and verify no Copilot NES decorations appear on unedited code. + +#### Expected Result + +- No red rectangular border (`"Text to be Deleted"` BOX annotation) from + Copilot appears on SonarQube issue locations. +- SonarQube markers display normally with their own styling. +- Hovering SonarQube markers shows SonarQube's message, not Copilot's. +- After applying a SonarQube quick fix, no Copilot annotation persists. +- On unedited files without SonarQube, no NES decorations appear either. + +#### 📸 Key Screenshots + +- [ ] **SonarQube file — no NES** — editor showing SonarQube markers with no + Copilot decorations overlapping. +- [ ] **SonarQube hover** — tooltip showing the SonarQube message, not a + Copilot annotation. + +--- + +## Notes on failure modes + +- No suggestion appears after editing → NES may be disabled in Preferences, or + the Copilot account is not signed in; verify both and retry. +- Red rectangular border appears on SonarQube markers → the NES annotation + type is still mapped to the root `textmarker`; check that `plugin.xml` does + not declare `markerType="org.eclipse.core.resources.textmarker"` on the NES + annotation types. +- Accepting a suggestion has no effect → the suggestion may have become stale + (the underlying code changed again before Tab was pressed); make a fresh + edit to trigger a new suggestion and accept immediately.