Skip to content

Commit

Permalink
Autocomplete: reproduce buggy prompt where line suffix is missing
Browse files Browse the repository at this point in the history
Observe how the prompt does not include the `= b + a` suffix. The LLM
generates the completion `finalResult = a + b`, which doesn't match the
existing code so we return an empty completion.
  • Loading branch information
olafurpg committed May 23, 2024
1 parent 9e083b4 commit 1a8ccbe
Show file tree
Hide file tree
Showing 3 changed files with 240 additions and 3 deletions.
230 changes: 230 additions & 0 deletions agent/recordings/defaultClient_631904893/recording.har.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion agent/src/__tests__/example-ts/src/sum.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
export function sum(a: number, b: number): number {
/* CURSOR */
const /* CURSOR */ = b + a

Check failure on line 2 in agent/src/__tests__/example-ts/src/sum.ts

View workflow job for this annotation

GitHub Actions / safe-prompts-lint

The file could not be parsed
return finalResult
}
10 changes: 8 additions & 2 deletions agent/src/index.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,9 @@ describe('Agent', () => {
workspaceRootUri: workspace.rootUri,
name: 'defaultClient',
credentials: TESTING_CREDENTIALS.dotcom,
extraConfiguration: {
'cody.autocomplete.advanced.provider': 'fireworks',
},
})

// Initialize inside beforeAll so that subsequent tests are skipped if initialization fails.
Expand Down Expand Up @@ -145,12 +148,15 @@ describe('Agent', () => {
expect(valid?.username).toStrictEqual('sourcegraphbot9k-fnwmu')
}, 10_000)

describe('Autocomplete', () => {
describe.only('Autocomplete', () => {

Check failure on line 151 in agent/src/index.test.ts

View workflow job for this annotation

GitHub Actions / build

Don't focus the test.
it('autocomplete/execute (non-empty result)', async () => {
await client.openFile(sumUri)
const doc = client.workspace.openUri(sumUri)
const position = doc.protocolDocument.selection!.start
console.log({ position, doc: doc.getText() })
const completions = await client.request('autocomplete/execute', {
uri: sumUri.toString(),
position: { line: 1, character: 3 },
position,
triggerKind: 'Invoke',
})
const texts = completions.items.map(item => item.insertText)
Expand Down

0 comments on commit 1a8ccbe

Please sign in to comment.