Skip to content

Commit

Permalink
Preview (#1430)
Browse files Browse the repository at this point in the history
* Add stream support for Bedrock Anthropic

* 🎨 make llamatokenizer commonjs compatible

* ➕ add esbuild optional deps

* 🚚 rename vendor/node_modules -> modules

* 🔖 update core version

* 🐛 fixes for transformers.js compatibility

* 🔖 update core version

* 🎨 set modelPath in constructor

* 🎨 fix transformers.js import

* Drafted max depth changes

* 🎨 eslint enforce import extensions

* 🎨 require -> import

* 🚸 notify user if not diff in /commit

* 💄 Improve colors of the IntelliJ tool window icon (#1273)

Without this fix, the continue icon sticks out from the other toolwindow icons,
resulting in an inconsistent appearance of the whole IDE and creates a feeling
that the continue plugin "doesn't fit, something must be broken".

According to
https://plugins.jetbrains.com/docs/intellij/icons.html#new-ui-icon-colors
specific colors are needed to work nicely with dark and light modes. Bonus is
that the active tool window icon color then changes automatically to white.

Co-authored-by: Lukas Baron <LukasBaron@gmail.com>

* ✨ send Bearer token to Ollama if apiKey set

* 🐛 fix slash command bug

* 🧑‍💻 add onnxruntime (--save-dev) for types

* 🐛 don't apply to file when running code block in terminal

* 🐛 avoid double paste

* ✨ gpt-4o

* 🎨 pass uniqueId to constructor

* 🚸 focus without scrolling into vie

* 🎨 refactoring for continue-proxy LLM (#1277)

* 🐛 continue server client fixes

* 🎨 refactor OpenAI _getHeaders

* 🎨 pass ideSettings to llmFromDescription

* ⚡️ improve add docstring command

* 💚 ci updates

* 🐛 merge fixes

* 💄 increase font size in JB

* 🐛 fix repeated paste bug

* 🐛 fix build script

* 🩹 various small improvements

* 📌 pin esbuild 0.17.19

* 🧑‍💻 pnpm i gui in prepackage

* 🐛 show all diff changes in vscode

* 🩹 getMetaKeyName

* 🐛 fix reading of unopened ipynb files

* ⚡️ gpt-4o system prompt

* 💄 make font size configurable in config.json ui.fontSize

* 🩹 properly dispose of diff handler

* 🐛 fix indexing status display

* ⚡️ context pruning

* 🎨 update free trial models

* fix: remove some backup files generated by pkg if present (#1287)

* Corrections to indexDocs params

* adjust toCopy (#1305)

Co-authored-by: Justin Milner <jmilner@jmilner-lt2.deka.local>

* Nate/prompt-file (#1308)

* ✨ .prompt files

* 🧑‍💻 back to npm  : (

* 🎨 nicer dropdown icon

* 🎨 full switch back to npm

* 🧑‍💻 uninstall script for fresh start

* 🎨 updated package-locks

* 🔥 remove hello from continuerc.json

* 🔥 remove example files

* 🎨 update test prompt

* 🎨 update prompt

* 🎨 update test prompt

* 👷 create out/node_modules in prepackage.js

* Log prompt-tokens to devdb and show in 'My Usage' view (#1309)

* Also log the number of prompt tokens to the dev-db

* Show prompt tokens in 'My Usage' view

* 🔥 remove console logs

* add lm studio tab-autocomplete walkthrough (#1298)

* 📝 update autocomplete LM Studio docs

* 💚 fix prepackage.js

* ⬆️ upgrade lancedb version

* 🚸 make it easier to continue past trial

* 🚸 improve model setup process

* 📌 update package-lock.jsons

* 🔀 merge changes

* 🩹 small fixes

* 🐛 fix jetbrains build

* 🩹 more jetbrains fixes

* 🎨 jetbrains improvements

* 🐛 fix lancedb prob in jetbrains build

* 👷 update jetbrains build process

* convert to use interface, fix type-related bug, more

* ✨ intellij problems context provider

* 👷 add script utils file

* 💚 fix jetbrains build regression

* add Docs wrong values

* 🎨 dynamic import transformers.js

* 🩹 small jetbrains updates

* ✨ folder context provider in jetbrains

* 🎨 many more jetbrains improvements

* protocol stuff

* 🐛 fix prompt file loading bug

* small corrections, some todos

* docs: add setup guide for OpenRouter (#1284)

* Update preIndexedDocs.ts (#1292)

Add Bootstrap and Alpine.js as pre indexed docs.

* add toString (#1324)

* add toString

* Fix indentation

---------

Co-authored-by: Justin Milner <jmilner@jmilner-lt2.deka.local>

* don't overwrite all models in onboarding

* 🔧 override headers with custom headers

* ✨ allow overwriting slash command description

* feature flag (#1327)

* ➕ feature flags dependency

* 🎨 make getHeaders awaitable

* 🔒️ token count for abuse monitoring

* 👷 remove win32-arm64 target

* Update build.js (#1330)

Fix error: EBUSY: resource busy or locked, rmdir '...\continue\binary\tmp\continue-node_modules-lancedb'

* 📌 pin onnxruntime version to match transformers.js

* 🎨 transformers cleanup, extensionVersion header

* 🔥 remove lingering pnpm

* 🎨 update default models

* 🎨 update defaults

* 👷 fix version in build post-check

* 🎨 test.js in dev

* 🧑‍💻 cleaning

* Add Gemini 1.5 Flash as a model (#1337)

* Fixes for dialog, remove logging, indexing corrections

* cleanup

* 🎨 log dev data for tokens generated

* Expose custom context provider registration through a vscode extension api. (#1288)

* Make github crawler find default branch

* Cleanup

* 🔀 merge fixes

* ⚡️ small improvements

* 🎨 tell users transformers.js not supported

* 🎨 trial updates

* 📝 update jetbrains readmej

* ⚡️ Use gptEditPrompt for Mistral

* ✨ Codestral FIM

* 🔖 update gradle version

* Fix typo "Experimantal" (#1353)

* 🎨 Refactor core (#1281)

* 🎨 refactor out react context

* 🎨 refactor local storage

* 🎨 refactor IdeMessenger

* 🔥 removing unused files

* 🚚 move scripts

* ✅ setup VSCode integration testing

* 🎨 fix test-related stuff

* 🚧 testing

* 🎨 tweak terminal cmd+L

* 🧑‍💻 install biome in core

* 🎨 run biome check on core

* 🎨 run biome check in vscode

* 🎨 run biome check in binary

* 🧑‍💻 global biome config

* 🎨 run biome check on core

* 🎨 run biome check in vscode

* 🎨 fix a few biome warnings

* 🚧 WIP on protocols

* 🚧 progress, organizing, about to remove webviewCore protocol

* 🚧 now running

* 🚧 move indexing code to core

* 🚧 WIP on JetBrains indexing implementation

* 🎨 finish listDir

* 🚧 gui fixes

* 🏷️ update IMessenger.on return type

* 🐛 fix jetbrains types + IDE detection

* 🧑‍💻 set up debugging for binary with TcpMessenger

* 👷 fix prepackage.js

* 🧑‍💻 turn off debug mode for intellij

* 🐛 merge fixes

* 🐛 fixes after refactor

* 🐛 merge fixes

* 💄 increase font size in JB

* 🔀 merge changes

* 🩹 small fixes

* 🐛 fix jetbrains build

* 🩹 more jetbrains fixes

* 🎨 jetbrains improvements

* 🐛 fix lancedb prob in jetbrains build

* 👷 update jetbrains build process

* ✨ intellij problems context provider

* 👷 add script utils file

* 💚 fix jetbrains build regression

* 🎨 dynamic import transformers.js

* 🩹 small jetbrains updates

* ✨ folder context provider in jetbrains

* 🎨 many more jetbrains improvements

* 🔀 merge fixes

* ⚡️ small improvements

* 🎨 tell users transformers.js not supported

* 🎨 trial updates

* 📝 update jetbrains readmej

* ⚡️ only use smaller context for local autocomplete models

* ⚡️ improve bracket filter

* ✨ global .continue/.prompts folder

* 💄 improved model setup process

* 🥅 display VSCode req errs

* ⚡️ improved autocomplete stopping

* 🎨 count cmd+I

* 🔥 remove duplicate import

* 🔥 remove unnecessary headers

* ⚡️ limit input history length

* ⚡️ limit submenu items until you can use disk-based search, to avoid high memory use

* 🔥 remove duplicate function def

* visibility not adjusting

* Smooth scrolling until mouse used

* Use smooth transition

* Cleanup

* Free Trial Auth (#1367)

* 🎨 pass gh auth token to free trial

* 💄 improved onboarding flow

* 🎨 make fewer auth reqs

* 🎨 refactor ordering of vscode deps

* 🐛 resolve confighandler

* ⚡️ autocomplete stopping improvements

* 🔥 remove unused auth code

* ⚡️ consider previous completion in bracket matching

* Build fixes: Remove duplicated code, re-add overwritten commit (#1358)

* readd overwritten commit

* Remove duplicated code

* 📝 update keywords

* ✨ /commit slash command

* 🎨 make transformers.js esm compatible

* ⚡️ fix jetbrains indexing perfomance issue

* 💄 autocomplete model onboarding

* ✨ mistral free trial autocomplete

* ✨ mistral description

* 🩹 comment out fireworks.png

* 🎨 trial fim model

* 🎨 get gh token for autocomplete model trial

* ✨ gh auth token jb

* 🔧 update config files

* 🎨 jetbrains onboarding

* 🧑‍💻 create gui/dist if it doesn't exist

* Fix sidebar indexing status timing-related bugs (#1368)

* readd overwritten commit

* Remove duplicated code

* Fix status load in and failed state update

* Late sidebar open bugs fixed

* Change ideMessenger post parameters

* debug pause

* Don't use global, fix table creation bug

* Creating new branch for config-related issues here

* cleanup

* 🚸 help user avoid login prompt if unwanted

* 🚸 update onboarding flow

* 📝 codestral as recommended in docs

* feat: Add cloudflare as provider

* 📝 set up codestral walkthrough

* codestral api base

* update codestral api link

* codestral api link

* bearer token

* 🐛 fix config loading

* cleaner error message

* fix codestral templating

* 🔥 remove unused kotlin test

* 🧑‍💻 ./gradlew build

* codestral fim template

* add additional llama.cpp params

* feat:Add promptPath configuration to use promptFolder from JSON file in load.ts (#1377)

Co-authored-by: catatapiafuentes <catalina.tapia@thoughtworks.com>

* Full project file path in context (#1407)

* Full workspace file path when using @OPEN Files

* Full project file path for @Files and 'active file' context

* FileTreeContextProvider uses splitPath from util instead of local copy

* relative-to-root file paths for rag retrieval

* ✨ jetbrains .continuerc.json support

* tweak

* ✨ allow non git tracked .prompts folder

* ✨ currentFile in prompt files

* explain the purpose of sign in

* help users remove free trial models

* ⚡️ improved codestral multi-file prompt template

* better filtering of model artifacts

* improve sign in flow

* small tweaks

* ✨ add gitRepo and completionId to dev data

* add unique id to autocomplete dev data

* fix build

* update pluginVersion

* fix intellij build

* feat(embeddings): add gemini provider (#1362)

* docs(walkthroughs): use correct table name for 'tag_catalog'

* feat(embeddings): add gemini

* duplicate code deletion

* docs(contributing): update incorrect instructions

* revert eslint changes

* docs(embeddings): use correct gemini model name

* Delete package.json

* Feat/gemini flash (#1352)

* Remove duplicate item from array

* Add references to Gemini Flash

* Update documentation to ref Gemini Flash

* Doc: Add IPEX-LLM model provider (#1417)

* Add related doc for IPEX-LLM accelerated Ollama provider

* Add description in providor title and other small updates

* small tweaks

* add span (#1374)

* 👷 add node_sqlite3 to binary/build

* fix build

* refactor build util

* fix terminal copying

* add getSession, updateSession

* fix build errors

* no proxy option

* fix potential config loading bug

* feat: Add support for Cloudflare AI Gateway (#1425)

Co-authored-by: jdelorey@cloudflare.com <jdelorey@cloudflare.com>

* install swc --save-dev

* fix template path

* 🧑‍💻 speed up debugging

* improving templating paths disambiguation

* autocomplete stopping improvement

* ⚡️ increase cache hit rate for autocomplete

* ⚡️ autocomplete improvements

* ⚡️ make autocomplete more token efficient

* 🐛 don't override max tokens for local models

* 🚑 fix onboarding loop

* build errs

---------

Co-authored-by: Ruben Kostandyan <kostard@amazon.com>
Co-authored-by: Justin Milner <jmilner@jmilner-lt2.deka.local>
Co-authored-by: tnglemongrass <113173292+tnglemongrass@users.noreply.github.com>
Co-authored-by: Lukas Baron <LukasBaron@gmail.com>
Co-authored-by: Fernando <fernando.sanchez.jr@gmail.com>
Co-authored-by: Justin Milner <justinmilner1@gmail.com>
Co-authored-by: Justin Milner <42585006+justinmilner1@users.noreply.github.com>
Co-authored-by: Tijs Zwinkels <tijs@tinkertank.eu>
Co-authored-by: DJ Johnson <mr.demarcus.johnson@gmail.com>
Co-authored-by: sam <1211977+sambarnes@users.noreply.github.com>
Co-authored-by: Jose Vega <bloguea.y.gana@gmail.com>
Co-authored-by: Pratik Parmar <steveparmar6nov2011@gmail.com>
Co-authored-by: Sam El-Husseini <sam.elhusseini@gmail.com>
Co-authored-by: Peter Zaback <pzaback@gmail.com>
Co-authored-by: James Delorey <jdelorey@cloudflare.com>
Co-authored-by: catatapiafuentes <catalina.tapia@thoughtworks.com>
Co-authored-by: Patrick Erichsen <patrick.a.erichsen@gmail.com>
Co-authored-by: Yuwen Hu <54161268+Oscilloscope98@users.noreply.github.com>
Co-authored-by: James Delorey <deloreyjmi@gmail.com>
  • Loading branch information
20 people committed Jun 6, 2024
1 parent 86339b2 commit aa18568
Show file tree
Hide file tree
Showing 69 changed files with 2,564 additions and 21,590 deletions.
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -163,4 +163,7 @@ continue_server.dist
Icon
Icon?

.continue
.continue

# Sandbox env default file
extensions/vscode/manual-testing-sandbox/example.ts
1 change: 1 addition & 0 deletions .vscode/launch.json
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@
"${workspaceFolder}/extensions/vscode/manual-testing-sandbox/test.js",
"--extensionDevelopmentPath=${workspaceFolder}/extensions/vscode"
],
"pauseForSourceMap": false,
"outFiles": [
"${workspaceFolder}/extensions/vscode/out/extension.js",
"/Users/natesesti/.continue/config.ts"
Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ nvm use
1. The new VS Code window with the extension is referred to as the _Host VS Code_
2. The window you started debugging from is referred to as the _Main VS Code_

4. To package the extension, run `npm package` in the `extensions/vscode` directory. This will generate `extensions/vscode/build/continue-patch.vsix`, which you can install by right-clicking and selecting "Install Extension VSIX".
4. To package the extension, run `npm run package` in the `extensions/vscode` directory. This will generate `extensions/vscode/build/continue-{VERSION}.vsix`, which you can install by right-clicking and selecting "Install Extension VSIX".

##### Debugging

Expand Down
23 changes: 19 additions & 4 deletions binary/build.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,11 @@ const fs = require("fs");
const path = require("path");
const ncp = require("ncp").ncp;
const { rimrafSync } = require("rimraf");
const { validateFilesPresent, execCmdSync } = require("../scripts/util");
const {
validateFilesPresent,
execCmdSync,
autodetectPlatformAndArch,
} = require("../scripts/util");

// Clean slate
const bin = path.join(__dirname, "bin");
Expand All @@ -23,6 +27,8 @@ let targets = [
"win32-x64",
];

const [currentPlatform, currentArch] = autodetectPlatformAndArch();

const assetBackups = [
"node_modules/win-ca/lib/crypt32-ia32.node.bak",
"node_modules/win-ca/lib/crypt32-x64.node.bak",
Expand Down Expand Up @@ -51,7 +57,7 @@ async function installNodeModuleInTempDirAndCopyToCurrent(packageName, toCopy) {
console.log(`Copying ${packageName} to ${toCopy}`);
// This is a way to install only one package without npm trying to install all the dependencies
// Create a temporary directory for installing the package
const adjustedName = toCopy.replace(/^@/, "").replace("/", "-");
const adjustedName = packageName.replace(/@/g, "").replace("/", "-");
const tempDir = path.join(
__dirname,
"tmp",
Expand All @@ -78,7 +84,6 @@ async function installNodeModuleInTempDirAndCopyToCurrent(packageName, toCopy) {
);

// Without this it seems the file isn't completely written to disk
// Ideally we validate file integrity in the validation at the end
await new Promise((resolve) => setTimeout(resolve, 2000));

// Copy the installed package back to the current directory
Expand Down Expand Up @@ -169,7 +174,7 @@ async function installNodeModuleInTempDirAndCopyToCurrent(packageName, toCopy) {
entryPoints: ["src/index.ts"],
bundle: true,
outfile: esbuildOutputFile,
external: ["esbuild", "./xhr-sync-worker.js", "vscode"],
external: ["esbuild", "./xhr-sync-worker.js", "vscode", "./index.node"],
format: "cjs",
platform: "node",
sourcemap: true,
Expand Down Expand Up @@ -214,6 +219,16 @@ async function installNodeModuleInTempDirAndCopyToCurrent(packageName, toCopy) {
`${targetDir}/build/Release/node_sqlite3.node`,
`${targetDir}/node_sqlite3.node`,
);

// Copy to build directory for testing
const [platform, arch] = target.split("-");

This comment has been minimized.

Copy link
@maxxrdrgz

maxxrdrgz Jun 11, 2024

Contributor

I believe this code here is causing darwin-arm64 to fail when trying to install dependencies for the first time.

if (platform === currentPlatform && arch === currentArch) {
fs.copyFileSync(
`${targetDir}/node_sqlite3.node`,
`build/node_sqlite3.node`,
);
}

fs.unlinkSync(`${targetDir}/build.tar.gz`);
fs.rmSync(`${targetDir}/build`, {
recursive: true,
Expand Down
1 change: 1 addition & 0 deletions binary/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

55 changes: 29 additions & 26 deletions core/.eslintrc.json
Original file line number Diff line number Diff line change
@@ -1,27 +1,30 @@
{
"root": true,
"parser": "@typescript-eslint/parser",
"parserOptions": {
"ecmaVersion": 6,
"sourceType": "module"
},
"extends": ["plugin:require-extensions/recommended"],
"plugins": ["@typescript-eslint", "import", "require-extensions"],
"rules": {
"quotes": ["warn", "double", {}],
"import/extensions": ["error", "always", {
"js": "never",
"jsx": "never",
"ts": "never",
"tsx": "never"
}],
"@typescript-eslint/naming-convention": "warn",
"@typescript-eslint/semi": "warn",
"curly": "warn",
"eqeqeq": "warn",
"no-throw-literal": "warn",
"semi": "off"
},
"ignorePatterns": ["out", "dist", "**/*.d.ts"]
}

"root": true,
"parser": "@typescript-eslint/parser",
"parserOptions": {
"ecmaVersion": 6,
"sourceType": "module"
},
"extends": ["plugin:require-extensions/recommended"],
"plugins": ["@typescript-eslint", "import", "require-extensions"],
"rules": {
"quotes": ["warn", "double", {}],
"import/extensions": [
"error",
"always",
{
"js": "never",
"jsx": "never",
"ts": "never",
"tsx": "never"
}
],
"@typescript-eslint/naming-convention": "warn",
"@typescript-eslint/semi": "warn",
"curly": "warn",
"eqeqeq": "warn",
"no-throw-literal": "warn",
"semi": "off"
},
"ignorePatterns": ["out", "dist", "**/*.d.ts"]
}
31 changes: 19 additions & 12 deletions core/autocomplete/cache.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,36 +29,43 @@ export class AutocompleteLruCache {
return new AutocompleteLruCache(db);
}

async get(key: string): Promise<string | undefined> {
async get(prefix: string): Promise<string | undefined> {
// NOTE: Right now prompts with different suffixes will be considered the same

// If the query is "co" and we have "c" -> "ontinue" in the cache,
// we should return "ntinue" as the completion.
// Have to make sure we take the key with shortest length
const result = await this.db.get(
"SELECT value FROM cache WHERE key = ?",
key,
"SELECT key, value FROM cache WHERE ? LIKE key || '%' ORDER BY LENGTH(key) DESC LIMIT 1",
prefix,
);

if (result) {
// Validate that the cached compeltion is a valid completion for the prefix
if (result && result.value.startsWith(prefix.slice(result.key.length))) {
await this.db.run(
"UPDATE cache SET timestamp = ? WHERE key = ?",
Date.now(),
key,
prefix,
);
return result.value;
// And then truncate so we aren't writing something that's already there
return result.value.slice(prefix.length - result.key.length);
}

return undefined;
}

async put(key: string, value: string) {
async put(prefix: string, completion: string) {
const result = await this.db.get(
"SELECT key FROM cache WHERE key = ?",
key,
prefix,
);

if (result) {
await this.db.run(
"UPDATE cache SET value = ?, timestamp = ? WHERE key = ?",
value,
completion,
Date.now(),
key,
prefix,
);
} else {
const count = await this.db.get("SELECT COUNT(*) as count FROM cache");
Expand All @@ -71,8 +78,8 @@ export class AutocompleteLruCache {

await this.db.run(
"INSERT INTO cache (key, value, timestamp) VALUES (?, ?, ?)",
key,
value,
prefix,
completion,
Date.now(),
);
}
Expand Down
4 changes: 3 additions & 1 deletion core/autocomplete/charStream.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
export async function* onlyWhitespaceAfterEndOfLine(
stream: AsyncGenerator<string>,
endOfLine: string[],
fullStop: () => void,
): AsyncGenerator<string> {
let pending = "";
for await (let chunk of stream) {
Expand All @@ -12,6 +13,7 @@ export async function* onlyWhitespaceAfterEndOfLine(
chunk[i + 1].trim() === chunk[i + 1]
) {
yield chunk.slice(0, i + 1);
fullStop();
return;
}
}
Expand All @@ -30,7 +32,7 @@ export async function* noFirstCharNewline(stream: AsyncGenerator<string>) {
for await (const char of stream) {
if (first) {
first = false;
if (char === "\n") {
if (char.startsWith("\n") || char.startsWith("\r")) {
return;
}
}
Expand Down
Loading

0 comments on commit aa18568

Please sign in to comment.