Skip to content

feat(sql-editor): add type-aware INSERT value hints#1933

Merged
openai0229 merged 11 commits into
OtterMind:mainfrom
auenger:feat/insert-value-hints
Jul 25, 2026
Merged

feat(sql-editor): add type-aware INSERT value hints#1933
openai0229 merged 11 commits into
OtterMind:mainfrom
auenger:feat/insert-value-hints

Conversation

@auenger

@auenger auenger commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Auto-fill type-aware default values for explicit-column INSERT ... VALUES ( statements.
  • Render field:type labels with Monaco injected text; labels are presentation-only and never enter executable SQL.
  • Preserve label groups across multiple INSERT statements, value edits, cursor movement, and SQL formatting without shifting Monaco's real text positions.
  • Keep MySQL's existing completion-candidate contract unchanged; editor hints are returned separately.
  • Limit the feature to MySQL and PostgreSQL, the two dialects validated with automated tests and real database execution.

Supported behavior contract

This PR deliberately uses a narrow, fail-closed contract:

  • Supported databases: MySQL and PostgreSQL only.

  • Supported statement: INSERT only; UPDATE and mixed/non-INSERT formatting retain existing behavior.

  • Required shape: an explicit column list, for example:

    INSERT INTO demo (id, enabled, name, created_at) VALUES (
  • INSERT INTO demo VALUES ( is not inferred because the SQL text does not establish a safe column mapping.

  • Every SQL column must resolve against database metadata. If it does not, no row is auto-filled.

  • Known scalar types receive dialect-specific executable defaults; unknown or unsafe-to-guess types use NULL.

  • PostgreSQL quoted identifiers keep exact case, while unquoted identifiers use PostgreSQL lowercase folding.

  • PostgreSQL single quotes, double quotes, comments, and dollar-quoted strings are excluded from statement/value delimiter handling.

  • The executable defaults and labels are separate response fields; injected labels never modify, copy with, format into, or execute as SQL.

The executable contract is covered directly by:

Review feedback addressed

  • Restored the MySQL VALUES-expression completion contract; all 176 scenario coverage cases pass.
  • Scoped backend hint registration and frontend enablement to MySQL/PostgreSQL only; the generic provider and unvalidated dialect opt-ins were removed.
  • Scoped PostgreSQL parsing to the current statement and stopped hints after the VALUES row closes.
  • Fixed later INSERT auto-fill, multi-INSERT label accumulation, post-edit rematerialization, and pre/post-format row mapping.
  • Kept PostgreSQL formatting's one-value-per-line layout for INSERT scripts only; SELECT and mixed scripts retain the prior default formatter.
  • Preserved PostgreSQL quoted/unquoted identifier semantics and handled dollar-quoted values.
  • Replaced PostgreSQL metadata-type regular expressions with deterministic scans to remove the CodeQL polynomial-backtracking findings.
  • Corrected the PostgreSQL empty bytea expression to '\x'::bytea after execution against PostgreSQL 16.

Verification

Local verification on the latest main:

  • MySQL owning module: 607 tests, 0 failures/errors (MysqlSqlCompletionScenarioCoverageTest: 176/176).
  • PostgreSQL owning module: 17 tests, 0 failures/errors.
  • PostgreSQL formatter regression: 4 tests, 0 failures/errors.
  • SQL editor focused TypeScript tests: passed.
  • yarn run lint: passed.
  • yarn run build:web:community --app_version=0.0.0: passed.
  • Community backend clean package: passed with -Dmaven.test.skip=true; this is a packaging check, not a test run.
  • Community/OFFLINE backend and frontend smoke: HTTP 200 on loopback.
  • Real database execution: MySQL 8.4 and PostgreSQL 16 passed, including PostgreSQL quoted identifiers, dollar-quoted values, and empty bytea.
  • git diff --check: passed.

Screenshots

UI screenshots and the prior manual regression notes are available in the PR comments.

@auenger

auenger commented Jul 22, 2026

Copy link
Copy Markdown
Contributor Author

实现说明与截图补充区

这次改动只针对带明确字段列表的 INSERT:

INSERT INTO demo (id, enabled, name, birthday, created_at) VALUES (

输入 VALUES ( 后,编辑器会根据数据库元数据自动插入一整组默认值并补齐右括号,例如整数 0、布尔值 FALSE/0、字符串 ''、日期和时间函数;未知或不适合猜测的类型使用 NULL。SQL Server 等方言使用各自可执行的日期表达式。

每个 value 后面的 字段名:类型 是 Monaco injected text,仅用于展示:

  • 不会写入编辑器文本,也不会进入实际执行 SQL;
  • 光标和删除操作不会被 label 阻挡;
  • SQL 格式化后会按 value 槽位重新定位;
  • 用户修改 value 时不会重填整行;
  • 删除单个 value 时只隐藏对应 label,重新输入后恢复;
  • UPDATE 已明确排除,不再提供这套自动填充或类型 label。

后端分层:

  • MySQL 保留专用元数据、默认值和候选值逻辑;
  • PostgreSQL 保留 JSON/JSONB、数组、bytea 等方言映射;
  • 其他采用相同显式字段 INSERT 结构的关系型数据库走 SPI 通用提供器;
  • MongoDB、Redis、Elasticsearch、Kylin 未开启本能力。

截图:
a95a7da4843961e55855ce024b0ddc1e

@Chat2DB-Pro Chat2DB-Pro left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the feature — the overall direction is good: injected-text labels that never enter executable SQL, solid async race guards (modelVersionId + requestId + epoch), undo grouping so one Ctrl+Z reverts the whole auto-fill, and the conservative explicit-column-list requirement are all the right calls. I verified the branch locally; requesting changes for the items below, ranked by severity.

1. Two existing contract tests fail on this branch (verified locally)

MysqlSqlCompletionScenarioCoverageTest has two contract cases asserting that the INSERT/REPLACE VALUES expression slot returns no completion candidates (expected: <EMPTY> but was: <SUCCESS>):

  • insertValuesExpressionSlotDoesNotUseInsertTargetColumns
  • replaceValuesExpressionSlotDoesNotUseReplaceTargetColumns

On main all 176 tests in that class pass; on this branch these two fail (full mysql module run: Tests run: 607, Failures: 2). The PR's verification list only ran MysqlSqlCompletionProviderTest (163 tests) and missed this suite in the same module, so the "0 failures" claim doesn't hold. If the behavior change is intentional, please update the contract tests and call it out in the description; otherwise adjust MysqlSqlCompletionValueCandidateProvider.

2. Multi-statement documents: the regex binds to the FIRST insert, and auto-fill silently stops working

StandardSqlInsertEditorHintProvider / PostgreSqlEditorHintProvider run matcher.find() over everything before the cursor, so the first INSERT INTO ... VALUES ( in the document wins — but the frontend sends the whole editor content. With any completed INSERT above the cursor (the most common workflow: writing several inserts in one console), the hint is computed against the wrong table and group(3) spans multiple statements. Knock-on effect: the value ranges are non-empty, so the frontend isEmptyRange guard rejects the auto-fill — auto-fill silently never fires whenever any complete INSERT precedes the current one. Suggest anchoring to the last unterminated INSERT before the cursor (or slicing at statement boundaries first). Related: after VALUES (1, 2); is closed, (.*)$ still matches and produces a hint — the provider should return empty once the VALUES parenthesis is balanced.

3. Hint stickiness: auto-fill effectively works once per document

findValuesRow (sqlInsertValueDefaults.ts, fallback branch) returns the ranges of the first VALUES row in the document whenever the exact-value match fails, and scanValuesRow never returns null. So once an auto-fill happened, rematerializeInsertValueHints stays non-empty as long as any values( text exists anywhere, and handleValueChange early-returns before the 'content'-sourced refreshBackendParameterHints — the only path that performs auto-fill. Consequences: typing VALUES ( in a second INSERT never auto-fills again, and after the user edits a value the labels can jump to the first (possibly unrelated) VALUES row in the document. Suggest clearing the hints when the exact match fails instead of falling back to the first row.

4. Default values are executable SQL — a few dialect mappings look wrong

  • Informix does not accept CURRENT_TIMESTAMP (it uses CURRENT / CURRENT YEAR TO SECOND), so the auto-filled row fails on execution.
  • TDengine conventionally uses NOW.
  • ZERO_BOOLEAN_DIALECTS contains both "INFORMIX" and "INFOMIX" — the duplicate spelling suggests the actual dbType enum values weren't verified. Please cross-check against DataSourceTypeEnum, or narrow the supported list to dialects whose defaults were actually validated.

5. Smaller items

  • ISqlSyntaxPlugin.getSqlEditorHints defaults to the standard provider for every dialect (including Redis/Mongo), relying on the regex not matching — this contradicts the description ("frontend enables for listed dialects"). Suggest defaulting to empty and opting in explicitly.
  • PostgreSqlEditorHintProvider duplicates ~300 lines of the standard provider with only quoting/2-part-name differences; consider parameterizing instead.
  • Backslash escapes aren't handled in the literal scanners: MariaDB/TiDB default to backslash escaping, so 'it\'s' desyncs the quote state machine on both backend and frontend (labels misplace; cosmetic but visible).
  • insideLiteralOrComment tracks doubleQuoted but only returns singleQuoted.

Items 1 and 2 are blocking; 3 is strongly recommended; 4 needs at least the Informix/TDengine check or a narrower dialect list. With those addressed this is a nice feature.

@auenger

auenger commented Jul 22, 2026

Copy link
Copy Markdown
Contributor Author

已根据 review 完成修复,提交:7b395383

主要调整:

  • 恢复 MySQL 原有补全契约:VALUES 表达式槽位继续返回 EMPTY,不再向候选列表注入字段默认值;编辑器所需的 editorHints 仍独立返回。
  • 标准 INSERT hint 只分析光标所在的当前 SQL 语句;VALUES 行已经闭合后不再继续生成后端提示。
  • 修复多条 INSERT 时第二条无法自动填充、label 跳到第一条/错误 value 的问题;用户修改已有 value 后 label 仍按最近的 VALUES 行重新定位。
  • 支持 MariaDB/TiDB/OceanBase 的反斜杠转义字符串扫描。
  • 将通用 INSERT hint 改为数据库插件显式 opt-in;Redis、MongoDB、Elasticsearch 等不会误用 SQL 提示。GBase8s 当前没有 SQL syntax plugin,因此从前端能力声明中移除。
  • PostgreSQL 合并到统一 provider,保留 array/json/jsonb/bytea 默认值;Informix 改用 CURRENT/TODAY,TDengine timestamp 改用 NOW

实际验证:

  • MySQL 模块完整测试:607 tests,0 failures,0 errors。
  • SPI 聚焦测试:9 tests,0 failures,0 errors。
  • 前端 test:sql-in-clipboard(含 INSERT value defaults):通过。
  • SQLEditor completion mode / insert highlight / parameter hint 聚焦测试:通过。
  • 前端 lint:通过。
  • Community Web build:通过。
  • Community 后端 48-module clean package:通过(该打包命令按项目约定跳过测试;测试结果来自上述独立测试命令)。
  • git diff --check:通过。

@auenger

auenger commented Jul 22, 2026

Copy link
Copy Markdown
Contributor Author

补充修复 PostgreSQL 多条 INSERT 场景:此前每次自动填充都会用最新一条 INSERT 的 label 集合覆盖已有集合,因此同一编辑器中仅最后一条保留提示。现在按 VALUES 行累积并重新定位各组 label,后续 INSERT 不再清除前面语句的提示;编辑既有值时也会保留全部提示组。新增连续 3 条 INSERT 回归覆盖。验证通过:test:sql-in-clipboard、ESLint、Stylelint、build:web:community。提交:12e1dfbc。

@auenger

auenger commented Jul 22, 2026

Copy link
Copy Markdown
Contributor Author

已完成本轮多 INSERT 回归修复并更新 PR:提交 12e1dfb 修复后续 INSERT 覆盖前序 label;提交 3fd9061 修复美化 SQL 后多组 label 同时落到第一条 VALUES 的问题。新逻辑会保留每个 INSERT 的独立提示组,并基于格式化前后的 VALUES 序号一一重新定位。自动回归覆盖连续 3 条 INSERT、修改前序 value、三条 SQL 美化为多行后的 label 位置;test:sql-in-clipboard、ESLint、Stylelint、build:web:community 和 git diff --check 均通过。PostgreSQL 页面人工回归已确认多条提示、value 编辑和 SQL 美化场景正常。

image

@openai0229 openai0229 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The blocking issue is the validation-to-scope mismatch. This PR changes executable INSERT defaults across roughly 30 dialects and 55 files, while the current evidence covers MySQL automated tests and PostgreSQL manual validation only. A wrong default here changes SQL that users may execute; it is not only presentation metadata.

Please either narrow the first PR to the dialects that are actually validated (preferably MySQL and PostgreSQL in focused changes), or add executable SQL coverage for every enabled dialect/default family. Please also link the product/behavior contract that defines which defaults are safe to auto-fill, then update the branch with main and run the full required CI suite.

auenger added 2 commits July 24, 2026 10:56
Keep INSERT value hints scoped to MySQL and PostgreSQL, stabilize injected labels during editing, and format PostgreSQL INSERT values vertically.
@auenger

auenger commented Jul 24, 2026

Copy link
Copy Markdown
Contributor Author

Updated as requested and narrowed the scope to the database dialects we have actually validated.

Scope

  • INSERT value hints are now enabled only for MySQL and PostgreSQL.
  • The generic standard-SQL hint SPI and the opt-in changes for the other untested dialects have been rolled back.
  • PostgreSQL-specific parsing/default-value behavior now lives in the PostgreSQL plugin instead of a shared SPI implementation.
  • The branch has been synchronized with the latest main.

Editor UX improvements

  • Kept the Monaco injected label presentation, so hint text remains visible but is not part of the executable SQL.
  • Changed decoration stickiness so typed values stay on the value side of the label and no longer briefly render after it.
  • Added a visible · delimiter as measured injected text and removed CSS margin-left / font scaling. This avoids the cumulative cursor-position drift that occurred when several labels were rendered on the same line.
  • Limited the injected-text cursor stop to the left/value side, making keyboard navigation and continued typing more predictable.
  • PostgreSQL INSERT formatting now places each column and each VALUES item on its own line, keeping values and hints visually aligned. MySQL formatting behavior is unchanged.

Verification

  • MySQL 8.4 Docker regression: passed — real local connection, INSERT value auto-fill, injected labels, value editing/navigation, and SQL formatting were checked.
  • MySQL plugin full test set: 607 tests, 0 failures, 0 errors.
  • PostgreSQL editor/formatting regression: passed — INSERT hints and per-value multiline formatting were checked.
  • PostgreSQL plugin full test set: 14 tests, 0 failures, 0 errors.
  • Formatter-focused backend tests: 2 tests, 0 failures, 0 errors; the PostgreSQL format API smoke test also passed.
  • Frontend focused test, lint, and Community web build all passed.
  • Full 48-module Community backend package passed (packaging command intentionally skips tests; plugin tests were run separately as listed above).

Commit: 67265a736c0e6da626f4a3d237764d1a11e4caca
f0980977-da16-42cc-9cd3-1b3d432b903f

# Conflicts:
#	chat2db-community-client/package.json
auenger added 2 commits July 24, 2026 15:40
Preserve quoted identifier semantics, handle dollar-quoted values, and keep PostgreSQL formatting changes scoped to INSERT scripts. Replace metadata-type regex matching with deterministic scans so CodeQL cannot hit polynomial backtracking.
@auenger

auenger commented Jul 24, 2026

Copy link
Copy Markdown
Contributor Author

已完成合并前的最终收敛,并针对现有 review / CodeQL 再做了一轮完整检查。

最终 head:2ffc09c8(功能补强提交:0c50ac52)。

本轮补强

  • 同步到最新 main
  • PostgreSQL 标识符按原生语义处理:双引号名称保持大小写,未加引号名称折叠为小写,避免 "ID"id 错配。
  • PostgreSQL $$...$$ / $tag$...$tag$ 内容作为单个 value 处理,内部逗号、分号和括号不会再影响 label range 或当前语句定位。
  • PostgreSQL 的一值一行格式化仅作用于纯 INSERT 脚本;SELECT、函数参数、IN (...) 和 INSERT/SELECT 混合脚本继续使用原有默认格式。
  • 修正空 bytea 默认值为 '\x'::bytea,并在 PostgreSQL 16 实际执行确认长度为 0。
  • 移除 PostgreSQL 类型判断中的回溯正则,改为确定性 token 扫描,处理 CodeQL 报出的 polynomial-regex findings。
  • 保持功能范围只覆盖 MySQL 和 PostgreSQL;其他未验证数据库没有重新启用。

验证证据

  • MySQL owning module:607 tests,0 failures/errors;其中 scenario coverage 176/176。
  • PostgreSQL owning module:17 tests,0 failures/errors。
  • PostgreSQL formatter regression:4 tests,0 failures/errors。
  • 前端 focused tests、lint、Community build:通过。
  • Community backend 48-module clean package:通过(该命令使用 -Dmaven.test.skip=true,仅作为产物验证)。
  • Community/OFFLINE 后端与前端 loopback HTTP smoke:200。
  • MySQL 8.4、PostgreSQL 16 真实数据库执行:通过。
  • git diff --check:通过。

PR body 已同步为当前实际范围,并补上了明确的行为契约与对应的可执行测试链接,方便 reviewer 直接核对。

@openai0229

Copy link
Copy Markdown
Contributor

Thanks for the extensive revisions and verification already completed on this feature. During the final maintainer review, we found one remaining reproducible edge case: when a real INSERT was preceded by text such as -- VALUES (0, FALSE, '\), frontend rematerialization treated the comment as the first VALUES row and moved all injected labels onto the comment line. The same lexical gap meant stale editor hints could allow auto-fill inside comment or string text, and PostgreSQL row scanning treated comment delimiters as executable commas or closing parentheses.

We pushed maintainer commit c43ecd93 to the contributor branch. It adds a shared frontend SQL code mask for strings, quoted identifiers, dollar quotes, line comments, and block comments; uses it for both auto-fill and label rematerialization; and makes the PostgreSQL VALUES scanner ignore delimiters inside comments.

Verification:

  • yarn test:sql-in-clipboard: passed
  • sqlCompletionModelMode.test.ts: passed
  • focused ESLint on the two changed TypeScript files: passed
  • PostgreSqlInsertEditorHintProviderTest: 9 tests, 0 failures/errors
  • git diff --check: passed

The new regression tests cover both commented and dollar-quoted fake VALUES rows, fail-closed auto-fill inside comments/strings, and PostgreSQL comment delimiters. We will wait for the refreshed full CI and Java CodeQL before the final merge review.

@openai0229 openai0229 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the latest fixes at c43ecd9. INSERT value-hint rematerialization now ignores strings and line/block comments, PostgreSQL comment delimiters are handled, the focused frontend and PostgreSQL tests pass, and the current head passes the required checks plus Java CodeQL. The previously identified blockers are resolved.

@openai0229
openai0229 dismissed Chat2DB-Pro’s stale review July 25, 2026 12:43

Addressed by subsequent commits: 7b39538 resolved the initial review feedback, 12e1dfb fixed multi-INSERT behavior, 67265a7 narrowed the feature to validated dialects, 0c50ac5 hardened PostgreSQL handling, and c43ecd9 fixed comment/string positioning with regression coverage. The latest head passes focused tests, required checks, and Java CodeQL.

@openai0229
openai0229 merged commit 9219040 into OtterMind:main Jul 25, 2026
15 of 16 checks passed
@openai0229 openai0229 moved this from In Review to Done in Chat2DB Community Jul 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

4 participants