fix(editor): stop Format Query crashing on an unterminated SQL literal

[datlechin]

What you see now

Format Query no longer crashes the app on a string literal that is still open and ends in a
backslash, and it no longer pushes the last character of an unclosed /* comment out of the comment
and reformats it as code.

Found while investigating #2158 (PR #2163). Different root cause, different trigger, so it ships on
its own.

Root cause

SQLTokenizer.tokenize has two branches that mishandle "the input ran out while we were inside a
construct", in opposite directions.

Overshoot, and it traps. Inside a quoted literal, if chars[i] == "\\" { i += 2; continue } was
unguarded. With the backslash at the last index i becomes count + 1, the while i < count loop
exits, and String(chars[start..<i]) slices past the end. Array index out of range. A trap is
uncatchable, so the do/catch in SQLEditorCoordinator does not help and the app dies.

Reproduced with a standalone probe extracting the exact loop: count=33, i=34, slice [29..<34],
SIGTRAP, exit 133. All three quote characters do it.

Type select * from t where c like 'C:\ and press Format Query.

Undershoot, and it corrupts. The block-comment scan is
while i + 1 < count && !(chars[i] == "*" && chars[i + 1] == "/"), and the i += 2 that skips the
closing */ is correctly gated. But when the comment is unterminated the loop stops at
i == count - 1 and nothing advances i, so the token is sliced short and the outer loop resumes on
the leftover character.

Measured: /* abc produced comment:"/* ab" plus identifier:"c". That last character is then
formatted as code, so it can be uppercased as a keyword or have whitespace and newlines inserted
around it. Silent corruption of the user's text by a formatting command.

The line-comment branch already gets this right with while i < count && chars[i] != "\n", which is
the shape the other two should have had. These are one root cause fifteen lines apart in one
function, so fixing only the trap would have left the corruption shipping.

The change

Two bounded exits, following the local convention in MongoShellFormatter.readStringLiteral:87
(bound the lookahead, keep the current character):

• string literal: i = min(i + 2, count)
• block comment: else { i = count } when the closing */ was never found

The formatter needs no decline path. An unterminated .string token already ships and already
works: 'abc with no backslash exits cleanly today and emits one .string holding the tail.
Assembly is verbatim, SQLFormatterService.swift:187 routes .string to appendToken, and
appendToken only prepends indent or a single space. Uppercasing is confined to .keyword. So
bounding the scanner is the smallest complete fix.

Blast radius

SQLTokenizer has exactly one production caller, SQLFormatterService.swift:52, so this reaches
Format Query only. Syntax highlighting does not use it. Entry points are the query menu, the editor
view, and the editor context menu, all funnelling into performFormatSQL.

Files

• TablePro/Core/Services/Formatting/SQLTokenizer.swift
• TableProTests/Core/Services/SQLTokenizerTests.swift
• TableProUITests/QueryFormatUnterminatedLiteralUITests.swift (new)
• CHANGELOG.md

Verification

Step	Result
generate, build	PASS
test SQLTokenizerTests pre-fix	FAIL, suite could not complete: Exceeded max restart count of 2 (Underlying Error: Crash: TablePro)
test 7 suites after the fix	PASS, 154 executed, 154 passed
uitest QueryFormatUnterminatedLiteralUITests pre-fix	FAIL
uitest QueryFormatUnterminatedLiteralUITests after	PASS
swiftlint on the production file	PASS, 0 violations

Suites run: SQLTokenizerTests, SQLFormatterServiceTests, FormatScopeResolverTests,
KeywordUppercaseHelperTests, SQLEditorCoordinatorTests, SQLEditorCoordinatorEscapeMenuTests,
SQLEditorCoordinatorCleanupTests.

The pre-fix red baseline is a host crash rather than a clean assertion, because a trap cannot be
caught by #expect, withKnownIssue, or XCTExpectFailure. That is the expected shape of the
evidence here, not an environment problem.

About the UI test

It types a backtick literal, not a single quote, and that is deliberate.
BracketPairs.allValues auto-closes ' and " through StandardOpenPairFilter, so a typed quote
gets a closing partner and the document no longer ends in the backslash that trips the tokenizer. An
auto-closed 'C:\' puts the backslash at index 3 of 5, i lands exactly on count, and nothing
traps. A first version of this test used ' and would have passed both before and after the fix.
Backticks are not paired and take the same tokenizer branch, so the backtick version builds the
trapping document deterministically. Proven by running it against the pre-fix code, where it fails.

Limitations

• SQLFormatterService trims the whole output, so trailing whitespace inside a trailing unterminated
  literal is still lost. Pre-existing and unchanged.
• Everything after an unterminated opening quote is still swallowed into one token and emitted
  verbatim rather than reformatted. That is the correct conservative behaviour for text the tokenizer
  cannot parse.
• Dialect-correct backslash escapes are out of scope. SQLTokenizer takes no dialect and treats \
  as an escape unconditionally, which is wrong for Postgres with standard_conforming_strings on.
  That is queued separately against SQLStatementScanner, which is on the execution path and needs
  its own review.
• JsonSyntaxParser.swift:200 has the same unguarded index += 2 shape but cannot trap: it fails
  closed, returning nil on exhaustion and only slicing after a validated in-bounds quote. Recorded
  so it is not re-raised.
• The required cross-vendor Codex review could not run: Codex returned a usage-limit error. An
  independent read-only Claude adversarial review ran in its place. Reported as not cross-vendor
  rather than as passed.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.