fix(normalizer): preserve bracket-quoted identifiers containing spaces#101
Merged
eric-weaver merged 1 commit intoJul 2, 2026
Conversation
31c17c2 to
18c16e6
Compare
b6e1d54 to
4122add
Compare
Bracket-quoted T-SQL identifiers whose content contains whitespace (e.g. [FPA EAM NAME]) were incorrectly de-bracketed in column/table reference position, producing invalid SQL like `CSC. FPA EAM NAME`. Two fixes: 1. shouldStripIdentifierQuotes: refuse to strip when token.Value contains whitespace — stripped content would yield multiple SQL tokens and break the query. 2. appendSpace: suppress the inter-token space when the preceding token ends with '.' and the current token is a QUOTED_IDENT — this handles the split that the lexer performs for constructs like `CSC.[FPA EAM NAME]` (lexed as `CSC.` + `[FPA EAM NAME]`). Simple identifiers ([public], [users]) and multi-part identifiers ([public].[users]) are unaffected — they contain no whitespace and continue to be de-bracketed as before. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4122add to
9990a30
Compare
This comment has been minimized.
This comment has been minimized.
eric-weaver
approved these changes
Jul 2, 2026
2 tasks
gh-worker-dd-mergequeue-cf854d Bot
pushed a commit
to DataDog/datadog-agent
that referenced
this pull request
Jul 7, 2026
### What does this PR do? Bumps `github.com/DataDog/go-sqllexer` from v0.2.2 to v0.2.3 across all modules that depend on it. ### Motivation Picks up [DataDog/go-sqllexer#101](DataDog/go-sqllexer#101), which preserves bracket-quoted T-SQL identifiers containing spaces (e.g. `[Column With Spaces]`) during SQL normalization instead of corrupting them. ### Describe how you validated your changes Bumped the version in each dependent module and ran `go work sync` + `go mod tidy`. The diff is limited to the go-sqllexer version and its `go.sum` hashes. Co-authored-by: ethan.perez <ethan.perez@datadoghq.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
Reported in SDBM-2749: T-SQL queries using bracket-quoted identifiers with spaces in the column name (e.g.
schema.[Column With Spaces]) were being normalized to invalid SQL. The brackets were stripped, leaving bare spaces in identifier position which breaks query structure.What was happening
shouldStripIdentifierQuotesonly protected non-simple identifiers in alias context. In column/table reference position, brackets were always stripped — including identifiers whose content contains spaces, which cannot appear unquoted in SQL.A secondary issue: the lexer splits
alias.[Spaced Name]into two tokens (alias.+[Spaced Name]), and the normalizer was inserting a space between them, producingalias. [Spaced Name]instead ofalias.[Spaced Name].Fix
shouldStripIdentifierQuotes: refuse to strip when the raw token value contains whitespace — unquoted, the content would become multiple SQL tokens.appendSpace: suppress the inter-token space when the preceding token ends with.and the next is aQUOTED_IDENT.Simple identifiers (
[schema],[table]) and dot-joined multi-part identifiers ([schema].[table]) contain no whitespace and continue to be de-bracketed as before.Test plan
SELECT t.[Column With Spaces] FROM dbo.SomeTable AS t→ brackets preserved, no spurious spaceSELECT [First Name], [Last Name] FROM [My Table]→ all brackets preservedSELECT * FROM [public].[users]→ still de-bracketed topublic.users(regression)go test ./...passes🤖 Generated with Claude Code