Skip to content

fix(normalizer): preserve bracket-quoted identifiers containing spaces#101

Merged
eric-weaver merged 1 commit into
DataDog:mainfrom
pierreln-dd:pierreln-dd/fix-strip-bracket-quoted-identifiers-with-spaces
Jul 2, 2026
Merged

fix(normalizer): preserve bracket-quoted identifiers containing spaces#101
eric-weaver merged 1 commit into
DataDog:mainfrom
pierreln-dd:pierreln-dd/fix-strip-bracket-quoted-identifiers-with-spaces

Conversation

@pierreln-dd

@pierreln-dd pierreln-dd commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Context

Reported in SDBM-2749: T-SQL queries using bracket-quoted identifiers with spaces in the column name (e.g. schema.[Column With Spaces]) were being normalized to invalid SQL. The brackets were stripped, leaving bare spaces in identifier position which breaks query structure.

What was happening

shouldStripIdentifierQuotes only protected non-simple identifiers in alias context. In column/table reference position, brackets were always stripped — including identifiers whose content contains spaces, which cannot appear unquoted in SQL.

A secondary issue: the lexer splits alias.[Spaced Name] into two tokens (alias. + [Spaced Name]), and the normalizer was inserting a space between them, producing alias. [Spaced Name] instead of alias.[Spaced Name].

Fix

  1. shouldStripIdentifierQuotes: refuse to strip when the raw token value contains whitespace — unquoted, the content would become multiple SQL tokens.
  2. appendSpace: suppress the inter-token space when the preceding token ends with . and the next is a QUOTED_IDENT.

Simple identifiers ([schema], [table]) and dot-joined multi-part identifiers ([schema].[table]) contain no whitespace and continue to be de-bracketed as before.

Test plan

  • SELECT t.[Column With Spaces] FROM dbo.SomeTable AS t → brackets preserved, no spurious space
  • SELECT [First Name], [Last Name] FROM [My Table] → all brackets preserved
  • SELECT * FROM [public].[users] → still de-bracketed to public.users (regression)
  • go test ./... passes

🤖 Generated with Claude Code

@pierreln-dd pierreln-dd force-pushed the pierreln-dd/fix-strip-bracket-quoted-identifiers-with-spaces branch from 31c17c2 to 18c16e6 Compare June 30, 2026 11:52
@pierreln-dd pierreln-dd marked this pull request as ready for review June 30, 2026 11:55
@pierreln-dd pierreln-dd requested a review from a team as a code owner June 30, 2026 11:55
@pierreln-dd pierreln-dd force-pushed the pierreln-dd/fix-strip-bracket-quoted-identifiers-with-spaces branch 2 times, most recently from b6e1d54 to 4122add Compare June 30, 2026 12:11
Bracket-quoted T-SQL identifiers whose content contains whitespace
(e.g. [FPA EAM NAME]) were incorrectly de-bracketed in column/table
reference position, producing invalid SQL like `CSC. FPA EAM NAME`.

Two fixes:
1. shouldStripIdentifierQuotes: refuse to strip when token.Value
   contains whitespace — stripped content would yield multiple SQL
   tokens and break the query.
2. appendSpace: suppress the inter-token space when the preceding
   token ends with '.' and the current token is a QUOTED_IDENT —
   this handles the split that the lexer performs for constructs
   like `CSC.[FPA EAM NAME]` (lexed as `CSC.` + `[FPA EAM NAME]`).

Simple identifiers ([public], [users]) and multi-part identifiers
([public].[users]) are unaffected — they contain no whitespace and
continue to be de-bracketed as before.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@pierreln-dd pierreln-dd force-pushed the pierreln-dd/fix-strip-bracket-quoted-identifiers-with-spaces branch from 4122add to 9990a30 Compare June 30, 2026 12:12
@datadog-prod-us1-3

This comment has been minimized.

@eric-weaver eric-weaver merged commit d86dde9 into DataDog:main Jul 2, 2026
7 of 8 checks passed
@pierreln-dd pierreln-dd mentioned this pull request Jul 3, 2026
2 tasks
gh-worker-dd-mergequeue-cf854d Bot pushed a commit to DataDog/datadog-agent that referenced this pull request Jul 7, 2026
### What does this PR do?

Bumps `github.com/DataDog/go-sqllexer` from v0.2.2 to v0.2.3 across all modules that depend on it.

### Motivation

Picks up [DataDog/go-sqllexer#101](DataDog/go-sqllexer#101), which preserves bracket-quoted T-SQL identifiers containing spaces (e.g. `[Column With Spaces]`) during SQL normalization instead of corrupting them.

### Describe how you validated your changes

Bumped the version in each dependent module and ran `go work sync` + `go mod tidy`. The diff is limited to the go-sqllexer version and its `go.sum` hashes.


Co-authored-by: ethan.perez <ethan.perez@datadoghq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants