v1.1.1

Column transformers and qualified stars, following ClickHouse 26.8.

• Column transformers without parentheses and with STRICT (#126):
  * EXCEPT a, * EXCEPT STRICT (a, b), * EXCEPT 'regex',
  * REPLACE a + 1 AS a, * APPLY toString, * APPLY x -> x + 1. STRICT
  is kept: ClickHouse fails on a missing column instead of ignoring it.
• SELECT db.t.* FROM db.t, a star qualified by database and table, also
  with transformers (#152).
• go.mod declares go 1.27, and any Go 1.27.x toolchain works.

⚠️ This patch release contains a breaking API change. SelectItem.Modifiers
is now []*ColumnTransformer (was []*FunctionExpr), and ASTVisitor has a new
method, VisitColumnTransformer. Visitors embedding DefaultASTVisitor or
PrintVisitor inherit it. See the Breaking section below (also in
COMPATIBILITY.md).

Breaking

• Column transformers have their own node (#126): SelectItem.Modifiers
  is []*ColumnTransformer (was []*FunctionExpr), with Kind ("EXCEPT",
  "REPLACE", "APPLY"), Strict, HasParen and Args. ASTVisitor has a
  new method, VisitColumnTransformer; visitors embedding DefaultASTVisitor
  or PrintVisitor inherit it.

Changed behavior

• Column transformers print with a space before parentheses, as in
  ClickHouse: * EXCEPT (a) APPLY (sum) (was EXCEPT(a) APPLY(sum)) (#126).

Newly rejected input

• An alias on a column matcher: SELECT * AS x, t.* AS x,
  COLUMNS(…) AS x, * EXCEPT (a) AS x (#126).
• Column transformer arguments ClickHouse rejects: EXCEPT (t.a), mixing a
  regex with names in EXCEPT, REPLACE (a) without AS, APPLY(f, g),
  and EXCEPT strict (STRICT is always the modifier; write EXCEPT (strict))
  (#126).

Additive

• Newly accepted input: column transformers without parentheses and with
  STRICT: * EXCEPT a, * EXCEPT STRICT (a, b), * EXCEPT 'regex',
  * REPLACE a + 1 AS a, * APPLY toString, * APPLY x -> x + 1 (#126).
• Newly accepted input: a star qualified by database and table,
  SELECT db.t.* FROM db.t, with column transformers (db.t.* EXCEPT (a))
  (#152). It parses as a *Path whose last field is the * identifier;
  t.* is still a *NestedIdentifier.

What's Changed

• Parser: accept a star qualified by database and table, db.t.* (#152) by @orian in #155
• Parser: model column transformers; accept STRICT and paren-free forms (#126) by @orian in #156
• Release v1.1.1 by @orian in #157

Full Changelog: v1.1.0...v1.1.1

v1.1.0

The parser now follows ClickHouse 26.8 much more closely.

• Silent SQL losses fixed: it no longer drops or changes clauses when printing. For example, DETACH used to print as DROP, SELECT 1, limit(1) became LIMIT (1), and INTERVAL … PRECEDING lost PRECEDING.
• Much more valid SQL accepted: EXPLAIN in all forms, INTERSECT, PASTE/NATURAL JOIN, CREATE/DROP INDEX, {name:Identifier} parameters, table-function expressions, and every SYSTEM SYNC/FLUSH/START|STOP form.
• SQL that ClickHouse rejects is rejected too.
• Formatting: PrintVisitor is about twice as fast.
• Traversal: Walk and the visitor reach every node.

⚠️ This minor release contains breaking API changes. Go 1.27 is required, ASTVisitor has new methods, some fields changed type or can now be nil, and positions and AST shapes changed. Read the Breaking section below (also in COMPATIBILITY.md) before upgrading.

Breaking

• Go 1.27 is required (go.mod said go 1.21.0).
• ASTVisitor has new methods: VisitStreamClause (#102), VisitCreateIndex
  and VisitDropIndex (#135). Visitors embedding DefaultASTVisitor or
  PrintVisitor inherit them; direct implementations must add them.
  StreamClause.Accept now dispatches to VisitStreamClause.
• Changed types: ShowStmt.Format is *FormatClause, from the embedded
  OutputClauses (name in Format.Format.Name; was *StringLiteral), and
  prints unquoted, FORMAT JSON rather than FORMAT 'JSON' (#87, #49).
• Fields that can now be nil, or hold other node types:
  • SubQuery.Select is nil for (EXPLAIN …); see SubQuery.Explain (#133);
  • TableIndex.Granularity is nil when GRANULARITY is omitted (#134);
  • ExplainStmt.Statement can be any statement, not only *SelectQuery, and
    is nil for EXPLAIN CURRENT TRANSACTION; ExplainStmt.Type is empty when
    the kind is omitted (#86);
  • SystemCtrlExpr.Cluster (the table) is nil when none is given (#108);
  • TableArgListExpr.Args can hold any expression (#129).
• Positions (#92):
  • Pos()/End() of a quoted StringLiteral/Ident include the quotes; the
    LiteralPos/LiteralEnd/NamePos/NameEnd fields still exclude them;
  • End() of nodes closed by ), ] or } points after the delimiter
    (ParamExprList, FunctionExpr, ArrayParamList, MapLiteral,
    TableSchemaClause, TableArgListExpr, ComplexType, QueryParam, …);
  • a statement's [Pos(), End()) covers exactly its text (it used to stop
    early or include the ;);
  • corrected fields: CompressionCodec.RightParenPos (the )), column
    COMMENT LiteralPos (the string), TypedPlaceholder.RightBracePos (the
    }), Fill.FillPos (FILL), IsNullExpr/IsNotNullExpr.IsPos (IS;
    Pos() is the operand), ShowStmt/GrantPrivilegeStmt.StatementEnd;
  • SystemSyncExpr.End(), SystemFlushExpr.End(), ExplainStmt.End() and
    TableIndex.End() cover the whole node (#62, #114, #86, #134).
• AST shapes:
  • ATTACH/DETACH keep their verb (IsAttach/IsDetach; DropStmt.Type()
    returns DETACH TABLE); DETACH TABLE t used to print as DROP TABLE t
    (#43, #52, #68);
  • a parenthesised set-operation operand is a group node: a SelectQuery with
    HasParen and the inner query in Group (#79, #93);
  • trailing FORMAT/query-level SETTINGS of a statement live in the embedded
    OutputClauses; after FORMAT on a SELECT in SelectQuery.OutputSettings
    (#87);
  • all parts of one TimeSeries target form a single TimeSeriesTargetClause
    (#42);
  • FROM t STREAM is TableExpr.Stream, not the alias STREAM (#81);
  • JoinExpr.Modifiers keeps GLOBAL, and can hold NATURAL/PASTE;
    a LOCAL JOIN b aliases a as LOCAL (#74, #89);
  • lower-case array join is an ARRAY JOIN (JoinExpr.Left is a
    *ColumnExprList), not a join on a table named by its first expression
    (#50);
  • SystemCtrlExpr.Type is the whole target, e.g. "REPLICATION QUEUES"
    (#108).

Changed behavior

• Traversal: Walk (and Find, FindAll, WalkWithBreak, Transform)
  and DefaultASTVisitor now reach every child node and agree with each other
  (#105, #106). They used to skip, among others, EXCEPT operands, INSERT … VALUES rows, TTL actions, named-collection parameters, DISTINCT ON
  columns, projection ORDER BY columns and RENAME pairs. Map keys are
  walked in place, so rewriting one changes the AST. IntervalFrom and
  WindowFrameParam no longer visit their child twice (#98).
• PrintVisitor prints every node like its String() (#98) and streams
  into one builder (#99). BeautifyVisitor emits every statement type (#51).
• Printed SQL fixed (the old output was invalid or meant something else):
  • WINDOW after HAVING (#88); IF [NOT] EXISTS before the name in DROP INDEX|PROJECTION and ADD INDEX (#64);
  • PRECEDING/FOLLOWING after an INTERVAL frame bound (#109);
  • SYSTEM SYNC REPLICA (#62), SYSTEM FLUSH DISTRIBUTED (#114), SYSTEM STOP|START DISTRIBUTED SENDS without … SENDS SENDS (#108);
  • SELECT 1, limit(1) keeps the column instead of becoming LIMIT (1)
    (#138); PASTE/NATURAL JOIN are joins, not aliases (#89);
  • (3,) keeps its comma (#70), f(p)(DISTINCT x) its DISTINCT (#71),
    INSERT INTO FUNCTION its FUNCTION (#72), PARTITION ID its ID (#76),
    WITH TOTALS without GROUP BY (#77), WITH FILL (#47);
  • USING is always parenthesised and comma joins print , (#74); -1::T
    keeps the sign on the literal (#83); TOP n WITH TIES (#90); no double
    spaces in CREATE TABLE t AS …/CREATE DATABASE … ENGINE (#45, #68);
  • CREATE TABLE keeps its query-level SETTINGS, and repeated CREATE USER
HOST/SETTINGS clauses are merged (#37).
• Parsing: INTERSECT is a keyword, not an implicit alias (#107);
  SELECT 1 EXCEPT (SELECT 2) is a set operation, not a column transformer
  (#107); in an expression { always starts a query parameter, and a
  {'k': v} map literal is valid only in INSERT … VALUES rows and as a
  SETTINGS value (#50).

Newly rejected input

• Select list and joins: a missing select list (SELECT, SELECT FROM t;
  a lone SELECT used to parse to zero statements) (#82); a comma before a
  clause other than FROM (SELECT a, ORDER BY a) (#138); a JOIN without
  ON/USING except CROSS, NATURAL, PASTE, ARRAY (#50); PASTE/
  NATURAL as an implicit alias, or with ON/USING, a strictness, CROSS
  or ARRAY (#89); GLOBAL ARRAY JOIN and LEFT GLOBAL JOIN (#74).
• Expressions: column transformers (EXCEPT, APPLY, REPLACE) after
  anything but *, t.*, COLUMNS(…) or * LIKE 'p' (#50); WITH CUBE/ROLLUP
  after GROUP BY CUBE(…)/ROLLUP(…) (#50); a map literal in an expression
  (#50); a trailing comma except in a one-element tuple (f(a,), [1,])
  (#70).
• Repeated or misplaced clauses: repeated engine clauses in CREATE TABLE,
  repeated CREATE USER clauses, GROUP BY modifiers out of order (#37);
  repeated TimeSeries target parts and RECENT without SAMPLES (#42); an
  alias, FINAL or SAMPLE after STREAM (#81); repeated WITH TIES and
  LIMIT … BY … WITH TIES (#90); FORMAT/SETTINGS after statements that
  take none (GRANT, SYSTEM, USE, SET, DELETE, USER/ROLE, CREATE FUNCTION/NAMED
  COLLECTION; used to be dropped silently), a quoted SHOW … FORMAT 'x', and
  INSERT … VALUES (…) FORMAT x (#87).
• DDL: an empty column list () (#45); ATTACH OR REPLACE,
  ATTACH/DETACH of FUNCTION/ROLE/USER/NAMED COLLECTION, DROP … PERMANENTLY, CREATE DICTIONARY d without a definition (#68); OR REPLACE
  with IF NOT EXISTS in CREATE TABLE/VIEW (#50); several pairs in RENAME DATABASE (#50); a PARTITION value other than a literal, query parameter,
  tuple or CAST of one (DROP PARTITION p, PARTITION toDate(…)) (#50); a
  malformed UUID '…' (ClickHouse: CANNOT_PARSE_UUID) (#50).
• SYSTEM: START|STOP DISTRIBUTED MERGES|FETCHES|TTL MERGES (used to print
  as … MERGES MERGES) (#108).
• Access control (#50): several roles in ALTER ROLE … RENAME TO; ON CLUSTER before the last role name; WITH ADMIN OPTION on a privilege grant;
  a column list on a wildcard target; the target *.table; GRANT ADMIN OPTION ON ….

Additive

• New types: OutputClauses (#87), StreamClause (#81), CreateIndex,
  DropIndex (#135).
• New fields: CreateTable.Settings, WithTimeoutClause.WithTimeoutEnd
  (#37); TimeSeriesTargetClause.InnerUUID, EngineShorthand (#42);
  IsAttach/IsDetach/Permanently, DropDatabase.Modifier (#68);
  ColumnExprList.HasTrailingComma (#70); AlterTableDelete.InPartition
  (#76); SelectQuery.HasParen, Group, OutputSettings, Union,
  Intersect, IntersectModifier, ExceptModifier (#79, #87, #93, #107,
  #125); TableExpr.Stream (#81); LimitClause.WithTies, WithTiesEnd (#90);
  OrderExpr.Nulls, Collate (#47); SystemSyncExpr.SyncEnd, Target,
  OnCluster, Database, IfExists, Mode, From, CacheName (#62);
  SystemCtrlExpr.OnCluster (#108); SystemFlushExpr.AsyncInsertQueue,
  OnCluster, Tables, Settings (#114); ExplainStmt.Settings,
  ExplainEnd (#86); Ident.Param for {name:Identifier} (#131);
  SubQuery.Explain (#133); span fields CastExpr.RightParenPos,
  SubQuery.RightParenPos, UsingClause.UsingEnd, OrderExpr.OrderEnd,
  NamedCollectionParam.ParamEnd, IsNullExpr/IsNotNullExpr.NullEnd (#92).
• New methods: VisitStreamClause, VisitCreateIndex, VisitDropIndex on
  DefaultASTVisitor and PrintVisitor; InsertStmt.IsTableFunction() (#72).
• Newly accepted input:
  • SELECT: INTERSECT [DISTINCT|ALL], EXCEPT DISTINCT|ALL, bare UNION
    (#1...

v1.0.4

What's Changed

• Add ClickHouse SQL corpora and fix parser panics by @orian in #28
• Add parser fuzzing and fix malformed-input crashes by @orian in #29
• Fix truncated SQL panics and expand manual fuzzing by @orian in #30
• Parser: support WITH RECURSIVE common table expressions by @orian in #32
• Expand parser fuzz coverage and fix AST panic paths by @orian in #34
• chore(parser): cover unterminated comments after CTEs by @orian in #33

Full Changelog: v1.0.3...v1.0.4

v1.0.3

What's Changed

• fix: support TTL GROUP BY SET rollups by @orian in #27

Full Changelog: v1.0.2...v1.0.3

v1.0.2

What's Changed

• fix: parse gcd codec chains by @rorylshanks in #26

New Contributors

• @rorylshanks made their first contribution in #26

Full Changelog: v1.0.1...v1.0.2

v1.0.1

What's Changed

• Parser: support TimeSeries engine-only targets by @orian in #25

Full Changelog: v1.0.0...v1.0.1

v1.0.0

What's Changed

• Parser: support bare TTL ... WHERE row-filter (fixes #2) by @orian in #4
• Parser: support INDEX TYPE kwargs e.g. text(tokenizer = ...) (fixes #3) by @orian in #5
• Parser: support CREATE VIEW DEFINER / SQL SECURITY / post-AS COMMENT (fixes #6) by @orian in #7
• Parser: support ENGINE = TimeSeries target clauses (fixes #8) by @orian in #9
• Rename module path to github.com/orian/clickhouse-sql-parser by @orian in #10
• Parser: support PROJECTION ... (SELECT ...) WITH SETTINGS (...) (fixes #11) by @orian in #12
• ci: run tests + build on main, modernize workflow by @orian in #13
• Parser: accept parameterless dictionary LAYOUT without inner parens (fixes #14) by @orian in #15
• test: cover every ClickHouse dictionary LAYOUT by @orian in #16
• Parser: accept CONSTRAINT ... ASSUME and preserve constraint kind (fixes #17) by @orian in #18
• BeautifyVisitor: render projection SELECT / ORDER BY (fixes #19) by @orian in #20
• Parser: apply column DEFAULT panic fix to canonical main by @orian in #23

Full Changelog: https://github.com/orian/clickhouse-sql-parser/commits/v1.0.0