Skip to content

ParserSQL 1.1.0

Choose a tag to compare

View all tags
@renecannao renecannao released this 09 Oct 20:59
v1.1.0
a123f6e
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

ParserSQL 1.1.0 expands native PostgreSQL and MySQL parsing, adds APIs for working with ASTs, and improves correctness and large-list parsing performance.

Grammar and parser features

  • Broader PostgreSQL expressions, casts, operators, named arguments, aggregates, window clauses, CTEs, TABLE/VALUES queries, DML, DDL, transaction control, COPY, and administrative/session commands. Support is bounded by the documented grammar; this is not complete PostgreSQL grammar parity.
  • Broader MySQL JSON and window syntax, aggregates, casts/conversions, full-text expressions, table modifiers, CTE-prefixed DML, INSERT sources, EXPLAIN, and structured table DDL.
  • MySQL CREATE TABLE LIKE and CREATE TABLE AS SELECT retain structured source relations/queries. CREATE/ALTER partitioning, JSON_TABLE, optimizer hints, and a bounded CREATE PROCEDURE implementation also gain AST support.
  • MySQL BEGIN [WORK] and START TRANSACTION options now have structured transaction ASTs. Unsupported transaction execution options are rejected before starting a local engine transaction.
  • parse_all() returns individual statement results and source spans for SQL batches.

AST APIs and correctness

  • Preorder visitor API with parent/depth/child context, skip/stop controls, and traversal limits.
  • Owned AST cloning, validated subtree replacement, and context-aware parameterization with extracted-literal and existing-placeholder mappings.
  • Canonical keyword/operator/function values while retaining original source spelling and quoted PostgreSQL function names.
  • Source ownership fixes for cached ASTs, allocation-failure handling, stricter malformed-input checks, and schema/table metadata fixes for catalog-qualified targets.
  • PostgreSQL generic SET values reject operator/function expressions, including malformed search_path values that could otherwise reach a proxy's session tracker.

Performance

Large IN-list child construction is linear in the number of elements. Token reuse and common PostgreSQL parsing paths also received optimizations. No new cross-parser speed ratio is claimed by this release.

Migration notes

  • The core Parser/ParseResult entry points and 48-byte AstNode layout are retained; statement, node, and token enums gain appended values. Rebuild consumers against matching headers and library sources.
  • AST structure and emitted text can change for newly structured or corrected grammar. PostgreSQL casts now use NODE_TYPE_CAST/NODE_TYPE_NAME; transaction commands expose structured options. Consumers that match exact tree shapes must review these changes.
  • Canonical values are not necessarily byte-for-byte input slices. Use source() where the original spelling is required, and respect parser-arena/input lifetimes. Owned clones copy both values and source strings.
  • MySQL function names canonicalize to uppercase; unquoted PostgreSQL function names fold to lowercase while quoted names retain case. SQL emission and AST-based digest text/hashes may change; validate persisted digest-based rules and snapshots during integration.
  • Parser acceptance is separate from query-engine execution support. New grammar does not imply that the local engine can execute it, and parameterization explicitly rejects unsupported contexts.

Coverage and limitations

The PostgreSQL compatibility harness uses pinned libpg_query/PostgreSQL sources and separates complete AST parsing from classification, partial parsing, errors, trailing input, and statement-type mismatches. The committed full-corpus report is a dated baseline, not a fresh release-wide coverage measurement.

The MySQL harness tracks 545 curated cases against grammar sources pinned to MySQL 8.4.8 and 9.7.2. The baseline includes 257 positive witnesses, 287 malformed controls, and one known CTAS/SELECT INTO OUTFILE gap. It is not an overall MySQL coverage percentage or live-server certification; native-oracle checks are optional and were not run for this release.

See AST APIs, PostgreSQL features, PostgreSQL baseline, and MySQL compatibility.

Validation

  • Final source tree: 1,773 tests passed, with 37 backend-dependent tests skipped; corpus harness built successfully.
  • PostgreSQL: 87 harness tests and 5,122 committed compatibility cases passed.
  • MySQL: 545 curated cases matched their expected outcomes; native-server oracle not run.
  • ProxySQL v3.0 adapter at 546abf5c: recompiled SET tests passed all 494 assertions. The digest smoke test passed 51 assertions with its existing buffer-cleanup error corrected in a temporary test copy. This was an adapter check, not a full ProxySQL deployment test.
  • CI: PR #70 CI passed all nine regular jobs: GCC and Clang on Ubuntu 22.04/24.04, macOS, corpus, benchmark, PostgreSQL compatibility, and MySQL compatibility. The scheduled-only full PostgreSQL job was skipped. The release commit has the exact same source tree as the tested PR revision.

Includes #50, #64, #66, #67 and #70, the release SET regression fix.

Full changelog

Release commit: a123f6e94dff79f0823909401cc7a3ef4eff9e56.

Assets 2
Loading