Skip to content

0.12.0

Choose a tag to compare

@ferenc-csaky ferenc-csaky released this 15 Oct 19:24
· 38 commits to main since this release
0.12.0
1be1eba

Upgrade Notes

  • The DataSQRL Docker images now default to TZ=UTC instead of America/Los_Angeles. Set TZ explicitly if you relied on the previous default (#2463)
  • The Vert.x server JVM now starts with -XX:+ExitOnOutOfMemoryError and -XX:MaxRAMPercentage=75. Both can be overridden through SQRL_JVM_ARGS, and an explicit -Xmx takes precedence (#2467)
  • A failed JBang UDF build now stops compilation with a fatal error instead of logging a warning (#2462)

Features

  • Added the core-agent Docker image: a DataSQRL coding agent built on Pi that bundles the SQRL CLI, the DataSQRL development skills and agent instructions, and selects the model provider from PI_PROVIDER, PI_MODEL or the available API keys (#2413, #2435)
  • Bundled the full documentation as llms-full.txt in the agent image, and generated llms.txt, llms-full.txt and a Markdown version of each page for every documentation version (#2451)
  • Warned at compile time (PRIMARY_KEY_NOT_UNIQUE) when a primary_key hint on a state table does not guarantee a unique key, for example after TRIM, LOWER or a lossy CAST on a key column. Such a key is hidden from Flink, so joins keep whole rows in state (#2426)
  • Extended the agent skills with guidance on cleaning primary keys once before using them, a DISTINCT decision table and watermark-free internal tables, and added a jwt helper for HS256 test tokens to the agent image (#2426)
  • Required slow-query logging (graphQLTailSampleTracingConfig) in the agent skills for every project that enables Vert.x, and taught index-friendly filters for API queries (#2461)
  • Aligned the agent skills and the documentation with the current compiler behavior (#2440)

Fixes

  • Made projects created with sqrl init compile without changes for all project types: prod sources declare the /*+ engine(kafka) */ hint, stream projects export to a Kafka topic, and dataset projects read from an external Iceberg table (#2462)
  • Declared the flink-table-common dependency in the add-func UDF templates, and reported failed JBang UDF builds with the javac output instead of a misleading "Could not find module" (#2462)
  • Defaulted the DataSQRL Docker images to UTC, so TIMESTAMP_LTZ values are no longer shifted when written to PostgreSQL during run and test (#2463)
  • Exited the Vert.x server on OutOfMemoryError so the container is restarted instead of staying up unhealthy, and sized its heap to 75% of the container memory instead of 25% (#2467)
  • Classified mysql-cdc and sqlserver-cdc sources as versioned-state connectors (#2440)

Documentation

  • Repositioned the README and the documentation around DataSQRL as a data engineering harness, with a rewritten getting-started guide and a new FAQ (#2399)
  • Versioned the documentation site by release, labeled pre-1.0 versions by minor version with a full-width version banner, and refreshed the site design (#2444, #2452, #2454)
  • Documented GraphQL and OpenAPI configuration and API versioning (#2370), Trino as an Iceberg query engine (#2369), batch job schedules (#2378), PostgreSQL and Vert.x size qualifiers and CPU factors (#2389), taskmanager-disk-size-gb for Flink cloud deployments (#2294), and the pg_partman_bgw.interval override (#2313)
  • Revamped the function documentation, and updated the language reference and deep-dive pages (#2280, #2321, #2427, #2437)
  • Fixed stdlib imports and stale commands and links, and retired the /flink and /ai pages (#2465)

Maintenance

  • Updated Flink SQL Runner from 0.11.3 to 0.12.0
  • Updated Apache Iceberg from 1.11.0 to 1.12.0 (#2429)
  • Ran documented commands verbatim and checked documented images in CI (#2464)
  • Built the agent image with the stdlib-docs submodule checked out at the Flink SQL Runner version (#2453, #2460)
  • Updated the automated PR review model and guide (#2322, #2450)