Repository navigation
Breaking Changes
- Cost ties always picks Flink from Flink and PostgreSQL, with
cost-modelset toDEFAULT(#2557)- Before: the order of an internal map picked the engine, so the plan could change with
enabled-enginesor a leftoverbuild/folder - Now: Flink on every compile
- Affected: deployed plans with such a table in PostgreSQL; a recompile moves it to Flink, and the new Flink job cannot restore the state of the old one
- Before: the order of an internal map picked the engine, so the plan could change with
Important
Before deploying a recompiled plan of a running pipeline:
- Compare the
Stage:line of each table inbuild/pipeline_explain.txtwith the deployed build - For a table that changed from
postgrestoflink, either keep it in PostgreSQL with/*+ engine(postgres) */, or deploy the new plan as a new Flink job without the old job's state
Features
- Database tables and indexes in
pipeline_explainfor every table that a database stores (#2554)- Output: a
--- physical ---part with oneTable <name> (<engine>):section per database table, listing its primary key and selected indexes - Index lines: type and columns as in the
indexhint, such asindex Orders_2_btree_c1c2 (btree): customerid, time;__pk_hashmarked(generated hash key) - JSON: a
database_tablesarray on each stored table inpipeline_explain.json - Changed format: the bare
---before each table's definition is now--- definition --- - Failed compiles:
pipeline_explain.*,pipeline_source.sqrlandpipeline_mutation_database.jsonno longer written when a compile fails between DAG planning and the backward compatibility check
- Output: a
- New function
typesafe_classifyinstdlib.typesafe, from Flink SQL Runner0.12.4: classifies each row of an insert-only table with TypeSafe AI's Jev model (DataSQRL/flink-sql-runner#411)- Setup:
IMPORT stdlib.typesafe.*; - Output: input columns, one typed
<question>_answercolumn per question,_model, and_errorwithon_error => 'NULL' - Question kinds:
noul,choice,score - API key:
pipeline.global-job-parameters.typesafe.api-keyinengines.flink.config, or theTYPESAFE_API_KEYenvironment variable - Details: module README
- Setup:
- Agent skills on indexes and engine selection in
implement-sqrlandconfigure-sqrl(#2554, #2557)- Index checks: the
api-query-indexesguide reads a table's indexes frompipeline_explain.txtinstead ofdeploy/plan/postgres.json - Engine selection: the
cost-modelvalues, the Flink pick on a tie, and theStage:line that shows each table's engine
- Index checks: the
Fixes
- Misplaced compile errors in
DISTINCT,IMPORTandEXPORTstatements, hints, and tables defined in other files (#2543)- Before: a
DISTINCTerror pointed into the query that the statement is rewritten into; anIMPORTerror pointed at the start of the statement - Now: errors point at the column, package path or hint name they are about
- Other files: Flink errors in a table from a
.table.sqlfile or an imported script point into that file - Empty definitions: fail with
Missing query or expression after ':=', at the table name - Output: a
%in a message is kept; no Java stack trace printed with a Flink error
- Before: a
Maintenance
- Flink SQL Runner
0.12.4, up from0.12.3(see its release notes) (7ba7d3614)- New function:
typesafe_classifyin the newstdlib-typesafemodule - Late records after a restore: fixed in
kafka-safeandupsert-kafka-safewith idle advancement on; needs Describe on the consumer group for fresh starts with'scan.startup.mode' = 'group-offsets' - Watermark alignment:
scan.watermark.alignment.*now applies withSOURCE_WATERMARK()inkafka-safeandupsert-kafka-safe - Zero idle timeout:
'scan.watermark.idle-timeout' = '0'withSOURCE_WATERMARK()disables idleness instead of failing
- New function:
| Dependency | From | To | PR |
|---|---|---|---|
| Maven Enforcer Plugin | 3.6.3 |
3.7.0 |
#2563 |
| JaCoCo | 0.8.15 |
0.8.16 |
#2561 |
| Node.js (MCP Inspector test image) | 26.12.1-alpine |
26.13.0-alpine |
#2564 |
Full Changelog: 0.12.3...0.12.4