v1.8.2

Starlake Starflow 1.8.2

This is the first release under the product's new name: Starflow. Starlake is the umbrella brand; the product you install and run is Starflow. The CLI version banner, help text, installer messages, and documentation now say Starflow, and the repository has moved to github.com/starlake-ai/starflow (old URLs redirect). Nothing technical changes: the binary is still starlake, packages and Maven coordinates stay ai.starlake, and SL_*/STARLAKE_* environment variables, config keys, and YAML formats are untouched. No action is needed on existing projects.

Bug fix

• OVERWRITE transforms insert by name, never by position (#1722): overwriting an existing table used a positional INSERT on most engines, so a SELECT whose column order diverged from the table's could silently write every value into the wrong column, or fail with a confusing count/cast error. All engines now pass the explicit column list resolved from the SELECT: values land in the columns of the same name regardless of SELECT order, and a column the table does not have fails loudly with a clear message. If you use OVERWRITE transforms, upgrading is strongly recommended.

Improvements

• Actionable schema-mismatch errors: when a transform fails because the SELECT returns columns the target table lacks, the error names the missing columns and prints the exact commands that fix it (starlake transform --name <task> --sync-apply, then re-run).
• Automatic SQL-to-YAML sync (opt-in): with SL_SYNC_SQL_WITH_YAML=true, a transform aligns the task's YAML attributes with its SQL before running, so adding a column anywhere in the SELECT evolves the target table with no manual step. Off by default because it rewrites .sl.yml files in your project.

What's Changed

• rename: Starflow product naming in code-owned prose by @hayssams in #1715
• rename: Starflow product naming in docs and samples by @hayssams in #1716
• chore: fix stale references surfaced by the naming audit by @hayssams in #1717
• rename: Starflow naming in installer and launchers (DO NOT MERGE until announcement) by @hayssams in #1718
• chore: canonicalize repo slug in release tooling by @hayssams in #1719
• fix: backtick the tables/.tmdl placeholder in semantic-export help by @hayssams in #1720
• fix: prefix guide links in CLI help with /starflow by @hayssams in #1721
• backport(1.8): OVERWRITE inserts by name, sync-apply hint, syncSqlWithYaml by @hayssams in #1724

Full Changelog: v1.8.1...v1.8.2

v1.7.2

Starlake 1.7.2 (maintenance release)

Bug fix

• OVERWRITE transforms insert by name, never by position (#1722): overwriting an existing table used a positional INSERT on most engines, so a SELECT whose column order diverged from the table's could silently write every value into the wrong column, or fail with a confusing count/cast error. All engines now pass the explicit column list resolved from the SELECT: values land in the columns of the same name regardless of SELECT order, and a column the table does not have fails loudly with a clear message. If you use OVERWRITE transforms on the 1.7 line, upgrading is strongly recommended.

Improvements (backported from 1.8.2)

• Actionable schema-mismatch errors: when a transform fails because the SELECT returns columns the target table lacks, the error names the missing columns and prints the exact commands that fix it (starlake transform --name <task> --sync-apply, then re-run).
• Automatic SQL-to-YAML sync (opt-in): with SL_SYNC_SQL_WITH_YAML=true, a transform aligns the task's YAML attributes with its SQL before running. Off by default because it rewrites .sl.yml files in your project.

What's Changed

• backport(1.7): OVERWRITE inserts by name, sync-apply hint, syncSqlWithYaml by @hayssams in #1725

Full Changelog: v1.7.1...v1.7.2

v1.8.1

Full Changelog: v1.8.0...v1.8.1

v1.8.0

What's Changed

• Setup: install the API for real - move the right zip, overwrite on re-install, verify the swap by @fupelaqu in #1696

Full Changelog: v1.7.1...v1.8.0

v1.7.1

What's Changed

• Windows setup: real java version check (JAVA_HOME-aware) + embedded JDK fallback in the install dir by @fupelaqu in #1692
• Unix setup: parity with the Windows java handling — version-dependent floor + embedded JDK (#1692) by @fupelaqu in #1693
• setup.ps1: java version probe dies under PowerShell 5.1 when the caller runs with ErrorActionPreference=Stop by @fupelaqu in #1694
• Windows: winutils that can actually start on bare images (hadoop-3.3.6 + vcruntime safety-copy) by @fupelaqu in #1695

Full Changelog: v1.7.0...v1.7.1

v1.7.0

What's Changed

• Update mariadb-java-client to 3.5.10 by @github-actions[bot] in #1681
• Update jsqlparser to 5.3.242 by @github-actions[bot] in #1680
• Update jsqlformatter to 5.4.1 by @github-actions[bot] in #1679
• Update duckdb_jdbc to 1.5.5.0 by @github-actions[bot] in #1675
• Update starlake-orchestration to 0.5.6.1 by @github-actions[bot] in #1673
• Update starlake-dagster to 0.5.9 by @github-actions[bot] in #1672
• Update starlake-airflow to 0.6.13.1 by @github-actions[bot] in #1671
• Add semantic-export command: export semantic models to Apache Ossie by @hayssams in #1682
• Update scalacheck-1-17 to 3.2.19.0 by @github-actions[bot] in #1691
• Update duckdb_jdbc to 1.5.5.1 by @github-actions[bot] in #1690
• Update starlake-airflow to 0.6.14 by @github-actions[bot] in #1685
• Update jsqltranspiler to 1.10 by @github-actions[bot] in #1641

Full Changelog: v1.6.0...v1.7.0

v1.6.0

What's Changed

• Update sbt-assembly to 0.15.0 by @github-actions[bot] in #1
• Update sbt-buildinfo to 0.10.0 by @github-actions[bot] in #2
• Update sbt-release to 1.0.15 by @github-actions[bot] in #3
• Update sbt-pgp to 2.0.2 by @github-actions[bot] in #6
• Update sbt-updates to 0.5.3 by @github-actions[bot] in #7
• Update sbt-plugin to 2.6.25 by @github-actions[bot] in #8
• Update sbt-git to 1.0.1 by @github-actions[bot] in #9
• Update sbt-native-packager to 1.3.25 by @github-actions[bot] in #10
• Update sbt-site to 1.3.3 by @github-actions[bot] in #11
• Update kafka-clients to 2.8.1 by @github-actions[bot] in #12
• Update scala-library, scala-reflect to 2.12.15 by @github-actions[bot] in #14
• Update sbt-scalafmt to 2.4.3 by @github-actions[bot] in #15
• Update scalatest to 3.2.10 by @github-actions[bot] in #17
• Update sbt-scoverage to 1.9.0 by @github-actions[bot] in #18
• Update sbt-sonatype to 3.9.10 by @github-actions[bot] in #19
• Update scalafmt-core to 3.0.4 by @github-actions[bot] in #16
• Update sbt-assembly to 1.1.0 by @github-actions[bot] in #21
• Update sbt-release to 1.1.0 by @github-actions[bot] in #22
• Update sbt-pgp to 2.1.1 by @github-actions[bot] in #23
• Update sbt-updates to 0.6.0 by @github-actions[bot] in #24
• Update sbt-native-packager to 1.8.1 by @github-actions[bot] in #26
• Update scala-library, scala-reflect to 2.13.6 by @github-actions[bot] in #28
• Add support for timestamp parsing in json & xml by @hayssams in #20
• tweak the logs by @seyguai in #29
• BugFix/ missing leading zeros when applying hash functions by @amine-hmila in #30
• bugfix - input lines by @mhdkassir in #31
• Update spark-xml to 0.13.0 by @github-actions[bot] in #33
• Update testcontainers-scala-elasticsearch, ... to 0.39.8 by @github-actions[bot] in #34
• Update spark-bigquery-with-dependencies to 0.22.2 by @github-actions[bot] in #35
• Update elasticsearch-spark-30 to 7.14.2 by @github-actions[bot] in #38
• Update scalafmt-core to 3.0.6 by @github-actions[bot] in #39
• Update sbt-pgp to 2.1.2 by @github-actions[bot] in #36
• Allow Audit sink name configuration from environment variable by @seyguai in #40
• Field relation/addition in BQ by @hayssams in #32
• Bugfix/schema update non partition table by @hayssams in #41
• Update sbt-git to 1.0.2 by @github-actions[bot] in #43
• Update elasticsearch-spark-30 to 7.15.0 by @github-actions[bot] in #44
• Bugfix/steward by @seyguai in #45
• Update sbt-scoverage to 1.9.1 by @github-actions[bot] in #49
• Update bigquery-connector to hadoop3-1.0.1 by @github-actions[bot] in #47
• Update google-cloud-bigquery to 1.137.2 by @github-actions[bot] in #46
• Feature/schema on write by @hayssams in #51
• Auto create audit table by @hayssams in #53
• Docker Apple m1 by @hayssams in #54
• feat(yml2ddl): Add Apply and Save SQL Script by @faresismail96 in #56
• Update samples by @hayssams in #57
• Include env vars with CLI options by @hayssams in #58
• Improve XLS handling: Add support for presql / postsql, tags, primary and foreign keys by @hayssams in #59
• codacy by @hayssams in #60
• Feature/query pushdown by @hayssams in #61
• Rename packages from com.ebiznext to ai.starlake by @hayssams in #62
• Update sbt-scalafmt to 2.4.4 by @github-actions[bot] in #72
• Update scalafmt-core to 3.0.8 by @github-actions[bot] in #73
• Update elasticsearch-spark-30 to 7.15.2 by @github-actions[bot] in #71
• Update gcs-connector to hadoop3-2.2.4 by @github-actions[bot] in #68
• Update bigquery-connector to hadoop3-1.2.0 by @github-actions[bot] in #67
• Update testcontainers-scala-elasticsearch, ... to 0.39.12 by @github-actions[bot] in #65
• Update google-cloud-bigquery to 2.4.1 by @github-actions[bot] in #66
• Update akka-http to 10.1.15 by @github-actions[bot] in #69
• Update jna to 5.10.0 by @github-actions[bot] in #70
• Get rid of RDD by @hayssams in #64
• Move directory / extensions /ack to metadata section by @hayssams in #74
• Update sbt-scalafmt to 2.4.5 by @github-actions[bot] in #79
• Update scalafmt-core to 3.2.1 by @github-actions[bot] in #80
• Update h2 to 2.0.202 by @github-actions[bot] in #78
• Update google-cloud-bigquery to 2.5.1 by @github-actions[bot] in #76
• Support Row Level / Column Level Policy by @hayssams in #75
• Support SQL in views folder by @hayssams in #81
• Databricks Support for Table ACL by @hayssams in #82
• Update spark-bigquery-with-dependencies to 0.23.1 by @github-actions[bot] in #83
• Update elasticsearch-spark-30 to 7.16.0 by @github-actions[bot] in #84
• Update sbt to 1.5.6 by @github-actions[bot] in #85
• Feature/blog by @hayssams in #86
• Update elasticsearch-spark-30 to 7.16.1 by @github-actions[bot] in #89
• Update jsqlparser to 4.3 by @github-actions[bot] in #88
• Update sbt to 1.5.7 by @github-actions[bot] in #90
• Feature/databricks support by @hayssams in #87
• Update h2 to 2.0.204 by @github-actions[bot] in #92
• Update sbt-updates to 0.6.1 by @github-actions[bot] in #93
• Update elasticsearch-spark-30 to 7.16.2 by @github-actions[bot] in #94
• Update sbt to 1.5.8 by @github-actions[bot] in #95
• Update sbt-scalafmt to 2.4.6 by @github-actions[bot] in #96
• Update scalafmt-core to 3.2.2 by @github-actions[bot] in #97
• WIP: Databricks Delta format support by @hayssams in #91
• Update sbt to 1.6.1 by @github-actions[bot] in #98
• Update scalafmt-core to 3.3.1 by @github-actions[bot] in #99
• Update jackson by @hayssams in #100
• Feature/custom application.conf by @hayssams in #101
• drop scala 2.11 support by @hayssams in #102
• Update spark-xml to 0.14.0 by @github-actions[bot] in #103
• Update google-cloud-bigquery to 2.6.1 by @github-actions[bot] in #105
• Update google-cloud-datacatalog to 1.6.2 by @github-actions[bot] in #106
• Update h2 to 2.0.206 by @github-actions[bot] in #107
• Update google-cloud-bigquery to 2.6.2 by @github-actions[bot] in #111
• Update elasticsearch-...

v1.5.16

Starlake 1.5.16. Distribution assets migrated from Maven Central to GitHub Releases.