0.12.4

Features

• New function typesafe_classify in the new stdlib-typesafe module: classifies each row of an insert-only table with TypeSafe AI's Jev model. (#411)
  • Output: input columns, one typed <question>_answer column per question, _model, and _error with on_error => 'NULL'
  • Question kinds: noul, choice, score; a column type that does not match its kind fails at planning, not at runtime
  • Choice options: one typed probability field each
  • API key: typesafe.api-key job parameter, or TYPESAFE_API_KEY environment variable
  • Registration: CREATE TEMPORARY FUNCTION typesafe_classify AS 'com.datasqrl.flinkrunner.stdlib.typesafe.typesafe_classify'
  • Details: module README

Fixes

• Late records after a restore in kafka-safe and upsert-kafka-safe, with idle advancement on. (#441)
  • Before: a partition with no row within scan.source-watermark.idle-advance-timeout was marked idle, so the subtask's other partitions pushed the watermark past its records
  • Now: marked idle only once every partition of the subtask without a row is read to the end of its log
  • New permission: Describe on the consumer group, for fresh starts with 'scan.startup.mode' = 'group-offsets'
• Watermark alignment ignored with SOURCE_WATERMARK() in kafka-safe and upsert-kafka-safe. (#443)
  • Before: scan.watermark.alignment.* ignored; sources in one group read at their own pace
  • Now: partitions more than max-drift ahead of the group's slowest source are paused, as with planner watermarks
  • Options: scan.watermark.alignment.group, scan.watermark.alignment.max-drift, scan.watermark.alignment.update-interval (default 1 s)
• Zero idle timeout failing with SOURCE_WATERMARK() in kafka-safe and upsert-kafka-safe, with idle advancement off. (#443)
  • Before: 'scan.watermark.idle-timeout' = '0' failed with idleTimeout must be greater than zero
  • Now: disables idleness, as with planner watermarks

Dependencies

Dependency	From	To	PR
Maven Enforcer Plugin	3.6.3	3.7.0	#438
JaCoCo	0.8.15	0.8.16	#439

---

Full Changelog: 0.12.3...0.12.4