0.12.3

Features

• kafka-safe sources now keep records without a key when the value contains every key field ('value.fields-include' = 'ALL'). Instead of dropping these records, the source reads the key fields from the value. To keep dropping them, set the new option scan.key-fields.read-from-value to false. upsert-kafka-safe always decodes the key. (#430)

Fixes

• Fixed idle source-watermark advancement in kafka-safe and upsert-kafka-safe. (#433)
  • When a partition's last record came from a backlog, its watermark no longer stays behind the wall clock and keeps event-time windows open.
  • A partition that is idle because the source is busy or rate-limited is no longer advanced past unread records, which were then dropped as late.
  • An idle partition now advances only after it has read to the end of its log. It advances to the time of that check, minus the out-of-orderness delay and the safety margin.
  • scan.source-watermark.idle-advance-broker-check-ttl now also limits how far an idle partition's watermark falls behind the wall clock. Keep it close to scan.source-watermark.idle-advance-timeout.
  • The safety margin must now cover clock skew between the brokers and the TaskManagers.
  • The log-end check needs Describe permission on the topics. Read access already includes it.
• Fixed the first row of a Kafka partition reaching downstream event-time operators after the watermark had already passed it. This happened with idle advancement enabled, typically after a job starts or restores. A partition's first row now immediately emits a watermark just below itself. This works around FLINK-40504 in Flink 2.3.0. (#434)
• Fixed value-format metadata columns reading as NULL in kafka-safe tables that set key.fields and 'value.fields-include' = 'ALL', for example debezium-json's value.source.timestamp. (#436)

Dependencies

• Updated Jackson BOM from 2.22.3 to 2.22.4. (#428)
• Updated Maven Surefire Plugin from 3.6.0 to 3.6.1. (#429)

---

Full Changelog: 0.12.2...0.12.3