Skip to content

v2.5.0-alpha.5

Pre-release
Pre-release

Choose a tag to compare

@ngjaying ngjaying released this 30 Oct 07:28
· 16 commits to master since this release
fd6bf37

Full Changelog: v2.5.0-alpha.4...v2.5.0-alpha.5

⚠️ Breaking Changes

  • Versioned checkpoint storage format (#4195): Checkpoint state now uses a versioned v2 storage format with frozen byte snapshots. Existing legacy checkpoints can still be restored upon upgrading to v2.5, but checkpoints generated by v2.5 cannot be restored on v2.4 or earlier.
  • Window boundary calculation for incremental tumbling and hopping windows (#4195): Incremental tumbling and hopping windows now compute their reported boundaries from scheduled logical deadlines rather than actual timer execution timestamps. Under delayed timer scheduling, window_end() may differ slightly from earlier versions to avoid boundary drift.

New Features

  • Built-in JSON Lines (jsonl) format support across all sources and sinks, with schema validation, type filtering, and batch framing support by @NopeCharon in #4218
  • Expose Kafka record metadata via meta(), including topic, partition (accessed as meta(`partition`)), offset, key, and message headers (e.g. meta(headers)->traceId) by @NopeCharon in #4214

Security Hardening

  • Harden HTTPS download transport verification by @ngjaying in #4210
  • Add automated Gitleaks workflow for repository PR and source scanning, and sanitize static credential examples and tutorial tokens by @ngjaying in #4209 and #4208

Bug Fixes

  • Checkpoint: capture task state snapshots synchronously at barrier points to eliminate data races between live mutations and asynchronous persistence, isolate restored operator state graphs, and serialize lifecycle save/discard operations by @ngjaying in #4195
  • Service: key executor pools by both service name and interface name and purge pooled executors upon service update or deletion, ensuring updated service endpoints take effect on rule start/restart by @NopeCharon in #4217
  • Graph Rules: preserve stream columns in graph rules when referencing source streams by planning with wildcard column pruning, fixing empty tuple {} output on schemaless and typed streams by @NopeCharon in #4213
  • Schedule Rules: prevent scheduled rules outside their active time window from running briefly and emitting real-time sink data upon creation or update by @ngjaying in #4204
  • Schedule Rules: quiet repeated scheduled stop log messages from INFO to DEBUG when rule is already stopped by @ngjaying in #4216
  • REST Sink: drain unread HTTP response bodies before closing connections when debugResp=false, enabling HTTP/1.1 connection reuse and avoiding ephemeral port exhaustion by @ngjaying in #4212
  • Functions: use deep equality comparison in array functions (array_position, array_last_position, array_contains, array_contains_any) to prevent panics when comparing nested slices or non-scalar types by @NUAAwqy in #4207
  • Functions: reject negative pad lengths in lpad and rpad with an error instead of panicking on strings.Repeat by @NUAAwqy in #4206

Performance

  • Checkpoint state snapshot deferral: defer incremental aggregation function state snapshots to the checkpoint barrier rather than re-evaluating on every row, reducing per-row publish overhead and skipping snapshot generation entirely for QoS 0 rules by @ngjaying in #4194

Dependencies and Chores

  • Bump github.com/klauspost/compress from 1.18.4 to 1.18.7 by @dependabot[bot] in #4205
  • Test: pin timezone in internal/xsql/valuer_test.go comparison tests by @NopeCharon in #4220

New Contributors