rekuiper v0.509-beta

Release Notes - rekuiper v0.509-beta

rekuiper v0.509-beta adds built-in video and ZeroMQ connectors, time-series sinks, upstream eKuiper compatibility fixes, security hardening, and revised system documentation.

Key highlights:

• Video Source Connector: Ingests live RTSP and HTTP video streams through external FFmpeg process pipes. Supports interval sampling and validates stream URLs to prevent local file access.

• ZeroMQ Sink and Source: Provides a built-in ZeroMQ connector (zmq). Supports PUB and SUB sockets with bind and connect modes.

• Image Sink Retention Pruning: Saves image frames (JPEG and PNG) to disk. Prunes files by age (maxAge) and file count (maxCount). Validates file names to prevent path traversal.

• Apache IoTDB REST Sink: Connects to the Apache IoTDB REST v1 service (/rest/v1/insertTablet). Converts numeric values to DOUBLE to prevent type conflicts. Parses timestamps strictly and sets invalid cells to null.

• TDengine 3.x REST Sink: Connects to the TDengine REST service. Supports super tables and batches multiple rows in single VALUES clauses.

• InfluxDB Sinks: Supports InfluxDB v1 and InfluxDB v2 Line Protocol formatting, dynamic measurement names, and authentication tokens.

• Random and Simulator Sources: Adds TYPE="random" compatibility alias to the simulator engine. Parses millisecond duration strings (such as 50ms and 1s) and configuration keys.

• Upstream eKuiper Compatibility:

  • Adds DEFAULT column values in stream DDL and fills missing fields automatically (#PR-4025 parity).
  • Validates BUFFER_FULL_POLICY settings across joined streams only when configured (#PR-4133 parity).
  • Adds the disable: true action option to skip sink execution without stopping rules (#PR-4100 parity).
  • Validates SQL table and field identifiers against SQL injection (#PR-4080 parity).
  • Adds the offsetReset setting for Kafka sources (#PR-4190 parity).
  • Binds network listeners to 127.0.0.1 by default for host security (#PR-4089 parity).

• Security and Fault Tolerance:

  • Treats HTTP status codes 408 and 429 as retryable errors and other 4xx codes as permanent errors.
  • Skips invalid batch rows with a warning log and continues processing valid rows.
  • Removes timer sleep delay when ruletest replay uses a zero interval.
  • Adds ARG WITH_FFMPEG="true" in the Dockerfile to let users build slim images without FFmpeg.

• Revised Documentation: Rewrites README.md and the documentation portal to describe data pipeline architecture, common design patterns, and connector compatibility.

• Docker image: ankurkrp/rekuiper:0.509-beta (and latest)

• Helm chart: deploy/chart/ekuiper (AppVersion: 0.509-beta)

---

What's Changed

• feat: video and ZeroMQ connectors, eKuiper parity, CI hardening, and v0.509.0-beta by @ankur-paan in #36

Full Changelog: v0.508-beta...v0.509-beta