Skip to content

v2.0.0

Latest

Choose a tag to compare

@gfinocchiaro gfinocchiaro released this 02 Jul 14:08

Changelog

[2.0.0] (2026-07-02)

Breaking Changes

  • Connector-managed snapshot replaces explicit COMMAND-mode parameters: The boolean parameters fields.evaluate.as.command.enable (introduced in 1.2.4) and fields.auto.command.mode.enable (introduced in 1.3.2) have been removed and superseded by the unified item.snapshot.enabled.mode parameter. COMMAND mode is now activated via item.snapshot.enabled.mode = COMMAND, which also enables connector-managed snapshot for the affected items. Manual COMMAND-mode subscriptions without connector-managed snapshot remain supported by mapping both field.key and field.command explicitly while leaving item.snapshot.enabled.mode at its default NONE. (#88)

  • Removed special-case snapshot signaling records: The convention of injecting Kafka records with key = snapshot and command ∈ {CS, EOS} to drive snapshot lifecycle from the topic side — only meaningful in combination with fields.evaluate.as.command.enable = true (introduced in 1.2.4) — has been removed. Snapshot lifecycle is now managed entirely by the connector based on item.snapshot.enabled.mode. (#88)

  • record.consume.from and record.extraction.error.strategy semantics under snapshot: When item.snapshot.enabled.mode is set to any value other than NONE, the connector manages partition positions explicitly (newly assigned partitions seek to beginning; re-assigned partitions resume from committed offset), making record.consume.from ineffective; and record.extraction.error.strategy is forced to IGNORE_AND_CONTINUE. (#88)

  • Minimum Lightstreamer Broker version bumped to 7.4.8: The connector now requires Lightstreamer Broker (also referred to as Lightstreamer Server) version 7.4.8 or newer, as the new connector-managed snapshot feature relies on APIs introduced in that release. (#88)

New Features

  • Connector-managed snapshot: Introduced first-class support for connector-managed snapshot via the new item.snapshot.enabled.mode parameter, with one mode per Lightstreamer subscription Mode:

    • MERGE — pins subscription Mode to MERGE; per-item store keeps the latest value; snapshot is a single event per item. See MERGE snapshot.
    • DISTINCT — pins subscription Mode to DISTINCT; per-item store is a FIFO bounded by item.snapshot.distinct.length; snapshot replays the most recent events in publish order. See DISTINCT snapshot.
    • COMMAND — pins subscription Mode to COMMAND; the connector synthesizes the command field (ADD / UPDATE / DELETE) from per-(item, key) state and tombstones; snapshot is the full row set currently materialized for the item. See COMMAND snapshot.

    When activated, the internal Kafka consumer starts eagerly at bind time, replays the topic from the beginning to pre-seed the Lightstreamer Server per-item store, then transitions to realtime tailing. Late subscribers receive the materialized snapshot followed by realtime updates. See Snapshot management and Connector-managed snapshot. (#88)

  • DISTINCT snapshot length parameter: Added the new item.snapshot.distinct.length parameter (positive integer, default 10) to cap the per-item FIFO when item.snapshot.enabled.mode = DISTINCT. (#88)

  • Per-item idle expiration: Added the new item.snapshot.max.idle.seconds parameter (non-negative integer; 0 disables) to discard the snapshot of an item after a configurable idle period, so that the next incoming record starts a fresh one. (#88)

Improvements

  • Poison-pill tolerance: A single malformed record no longer aborts consumption. Records that fail deserialization are individually skipped and logged at WARN level (with topic, partition, and offset), and consumption continues with the remaining good records. As a safety net, when all records in a non-empty batch fail — a pattern indicating a systemic misconfiguration (e.g., wrong deserializer) rather than isolated corrupt records — the connector fails fast instead. (#88)

  • Upgraded Gradle to version 9.6.1. (#88)

Third-Party Library Updates

  • Upgraded the ls-adapter-inprocess SDK dependency to version 8.1.0. (#88)

  • Upgraded the base Lightstreamer Docker image to version 7.4.8. (#88)

  • Upgraded the slf4j-reload4j dependency to version 2.0.18. (#88)

  • Upgraded the Spotless plugin for Gradle dependency to version 8.8.0. (#88)

Examples and Documentation

  • Snapshot management documentation: Added the new Snapshot management chapter covering the default NONE behavior, the connector-managed snapshot lifecycle, per-Mode snapshot shape, extraction-layout rules for snapshot correctness, idle expiration, and caveats; plus a new Item snapshot settings parameter reference. (#88)

  • COMMAND mode field mapping documentation: Added the new COMMAND mode field mapping subsection describing both routes — connector-managed snapshot (item.snapshot.enabled.mode = COMMAND, with synthesized command field) and manual mapping (field.command mapped explicitly, default NONE snapshot mode) — and the trade-offs between them. (#88)

  • Updated the factory adapters.xml with the new item.snapshot.* parameters and aligned field.key / field.command documentation. (#88)

  • Airport Demo overhaul: Updated the Airport Demo to showcase COMMAND-mode connector-managed snapshot, alongside a broader rewrite of the demo producer, its JSON payload schema, and the corresponding adapters.xml field mappings; the demo README has been re-aligned accordingly. (#88)

  • Sample producer Maven coordinates: The Maven group published to GitHub Packages has been unified to com.lightstreamer.kafka.examples for both sample producer projects — quickstart-producer (previously com.lightstreamer.kafka) and the airport-demo producer example-kafka-connector-demo-publisher (previously com.lightstreamer.examples). Consumers pinning the old coordinates must update them. (#88)

  • Quickstart Compose modernization: Rewrote every quickstart docker-compose.yml to the long-form depends_on syntax with explicit condition: clauses; added broker healthchecks where the broker image supports them (Redpanda-based quickstarts); upgraded the bundled Redpanda Console image from v2.4.5 to v3.8.0 and migrated its embedded config to the v3 schema. (#88)

  • Applied style and formatting consistency pass across all updated README.md files (root, docker/, examples/, and every examples/**/README.md) and across the factory adapters.xml: unified heading hierarchy and Table of Contents indentation, parameter naming and link conventions, code-block fencing, and inline-code styling for parameters, values, and identifiers. (#88)