Skip to content

feat: Add broker communication mode#4255

Merged
dominikriemer merged 12 commits intoremove-supported-groundingfrom
add-broker-communication-mode
Mar 18, 2026
Merged

feat: Add broker communication mode#4255
dominikriemer merged 12 commits intoremove-supported-groundingfrom
add-broker-communication-mode

Conversation

@dominikriemer
Copy link
Member

Purpose

This PR introduces the initial broker-based communication mode for core-to-extension requests using NATS request/reply, as a foundation to remove the need for an open incoming HTTP port on extension services.

  • Added transport-mode configuration (http, nats, auto/dual) via environment support in commons.
  • Added shared broker transport model in streampipes-model
  • Added new module streampipes-nats-extensions for extension-side broker request handling.
  • Added core-side NATS request/reply client and transport-aware request manager wiring in streampipes-service-core.
  • Extended ExtensionServiceRequestTarget/ExtensionServiceRequestTargets to carry operation-aware routing (HTTP path + broker topic semantics).
  • Integrated broker receiver startup/shutdown into extension service bootstrap (streampipes-service-extensions).
  • Implemented NATS handlers for SERVICE_LOAD, ADAPTER_STATE_CHANGE (start/stop)
  • Added support in core manager to send requestAdapterStateChange via NATS (including auth token in envelope).
  • Kept HTTP flow intact for backward compatibility and migration safety.

Behavior

Core transport selection is now mode-based:
http: always HTTP
nats: always NATS
auto: prefer NATS for services advertising broker support, otherwise HTTP
Extension runtime can be configured to run HTTP-only, NATS-only, or dual mode.

Remarks

PR introduces (a) breaking change(s): no

PR introduces (a) deprecation(s): no

@github-actions github-actions bot added dependencies Pull requests that update a dependency file java Pull requests that update Java code ui Anything that affects the UI pipeline elements Relates to pipeline elements backend Everything that is related to the StreamPipes backend testing Relates to any kind of test (unit test, integration, or E2E test). go Pull requests that update Golang code labels Mar 13, 2026
@dominikriemer dominikriemer changed the base branch from dev to remove-supported-grounding March 13, 2026 16:40
@dominikriemer dominikriemer changed the title Add broker communication mode feat: Add broker communication mode Mar 13, 2026
@github-actions github-actions bot removed ui Anything that affects the UI testing Relates to any kind of test (unit test, integration, or E2E test). go Pull requests that update Golang code labels Mar 13, 2026
@github-actions github-actions bot added documentation Everything related to documentation testing Relates to any kind of test (unit test, integration, or E2E test). labels Mar 15, 2026
@dominikriemer dominikriemer marked this pull request as ready for review March 18, 2026 15:09
@dominikriemer dominikriemer merged commit f61a43f into remove-supported-grounding Mar 18, 2026
21 of 23 checks passed
@dominikriemer dominikriemer deleted the add-broker-communication-mode branch March 18, 2026 15:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend Everything that is related to the StreamPipes backend dependencies Pull requests that update a dependency file documentation Everything related to documentation java Pull requests that update Java code pipeline elements Relates to pipeline elements testing Relates to any kind of test (unit test, integration, or E2E test).

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant