OPC UA Subscription Mode Concept

[tenthe]

GitHub Discussion - OPC UA Subscription Mode Concept

Purpose

This note outlines the OPC UA subscription mode idea for discussion. It focuses on the problem and the expected user-facing behavior, while leaving out the implementation details.

Problem

The current OPC UA adapter has a conceptual mismatch in subscription mode.

OPC UA subscriptions are based on monitored items. A monitored item observes one node attribute and reports notifications when the value changes according to the configured sampling interval, filter, queue size, and subscription publishing interval.

StreamPipes adapters usually produce events with a fixed schema where one event contains multiple fields. For the OPC UA adapter, this means a user can select several OPC UA nodes and expects one StreamPipes event containing all selected node values.

This is straightforward in pull mode:

1. The adapter reads all selected nodes at a configured interval.
2. The adapter emits one event with all values.
3. The event can be interpreted as a periodic snapshot.

In subscription mode this is not naturally true:

1. The OPC UA server reports updates per monitored item.
2. One node may change while other selected nodes do not.
3. The adapter currently updates an internal latest-value map and emits the whole event again once all fields are available.

The resulting event looks like a complete multi-field measurement, but only one or a few fields may be fresh. The other fields may be cached values from earlier notifications. In addition, the event stream has no fixed interval because events are emitted whenever any monitored node changes.

This is problematic for downstream analytics:

• Events can falsely look synchronized although fields may have different freshness.
• Event frequency depends on value changes, not on a stable sampling grid.
• Fast-changing nodes cause repeated emission of slow-changing values.
• Aggregations and charts may unintentionally overweight values that are repeated due to unrelated node changes.
• Detection algorithms often become easier and more predictable with fixed-interval input.

Current User Configuration

The regular OPC UA adapter currently exposes these relevant options:

• OPC UA server connection by URL or host/port
• Security mode and security policy
• Authentication by anonymous login, username/password, or X.509
• Adapter type: pull mode or subscription mode
• Selected OPC UA variable nodes
• Naming strategy for StreamPipes event fields
• Pull interval, only in pull mode
• Incomplete event handling, only in pull mode

Subscription mode currently does not expose OPC UA subscription settings such as:

• StreamPipes output interval
• OPC UA sampling interval
• OPC UA publishing interval
• Monitored item queue size
• Discard policy
• Deadband or data-change filter
• Initial value handling
• Stale value handling
• Change-event vs snapshot-event semantics

The current implementation hardcodes subscription behavior:

• One OpcUaSubscription is created for the selected nodes.
• One OpcUaMonitoredItem is created per selected node.
• Subscription publishing interval is 1000 ms.
• Monitored item sampling interval is 1000 ms.
• Queue size is 10.
• discardOldest is true.
• Each data-change callback updates a shared event map.
• Once all expected fields are present, the adapter emits the full event on every monitored item update.

Proposed Direction

Introduce a subscription-backed periodic snapshot mode.

The adapter should still acquire data from OPC UA using subscriptions, but it should emit StreamPipes events at a fixed user-defined output interval.

Conceptually:

OPC UA server
  -> monitored item notifications when nodes change
  -> adapter latest-value cache
  -> StreamPipes event emitted every configured output interval

This provides a better compromise:

• OPC UA acquisition remains efficient and change-driven.
• StreamPipes output becomes regular and easier for analytics.
• Dashboards, charts, windows, and detection algorithms receive fixed-interval events.
• The event semantics can be described as "latest known values at output time".

Important: this mode does not produce a guaranteed synchronized read of all nodes at the output timestamp. It produces a periodic latest-value snapshot based on cached subscription values.

Recommended User-Facing Modes

There are two possible product choices.

Option A: Change Existing Subscription Mode

Change the current subscription mode so it always emits at a configured output interval.

Advantages

• Simpler user interface
• Avoids the current irregular full-event emission behavior
• Subscription mode becomes analytics-friendly by default

Disadvantages

• Changes existing behavior for users who rely on immediate event emission after a node change
• The name "subscription mode" still needs explanation because output is periodic, not event-per-notification

Option B: Add a New Subscription Snapshot Mode

Keep the current idea of subscription mode but split it into explicit modes:

• Pull snapshot: read all nodes periodically
• Subscription snapshot: subscribe to nodes, cache latest values, emit full events periodically
• Subscription changes: subscribe to nodes and emit change events when notifications arrive

Advantages

• Clear semantics
• Avoids silently changing existing behavior
• Supports both analytics use cases and change-log or alerting use cases

Disadvantages

• More configuration complexity
• Requires clearer UI text and documentation

Recommended approach: add Subscription snapshot as the primary new mode. The current subscription behavior can either be migrated to this mode or later replaced by a dedicated Subscription changes mode.

Recommended Configuration

Required for Subscription Snapshot

Output interval

Controls how often StreamPipes emits an event. This is the analytics-facing interval.

Example: emit one event every 1000 ms, independent of how often individual OPC UA nodes change.

Sampling interval

Controls how often the OPC UA server samples each monitored item. This should be configurable because different servers and PLCs have different limits.

Default proposal: 1000 ms.

Publishing interval

Controls how often the OPC UA subscription publishes queued notifications to the client.

Default proposal: same as sampling interval or 1000 ms.

Initial event behavior

Defines what happens before all selected nodes have produced at least one valid value.

Recommended options:

• Wait for complete cache: emit no events until all selected nodes have a first value
• Emit incomplete snapshots: emit events at the output interval and leave missing values unset or null

Default proposal: wait for complete cache.

Stale value handling

Defines what happens when a node has not received an update for a configured time.

Recommended options:

• Keep last value
• Set stale field to null
• Skip event if any required field is stale

Default proposal: keep last value, but provide optional stale metadata in a later iteration.

Useful Advanced Options

Queue size

Controls how many data-change notifications the server queues per monitored item.

For periodic snapshot output, a queue size of 1 may be sufficient if only the latest value matters. A larger queue is useful only if the adapter later processes all intermediate changes.

Default proposal for snapshot mode: 1.

Discard policy

Controls whether the server discards oldest or newest queued values when the queue is full.

For latest-value snapshots, discardOldest = true is usually appropriate because the newest value should win.

Default proposal: discardOldest = true.

Deadband/filter

Allows the user to reduce update traffic for numeric values by only reporting meaningful changes.

This is useful but should probably be an advanced option because it adds complexity and depends on server support.

Status handling

Defines how bad OPC UA status codes are represented.

Recommended options:

• Ignore bad updates and keep previous good value
• Store null for bad updates
• Mark the whole event incomplete

Default proposal: ignore bad updates and keep previous good value in snapshot mode, but log or debug status and consider metadata later.

Event Semantics

The event produced by subscription snapshot mode should be documented as:

A periodic latest-value snapshot of all selected OPC UA nodes, based on values received through OPC UA subscriptions.

This means:

• The event interval is fixed by the StreamPipes output interval.
• Values may have been updated at different times.
• Repeated values are expected and intentional.
• The mode is optimized for analytics and charts, not for preserving every individual OPC UA change.

This distinction is important:

• Pull snapshot means "read all selected nodes now".
• Subscription snapshot means "emit latest known values now".
• Subscription changes means "this node changed".