From c1627cd54a8823978e2211b570603a3944e54ef3 Mon Sep 17 00:00:00 2001 From: Michael Date: Wed, 6 May 2026 12:09:10 +0200 Subject: [PATCH 1/3] Document redispatch attributes, state machine, resilience, and selection rules Fills the gaps between the redispatch implementation and ems/README.md: - Adds a Mermaid sequence diagram mirroring the UFTP one - Adds a 'Selection rules' note (MANDATORY preferred over VOLUNTARY) - Adds an 'Asset attributes' table covering all 19 redispatch attributes - Enumerates redispatchBidStatus values (NONE, PENDING_CONFIRMATION, CONFIRMED) - Adds a 'Resilience and polling' subsection covering the polling floor, fetch-failure semantics, mandatory API key, and handler-restart triggers - Clarifies that announcement history records every polled announcement (audit trail) and notes the 10k LRU bound - Cross-links the prerequisites back to the UFTP Getting Started section --- ems/README.md | 69 ++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 68 insertions(+), 1 deletion(-) diff --git a/ems/README.md b/ems/README.md index f052169..5e59b85 100644 --- a/ems/README.md +++ b/ems/README.md @@ -125,6 +125,8 @@ ISPs (Imbalance Settlement Periods) are 15-minute intervals. `FlexRequestISPType In addition to the UFTP day-ahead flex trading described above, GOPACS provides a **Redispatch** mechanism for intraday congestion management. When a congestion situation is expected today, grid operators publish announcements requesting flexibility from market participants. +> Prerequisites are the same as UFTP — see [Getting Started](#getting-started) for the GOPACS account and contracted EAN. Redispatch additionally requires an API key (see [Configuration](#configuration-1) below). + The Redispatch flow is different from the UFTP flow: 1. **Announcements** — GOPACS publishes congestion announcements via a REST API @@ -134,6 +136,30 @@ The Redispatch flow is different from the UFTP flow: 5. **Activation** — The trading platform notifies the CSP when an order is filled 6. **Delivery** — The CSP adjusts power as agreed +```mermaid +sequenceDiagram + participant Op as Operator + participant OR as OpenRemote (CSP) + participant API as GOPACS Redispatch API + participant TP as Trading Platform (future) + + loop every ≥ 5 min + OR->>API: GET /machineannouncements (CONGESTIONMANAGEMENT, OPEN) + API-->>OR: announcements + Note right of OR: Record every polled announcement in history + OR->>API: GET .../eansolvingeffectivity per announcement + API-->>OR: EAN categories per announcement + Note right of OR: Keep announcements where the contracted EAN
is listed; prefer MANDATORY over VOLUNTARY + OR->>OR: Update redispatch* asset attributes,
set redispatchBidStatus = PENDING_CONFIRMATION + end + + Op->>OR: Set redispatchBidPrice, toggle redispatchConfirmBid = true + OR->>OR: Log bid, set redispatchBidStatus = CONFIRMED + OR-->>TP: Place order (not yet implemented) +``` + +**Selection rules:** per poll, the handler keeps only `CONGESTIONMANAGEMENT` / `ANNOUNCEMENT_OPEN` announcements where the contracted EAN appears in some EAN-effectivity category, then prefers `MANDATORY` over `VOLUNTARY` compliance type when more than one matches. + ### Configuration | Variable | Required | Description | @@ -148,6 +174,38 @@ On the **EMS GOPACS Asset**, configure: - **`redispatchEnabled`** — Set to `true` to start polling for announcements +### Asset attributes + +Every redispatch attribute on `EmsGOPACSAsset`. All status, bid-suggestion and history attributes are written by the handler and surfaced read-only in the UI; only `redispatchEnabled`, `redispatchBidPrice` and `redispatchConfirmBid` are operator-editable. + +| Group | Attribute | Type | RO | Purpose | +|---|---|---|---|---| +| Configuration | `redispatchEnabled` | boolean | | Master switch — toggle off/on to (re)start the polling handler. | +| Status | `redispatchAnnouncementId` | text | ✓ | ID of the currently selected announcement, if any. | +| Status | `redispatchComplianceType` | text | ✓ | `MANDATORY` or `VOLUNTARY`. | +| Status | `redispatchAnnouncementMessage` | text (multiline) | ✓ | Free-text description from the DSO. | +| Status | `redispatchStartTime` | timestamp | ✓ | Start of the problem period. | +| Status | `redispatchEndTime` | timestamp | ✓ | End of the problem period. | +| Status | `redispatchBidValidityEnd` | timestamp | ✓ | Latest moment a bid can still be submitted for this announcement. | +| Status | `redispatchRequestedPower` | number (kW) | ✓ | Remaining problem profile, written as predicted data points (15-min ISP grid, 7-day retention). | +| Status | `redispatchEanEffectivity` | text | ✓ | Effectivity category in which the contracted EAN was matched. | +| Status | `redispatchRequestAreaBuy` | text | ✓ | DSO-supplied area description for buy orders. | +| Status | `redispatchRequestAreaSell` | text | ✓ | DSO-supplied area description for sell orders. | +| Status | `redispatchLastPoll` | timestamp | ✓ | Timestamp of the last successful poll cycle. | +| Bid | `redispatchSuggestedPower` | number (kW) | ✓ | _Not yet populated — pending bid pricing strategy follow-up._ | +| Bid | `redispatchSuggestedVolume` | number (kWh) | ✓ | _Not yet populated — pending bid pricing strategy follow-up._ | +| Bid | `redispatchBidPrice` | number (EUR/MWh) | | Operator-supplied bid price. | +| Workflow | `redispatchConfirmBid` | boolean | | Operator toggles to `true` to confirm the active bid; handler resets it after processing. | +| Workflow | `redispatchBidStatus` | text | ✓ | State machine — see below. | +| History | `redispatchAnnouncementHistory` | JSON object | ✓ | One data point per polled announcement (90-day retention). | +| History | `redispatchBidHistory` | JSON object | ✓ | One data point per confirmed bid (90-day retention). | + +`redispatchBidStatus` values: + +- `NONE` — no active announcement +- `PENDING_CONFIRMATION` — operator action required +- `CONFIRMED` — bid logged (and, in future, sent to the trading platform) + ### Operator Workflow (Pilot Phase) 1. When a relevant congestion announcement is detected, the asset attributes are updated with the announcement details @@ -156,6 +214,13 @@ On the **EMS GOPACS Asset**, configure: 4. The operator sets `redispatchBidPrice` (EUR/MWh) and toggles `redispatchConfirmBid` to `true` 5. The bid is confirmed and logged (trading platform integration is pending) +### Resilience and polling + +- The polling interval is clamped to a minimum of 5 minutes because GOPACS recommends spacing requests at least that far apart. +- HTTP errors and exceptions on the announcements endpoint **skip the poll and preserve current attributes**, so transient API hiccups do not flap the bid status. Only a confirmed-empty response (HTTP 200 with no announcements) clears the active announcement and resets `redispatchBidStatus` to `NONE`. +- The handler **refuses to start** (logs `SEVERE`) when `GOPACS_REDISPATCH_API_KEY` is unset — without it there is no way to resolve EAN effectivity per announcement. +- Toggling `redispatchEnabled` off then on restarts the handler; the same applies when `contractedEAN` is changed. Useful when you need to force a clean state. + ### Components ``` @@ -170,7 +235,9 @@ gopacs/ ### History -Announcement and bid history are stored as time-series data points on `redispatchAnnouncementHistory` and `redispatchBidHistory` attributes, retained for 90 days. These are viewable in the OpenRemote history panel. +Announcement and bid history are stored as time-series data points on `redispatchAnnouncementHistory` and `redispatchBidHistory`, retained for 90 days and viewable in the OpenRemote history panel. + +`redispatchAnnouncementHistory` records **every** polled announcement, including ones that the EAN-effectivity check rejected, so the audit trail captures everything GOPACS returned during the handler's lifetime — not just the announcements that became active. To keep memory bounded for long-running handlers, the running set of already-recorded announcement IDs is capped at 10 000 entries (LRU-evicted). ### Future From 95e58b7762a9b142cfa9b25815def48efbe7aac1 Mon Sep 17 00:00:00 2001 From: Michael Date: Wed, 6 May 2026 12:29:15 +0200 Subject: [PATCH 2/3] Address review feedback on redispatch documentation - Clarify that redispatchAnnouncementHistory records announcements twice for selections (first-sight without effectivity, then a richer entry when selected) so the audit trail does not look duplicated to operators. - Note that the >=5 min poll floor clamps the configured interval, not a runtime backoff. - Spell out that persistent non-200 responses (e.g. 401 from a bad API key) keep stale announcements visible indefinitely; point to the log strings to grep for. - Add an example category for redispatchEanEffectivity and tighten the redispatchLastPoll description to "completed poll where the API responded". - Note that the diagram's selection-side actions only fire on a new selection, not every iteration. --- ems/README.md | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/ems/README.md b/ems/README.md index 5e59b85..e810142 100644 --- a/ems/README.md +++ b/ems/README.md @@ -143,14 +143,14 @@ sequenceDiagram participant API as GOPACS Redispatch API participant TP as Trading Platform (future) - loop every ≥ 5 min + loop every poll interval (≥ 5 min) OR->>API: GET /machineannouncements (CONGESTIONMANAGEMENT, OPEN) API-->>OR: announcements - Note right of OR: Record every polled announcement in history + Note right of OR: Record every newly-seen announcement in history OR->>API: GET .../eansolvingeffectivity per announcement API-->>OR: EAN categories per announcement Note right of OR: Keep announcements where the contracted EAN
is listed; prefer MANDATORY over VOLUNTARY - OR->>OR: Update redispatch* asset attributes,
set redispatchBidStatus = PENDING_CONFIRMATION + OR->>OR: On a new selection: update redispatch* attributes,
record a second history entry with effectivity,
set redispatchBidStatus = PENDING_CONFIRMATION end Op->>OR: Set redispatchBidPrice, toggle redispatchConfirmBid = true @@ -188,16 +188,16 @@ Every redispatch attribute on `EmsGOPACSAsset`. All status, bid-suggestion and h | Status | `redispatchEndTime` | timestamp | ✓ | End of the problem period. | | Status | `redispatchBidValidityEnd` | timestamp | ✓ | Latest moment a bid can still be submitted for this announcement. | | Status | `redispatchRequestedPower` | number (kW) | ✓ | Remaining problem profile, written as predicted data points (15-min ISP grid, 7-day retention). | -| Status | `redispatchEanEffectivity` | text | ✓ | Effectivity category in which the contracted EAN was matched. | +| Status | `redispatchEanEffectivity` | text | ✓ | Effectivity category in which the contracted EAN was matched (e.g. `THREE_PHASE_NETWORK_REDUCE`). | | Status | `redispatchRequestAreaBuy` | text | ✓ | DSO-supplied area description for buy orders. | | Status | `redispatchRequestAreaSell` | text | ✓ | DSO-supplied area description for sell orders. | -| Status | `redispatchLastPoll` | timestamp | ✓ | Timestamp of the last successful poll cycle. | +| Status | `redispatchLastPoll` | timestamp | ✓ | Timestamp of the last completed poll cycle (only updated when the API responded). | | Bid | `redispatchSuggestedPower` | number (kW) | ✓ | _Not yet populated — pending bid pricing strategy follow-up._ | | Bid | `redispatchSuggestedVolume` | number (kWh) | ✓ | _Not yet populated — pending bid pricing strategy follow-up._ | | Bid | `redispatchBidPrice` | number (EUR/MWh) | | Operator-supplied bid price. | | Workflow | `redispatchConfirmBid` | boolean | | Operator toggles to `true` to confirm the active bid; handler resets it after processing. | | Workflow | `redispatchBidStatus` | text | ✓ | State machine — see below. | -| History | `redispatchAnnouncementHistory` | JSON object | ✓ | One data point per polled announcement (90-day retention). | +| History | `redispatchAnnouncementHistory` | JSON object | ✓ | One data point on first sight of each polled announcement, plus a richer entry (with effectivity details) when one is selected (90-day retention). | | History | `redispatchBidHistory` | JSON object | ✓ | One data point per confirmed bid (90-day retention). | `redispatchBidStatus` values: @@ -218,6 +218,7 @@ Every redispatch attribute on `EmsGOPACSAsset`. All status, bid-suggestion and h - The polling interval is clamped to a minimum of 5 minutes because GOPACS recommends spacing requests at least that far apart. - HTTP errors and exceptions on the announcements endpoint **skip the poll and preserve current attributes**, so transient API hiccups do not flap the bid status. Only a confirmed-empty response (HTTP 200 with no announcements) clears the active announcement and resets `redispatchBidStatus` to `NONE`. +- A *persistent* non-200 (e.g. an invalid API key returning 401, or a sustained outage) keeps the previously selected announcement on screen indefinitely. If `redispatchLastPoll` falls behind the configured interval, check the manager logs for `Failed to fetch announcements: HTTP …` (warning) or `Error fetching announcements` (severe). - The handler **refuses to start** (logs `SEVERE`) when `GOPACS_REDISPATCH_API_KEY` is unset — without it there is no way to resolve EAN effectivity per announcement. - Toggling `redispatchEnabled` off then on restarts the handler; the same applies when `contractedEAN` is changed. Useful when you need to force a clean state. @@ -237,7 +238,7 @@ gopacs/ Announcement and bid history are stored as time-series data points on `redispatchAnnouncementHistory` and `redispatchBidHistory`, retained for 90 days and viewable in the OpenRemote history panel. -`redispatchAnnouncementHistory` records **every** polled announcement, including ones that the EAN-effectivity check rejected, so the audit trail captures everything GOPACS returned during the handler's lifetime — not just the announcements that became active. To keep memory bounded for long-running handlers, the running set of already-recorded announcement IDs is capped at 10 000 entries (LRU-evicted). +`redispatchAnnouncementHistory` records **every** polled announcement on first sight (including ones that the EAN-effectivity check later rejects), so the audit trail captures everything GOPACS returned during the handler's lifetime — not just the announcements that became active. When an announcement is then *selected* on a poll, a second, richer history entry is recorded with the matched effectivity details, so an active announcement will appear twice in the timeline (once at first sight, once on selection). To keep memory bounded for long-running handlers, the running set of already-recorded announcement IDs is capped at 10 000 entries (LRU-evicted). ### Future From 803c773c737907753a5d0c7e57afa96f063bb7dc Mon Sep 17 00:00:00 2001 From: Michael Date: Wed, 3 Jun 2026 16:06:12 +0200 Subject: [PATCH 3/3] Correct redispatch doc clearing conditions and FIFO eviction wording Address Copilot review feedback on the redispatch documentation: - Resilience note now describes all three HTTP-200 conditions that clear the active announcement (empty response, no open CONGESTIONMANAGEMENT announcements, or contracted EAN absent from effectivity categories), instead of only the empty-response case. Clarifies that only a failed fetch preserves prior state. - Replace inaccurate "LRU-evicted" wording with insertion-order/FIFO in both the README and the recordedAnnouncementIds code comment; the set uses a LinkedHashMap with accessOrder=false. - Align the sequence diagram announcement state label with the actual ANNOUNCEMENT_OPEN constant used by the handler and prose. --- ems/README.md | 6 +++--- .../ems/manager/gopacs/GOPACSRedispatchHandler.java | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/ems/README.md b/ems/README.md index e810142..4e222a1 100644 --- a/ems/README.md +++ b/ems/README.md @@ -144,7 +144,7 @@ sequenceDiagram participant TP as Trading Platform (future) loop every poll interval (≥ 5 min) - OR->>API: GET /machineannouncements (CONGESTIONMANAGEMENT, OPEN) + OR->>API: GET /machineannouncements (CONGESTIONMANAGEMENT, ANNOUNCEMENT_OPEN) API-->>OR: announcements Note right of OR: Record every newly-seen announcement in history OR->>API: GET .../eansolvingeffectivity per announcement @@ -217,7 +217,7 @@ Every redispatch attribute on `EmsGOPACSAsset`. All status, bid-suggestion and h ### Resilience and polling - The polling interval is clamped to a minimum of 5 minutes because GOPACS recommends spacing requests at least that far apart. -- HTTP errors and exceptions on the announcements endpoint **skip the poll and preserve current attributes**, so transient API hiccups do not flap the bid status. Only a confirmed-empty response (HTTP 200 with no announcements) clears the active announcement and resets `redispatchBidStatus` to `NONE`. +- HTTP errors and exceptions on the announcements endpoint **skip the poll and preserve current attributes**, so transient API hiccups do not flap the bid status. Any *successful* poll (HTTP 200) that yields no announcement selected for the contracted EAN clears the active announcement and resets `redispatchBidStatus` to `NONE`. That covers three cases: the response is empty, the response has announcements but none are open `CONGESTIONMANAGEMENT`, or some are but the contracted EAN is not listed in their EAN-effectivity categories. Only a failed fetch (HTTP error / exception) leaves the previous announcement untouched. - A *persistent* non-200 (e.g. an invalid API key returning 401, or a sustained outage) keeps the previously selected announcement on screen indefinitely. If `redispatchLastPoll` falls behind the configured interval, check the manager logs for `Failed to fetch announcements: HTTP …` (warning) or `Error fetching announcements` (severe). - The handler **refuses to start** (logs `SEVERE`) when `GOPACS_REDISPATCH_API_KEY` is unset — without it there is no way to resolve EAN effectivity per announcement. - Toggling `redispatchEnabled` off then on restarts the handler; the same applies when `contractedEAN` is changed. Useful when you need to force a clean state. @@ -238,7 +238,7 @@ gopacs/ Announcement and bid history are stored as time-series data points on `redispatchAnnouncementHistory` and `redispatchBidHistory`, retained for 90 days and viewable in the OpenRemote history panel. -`redispatchAnnouncementHistory` records **every** polled announcement on first sight (including ones that the EAN-effectivity check later rejects), so the audit trail captures everything GOPACS returned during the handler's lifetime — not just the announcements that became active. When an announcement is then *selected* on a poll, a second, richer history entry is recorded with the matched effectivity details, so an active announcement will appear twice in the timeline (once at first sight, once on selection). To keep memory bounded for long-running handlers, the running set of already-recorded announcement IDs is capped at 10 000 entries (LRU-evicted). +`redispatchAnnouncementHistory` records **every** polled announcement on first sight (including ones that the EAN-effectivity check later rejects), so the audit trail captures everything GOPACS returned during the handler's lifetime — not just the announcements that became active. When an announcement is then *selected* on a poll, a second, richer history entry is recorded with the matched effectivity details, so an active announcement will appear twice in the timeline (once at first sight, once on selection). To keep memory bounded for long-running handlers, the running set of already-recorded announcement IDs is capped at 10 000 entries (oldest inserted IDs are evicted first — insertion-order/FIFO). ### Future diff --git a/ems/src/main/java/org/openremote/extension/ems/manager/gopacs/GOPACSRedispatchHandler.java b/ems/src/main/java/org/openremote/extension/ems/manager/gopacs/GOPACSRedispatchHandler.java index 2d36388..0ef16e8 100644 --- a/ems/src/main/java/org/openremote/extension/ems/manager/gopacs/GOPACSRedispatchHandler.java +++ b/ems/src/main/java/org/openremote/extension/ems/manager/gopacs/GOPACSRedispatchHandler.java @@ -98,7 +98,7 @@ protected record EanEffectivityResult( private ScheduledFuture pollingFuture; private String lastProcessedAnnouncementId; - // Bounded LRU set so a long-running handler does not accumulate every announcement ID it has ever seen. + // Bounded set (insertion-order/FIFO eviction) so a long-running handler does not accumulate every announcement ID it has ever seen. private final Set recordedAnnouncementIds = Collections.newSetFromMap( new LinkedHashMap<>(16, 0.75f, false) { @Override