Make metadata a obligatory requirement

[kaiwirt]

I'm not sure if this is the correct place for this issue.

As far as i recall, we decided, that metadata should be a prerequisite before transmitting data over WIS2. Somehow this requirement should be enforced. Following the discussions here
wmo-im/wis2-notification-message#25
and here
wmo-im/wis2-notification-message#31

i currently don't see a place in the protocol / message flow, where we can check if metadata is available.

We either need to decide at which place we can put such a check, or we need to agree, that we live with a strong requirement for metadata potentially allowing for data without metadata.

cc: @golfvert @antje-s @josusky @tomkralidis

[6a6d74]

My view is that we state it as a strong requirement, rather than force Global Services to check before doing anything.

I think this is OK because if there's no discovery metadata, data consumers won't be able to find the data or where to subscribe to notifications about the data. Yes - I admit that some people are likely to guess (by which I mean, browse the topic hierarchy on their favourite Global Broker), or simply benefit from using wild-cards in their subscription (which means they would get data that they don't have any information about).

I suggest we mitigate this problem via the Global Monitor, e.g., doing a consistency check of looking for the discovery metadata associated with a notification message, and publishing an alert if there's a failure. Alert should be visible on the dashboard and directly notifying the originating WIS2node.

Aside: how do alerts get pushed back to the originating WIS2node?

[efucile]

Decision

To be implemented by GBs by Q3 2024

[golfvert]

It will be enforced on Oct. 1st 2024. One quarter before the end of the pre-operational phase.

[6a6d74]

@6a6d74 (Jeremy) to make sure the Guide text is updated to reflect the decision.

[amilan17]

@tomkralidis does this change the requirement in WNM from "SHOULD provide.." to "SHALL provide.." a metadata_id?

[tomkralidis]

@amilan17 we are keeping as an option (and SHOULD in WCMP2 proper) to facilitate data publication until 01 October 2024 as agreed at the W2AT November meeting in Washington the beginning of the operational phase (cc @hhaddouch @Haddouch101 to verify in q1 2025).

[amilan17]

If I remember correctly, discovery medata is a requirement BEFORE exchanging data. So why not make the reference to the metadata identifier a requirement in the WNM too?

[6a6d74]

Guide now states that discovery metadata must be published before data is exchanged. See PR #116