Minimal dataspace participant

[SimonStier]

We are defining tiers of DSP implementation for MaterialsCommons, a Horizon Europe project building a federated data space for materials science (drafts, rendered). A recurring question there is what the smallest thing is that can still count as a participant.

The motivation is that a research group usually cannot operate a connector. What it does have is a web server, and often only static hosting such as GitHub Pages.

Our reading is that static hosting cannot be a DSP provider today, for three independent reasons:

• the provider has to originate callbacks to the consumer's callbackAddress (ContractAgreementMessage, the FINALIZED event, TransferStartMessage), and static hosting cannot make outbound requests;
• providerPid is minted per process and both state machines require mutation, so nothing can be pre-generated;
• the catalogue request is a POST, while the dataset request is a GET.

If a subset were defined that did not require those, publishing a catalogue would cost a lab one file. That is a very low entry barrier, and for research adoption it is the difference between participating and not.

A harvester or broker with a proper DSP implementation is of course a solution, and the spec already describes catalog brokers. A shared connector can advertise datasets on behalf of publishers that run nothing, and since the DataAddress in a Transfer Start Message need not point at the connector's own host, the bytes stay with the publisher. But it means more centralization: someone has to operate that connector, and the publisher becomes content inside another participant's catalogue rather than holding its own participantId (related: #253).

Two questions:

1. Is a catalogue-only participant something DSP wants to support, or is operating a connector meant to be the definition of participation?
2. If the latter, is it intended that an advertised DataService may be operated by a third party on the publisher's behalf?

Related threads: #253 on multi-tenancy and participantId, #240 on catalogue queries and how a consumer finds a provider it does not know in advance, and #250 on the schemas and the JSON-LD context. #280 covers the POST-versus-GET asymmetry mentioned above.

[SimonStier]

cc @rartino @linnosma @karikolehmainen @MBueschelberger @schaarj, this is the entry-level question from our WP11 tier work.

[jimmarino]

Thanks for the detailed question.

On the use case: I would question the premise that this tier is "the difference between participating and not."

Static hosting makes no per-request decision. Whatever it serves, it provides to anyone who can reach it. So a lab on static hosting is in one of two situations. Either its data is open, in which case there is no policy to enforce, no agreement to reach, and no reason for a consumer to go through DSP to get it. A DCAT document and a URL are sufficient, and that is what open data portals have done for years. Or its data needs governing (criteria satisfied, license terms, restricted access, etc.), in which case something must evaluate policy and make an access decision at request time. That is what a connector is.

The obstacle is not the connector as a piece of software. It is that static hosting cannot decide anything: it returns the same bytes to every request. Governed access is a sequence of decisions that depend on who is asking, whether the policy admits them, what terms they accept, and how access is then bound to those terms. The three reasons you list are all consequences of that: process identifiers and state exist because the decision has state, callbacks exist because the provider makes the decision and reports it, and the POST exists because the request carries the consumer's identity. A smaller protocol cannot remove the decision-maker. It can only change who operates it, and an operator running a connector under the lab's identity is exactly that.

The same applies to the broker model. If the bytes stay on public static hosting, the agreement the broker negotiates protects nothing, because anyone can fetch the bytes without one. The DataAddress indirection is only useful when the endpoint it points to enforces the agreement.

Being listed in a catalog gives a lab visibility. It has that already with a DCAT document. Being a participant gives it something different: it becomes a party to agreements that can be authenticated. Neither is available to something that cannot hold credentials or run a state machine, and no subset of the protocol changes that.

In the end, running a DSP implementation such as the EDC or TNO Connector is quite simple given the design center of DSP as described above.

[SimonStier]

That is a fair correction on the broker, and it is the part I had wrong: if the bytes sit on public static hosting, the agreement the broker negotiates protects nothing, so the indirection buys interoperability at best and not governance.

One thing worth pinning down, since it is the part that helps us most. You write that a smaller protocol "can only change who operates it, and an operator running a connector under the lab's identity is exactly that". Read literally, an advertised DataService may then be operated by a third party on the publisher's behalf, as long as it can hold credentials and decide. Is that a correct reading?

[SimonStier]

On "a DCAT document and a URL are sufficient", DCAT standardises the vocabulary rather than the retrieval. Protocols do exist that fill that gap, FAIR Data Point and OGC API - Records among them, but DCAT-AP prescribes none of them and none has become universal, so a consumer federating across providers still needs per-provider integration or has to speak several. That fragmentation, rather than an absence, is what we were reacting to.

So the question underneath this is where a data space begins. DSP puts it at a minimal level of governance, where a decision gets made. We came at it from a common protocol, where retrieval is uniform whether or not anything is governed. Both are defensible, and if it is the former, our entry level is a different thing and should not be called DSP. Not settled on our side yet.

One reason it might matter beyond our case: DSP is on its way to ISO. If the open-data case were a lower conformance tier inside the standard rather than something outside it, a single protocol would span both sides. Research groups would get a path into industrial data spaces without first standing up governance they do not need, and industry would reach research data through the stack it already operates rather than through a separate DCAT integration.