-
Notifications
You must be signed in to change notification settings - Fork 14
/
transfer.process.protocol.md
208 lines (118 loc) · 9.78 KB
/
transfer.process.protocol.md
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
# Transfer Process Protocol
## Introduction: Terms
This document outlines the key elements of the transfer process protocol. The following terms are used:
- A _**message type**_ defines the structure of a _message_.
- A _**message**_ is an instantiation of a _message type_.
- The _**transfer process protocol**_ is the set of allowable message type sequences and is defined as a state machine (TP-SM).
- A _**transfer process (TP)**_ is an instantiation of the CNP-TP.
- A _**provider**_ is a participant agent that offers an asset.
- A _**consumer**_ is a participant agent that requests access to an offered asset.
- A _**Connector**_ is a `PariticipantAgent` that produces `Agreements` and manages `Asset` sharing.
- An _**Asset**_ is data or a service a provider grants access to.
- An _**Agreement**_ is a result of a [Contract Negotiation](../negotiation/contract.negotiation.protocol.md) and is associated with _exactly one_ `Asset`.
## Transfer Process Protocol
A transfer process (TP) involves two parties, a _provider_ that offers one or more assets under a usage policy and _consumer_ that requests assets. A TP progresses through
a series of states, which are tracked by the provider and consumer using messages. A TP transitions to a state in response to a message from the counter-party.
### Connector Components: Control and Data Planes
A TP is managed by a `Connector`. The connector consists of two logical components, a `Control Plane` and a `Data Plane`. The control plane serves as a coordinating layer that
receives counter-party messages and manages the TP state. The data plane performs the actual transfer of asset data using a wire protocol. Both participants run control and data
planes.
It is important to note that the control and data planes are logical constructs. Implementations may choose to deploy both components within a single process or across
heterogeneous clusters.
### Asset Transfer Types
Asset transfers are characterized as `push` or `pull` transfers and asset data is either `finite` or `non-finite`. This section describes the difference between these types.
#### Push Transfer
A push transfer is when the provider data plane initiates sending of asset data to a consumer endpoint. For example, after the consumer has issued an `TransferRequestMessage,` the
provider begins data transmission to an endpoint specified by the consumer using an agreed-upon wire protocol.
<< TODO: Include example diagram >>
#### Pull Transfer
A pull transfer is when the consumer data plane initiates retrieval of asset data from a provider endpoint. For example, after the consumer has issued an `TransferProcessStart,`
message, the consumer requests the data from the provider-specified endpoint.
<< TODO: Include example diagram >>
#### Finite and Non-Finite Asset Data
Asset data may be `finite` or `non-finite.` Finite data is data that is defined by a finite set, for example, machine learning data or images . After finite data transmission has
finished, the transfer process is completed. Non-finite data is data that is defined by an infinite set or has no specified end, for example streams or an API endpoint. With
non-finite data, a TP will continue indefinitely until either the consumer or provider explicitly terminates the transmission.
### Transfer Process States
The TP states are:
- **REQUESTED** - An asset has been requested under an `Agreement` by the consumer and the provider has sent an ACK response.
- **STARTED** - The asset is available for access by the consumer or the provider has begun pushing the asset to the consumer endpoint.
- **COMPLETED** - The transfer has been completed by either the consumer or the provider.
- **SUSPENDED** - The transfer has been suspended by the consumer or the provider.
- **TERMINATED** - The transfer process has been terminated by the consumer or the provider.
### Transfer Process State Machine
## Message Types
### 1. TransferRequestMessage
**Sent by**: Consumer
**Resulting State**: REQUESTED
**Example**: [TransferRequestMessage](./message/transfer-request-message.json)
**Response**: [TransferProcess](./message/transfer-process.json) containing the transfer process id or ERROR.
**Schema**: [TransferRequestMessage](./message/shape/transfer-request-message-shape.ttl), [TransferRequestMessage JSON Schema](./message/schema/transfer-request-message-schema.json), [TransferProcess Shape](./message/shape/transfer-process-shape.ttl) and the [TransferProcess JSON Schema](./message/schema/transfer-process-schema.json)
#### Description
The `TransferRequestMessage` is sent by a consumer to initiate a transfer process.
#### Notes
- The `agreementId` property refers to an existing contract agreement between the consumer and provider.
- The `dct:format` property is a format specified by a `Distribution` for the `Asset` associated with the agreement. This is generally obtained from the provider `Catalog`.
- The `dataAddress` property must only be provided if the `dct:format` requires a push transfer.
- `callbackAddress` is a URI indicating where messages to the consumer should be sent. If the address is not understood, the provider MUST return an UNRECOVERABLE error.
Providers should implement idempotent behavior for `TransferRequestMessage` based on the value of `@id`. Providers may choose to implement idempotent behavior for a certain period of
time. For example, until a transfer processes has completed and been archived after an implementation-specific expiration period. If a request for the given `@id` has already been
received *and* the same consumer sent the original message, the provider should respond with an appropriate `DataAddressMessage`.
Once a transfer process have been created, all associated callback messages must include a `correlationId` set to the `TransferRequestMessage` `@id` value.
Providers must include a `correlationId` property in the `TransferProcessMessage` with a value set to the `@id` of the corresponding `TransferRequestMessage`.
#### Notes
- The `dataAddress` contains a transport-specific endpoint address for pushing the asset. It may include a temporary authorization token.
- Valid states of a `TransferProcess` are `REQUESTED`, `STARTED`, `TERMINATED`, `COMPLETED`, and `SUSPENDED`.
### 2. TransferStartMessage
**Sent by**: Provider
**Resulting State**: STARTED
**Example**: [TransferStartMessage](./message/transfer-start-message.json)
**Response**: ACK or ERROR.
**Schema**: [TransferStartMessageShape](./message/shape/transfer-start-message-shape.ttl) and the [TransferStartMessage JSON Schema](./message/schema/transfer-start-message-schema.json)
#### Description
The `TransferStartMessage` is sent by the provider to indicate the asset transfer has been initiated.
#### Notes
- The `dataAddress` is only provided if the current transfer is a pull transfer and contains a transport-specific endpoint address for obtaining the asset. It may include a
temporary authorization token.
### 3. TransferSuspensionMessage
**Sent by**: Provider or Consumer
**Resulting State**: SUSPENDED
**Example**: [TransferSuspensionMessage](./message/transfer-suspension-message.json)
**Response**: ACK or ERROR.
**Schema**: [TransferSuspensionMessageShape](./message/shape/transfer-suspension-message-shape.ttl) and the [TransferSuspensionMessage JSON Schema](./message/schema/transfer-suspension-message-schema.json)
#### Description
The `TransferSuspensionMessage` is sent by the provider or consumer when either of them needs to temporarily suspend the data transfer.
### 4. TransferCompletionMessage
**Sent by**:
**Resulting State**:
**Example**: [TransferCompletionMessage](./message/transfer-completion-message.json)
**Response**: ACK or ERROR.
**Schema**: [TransferCompletionMessageShape](./message/shape/transfer-completion-message-shape.ttl) and the [TransferCompletionMessage JSON Schema](./message/schema/transfer-completion-message-schema.json)
#### Description
The `TransferCompletionMessage` is sent by the provider or consumer when asset transfer has completed. Note that some data plane implementations may optimize completion
notification by performing it as part of its wire protocol. In those cases, a `TransferCompletionMessage` message does not need to be sent.
### 5. TransferTerminationMessage
**Sent by**: Provider or Consumer
**Resulting State**: TERMINATED
**Example**: [TransferTerminationMessage](./message/transfer-termination-message.json)
**Response**: ACK or ERROR.
**Schema**: [TransferTerminationMessageShape](./message/shape/transfer-termination-message-shape.ttl) and the [TransferTerminationMessage JSON Schema](./message/schema/transfer-termination-message-schema.json)
#### Description
The `TransferTerminationMessage` is sent by the provider or consumer at any point except a terminal state to indicate the data transfer process should stop and be placed in
a terminal state. If the termination was due to an error, the sender may include error information.
## TransferError
**Sent by**: Consumer or Provider
**Example**: [TransferError](./message/transfer-error.json)
**Schema**: [TransferErrorShape](./message/shape/transfer-error-shape.ttl) and the [TransferError JSON Schema](./message/schema/transfer-error-schema.json)
#### Description
The `TransferError` is an object returned by a consumer or provider indicating an error has occurred. It does not cause a state transition.
#### Notes
- A `TransferError` is different to an error response. A `TransferError` does not necessarily finish the negotiation but can continue afterwards.