Improve and document data-plane-transfer

CHANGELOG.md

@@ -21,6 +21,7 @@ in the detailed section referring to by linking pull requests or issues.
 * Event Framework for PolicyDefinition entity (#1437)
 * SQL Translation layer (#1357, #1459)
 * Permit API verbose error response (#1479)
+* Fix TODO and document `:extensions:data-plane-transfer` (#1519)
 
 #### Changed
 
@@ -45,7 +46,7 @@ in the detailed section referring to by linking pull requests or issues.
 * Avoid endless loops in `ContractNegotiationManager` (#1487)
 * Fix race condition in `ContractNegotiationIntegrationTest` (#1505)
 * Fix for change in Cosmos DB behavior on missing sort fields (#1514)
-* Effectively removed default LIMIT in SQL Contract Def Store (#1515) 
+* Effectively removed default LIMIT in SQL Contract Def Store (#1515)
 
 ## [milestone-4] - 2022-06-07
 

extensions/data-plane-transfer/README.md

@@ -1,13 +1,9 @@
-# Data Plane Transfer extension
+# Data Plane Transfer
 
-This extension provides resources used to delegate data transfer to the Data Plane, or to use the Data Plane as a proxy for querying the data.
+This module provides extensions for performing data transfer through the Data Plane. Each extension is designed to
+support one type of data transfer.
 
-The setting parameters of this extension are listed below:
+## Extensions
 
-| Parameter name                                      | Description                                                                                                                        | Mandatory | Default value                          |
-|:----------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------|:----------|:---------------------------------------|
-| `edc.transfer.proxy.token.signer.privatekey.alias`  | Alias of private key used to sign token used to hit Data Plane public API                                                          | true      |                                        |
-| `edc.transfer.proxy.token.verifier.publickey.alias` | Alias of public key used to verify tokens hitting the Data Plane public API (public key must be in the Vault)                      | false     | private key alias suffixed with "-pub" |
-| `edc.transfer.proxy.token.validity.seconds`         | Validity of tokens generated for hitting Data Plane public API (in seconds)                                                        | false     | 600                                    | 
-| `edc.transfer.proxy.endpoint`                       | Public API endpoint of the Data Plane (TODO: this has to be dynamically generated by the Data Plane Selector) based on the request | true      |                                        |
-| `edc.transfer.client.selector.strategy`             | Selection strategy used by the client to determine to which Data Plane instance data transfer should be delegated                  | true      |                                        |
+- [Data Plane Transfer Client](./data-plane-transfer-client/README.md) provides a client to delegate data transfer to the Data Plane.
+- [Data Plane Transfer Sync](./data-plane-transfer-sync/README.md) provides services to use the Data Plane as a proxy for querying data from the provider data source.

extensions/data-plane-transfer/data-plane-transfer-client/README.md

@@ -0,0 +1,52 @@
+# Data Plane Transfer Client
+
+This extension provides a client for delegating data transfer to a Data Plane instance.
+
+## Background
+
+After successful negotiation and contract agreement between a consumer and a provider, the Data Plane must be triggered to perform the actual data transfer.
+
+### Scope
+
+Data transfer from source to destination
+
+### Use Cases
+
+This extension is dedicated to use-cases wherein the consumer wants specific data from a given provider to be pushed in a recipient storage in its own environment.
+
+## Technical Details
+
+### Interfaces
+
+### Dependencies
+
+| Name                                        | Description                             |
+|:--------------------------------------------|:----------------------------------------|
+| extensions:data-plane:data-plane-spi        | Required for `DataPlaneManager`         |
+| extensions:data-plane-selector:selector-spi | Required for `DataPlaneSelectorService` |
+
+### Configurations
+
+| Parameter name                                      | Description                                                                                                                        | Mandatory | Default value                          |
+|:----------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------|:----------|:---------------------------------------|
+| `edc.transfer.client.selector.strategy`             | Selection strategy used by the client to determine to which Data Plane instance data transfer should be delegated                  | false     | random                                 |
+
+## Terminology
+
+## Design Principles
+
+This extension provides a client for delegating a data transfer to the data plane. As Data Plane exposes both a Java and a REST API, the client
+is designed to support both. The extension is centered around the `DataPlaneTransferFlowController` which is triggered for any data request
+whose destination type is different from `HttpProxy` (this one being reserved for Client Pull data transfers). This controller maps the incoming
+`DataRequest` into a `DataFlowRequest`, which is mainly composed of source/destination data address pair. This `DataFlowRequest` is
+finally forwarded to the Data Plane by the client.
+
+#### Flow diagram
+
+![alt text](../../../docs/architecture/data-transfer/diagrams/data-plane-transfer-client.png)
+
+## Decisions
+
+
+
+

extensions/data-plane-transfer/data-plane-transfer-client/src/main/java/org/eclipse/dataspaceconnector/transfer/dataplane/flow/DataPlaneTransferFlowController.java

@@ -22,18 +22,19 @@
 import org.eclipse.dataspaceconnector.spi.types.domain.DataAddress;
 import org.eclipse.dataspaceconnector.spi.types.domain.transfer.DataFlowRequest;
 import org.eclipse.dataspaceconnector.spi.types.domain.transfer.DataRequest;
+import org.eclipse.dataspaceconnector.transfer.dataplane.spi.DataPlaneTransferConstants;
 import org.eclipse.dataspaceconnector.transfer.dataplane.spi.client.DataPlaneTransferClient;
 import org.jetbrains.annotations.NotNull;
 
 import java.util.UUID;
 
-import static org.eclipse.dataspaceconnector.transfer.dataplane.spi.DataPlaneTransferType.HTTP_PROXY;
+import static org.eclipse.dataspaceconnector.transfer.dataplane.spi.DataPlaneTransferConstants.HTTP_PROXY;
 
 /**
  * Implementation of {@link DataFlowController} that delegates data transfer to Data Plane instance.
  * Note that Data Plane can be embedded in current runtime (test, samples...) or accessed remotely.
  * The present {@link DataFlowController} is triggered when destination type in the {@link DataRequest} is different from
- * {@link org.eclipse.dataspaceconnector.transfer.dataplane.spi.DataPlaneTransferType#HTTP_PROXY}, as this one is reserved for synchronous data transfers.
+ * {@link DataPlaneTransferConstants#HTTP_PROXY}, as this one is reserved for synchronous data transfers.
  */
 public class DataPlaneTransferFlowController implements DataFlowController {
     private final DataPlaneTransferClient client;

extensions/data-plane-transfer/data-plane-transfer-client/src/test/java/org/eclipse/dataspaceconnector/transfer/dataplane/flow/DataPlaneTransferFlowControllerTest.java

@@ -27,7 +27,7 @@
 import org.mockito.ArgumentCaptor;
 
 import static org.assertj.core.api.Assertions.assertThat;
-import static org.eclipse.dataspaceconnector.transfer.dataplane.spi.DataPlaneTransferType.HTTP_PROXY;
+import static org.eclipse.dataspaceconnector.transfer.dataplane.spi.DataPlaneTransferConstants.HTTP_PROXY;
 import static org.mockito.ArgumentMatchers.any;
 import static org.mockito.Mockito.mock;
 import static org.mockito.Mockito.times;

extensions/data-plane-transfer/data-plane-transfer-spi/src/main/java/org/eclipse/dataspaceconnector/transfer/dataplane/spi/DataPlaneTransferConstants.java

@@ -0,0 +1,37 @@
+/*
+ *  Copyright (c) 2022 Amadeus
+ *
+ *  This program and the accompanying materials are made available under the
+ *  terms of the Apache License, Version 2.0 which is available at
+ *  https://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  SPDX-License-Identifier: Apache-2.0
+ *
+ *  Contributors:
+ *       Amadeus - Initial implementation
+ *
+ */
+
+package org.eclipse.dataspaceconnector.transfer.dataplane.spi;
+
+/**
+ * Type of Data Plane transfer.
+ */
+public interface DataPlaneTransferConstants {
+    /**
+     * {@link org.eclipse.dataspaceconnector.spi.types.domain.transfer.DataFlowRequest} type that
+     * triggers data proxy transfer.
+     */
+    String HTTP_PROXY = "HttpProxy";
+
+    /**
+     * Claim of the token used in input of Data Plane public API containing the address of the
+     * data source as an encrypted string.
+     */
+    String DATA_ADDRESS = "dad";
+
+    /**
+     * Claim of the token used in input of Data Plane public API containing the contract id.
+     */
+    String CONTRACT_ID = "cid";
+}