MXD Backend Service API #172

ravinderkumarsap · 2023-12-03T23:27:04Z

Description

Please describe your PR:
Created a http backend service to validate the transfer.
Link for issue : #128

What does this PR introduce?

Here's a step-by-step Explanation of created a REST API application to complete and validate a successful asset transfer:

Create a Content Storage API:

POST API /api/v1/contents:
{{backend-service-url}}/api/v1/content/

Endpoint to store assets.
Accepts asset data in the request body.
Persists the asset data and returns a URL or ID in the response.

GET API /api/v1/contents/{id}:
{{backend-service-url}}/api/v1/content/1

Endpoint to fetch content by ID.
Takes the content ID as a path parameter.
Returns the stored content associated with the given ID.
Create a Transfer API:

POST API /api/v1/transfers:
{{backend-service-url}}/api/v1/transfer/
Endpoint to accept transfer data from the connector.
Expects a JSON payload similar to:
{
"id": "",
"endpoint": "http://alice-tractusx-connector-dataplane:8081/api/public",
"authKey": "Authorization",
"authCode": "",
"properties": {}
}

Persists the transfer data with the transfer ID.
GET API /api/v1/transfers/{id}:
{{backend-service-url}}/api/v1/transfer/52ba4b2f-ae3a-46ae-96f2-6781aa819386/

Endpoint to retrieve transfer data by ID.
Takes the transfer ID as a path parameter.
Returns the JSON data pushed by the connector for the specified transfer.

GET API /api/v1/transfers/{id}/contents:
{{backend-service-url}}/api/v1/transfer/52ba4b2f-ae3a-46ae-96f2-6781aa819386/contents/

Endpoint to retrieve the actual assets content.
Uses the stored endpoint URL from the transfer data to fetch content.
Returns the content fetched from the specified endpoint.

GET API /api/v1/content/random
{{backend-service-url}}/api/v1/content/random
Return the Sample Random Generated JSON

Does it add a new feature? yes
Is it enhancing documentation : yes

It has the postman collections inside postman folder inside backend-service folder.

Closes #128

Pre-review checks

Please ensure to do as many of the following checks as possible, before asking for committer review:

DEPENDENCIES are up-to-date. Dash license tool. Committers can open IP issues for restricted libs.
Copyright and license header are present on all affected files

mxd/backend-service/Dockerfile

mxd/backend-service/src/main/java/com/eclipse/mxd/BackendApplication.java

mxd/backend-service/src/main/java/com/eclipse/mxd/Main.java

mxd/backend-service/src/main/java/com/eclipse/mxd/model/TransferRequest.java

mxd/backend-service/src/main/java/com/eclipse/mxd/model/TransfersModel.java

mxd/backend-service/src/main/java/com/eclipse/mxd/repository/Impl/ContentRepositoryImpl.java

mxd/backend-service/Dockerfile

mxd/backend-service/build.gradle

mxd/backend-service/src/main/java/org/eclipse/mxd/Main.java

mxd/backend-service/src/main/java/org/eclipse/mxd/entity/Transfer.java

mxd/backend-service/src/main/java/org/eclipse/mxd/service/Impl/TransferServiceImpl.java

mxd/backend-service/src/main/java/org/eclipse/mxd/util/DateTimeUtil.java

mxd/backend-service/src/main/java/org/eclipse/mxd/util/HibernateUtil.java

mxd/backend-service/src/main/java/org/eclipse/mxd/Main.java

mxd/backend-service/src/main/java/org/eclipse/mxd/model/TransfersRequest.java

mxd/backend-service/src/main/java/org/eclipse/mxd/util/SettingResolver.java

mxd/backend-service/src/main/java/org/eclipse/mxd/util/SingletonObjectMapper.java

mxd/backend-service.tf

paullatzelsperger

Package declarations are wrong, and many copyright headers are likely wrong. Further, there is no reason to introduce J2EE when EDC modules are perfectly capable of implementing a REST API.

mxd/backend-service/src/main/java/org/eclipse/mxd/entity/Content.java

ravinderkumarsap · 2023-12-11T08:21:58Z

Package declarations are wrong, and many copyright headers are likely wrong. Further, there is no reason to introduce J2EE when EDC modules are perfectly capable of implementing a REST API.

I have updated the package declarations as discussed and updated the copyright headers as suggested.

paullatzelsperger · 2023-12-18T07:32:03Z

Package declarations are wrong, and many copyright headers are likely wrong. Further, there is no reason to introduce J2EE when EDC modules are perfectly capable of implementing a REST API.

Package declarations are wrong, and many copyright headers are likely wrong. Further, there is no reason to introduce J2EE when EDC modules are perfectly capable of implementing a REST API.

I have updated the package declarations as discussed and updated the copyright headers as suggested.

you are still using Jakarta EE, for which there is no reason and which I asked you to replace earlier. Please use EDC modules and components, a lightweight REST API can easily be built with it, checkout this.

ravinderkumarsap · 2023-12-18T19:53:22Z

Package declarations are wrong, and many copyright headers are likely wrong. Further, there is no reason to introduce J2EE when EDC modules are perfectly capable of implementing a REST API.

Package declarations are wrong, and many copyright headers are likely wrong. Further, there is no reason to introduce J2EE when EDC modules are perfectly capable of implementing a REST API.

I have updated the package declarations as discussed and updated the copyright headers as suggested.

you are still using Jakarta EE, for which there is no reason and which I asked you to replace earlier. Please use EDC modules and components, a lightweight REST API can easily be built with it, checkout this.

We have started with converting to EDC modules. We will update you once completed .

…ractusx_tutorial-resources into feat/128-backend-service

mxd/backend-service/Dockerfile

mxd/backend-service.tf

mxd/backend-service/Dockerfile

mxd/backend-service/build.gradle.kts

...e/src/main/java/org/eclipse/tractusx/mxd/backendservice/controller/ContentApiController.java

...ice/src/main/java/org/eclipse/tractusx/mxd/backendservice/service/HttpConnectionService.java

ravinderkumarsap · 2024-01-04T14:13:15Z

Package declarations are wrong, and many copyright headers are likely wrong. Further, there is no reason to introduce J2EE when EDC modules are perfectly capable of implementing a REST API.

Package declarations are wrong, and many copyright headers are likely wrong. Further, there is no reason to introduce J2EE when EDC modules are perfectly capable of implementing a REST API.

I have updated the package declarations as discussed and updated the copyright headers as suggested.

you are still using Jakarta EE, for which there is no reason and which I asked you to replace earlier. Please use EDC modules and components, a lightweight REST API can easily be built with it, checkout this.

We have started with converting to EDC modules. We will update you once completed.

I have completed the backend service using EDC modules. Let me know the feedback.

ndr-brt

@ravinderkumarsap as I said before, please add E2E test coverage that demonstrates that the integration is working, after that I'll review the PR properly.

@ndr-brt I have completed the review comments and updated on PR. I am working on the E2E test coverage.

when the tests will be ready please re-request my review

Added e2e tests, refactored code

ndr-brt

please ensure that tests are running as part of the CI

...d-service/src/main/java/org/eclipse/tractusx/mxd/backendservice/BackendServiceExtension.java

mxd/backend-service/src/test/java/org/eclipse/tractusx/mxd/e2e/ContentApiEndToEndTest.java

Updated workflow, fixed issue with db name

ndr-brt · 2024-06-12T12:12:11Z

mxd/backend-service/build.gradle.kts

+    implementation(libs.restAssured)
+    implementation(libs.edc.configuration.filesystem)
+    implementation(libs.edc.boot)
+    implementation(libs.edc.json.ld)
+    implementation(libs.edc.web.spi)
+    implementation(libs.edc.api.core)
+    implementation(libs.edc.core)
+    implementation(libs.edc.http)
+    implementation(libs.edc.http.lib)
+    implementation(libs.edc.http.spi)
+    implementation(libs.edc.jersey.core)
+    implementation(libs.swagger.core)
+    implementation(libs.edc.transaction.spi)
+    implementation(libs.edc.connector.core)
+    implementation(libs.edc.sql.core)
+    implementation(libs.apache.commons)
+    implementation(libs.postgresql)
+    implementation(libs.edc.transform)
+    implementation(libs.edc.transaction)
+    implementation(libs.edc.util)


there are some dependencies that are not needed, e.g.:

libs.edc.api.core libs.edc.jersey.core libs.swagger.core libs.edc.transform libs.edc.transaction libs.edc.util

please clean them up

Removed unused libraries

ndr-brt · 2024-06-12T12:14:03Z

mxd/backend-service/gradle/libs.versions.toml

+format.version = "1.1"
+[versions]
+assertj = "3.25.3"
+edc = "0.6.0"


is there a particular reason to not use the latest EDC version? (0.7.0)

Updated EDC version to 0.7.0

ndr-brt · 2024-06-12T12:15:53Z

mxd/backend-service/build.gradle.kts

+    testImplementation(libs.assertj)
+    testImplementation(libs.edc.junit)
+    implementation(libs.edc.junit.base)
+    testImplementation(libs.testng)


I don't think we are using testng, please cleanup test dependencies as well

Removed unused libraries.

ndr-brt · 2024-06-12T12:16:37Z

mxd/backend-service/build.gradle.kts

+    testImplementation(libs.postgres.containers)
+
+    testImplementation(testFixtures(libs.edc.sql.core))
+    testImplementation(libs.edc.core)


Removed unused libraries.

ndr-brt · 2024-06-12T12:17:43Z

...d-service/src/main/java/org/eclipse/tractusx/mxd/backendservice/BackendServiceExtension.java

+    @Inject(required = false)
+    private EventListener okHttpEventListener;


ndr-brt · 2024-06-12T13:37:30Z

...n/java/org/eclipse/tractusx/mxd/backendservice/statements/TransferStatementsServiceImpl.java

+    @Override
+    public SqlQueryStatement createQuery(QuerySpec querySpec) {
+        var select = format("SELECT * FROM %s", getTransferTable());
+        //return new SqlQueryStatement(select, querySpec, null);


Removed commented codes.

ndr-brt · 2024-06-12T13:38:27Z

.../src/main/java/org/eclipse/tractusx/mxd/backendservice/store/SqlContentStoreServiceImpl.java

+    @Override
+    public StoreResult<Void> update(Content definition) {
+        return null;
+    }
+
+    @Override
+    public StoreResult<Content> deleteById(String id) {
+        return null;
+    }


why do define these methods if the implementation is not provided? please delete them from both interface and implementation

Removed unused codes.

ndr-brt · 2024-06-12T13:40:07Z

.../src/main/java/org/eclipse/tractusx/mxd/backendservice/store/SqlContentStoreServiceImpl.java

+            monitor.warning(exception.getMessage());
+            throw new EdcPersistenceException(exception.getMessage(), exception);
+        }
+        return null;


what's the point of returning null here? I guess if affectedRow is 0 then it should throw an exception/return a failed result

Updated to throw exception.

ndr-brt · 2024-06-12T13:40:28Z

...src/main/java/org/eclipse/tractusx/mxd/backendservice/store/SqlTransferStoreServiceImpl.java

+    @Override
+    public StoreResult<Void> update(Transfer transfer) {
+        return null;
+    }
+
+    @Override
+    public StoreResult<Transfer> deleteById(String id) {
+        return null;
+    }


same as the other class, please remove if not needed

Removed unused codes.

ndr-brt · 2024-06-12T13:42:37Z

mxd/backend-service/src/test/java/org/eclipse/tractusx/mxd/testfixtures/PostgresRuntime.java

+
+    @RegisterExtension
+    public static final EdcRuntimeExtension RUNTIME = new EdcRuntimeExtension(
+            "",


this empty string parameter can be removed

This parameter is needed by EdcRuntimeExtension because of constructor overloading.

@ieuna nope, there's a different constructor where the modules are passed as varargs as last argument, so you could use that one instead (way cleaner: an empty string is confusing to read)

not passing any argument to the varargs wasn't working, so "new String()" is passed to the third argument(varargs modules) now.

@RegisterExtension EdcRuntimeExtension RUNTIME = new EdcRuntimeExtension( "backend", Map.of(....), new String() );

@ieuna you don't need the third argument, just two: runtime name and configuration

Removed redundant codes, Updated EDC version

ndr-brt

LGTM

ndr-brt · 2024-06-19T12:49:48Z

please fix the conflicts and I will merge this

Resolved conflict

ieuna · 2024-06-24T05:20:56Z

please fix the conflicts and I will merge this

resolved

ndr-brt · 2024-06-24T06:47:03Z

@ieuna there's a validate-terraform-format check failing

ieuna · 2024-06-24T07:53:11Z

@ieuna there's a validate-terraform-format check failing
@ndr-brt
fixed terraform format. validate terraform check is passing now

backend-service-changes

6925a39

ravinderkumarsap requested a review from paullatzelsperger as a code owner December 3, 2023 23:27

hemantxpatel reviewed Dec 4, 2023

View reviewed changes

ravinderkumarsap added 2 commits December 4, 2023 23:54

feedback changes completed

3b0b512

feedback changes completed

dd29d45

ravinderkumarsap requested a review from hemantxpatel December 4, 2023 18:35

feedback changes

f28be31

hemantxpatel reviewed Dec 5, 2023

View reviewed changes

ravinderkumarsap added 2 commits December 6, 2023 16:08

feedback changes

30cc021

feedback changes

728d7bb

ravinderkumarsap requested a review from hemantxpatel December 6, 2023 11:08

hemantxpatel reviewed Dec 7, 2023

View reviewed changes

feedback changes

bdef4fa

ravinderkumarsap requested a review from hemantxpatel December 8, 2023 11:17

paullatzelsperger requested changes Dec 11, 2023

View reviewed changes

mxd/backend-service/src/main/java/org/eclipse/mxd/entity/Content.java Outdated Show resolved Hide resolved

mxd/backend-service/src/main/java/org/eclipse/mxd/entity/Content.java Outdated Show resolved Hide resolved

feedback changes

ae38a3e

ravinderkumarsap requested a review from paullatzelsperger December 11, 2023 07:33

feedback changes

090e8f2

ravinderkumarsap added 5 commits December 28, 2023 19:18

Merge branch 'main' of https://github.com/sap-contributions/eclipse-t…

e23e9f9

…ractusx_tutorial-resources into feat/128-backend-service

Merge branch 'main' of https://github.com/sap-contributions/eclipse-t…

3c83912

…ractusx_tutorial-resources into feat/128-backend-service

Implemented EDC Modules for Rest API

e4702eb

Pipelines changes

2b10726

pipeline changes

ec30769

hemantxpatel suggested changes Jan 4, 2024

View reviewed changes

feedback changes

3789aa0

ravinderkumarsap requested a review from hemantxpatel January 4, 2024 13:17

ndr-brt requested changes Mar 12, 2024

View reviewed changes

paullatzelsperger removed their request for review March 15, 2024 11:30

ieuna and others added 4 commits May 13, 2024 13:09

Added e2e tests, refactored code

19573f8

Updated readme file

5501e56

Removed hardcoded URL

ffa16bf

Merge pull request #41 from sap-contributions/backend-service

d0d5b1b

Added e2e tests, refactored code

hemantxpatel requested a review from ndr-brt May 17, 2024 08:15

ndr-brt requested changes May 21, 2024

View reviewed changes

ieuna and others added 3 commits May 27, 2024 14:37

Updated workflow, fixed issue with db name

9d802c9

Removed braces

09dfcd8

Merge pull request #45 from sap-contributions/backend-service

5ea9e95

Updated workflow, fixed issue with db name

hemantxpatel mentioned this pull request Jun 11, 2024

EDC - MXD - Add HTTP Backend Service eclipse-tractusx/sig-release#314

Open

1 task

ieuna requested a review from ndr-brt June 12, 2024 09:27

ndr-brt requested changes Jun 12, 2024

View reviewed changes

ieuna and others added 2 commits June 18, 2024 14:54

Removed redundant codes, Updated EDC version

103be8a

Merge pull request #60 from sap-contributions/feat/backend-service

1053ab7

Removed redundant codes, Updated EDC version

ieuna requested a review from ndr-brt June 18, 2024 12:34

ndr-brt approved these changes Jun 19, 2024

View reviewed changes

ieuna and others added 3 commits June 20, 2024 08:45

Merged branch main into backend-service

3401262

Updated backend service terraform config

13e3443

Merge pull request #67 from sap-contributions/feat/backend-service

6561a72

Resolved conflict

ieuna requested a review from ndr-brt June 24, 2024 05:21

Fixed terraform file format

07b807b

ndr-brt approved these changes Jun 25, 2024

View reviewed changes

ndr-brt merged commit 4d7bf0b into eclipse-tractusx:main Jun 25, 2024
4 of 5 checks passed

ieuna mentioned this pull request Jul 5, 2024

MXD 1.0 Develop Backend App sap-contributions/eclipse-tractusx_tutorial-resources#25

Closed

3 tasks

		@Inject(required = false)
		private EventListener okHttpEventListener;

MXD Backend Service API #172

MXD Backend Service API #172

Conversation

ravinderkumarsap commented Dec 3, 2023 • edited Loading

Description

Pre-review checks

paullatzelsperger left a comment

Choose a reason for hiding this comment

ravinderkumarsap commented Dec 11, 2023

paullatzelsperger commented Dec 18, 2023

ravinderkumarsap commented Dec 18, 2023

ravinderkumarsap commented Jan 4, 2024

ndr-brt left a comment

Choose a reason for hiding this comment

ndr-brt left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ndr-brt left a comment

Choose a reason for hiding this comment

ndr-brt commented Jun 19, 2024

ieuna commented Jun 24, 2024

ndr-brt commented Jun 24, 2024

ieuna commented Jun 24, 2024 • edited Loading

ravinderkumarsap commented Dec 3, 2023 •

edited

Loading

ieuna commented Jun 24, 2024 •

edited

Loading