feat(document): document deserialization & serialization #3066

chillleader · 2024-08-07T19:47:22Z

Description

This PR will add the first implementation of serialization and deserialization of document references.

Highlights:

Jackson deserializer to handle the simple case when connector wants to get a Document object
Jackson deserializer to handle more complex cases when target type is not a Document but the document can be converted to the target type
Operation execution
- Lazy and eager models are available
- Lazy is applied by default when the connector doesn't specify its target type (e.g. just Object). The operation will be evaluated on serialization instead in this case.
Jackson serializer that can execute operations
Refined document class structure
DocumentStore interface and in-memory impl

Related issues

related https://github.com/camunda/team-connectors/issues/873

(cherry picked from commit a57e940)

(cherry picked from commit 5634aea)

(cherry picked from commit 92ec79f)

(cherry picked from commit 5a808bd)

(cherry picked from commit 0b0a558)

...core/src/main/java/io/camunda/connector/runtime/core/document/DocumentOperationExecutor.java

...c/main/java/io/camunda/connector/runtime/core/document/jackson/DocumentDeserializerBase.java

...ntime-core/src/main/java/io/camunda/connector/runtime/core/document/DocumentFactoryImpl.java

...ore/src/main/java/io/camunda/connector/api/document/operation/DocumentOperationExecutor.java

.../io/camunda/connector/document/annotation/jackson/deserializer/DocumentDeserializerBase.java

...ors/http/http-base/src/main/java/io/camunda/connector/http/base/model/HttpCommonRequest.java

mathias-vandaele · 2024-08-16T12:42:19Z

...ntime-core/src/main/java/io/camunda/connector/runtime/core/document/DocumentFactoryImpl.java

+  private final DocumentStore documentStore;
+
+  public DocumentFactoryImpl(DocumentStore documentStore) {
+    this.documentStore = documentStore;


Will this class carry all of the future possible store ? In this case, how do you make the link between the document creation request and the corresponding store?

In the future we will generally use only one store here for connecting to the C8 document API. The in-memory implementation will be reserved for tests. As of now, I don't see the need to have multiple store implementations active at the same time - different stores are anyway hidden behind the C8 REST API.

One thing where a different approach is needed is external documents (such as something only available to us via an external URL). But this probably will be handled separately in the DocumentFactory

mathias-vandaele · 2024-08-16T12:52:26Z

...runtime-core/src/main/java/io/camunda/connector/runtime/core/outbound/JobHandlerContext.java

+        job,
+        secretProvider,
+        validationProvider,
+        new DocumentFactoryImpl(new InMemoryDocumentStore()),


How will it work when new store comes?

connector-sdk/core/src/main/java/io/camunda/connector/api/document/DocumentFactory.java

sbuettner · 2024-08-19T07:35:37Z

connector-sdk/core/src/main/java/io/camunda/connector/api/document/DocumentReference.java

+ */
+package io.camunda.connector.api.document;
+
+public interface DocumentReference {


Should we seal this interface?

It's being extended / implemented by classes in a different package (DocumentReferenceModel where operation is added) so I couldn't seal it here

connector-sdk/core/src/main/java/io/camunda/connector/api/document/store/DocumentStore.java

sbuettner

Great work! Some minor comments.

...t/src/main/java/io/camunda/connector/document/annotation/jackson/DocumentReferenceModel.java

* wip (cherry picked from commit a57e940) * wip (cherry picked from commit 5634aea) * prepare end-to-end draft (cherry picked from commit 92ec79f) * prepare end-to-end draft (cherry picked from commit 5a808bd) * rework deserializers (cherry picked from commit 0b0a558) * incorporate feedback, move jackson code into a separate module * clean up accidental change * naming adjustments * lint * remove extra dollar sign * remove extra dollar sign

* feat(sdk): first iteration of document support * apply review suggestions * feat(document): document deserialization & serialization (#3066) * wip (cherry picked from commit a57e940) * wip (cherry picked from commit 5634aea) * prepare end-to-end draft (cherry picked from commit 92ec79f) * prepare end-to-end draft (cherry picked from commit 5a808bd) * rework deserializers (cherry picked from commit 0b0a558) * incorporate feedback, move jackson code into a separate module * clean up accidental change * naming adjustments * lint * remove extra dollar sign * remove extra dollar sign * feat(document): restructure the project and move document stuff to a different package (#3149) * refactor: move document stuff to a different module --------- Co-authored-by: Jonathan Roques <jonathan.roques@camunda.com> * fix(document): document serializer doesn't handle sdk document model (#3205) * feat(doc-handling): Some investigations to use documents in the REST … (#2976) * feat(doc-handling): Some investigations to use documents in the REST connector --------- Co-authored-by: Pavel Kotelevsky <pavel.kotelevskii@camunda.com> Co-authored-by: Pavel Kotelevsky <38818382+chillleader@users.noreply.github.com>

* wip (cherry picked from commit a57e940) * wip (cherry picked from commit 5634aea) * prepare end-to-end draft (cherry picked from commit 92ec79f) * prepare end-to-end draft (cherry picked from commit 5a808bd) * rework deserializers (cherry picked from commit 0b0a558) * incorporate feedback, move jackson code into a separate module * clean up accidental change * naming adjustments * lint * remove extra dollar sign * remove extra dollar sign

chillleader added 5 commits August 7, 2024 21:38

wip

523a60a

(cherry picked from commit a57e940)

wip

4330114

(cherry picked from commit 5634aea)

prepare end-to-end draft

d476f1e

(cherry picked from commit 92ec79f)

prepare end-to-end draft

04e42f7

(cherry picked from commit 5a808bd)

rework deserializers

f8cbca0

(cherry picked from commit 0b0a558)

chillleader mentioned this pull request Aug 7, 2024

feat(document): serialization & deserialization #3051

Closed

chillleader self-assigned this Aug 7, 2024

chillleader requested review from sbuettner and mathias-vandaele August 7, 2024 19:49

chillleader marked this pull request as ready for review August 7, 2024 19:49

chillleader requested a review from a team as a code owner August 7, 2024 19:49

github-advanced-security bot found potential problems Aug 7, 2024

View reviewed changes

chillleader changed the title ~~Document serialization~~ feat(document): document deserialization & serialization Aug 8, 2024

sbuettner approved these changes Aug 12, 2024

View reviewed changes

incorporate feedback, move jackson code into a separate module

c466d72

chillleader requested a review from sbuettner August 15, 2024 11:30

github-advanced-security bot found potential problems Aug 15, 2024

View reviewed changes

mathias-vandaele reviewed Aug 16, 2024

View reviewed changes

mathias-vandaele approved these changes Aug 16, 2024

View reviewed changes

clean up accidental change

706f591

sbuettner reviewed Aug 19, 2024

View reviewed changes

connector-sdk/core/src/main/java/io/camunda/connector/api/document/DocumentFactory.java Outdated Show resolved Hide resolved

sbuettner reviewed Aug 19, 2024

View reviewed changes

connector-sdk/core/src/main/java/io/camunda/connector/api/document/store/DocumentStore.java Outdated Show resolved Hide resolved

sbuettner approved these changes Aug 19, 2024

View reviewed changes

chillleader added 2 commits August 19, 2024 09:50

naming adjustments

aa44e63

lint

19d0f0e

sbuettner approved these changes Aug 19, 2024

View reviewed changes

sbuettner reviewed Aug 19, 2024

View reviewed changes

...t/src/main/java/io/camunda/connector/document/annotation/jackson/DocumentReferenceModel.java Outdated Show resolved Hide resolved

chillleader added 2 commits August 19, 2024 10:23

remove extra dollar sign

e55a7e6

remove extra dollar sign

a1f2086

chillleader merged commit a8b8e07 into epic-1748-doc-ref-handling Aug 19, 2024
7 checks passed

chillleader deleted the document-serialization branch August 19, 2024 09:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(document): document deserialization & serialization #3066

feat(document): document deserialization & serialization #3066

chillleader commented Aug 7, 2024 •

edited

Loading

mathias-vandaele Aug 16, 2024

chillleader Aug 16, 2024

mathias-vandaele Aug 16, 2024

sbuettner Aug 19, 2024

chillleader Aug 19, 2024 •

edited

Loading

sbuettner left a comment

feat(document): document deserialization & serialization #3066

feat(document): document deserialization & serialization #3066

Conversation

chillleader commented Aug 7, 2024 • edited Loading

Description

Related issues

mathias-vandaele Aug 16, 2024

Choose a reason for hiding this comment

chillleader Aug 16, 2024

Choose a reason for hiding this comment

mathias-vandaele Aug 16, 2024

Choose a reason for hiding this comment

sbuettner Aug 19, 2024

Choose a reason for hiding this comment

chillleader Aug 19, 2024 • edited Loading

Choose a reason for hiding this comment

sbuettner left a comment

Choose a reason for hiding this comment

chillleader commented Aug 7, 2024 •

edited

Loading

chillleader Aug 19, 2024 •

edited

Loading