Dynapi parameters, io re-engineering #319

litvinovg · 2022-07-20T16:31:18Z

What does this pull request do?

Align dynamic API ontology class names with convention
Convert dynamic API ontology to n3
renamed DefaultAction into NullAction
Re-engineering of OperationData and Converters in progress
Use jsonPath to load data from requests
Use json schema validation to validate input and output json.
Resource id is injected into input json body before processing by JsonConverter
Parameter can be declared as local. Local parameters are auto-created by api before processing input data.
ModelComponent was removed. Instead of ModelComponent DynapiModelFactory is used to create links to Vitro|VIVO models. Links are created by API for each request it required as done for all other local parameters.
Old data types were removed
ParameterTypes now store SerializationType and ImplementationType, also can contain type in case of arrays.
ImplementationType store serialisation and deserialization configs and class name to be created as a result of de-serialization.
View classes created to sort Data in DataStore, get data in most specific types required for components
Created Implementation configurations to store information how serialization and de-serialization should be done
Parameter conversion is done with information stored in ImplementationConfig. ConversionMethod that should do the conversion is created during Action initialization and stored in ImplementationType. Added tests for conversion.
Old converters were removed
requiredParameters were renamed to InputParameters
providedParameters were renamed to OutputParameters
DataStore replaced OperationData
Data class was removed and replaced with new Data class that stores Object data and serialized value provided by converters.
Defined parameter type JsonObjectArrayOfPairs to store all results of SPARQL Query.
Created ModelWriter component + unit and integration tests
Implemented content type negotiation in converter, added tests

… check action json paths, if not set use default.

…Type

…moved rdf related checks from Parameters class

…s and providedParams to outputParams

…RQL Query and N3Template

chenejac

@litvinovg I have reviewed your PR, it looks very well. I have provided a couple of comments. Moreover, the list of classes in the data.types package is a little bit strange, maybe that should be split into two packages.

home/src/main/resources/rdf/tbox/filegraph/vitro-dynamic-api.n3

api/src/main/java/edu/cornell/mannlib/vitro/webapp/dynapi/AbstractPool.java

api/src/main/java/edu/cornell/mannlib/vitro/webapp/dynapi/ActionPool.java

api/src/main/java/edu/cornell/mannlib/vitro/webapp/dynapi/DynamicAPIDocumentation.java

chenejac · 2022-08-22T12:05:42Z

api/src/main/java/edu/cornell/mannlib/vitro/webapp/dynapi/components/Action.java

+    public Parameters getInternalParams() {
+        return internalParams;
+    }


What is the idea for definition of those internalParams? Is that also defined in n3 description of a dynamic action?

Parameters not required in request, but created for each action. VIVO models is one of such examples

api/src/main/java/edu/cornell/mannlib/vitro/webapp/dynapi/components/Parameter.java

chenejac · 2022-08-22T12:33:43Z

...ornell/mannlib/vitro/webapp/dynapi/components/serialization/JsonObjectSerializationType.java

+import edu.cornell.mannlib.vitro.webapp.dynapi.components.Parameters;
+import edu.cornell.mannlib.vitro.webapp.utils.configuration.Property;
+
+public class JsonObjectSerializationType extends SerializationType {


Why part of this serialization type is Json, and it is not the case for primitive and array serialization types (other classes in this package)?

It represents tree structures. So it should be separate, but maybe renamed in future.

chenejac · 2022-08-22T12:38:51Z

api/src/main/java/edu/cornell/mannlib/vitro/webapp/dynapi/components/validators/IsInteger.java

+			}
+    	} else {
+    		try {
+				if (!isInteger(data.getJsonValue())) {


Not sure this is the best way to decouple validator and data, I mean here is explicitly mentioned JSON format.

Not really. It is serialized value, I will rename the method. It is also a json representation. So the validator always work with Strings.
Still we will need data specific validators to be applied only to specific data types. That should be resolved in future.

chenejac · 2022-08-22T12:47:52Z

api/src/main/java/edu/cornell/mannlib/vitro/webapp/dynapi/data/SimpleDataView.java

+import edu.cornell.mannlib.vitro.webapp.dynapi.components.Parameter;
+import edu.cornell.mannlib.vitro.webapp.dynapi.components.Parameters;
+
+public class SimpleDataView {


This is not an extension of View?

No, interface wasn't used. Should be removed.

chenejac · 2022-08-22T12:50:31Z

api/src/main/java/edu/cornell/mannlib/vitro/webapp/dynapi/data/View.java

@@ -0,0 +1,5 @@
+package edu.cornell.mannlib.vitro.webapp.dynapi.data;
+
+public interface View {


It would be better that we can really define some interface here (I mean some methods), in this way I am not sure what is the benefit of making hierarchy of object implementing this view

I don't think it is possible. I think that interface should be removed.

chenejac · 2022-08-22T13:14:55Z

Parameter can be declared as local. Local parameters are auto-created by api before processing input data.

Is this idea for InternalParameters?

litvinovg · 2022-08-22T14:31:13Z

Parameter can be declared as local. Local parameters are auto-created by api before processing input data.

Is this idea for InternalParameters?
Yes

litvinovg · 2022-08-22T17:12:40Z

@litvinovg I have reviewed your PR, it looks very well. I have provided a couple of comments. Moreover, the list of classes in the data.types package is a little bit strange, maybe that should be split into two packages.

Yes. I split that files into two packages.

chenejac

It looks ok for me now. I still don't like the name JsonObjectSerializationType, but we can consider renaming that later.

* renamed classes * Converted dynamic api ontology to n3 * renamed DefaultAction to NullAction * Moved RESOURCE_ID to RESTEndpoint class * added data view classes * local parameters * started working with json converter * Json validation and jsonPath data loading * get serialized type from ParameterType * tests: fixes for REST endpoint integration test * set required parameters in json schema * inject resource id from url path into json request before validation * refact: move prepareResponse to Converter class from OperationResult * if no acceptContentTypes found use the same content type as input * tests fixes, refactoring, form data converter improvements * Started working on JSON output conversion * Check param json paths for params input/output conversion, if not set check action json paths, if not set use default. * save initializer draft code * fix: fix NPE in RPCEndpointTest * renamed old ParameterType to SerializationType. Created new ParameterType * get serialization info from serialization type name * Extracted RDFType from SerializationType * refact: created isLiteral, isUri, isRdfType methods in ParamterType, moved rdf related checks from Parameters class * feat: added implementation type * added parameter implementation configuration * created parameter converter * array conversion * Removed old converters, operation data, model component. * refact: renamed RawData to Data, renamed requiredParams to inputParams and providedParams to outputParams * Refactored parameter converter. Cache conversion method. * added ModelWriter, restricted access to Data.setObject * content type negotiation * n3 syntax fix * created a few SPARQL query tests, refactored and improved code in SPARQL Query and N3Template * fixes related to review comments * refact: moved implementation related classes to separate package * added sparql query uri input test * refact: removed unused imports and variables. * restricted Data.getObject access modifier to protected

litvinovg added 4 commits July 11, 2022 17:04

renamed classes

4303b49

Converted dynamic api ontology to n3

a152e8c

renamed DefaultAction to NullAction

6166d33

Moved RESOURCE_ID to RESTEndpoint class

18bae04

litvinovg force-pushed the params branch 4 times, most recently from c9d7ac5 to 428b182 Compare July 29, 2022 13:20

litvinovg force-pushed the params branch from a46a105 to 872409a Compare August 10, 2022 15:45

litvinovg marked this pull request as ready for review August 10, 2022 15:45

litvinovg added 20 commits August 15, 2022 16:11

added data view classes

8aafac7

local parameters

19e8f20

started working with json converter

12480d8

Json validation and jsonPath data loading

bafa09d

get serialized type from ParameterType

c17760a

tests: fixes for REST endpoint integration test

fe953f7

set required parameters in json schema

cf7b2c6

inject resource id from url path into json request before validation

552a333

refact: move prepareResponse to Converter class from OperationResult

3c760c4

if no acceptContentTypes found use the same content type as input

875cfee

tests fixes, refactoring, form data converter improvements

74ece92

Started working on JSON output conversion

0e4f7da

Check param json paths for params input/output conversion, if not set…

f907e42

… check action json paths, if not set use default.

save initializer draft code

ec33799

fix: fix NPE in RPCEndpointTest

25d6c5b

renamed old ParameterType to SerializationType. Created new Parameter…

679f65d

…Type

get serialization info from serialization type name

1f6488e

Extracted RDFType from SerializationType

ba4d332

refact: created isLiteral, isUri, isRdfType methods in ParamterType, …

4cfe461

…moved rdf related checks from Parameters class

feat: added implementation type

63fc974

litvinovg added 6 commits August 15, 2022 16:13

added parameter implementation configuration

055f155

created parameter converter

9aa1316

array conversion

cf96468

Removed old converters, operation data, model component.

e159ef0

refact: renamed RawData to Data, renamed requiredParams to inputParam…

4156cd9

…s and providedParams to outputParams

Refactored parameter converter. Cache conversion method.

cb98b82

litvinovg force-pushed the params branch from 229d157 to cb98b82 Compare August 15, 2022 14:13

litvinovg added 4 commits August 16, 2022 15:12

added ModelWriter, restricted access to Data.setObject

2ec3d52

content type negotiation

bd5eac7

n3 syntax fix

3b4c6e4

created a few SPARQL query tests, refactored and improved code in SPA…

b77a780

…RQL Query and N3Template

chenejac requested changes Aug 22, 2022

View reviewed changes

fixes related to review comments

646599a

refact: moved implementation related classes to separate package

e138a5d

litvinovg added 3 commits August 22, 2022 19:28

added sparql query uri input test

cd36a8c

refact: removed unused imports and variables.

075110f

restricted Data.getObject access modifier to protected

0211022

chenejac approved these changes Aug 23, 2022

View reviewed changes

chenejac merged commit 5a1e1f5 into vivo-project:sprint-dynapi-2022-feb-staging Aug 23, 2022

This was referenced Sep 14, 2022

Working on SPARQQuery Mocktest #306

Closed

N3 operation retractions #310

Closed

chenejac mentioned this pull request Oct 6, 2022

3623 validation of data #287

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Dynapi parameters, io re-engineering #319

Dynapi parameters, io re-engineering #319

litvinovg commented Jul 20, 2022 •

edited

chenejac left a comment

chenejac Aug 22, 2022

litvinovg Aug 22, 2022

chenejac Aug 22, 2022

litvinovg Aug 22, 2022

chenejac Aug 22, 2022

litvinovg Aug 22, 2022

chenejac Aug 22, 2022

litvinovg Aug 22, 2022

chenejac Aug 22, 2022

litvinovg Aug 22, 2022

chenejac commented Aug 22, 2022

litvinovg commented Aug 22, 2022

litvinovg commented Aug 22, 2022

chenejac left a comment

		@@ -0,0 +1,5 @@
		package edu.cornell.mannlib.vitro.webapp.dynapi.data;

		public interface View {

Dynapi parameters, io re-engineering #319

Dynapi parameters, io re-engineering #319

Conversation

litvinovg commented Jul 20, 2022 • edited

What does this pull request do?

chenejac left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chenejac commented Aug 22, 2022

litvinovg commented Aug 22, 2022

litvinovg commented Aug 22, 2022

chenejac left a comment

Choose a reason for hiding this comment

litvinovg commented Jul 20, 2022 •

edited