feature(api-v2): Generate client test data for ontologies, resources, search, and lists #1549

benjamingeer · 2019-12-06T09:18:18Z

This PR incorporates more API v2 routes into the client code generation framework so they can return client test data.

ontologies route
resources route
search route
lists route

Resolves #1548.

tobiasschweizer · 2019-12-10T10:01:54Z

In v2 test data for resources, I cannot find JSON data for creation and metadata update ops.

tobiasschweizer · 2019-12-10T10:04:30Z

see https://github.com/dasch-swiss/knora-api-js-lib/blob/9e16288916c2e2a5a66b7d0d9f4448c7dd911438/src/api/v2/resource/resources-endpoint.spec.ts#L96-L648 for the tests I have written so far (test data hard coded)

tobiasschweizer · 2019-12-10T10:06:30Z

I think the the values themselves do not have to be tested thoroughly because this is already done in https://github.com/dasch-swiss/knora-api-js-lib/blob/wip/create-resource/src/api/v2/values/values-endpoint.spec.ts with test data generated by Knora.

benjamingeer · 2019-12-10T10:06:56Z

In v2 test data for resources, I cannot find JSON data for creation and metadata update ops.

I just included the stuff you asked for 😜:

#1546 (comment)

benjamingeer · 2019-12-10T10:08:25Z

Will add test data for the update operations, too.

tobiasschweizer · 2019-12-10T10:09:32Z

Will add test data for the update operations, too.

Thanks!

tobiasschweizer · 2019-12-10T10:15:39Z

In the meantime I am changing from static to generated v2 test data files. Works already for the ontologies! :-)

My goal is to get rid of static test data at all.

tobiasschweizer · 2019-12-10T10:31:51Z

there is a failing test in ExceptionHandlerR2RSpec: https://github.com/dasch-swiss/knora-api/pull/1549/checks?check_run_id=341570711#step:6:1310

tobiasschweizer · 2019-12-10T15:05:35Z

webapi/src/main/scala/org/knora/webapi/routing/v2/ListsRouteV2.scala

+    override val description: String = "An endpoint for working with Knora lists."
+    override val functions: Seq[ClientFunction] = Seq.empty
+
+    def knoraApiPath: Route = getList ~ getNode


Could you add a comment that this is actually the route definition?

tobiasschweizer · 2019-12-10T15:06:49Z

webapi/src/main/scala/org/knora/webapi/routing/v2/OntologiesRouteV2.scala

    private val ALL_LANGUAGES = "allLanguages"
    private val LAST_MODIFICATION_DATE = "lastModificationDate"

-    def knoraApiPath: Route = {
+    def knoraApiPath: Route = dereferenceOntologyIri ~ getOntologyMetadata ~ updateOntologyMetadata ~ getOntologyMetadataForProjects ~


Could you add a comment that this is actually the route definition?

tobiasschweizer · 2019-12-10T15:07:24Z

webapi/src/main/scala/org/knora/webapi/routing/v2/ResourcesRouteV2.scala

@@ -127,138 +147,61 @@ class ResourcesRouteV2(routeData: KnoraRouteData) extends KnoraRoute(routeData)
        }
    }

+    def knoraApiPath: Route = createResource ~ updateResourceMetadata ~ getResourcesInProject ~


Could you add a comment that this is actually the route definition?

tobiasschweizer · 2019-12-10T15:07:41Z

webapi/src/main/scala/org/knora/webapi/routing/v2/SearchRouteV2.scala

    private val LIMIT_TO_PROJECT = "limitToProject"
    private val LIMIT_TO_RESOURCE_CLASS = "limitToResourceClass"
    private val OFFSET = "offset"
    private val LIMIT_TO_STANDOFF_CLASS = "limitToStandoffClass"

+    def knoraApiPath: Route = fullTextSearchCount ~ fullTextSearch ~ gravsearchCountGet ~ gravsearchCountPost ~


Could you add a comment that this is actually the route definition?

benjamingeer · 2019-12-10T15:15:45Z

there is a failing test in ExceptionHandlerR2RSpec

That's not a failing test, the test is supposed to throw an exception. The GitHub CI build is actually failing here, but I don't know why:

https://github.com/dasch-swiss/knora-api/pull/1549/checks?check_run_id=341570711#step:6:2012

benjamingeer · 2019-12-11T12:19:04Z

Could you add a comment that this is actually the route definition?

I've added it to all routes.

I added more test requests/responses for the resources route. Do you want update requests for the ontologies route too?

tobiasschweizer · 2019-12-11T12:38:38Z

I've added it to all routes.

Great, thanks.

I added more test requests/responses for the resources route. Do you want update requests for the ontologies route too?

Yes, we will need that too.

Could you also create JSON test data for a resource preview (response to a creation request)?

tobiasschweizer · 2019-12-11T12:44:36Z

The preview JSON data has a higher priority to finish dasch-swiss/dsp-js-lib#115, so if you need more time for the ontologies and want to do this in a subsequent PR, that's also fine.

benjamingeer · 2019-12-11T14:41:22Z

if you need more time for the ontologies and want to do this in a subsequent PR

Yes, because I'd like to finish #1509 first.

tobiasschweizer · 2019-12-11T14:48:53Z

When I access http://localhost:3333/clientapi/typescript I now get

{
"error": "akka.http.scaladsl.model.IllegalUriException: Illegal URI reference: Invalid input '/', expected userinfo-char, pct-encoded or '@' (line 1, column 36): http://0.0.0.0:3333resourcespreview/http%3A%2F%2Frdfh.ch%2F0001%2Fa-thing\n ^"
}

tobiasschweizer · 2019-12-11T15:30:30Z

"anything:hasDecimal" : {
    "@type" : "knora-api:DecimalValue",
    "knora-api:decimalValueAsDecimal" : {
      "@type" : "x  sd:decimal",
      "@value" : "100000000000000.000000000000001"
    }
  }

I think this is too much precision, JS serializes this to "100000000000000".

tobiasschweizer · 2019-12-11T15:34:10Z

Maybe I could look at https://github.com/royNiladri/js-big-decimal or some similar lib.

tobiasschweizer · 2019-12-11T15:52:02Z

if possible, could you also generate data for

update a resource's metadata submitting the last modification date
response to metadata update
response to resource deletion

benjamingeer · 2019-12-11T16:28:56Z

"error": "akka.http.scaladsl.model.IllegalUriException

Sorry, stupid mistake.

tobiasschweizer · 2019-12-12T07:56:52Z

When I try to generate client api code I now get:

{"error":"org.knora.webapi.ClientApiGenerationException: Failed to get test data: The requested resource could not be found."}

However, I can get http://localhost:3333/v2/resourcespreview/http%3A%2F%2Frdfh.ch%2F0001%2Fa-thing

tobiasschweizer · 2019-12-12T08:46:13Z

webapi/src/main/scala/org/knora/webapi/routing/v2/ResourcesRouteV2.scala

@@ -376,7 +376,7 @@ class ResourcesRouteV2(routeData: KnoraRouteData) extends KnoraRoute(routeData)

    private def getResourcesPreviewTestResponse: Future[SourceCodeFileContent] = {
        for {
-            responseStr <- doTestDataRequest(Get(s"${baseApiUrl}resourcespreview/${SharedTestDataADM.AThing.iriEncoded}"))
+            responseStr <- doTestDataRequest(Get(s"$baseApiUrl/resourcespreview/${SharedTestDataADM.AThing.iriEncoded}"))


you are still missing v2: http://0.0.0.0:3333/resourcespreview/http%3A%2F%2Frdfh.ch%2F0001%2Fa-thing

Shall I fix it?

I'm doing it now along with the other stuff you asked for.

Ok, thank you.

benjamingeer · 2019-12-12T08:46:37Z

org.knora.webapi.ClientApiGenerationException

Yet another stupid mistake. I think I need to hibernate for the winter.

Can I change the file extensions to .jsonld instead of .json, or would that be a pain?

tobiasschweizer · 2019-12-12T08:53:31Z

Can I change the file extensions to .jsonld instead of .json, or would that be a pain?

I experienced some problems earlier with that extension in the past when loading the files in a JS env. But we can give it a try in a separate PR. I am supposed to publish a new version of knora-api-js-lib today for @lrosenth, so I just want to finish resource creation.

benjamingeer · 2019-12-12T09:05:43Z

OK please try it now.

tobiasschweizer · 2019-12-12T09:53:06Z

Ok, that's all I need to finish dasch-swiss/dsp-js-lib#115, thanks!

I spotted a potential issue for the automatic integration tests in Knora we want to set up (#1554): some of the data is dynamic (such as creation / modification dates) and the tests will fail because the assertions are hard-coded (but they won't fail because of an actual problem we would have to fix).

benjamingeer · 2019-12-12T10:25:27Z

some of the data is dynamic (such as creation / modification dates)

I think I've fixed this now. Let me know if it's OK to merge.

tobiasschweizer · 2019-12-12T11:44:07Z

I think I've fixed this now. Let me know if it's OK to merge.

Works :-)

benjamingeer · 2019-12-12T11:49:37Z

Works :-)

Did you forget to press the "Approve" button, or is there something else I need to do?

tobiasschweizer

Ok, I did a final check. Good to go!

benjamingeer · 2019-12-12T12:29:35Z

Thanks for your patience!

tobiasschweizer · 2019-12-12T13:49:00Z

Thanks for your patience!

No worries!

tobiasschweizer · 2019-12-12T14:11:19Z

Once this is merged, could you update the branches for the modification date and the time value in Knora. I then have to regenerate the client data from these branches to integrate them in the PR is knora-api-js-lib.

feature(api-v2): Return API v2 test data for resources (ongoing).

cde41cd

benjamingeer self-assigned this Dec 6, 2019

benjamingeer added API/V2 clientapi frontend: Salsah, DSP-APP, BEOL, MLS, etc. labels Dec 6, 2019

benjamingeer added this to the 2019-12 milestone Dec 6, 2019

Benjamin Geer added 3 commits December 9, 2019 10:51

test(clientapi): Add more test data.

83d150c

test(clientapi): Load incunabula test data.

cec2da3

test(clientapi): Return more test data.

5e12353

benjamingeer changed the title ~~feature(api-v2): Generate clientapi test data for resources, ontologies, and lists~~ feature(api-v2): Generate clientapi test data for ontologies, resources, search, and lists Dec 9, 2019

benjamingeer changed the title ~~feature(api-v2): Generate clientapi test data for ontologies, resources, search, and lists~~ feature(api-v2): Generate client test data for ontologies, resources, search, and lists Dec 9, 2019

test(clientapi): Return more test data.

fa7f264

benjamingeer requested a review from tobiasschweizer December 9, 2019 14:06

benjamingeer mentioned this pull request Dec 9, 2019

Ben's PR history #571

Open

Merge branch 'develop' into wip/1548-clientapi-testdata

9e0951b

tobiasschweizer reviewed Dec 10, 2019

View reviewed changes

Benjamin Geer added 3 commits December 10, 2019 16:35

style(routing): Add consistent comment to all knoraApiPath methods.

31e2846

build(routing): Fix compile error caused by reformatting.

2386d08

test(clientapi): Add more test data.

608d317

Benjamin Geer added 2 commits December 11, 2019 15:30

test(clientapi): Increase timeout.

44ecb54

test(clientapi): Add resource preview test response.

62caf30

tobiasschweizer mentioned this pull request Dec 11, 2019

add method to create a resource dasch-swiss/dsp-js-lib#115

Merged

Benjamin Geer added 2 commits December 11, 2019 17:23

Merge branch 'develop' into wip/1548-clientapi-testdata

e6f5b00

fix(clientapi): Fix bad URL.

6273366

tobiasschweizer reviewed Dec 12, 2019

View reviewed changes

test(clientapi): Fix silly mistakes, add more test data.

b40aefb

fix(clientapi): Use static modification dates.

88b9141

tobiasschweizer approved these changes Dec 12, 2019

View reviewed changes

benjamingeer merged commit aa6b773 into develop Dec 12, 2019

benjamingeer deleted the wip/1548-clientapi-testdata branch December 12, 2019 14:28

feature(api-v2): Generate client test data for ontologies, resources, search, and lists #1549

feature(api-v2): Generate client test data for ontologies, resources, search, and lists #1549

Conversation

benjamingeer commented Dec 6, 2019 • edited

tobiasschweizer commented Dec 10, 2019

tobiasschweizer commented Dec 10, 2019

tobiasschweizer commented Dec 10, 2019

benjamingeer commented Dec 10, 2019

benjamingeer commented Dec 10, 2019

tobiasschweizer commented Dec 10, 2019

tobiasschweizer commented Dec 10, 2019

tobiasschweizer commented Dec 10, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

benjamingeer commented Dec 10, 2019

benjamingeer commented Dec 11, 2019

tobiasschweizer commented Dec 11, 2019

tobiasschweizer commented Dec 11, 2019

benjamingeer commented Dec 11, 2019

tobiasschweizer commented Dec 11, 2019

tobiasschweizer commented Dec 11, 2019 • edited

tobiasschweizer commented Dec 11, 2019

tobiasschweizer commented Dec 11, 2019

benjamingeer commented Dec 11, 2019 • edited

tobiasschweizer commented Dec 12, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

benjamingeer commented Dec 12, 2019

tobiasschweizer commented Dec 12, 2019

benjamingeer commented Dec 12, 2019

tobiasschweizer commented Dec 12, 2019

benjamingeer commented Dec 12, 2019

tobiasschweizer commented Dec 12, 2019

benjamingeer commented Dec 12, 2019

tobiasschweizer left a comment

Choose a reason for hiding this comment

benjamingeer commented Dec 12, 2019

tobiasschweizer commented Dec 12, 2019

tobiasschweizer commented Dec 12, 2019

benjamingeer commented Dec 6, 2019 •

edited

tobiasschweizer commented Dec 11, 2019 •

edited

benjamingeer commented Dec 11, 2019 •

edited