INSPIRE JRC validation #2444

PascalLike · 2017-12-19T12:35:40Z

This pull request introduces an extra validation functionality for ISO19139 metadata, based on ETF web application.

The validation is designed like a remote validation service, by using the API on a configured ETF web application. This ETF endpoint must be configured in settings:

A check on the checkbox and a valid endpoint are both required.

The interaction with this endpoint works in two steps, implemented in two new methods in GeoNetwork API:

The first method is used for the submission of a metadata record to the remote service for the validation.
The validation is not in real time, so the second method is needed to check asynchronously the progress.

When the INSPIRE validation is set in GeoNetwork settings, the validation button change in a dropdown with local validation and INSPIRE one:

When an INSPIRE validation is completed this panel is shown:

With the exit status of the validation and a link to a report with a detailed report (on the ETF web application):

This pull request is made with the intention to improve the integration of INSPIRE directives in GeoNetwork.

PascalLike · 2017-12-19T12:49:40Z

@fxprunayre and @josegar74 please check if it's ok.

josegar74 · 2017-12-20T07:34:07Z

web-ui/src/main/resources/catalog/locales/en-admin.json

@@ -772,6 +772,8 @@
    "system/inspire/atomDisabled": "Disabled",
    "system/inspire/enable-help": "Enable INSPIRE CSW (ie. language support and INSPIRE GetCapabilities document) and INSPIRE indexing. INSPIRE themes thesaurus MUST be installed to properly index themes and annexes. Note: It does not enable the INSPIRE editor view mode (see iso19139/layout/config-editor.xml).",
    "system/inspire/enableSearchPanel-help": "Enable search panel (Ext search only).",
+    "system/inspire/remotevalidation/url": "INSPIRE remote validation URL",


I would add also a placeholder value for the field with the url of actual service, for now is in a sandbox, but can be updated later when the service is "official". In any case the users can setup the service locally also.

This should help the users with the url format. Placeholder: http://inspire-sandbox.jrc.ec.europa.eu/etf-webapp/

josegar74 · 2017-12-20T07:35:45Z

services/src/main/java/org/fao/geonet/api/records/InspireValidationApi.java

+        try {
+            Element md = (Element) ApiUtils.getUserSession(session).getProperty(Geonet.Session.METADATA_EDITING + id);
+            if (md == null) {
+                throw new ResourceNotFoundException(String.format("Requested metadata with id '%s' is not available in current session. "


Check to move the message to message bundle files. Not sure if done in other services, but should be good to use them to provide localised error messages.

josegar74 · 2017-12-20T07:35:59Z

services/src/main/java/org/fao/geonet/api/records/InspireValidationApi.java

@@ -0,0 +1,237 @@
+package org.fao.geonet.api.records;


Add file header

josegar74 · 2017-12-20T07:38:40Z

services/src/main/java/org/fao/geonet/api/records/editing/InspireValidatorUtils.java

@@ -0,0 +1,494 @@
+package org.fao.geonet.api.records.editing;


Add file header.

Also a unit test for this class can be useful.

josegar74 · 2017-12-20T07:39:55Z

...main/resources/catalog/components/inspirevalidationservice/partials/mdInspireValidation.html

@@ -0,0 +1,44 @@
+<span>


Check the indentation of the file, afaik we use 2 spaces in html files.

fxprunayre · 2017-12-20T07:46:55Z

services/src/main/java/org/fao/geonet/api/records/InspireValidationApi.java

+                    + "This activates an asyncronous process, this method does not return any report. "
+                    + "This method returns an id to be used to get the report.",
+                    nickname = "submitValidate")
+    @RequestMapping(value = "/{metadataUuid}/validate/submit",


Would it be better to have the API like

PUT /srv/api/0.1/records/{metadataUuid}/validate/internal (current one XSD + schematron) PUT /srv/api/0.1/records/{metadataUuid}/validate/inspire

?

So we can add new validator in the long term.

fxprunayre · 2017-12-20T07:53:57Z

...main/resources/catalog/components/inspirevalidationservice/GnmdInspireValidationDirective.js

+                  gnPopup.createModal({
+                    class: 'disclaimer-popup',
+                    title: $translate.instant('inspirePopupReportTitle'),
+                    content: '<div>' + $translate.instant('inspirePopupReportText') + status + '</br></br>' + '<a href=\''+url+'\' target=\'_blank\'>'+$translate.instant('inspirePopupReportLink')+'</a></div>'


You should run

cd web-ui mvn clean install -Pjslint

to clean up JS formatting.

fxprunayre · 2017-12-20T07:59:28Z

LGTM, maybe one change to apply to the API ?

In the long term, we should maybe rework the validation to make them more configurable in the DataManager but we should wait for Maria's refactor first.

fxprunayre · 2017-12-20T08:05:49Z

This is related to #2380

PascalLike · 2017-12-20T20:06:46Z

Thanks @josegar74 @fxprunayre , fixed 👍

fxprunayre · 2018-01-05T17:49:22Z

services/src/main/java/org/fao/geonet/api/records/InspireValidationApi.java

+                    + "This activates an asyncronous process, this method does not return any report. "
+                    + "This method returns an id to be used to get the report.",
+                    nickname = "submitValidate")
+    @RequestMapping(value = "/{metadataUuid}/validate/inspire/submit",


Thanks Antonio for the API layout change. I would even remove /submit and /check as PUT will mean start and GET check ? What do you think ?

fxprunayre

Need a rebase too. It would be good to merge this next week.

PascalLike · 2018-01-08T09:45:45Z

@fxprunayre, why it's needed a rebase?
I did requested changes on REST methods signatures.
The checks failed, but seems to be a travis issue.

PascalLike · 2018-01-08T12:08:57Z

I did a rebase (juan told me that there where present conficts on the reviewer interface) and cleaned the commit history

fxprunayre · 2018-01-08T18:15:52Z

Testing with http://inspire-sandbox.jrc.ec.europa.eu/etf-webapp, I've quite often 4 times for 5 calls

A null test id is returned. Do you have an idea why ? Did you test with local docker install of ETF ?

It's also quite slow around 2min for a successfull or not call.

PascalLike · 2018-01-09T08:37:27Z

I'll investigate how is managed this error scenario

I suggest you to use the docker image. The official etf-webapp is slower and this time I suspect not working
docker run --name etf -d -p 80:8080 -v ~/etf:/etf iide/etf-webapp:latest
(It need some time to be ready after started)

PascalLike · 2018-01-09T17:03:08Z

@fxprunayre , I managed the case where etf returns null but with no HTTP error code. Please let me know if it works now.
The validation task status is checked every 10 seconds, do you think it's too slow?

Delawen · 2018-01-11T12:29:22Z

@fxprunayre if you still think this is ok, I will merge. You were the one experiencing weird behaviour.

josegar74 · 2018-01-17T11:31:29Z

web-ui/src/main/resources/catalog/components/catalog/CatalogService.js

@@ -421,6 +421,7 @@
          isXLinkEnabled: 'system.xlinkResolver.enable',
          isSelfRegisterEnabled: 'system.userSelfRegistration.enable',
          isFeedbackEnabled: 'system.userFeedback.enable',
+          isInspireEnabled: 'system.inspireValidation.enable',


This is in a comment section, but the value is not fine, should be the same as in line 440: isInspireEnabled: 'system.inspire.enable',

josegar74 · 2018-01-25T07:24:47Z

@PascalLike something to check also would be the naming of the directive/html template as currently seem handling both the internal and INSPIRE validation, but the naming of the files/directive refers only to INSPIRE and it's confusing users.

Apologies as I didn't notice in the review about this.

PascalLike · 2018-01-29T09:35:15Z

@josegar74 I fixed this here #2507

PascalLike changed the title ~~Integrate INSPIRE JRC validation~~ INSPIRE JRC validation Dec 19, 2017

josegar74 reviewed Dec 20, 2017

View reviewed changes

josegar74 requested a review from fxprunayre December 20, 2017 07:43

josegar74 added this to the 3.4.2 milestone Dec 20, 2017

fxprunayre requested changes Dec 20, 2017

View reviewed changes

fxprunayre added the enhancement label Dec 20, 2017

fxprunayre reviewed Jan 5, 2018

View reviewed changes

fxprunayre approved these changes Jan 5, 2018

View reviewed changes

fxprunayre reviewed Jan 5, 2018

View reviewed changes

PascalLike force-pushed the etf-remote-validation branch from 73eaa57 to 8fe5c64 Compare January 8, 2018 12:06

PascalLike added 3 commits January 9, 2018 15:20

API for remote validation

d68d9dd

Inspire Validation Service - Metadata editor directive and integration

eb8d209

Inspire Validation - Check for null test ids

6ff94f8

PascalLike force-pushed the etf-remote-validation branch from 943d211 to 6ff94f8 Compare January 9, 2018 16:08

josegar74 reviewed Jan 17, 2018

View reviewed changes

fxprunayre merged commit 9e9c543 into geonetwork:3.4.x Jan 22, 2018

PascalLike deleted the etf-remote-validation branch January 23, 2018 08:02

PascalLike mentioned this pull request Jan 29, 2018

Rename inspire validator directive to a generic one #2507

Merged

pvgenuchten mentioned this pull request Jun 24, 2019

make reports available in machine parsable format etf-validator/governance#79

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

INSPIRE JRC validation #2444

INSPIRE JRC validation #2444

PascalLike commented Dec 19, 2017 •

edited

Loading

PascalLike commented Dec 19, 2017

josegar74 Dec 20, 2017

josegar74 Dec 20, 2017

josegar74 Dec 20, 2017

josegar74 Dec 20, 2017

josegar74 Dec 20, 2017

fxprunayre Dec 20, 2017

fxprunayre Dec 20, 2017

fxprunayre commented Dec 20, 2017

fxprunayre commented Dec 20, 2017

PascalLike commented Dec 20, 2017

fxprunayre Jan 5, 2018

fxprunayre left a comment

PascalLike commented Jan 8, 2018

PascalLike commented Jan 8, 2018

fxprunayre commented Jan 8, 2018

PascalLike commented Jan 9, 2018

PascalLike commented Jan 9, 2018

Delawen commented Jan 11, 2018

josegar74 Jan 17, 2018

josegar74 commented Jan 25, 2018

PascalLike commented Jan 29, 2018

		@@ -0,0 +1,494 @@
		package org.fao.geonet.api.records.editing;

INSPIRE JRC validation #2444

INSPIRE JRC validation #2444

Conversation

PascalLike commented Dec 19, 2017 • edited Loading

PascalLike commented Dec 19, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fxprunayre commented Dec 20, 2017

fxprunayre commented Dec 20, 2017

PascalLike commented Dec 20, 2017

Choose a reason for hiding this comment

fxprunayre left a comment

Choose a reason for hiding this comment

PascalLike commented Jan 8, 2018

PascalLike commented Jan 8, 2018

fxprunayre commented Jan 8, 2018

PascalLike commented Jan 9, 2018

PascalLike commented Jan 9, 2018

Delawen commented Jan 11, 2018

Choose a reason for hiding this comment

josegar74 commented Jan 25, 2018

PascalLike commented Jan 29, 2018

PascalLike commented Dec 19, 2017 •

edited

Loading