As a user, I want to validate a bundle that uses multiple versions of the Information Model / Discipline LDDs

[jordanpadams]

...so that I can ensure the validity of the archive, even if I needed to use multiple versions of data dictionaries that may not be compatible.

Describe the bug
When reading in LDDs that are incompatible with older versions of the IM, validate appears to be ignoring the schemaLocation versions.

See #208 for more details

To Reproduce

$ validate-1.20.0/bin/validate -t ~/Downloads/test2/bundle_cassini-huygens-coradar.xml ~/Downloads/test2/BILQH07S314_D065_T008S02_V02.xml

PDS Validate Tool Report

Configuration:
   Version                       1.20.0
   Date                          2020-02-20T01:27:00Z

Parameters:
   Targets                       [file:/Users/jpadams/Downloads/test2/bundle_cassini-huygens-coradar.xml, file:/Users/jpadams/Downloads/test2/BILQH07S314_D065_T008S02_V02.xml]
   Severity Level                WARNING
   Recurse Directories           true
   File Filters Used             [*.xml, *.XML]
   Data Content Validation       on
   Product Level Validation      on
   Max Errors                    100000
   Registered Contexts File      /Users/jpadams/Downloads/validate-1.20.0/resources/registered_context_products.json



Product Level Validation Results

  PASS: file:/Users/jpadams/Downloads/test2/bundle_cassini-huygens-coradar.xml

  FAIL: file:/Users/jpadams/Downloads/test2/BILQH07S314_D065_T008S02_V02.xml
    Begin Schema: https://pds.nasa.gov/pds4/geom/v1/PDS4_GEOM_1B10_1700.xsd
      ERROR  [error.label.schema]   line 383, 82: src-resolve: Cannot resolve the name 'pds:local_identifier' to a(n) 'element declaration' component.
      ERROR  [error.label.schema]   line 573, 74: src-resolve: Cannot resolve the name 'pds:local_identifier' to a(n) 'element declaration' component.
      ERROR  [error.label.schema]   line 605, 74: src-resolve: Cannot resolve the name 'pds:local_identifier' to a(n) 'element declaration' component.
      ERROR  [error.label.schema]   line 637, 74: src-resolve: Cannot resolve the name 'pds:local_identifier' to a(n) 'element declaration' component.
      ERROR  [error.label.schema]   line 669, 74: src-resolve: Cannot resolve the name 'pds:local_identifier' to a(n) 'element declaration' component.
      ERROR  [error.label.schema]   line 1242, 74: src-resolve: Cannot resolve the name 'pds:local_identifier' to a(n) 'element declaration' component.
    End Schema: https://pds.nasa.gov/pds4/geom/v1/PDS4_GEOM_1B10_1700.xsd

Summary:

  6 error(s)
  0 warning(s)

  Message Types:
    6            error.label.schema

End of Report
Completed execution in 4374 ms

$ validate-1.20.0/bin/validate -t ~/Downloads/test2/BILQH07S314_D065_T008S02_V02.xml ~/Downloads/test2/bundle_cassini-huygens-coradar.xml 

PDS Validate Tool Report

Configuration:
   Version                       1.20.0
   Date                          2020-02-20T01:33:29Z

Parameters:
   Targets                       [file:/Users/jpadams/Downloads/test2/BILQH07S314_D065_T008S02_V02.xml, file:/Users/jpadams/Downloads/test2/bundle_cassini-huygens-coradar.xml]
   Severity Level                WARNING
   Recurse Directories           true
   File Filters Used             [*.xml, *.XML]
   Data Content Validation       on
   Product Level Validation      on
   Max Errors                    100000
   Registered Contexts File      /Users/jpadams/Downloads/validate-1.20.0/resources/registered_context_products.json



Product Level Validation Results

  FAIL: file:/Users/jpadams/Downloads/test2/BILQH07S314_D065_T008S02_V02.xml
      ERROR  [error.label.schema]   line 79, 68: cvc-complex-type.2.4.c: The matching wildcard is strict, but no declaration can be found for element 'Mission_Information'.
      ERROR  [error.label.missing_file]   URI reference does not exist: file:/Users/jpadams/Downloads/test2/BILQH07S314_D065_T008S02_V02.IMG
    Begin Content Validation: file:/Users/jpadams/Downloads/test2/BILQH07S314_D065_T008S02_V02.IMG
      FATAL_ERROR  [error.array.bad_file_read]   array 1: Error reading data file 'file:/Users/jpadams/Downloads/test2/BILQH07S314_D065_T008S02_V02.IMG': /Users/jpadams/Downloads/test2/BILQH07S314_D065_T008S02_V02.IMG (No such file or directory)
    End Content Validation: file:/Users/jpadams/Downloads/test2/BILQH07S314_D065_T008S02_V02.IMG

  PASS: file:/Users/jpadams/Downloads/test2/bundle_cassini-huygens-coradar.xml

Summary:

  3 error(s)
  0 warning(s)

  Message Types:
    1            error.label.missing_file
    1            error.array.bad_file_read
    1            error.label.schema

End of Report
Completed execution in 5155 ms

Test Data
BILQH07S314_D065_T008S02_V02.xml.txt
bundle_cassini-huygens-coradar.xml.txt

[jordanpadams]

After some investigating this is a bigger issue under the hood for SAX LSResourceResolver. Basically it is caching the old version of the IM and resolving based upon the namespace. i think the solution will need to be to check in the IM versions. if it ever switches, dump the cache.

[jordanpadams]

June schedule status: Delayed due to higher priority bug fixes and limited availability of resources.

[jordanpadams]

Build 11.0 Descope Rationale: Staff pulled off onto other flight projects. This will no impact the build and this ticket will be added to Build 11.1

[qchaupds]

After more investigations, we learn that because validate was not updating the list of schema locations for each label, a subsequent label was using the previous schema location(s). The error is very subtle and only shows up when the labels uses different Information Model (IM) version. For the case above, the first label process was bundle_cassini-huygens
-coradar.xml with IM version 1.10.0.0 whereas the second label has IM version 1.12.0.0 thus the error observed and reported in the report.

When each label is being validated, this list of schema location SHOULD be updated should solve the issue.

This class need updating is src/main/java/gov/nasa/pds/tools/validate/rule/pds4/LabelValidationRule.java

and the following code snippet:

// Each label may contain different schemas and different versions between each other. // The list of schema locations must be updated for every label in the schemaValidator. if (getContext().isForceLabelSchemaValidation()) { try { schemaValidator.setExternalLocations( getExtractor(target).getSchemaLocation()); } catch (Exception ignore) { //Should not throw an exception } }

can be at the beginning of function validateLabel(). This issue is very closely related to issue #210 so they both should be worked on together in one work unit.

A run in DEV shows both labels are valid:

% validate --skip-content-validation -r report_github188_label_valid_both.json -s json -t src/test/resources/github188/bundle_cassini-huygens-coradar.xml src/test/resources/github188/BILQH07S314_D065_T008S02_V02_without_Missing_Area_tag.xml

% egrep "status|label" report_github188_label_valid_both.json
"status": "PASS",
"label": "file:/home/qchau/sandbox/validate/src/test/resources/github188/bundle_cassini-huygens-coradar.xml",
"status": "PASS",
"label": "file:/home/qchau/sandbox/validate/src/test/resources/github188/BILQH07S314_D065_T008S02_V02_without_Missing_Area_tag.xml",