Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Product referential integrity check throws invalid WARNINGs #368

Closed
2 tasks
jordanpadams opened this issue Jun 30, 2021 · 6 comments
Closed
2 tasks

Product referential integrity check throws invalid WARNINGs #368

jordanpadams opened this issue Jun 30, 2021 · 6 comments
Assignees
Labels
B12.0 bug Something isn't working s.medium

Comments

@jordanpadams
Copy link
Member

jordanpadams commented Jun 30, 2021

🐛 Describe the bug

When I try to validate against the insight documents bundle (as an example), I am getting WARNINGs about LID references not found for products that do not pertain to this bundle.

      WARNING  [info.validation.general]   A LID reference urn:nasa:pds:context:instrument:seis.insight is referencing a logical identifier for a product not found in this bundle

📜 To Reproduce

Steps to reproduce the behavior:

$ validate-2.1.0-SNAPSHOT/bin/validate -t /data/pds4/test-data/registry/urn-nasa-pds-insight_documents/bundle_insight_documents.xml -R pds4.bundle
...
  PASS: file:/data/pds4/test-data/registry/urn-nasa-pds-insight_documents/document_hp3rad/release_notes.xml
      WARNING  [info.validation.general]   A LID reference urn:nasa:pds:context:instrument:hp3.insight is referencing a logical identifier for a product not found in this bundle
...

🕵️ Expected behavior

  • Validate should ignore these references for the time being. We can throw an INFO or DEBUG message here if we want, and then eventually replace with some registry integration, but not now. This will definitely dramatically inflate the number of warning messages coming across (especially for context products).
  • For WARNINGs that should exist, the problem type is incorrect. Current problem type: info.validation.general. Replace with new problem type: warning.integrity.reference_not_found

📚 Version of Software Used

2.1.0-SNAPSHOT

🩺 Test Data / Additional context

Really any bundle, but https://pds.nasa.gov/data/pds4/test-data/registry/urn-nasa-pds-insight_documents/ also

🏞Screenshots

🖥 System Info

  • OS: [e.g. iOS]
  • Browser [e.g. chrome, safari]
  • Version [e.g. 22]

🦄 Applicable requirements

🦄 #308

@tloubrieu-jpl
Copy link
Member

@qchaupds will branch off #69 for this ticket ()

@jordanpadams
Copy link
Member Author

@qchaupds per this ticket, just added another update that should be include here:

For WARNINGs that should exist, the problem type is incorrect. Current problem type: info.validation.general. Replace with new problem type: warning.integrity.reference_not_found

@qchaupds
Copy link
Contributor

qchaupds commented Aug 2, 2021

Because the code from branch #69 has been merged back into main branch on July 17, 2021, the work for this ticket will be branched off a fresh copy of main.

qchaupds pushed a commit that referenced this issue Aug 2, 2021
1. Add new test resource: src/test/resources/github368
2. Add logic to make throwing the warning optional for future work: src/main/java/gov/nasa/pds/tools/util/ReferentialIntegrityUtil.java
3. Add new enum type to report the warning to a specific type instead of using general: src/main/java/gov/nasa/pds/tools/validate/ProblemType.java
4. Modify test to reflect the correct number of warnings and add new test: src/test/resources/features/validate.feature

Refs:

#368 Product referential integrity check throws invalid WARNINGs
jordanpadams added a commit that referenced this issue Aug 9, 2021
jordanpadams added a commit that referenced this issue Aug 9, 2021
Product referential integrity check now does not throw invalid WARNINGs
@jordanpadams
Copy link
Member Author

closed per #383

@rchenatjpl
Copy link

@jordanpadams @qchaupds
Am I missing something? This is the same as #308, right?

@jordanpadams
Copy link
Member Author

@rchenatjpl this was a bug identified after #308 was implemented. as long as you feel #308 was testing sufficiently, I think we can say this was tested as well

al-niessner pushed a commit that referenced this issue Nov 17, 2023
Through many of the tickets things had purposely been turned off due to a large amount of noise #368. When the reporting was turned on for this ticket got 8 warnings when the expectation was one or two. Found that the reference for some were not for this bundle. Added further filtering that the reference had to be contained in this bundle as well.

Found many of the lid/lidvid checks where contains() instead of startsWith(). Change to startsWith() to prevent some messed up thing where there is a match in the middle of the string when it clearly must startWith() for correctness. This too may have some cleanup to the output.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
B12.0 bug Something isn't working s.medium
Projects
None yet
Development

No branches or pull requests

4 participants