Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Batch validation of computer samples against their XSD #102

Merged
merged 7 commits into from
Mar 15, 2024

Conversation

wendellpiez
Copy link
Collaborator

Committer Notes

Addresses #101

All Submissions:

  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same update/change?
  • Have you squashed any non-relevant commits and commit messages? [instructions]
  • Do all automated CI/CD checks pass?

Changes to Core Features:

  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your core changes, as applicable?
  • Have you included examples of how to use your new feature(s)?
  • Have you updated all website and readme documentation affected by the changes you made? Changes to the website can be made in the website/content directory of your branch.

@wendellpiez wendellpiez requested a review from a team as a code owner February 16, 2024 20:49
Copy link

github-actions bot commented Feb 16, 2024

XSpec Test Results

  2 files  ±0  40 suites  ±0   0s ⏱️ ±0s
105 tests ±0  90 ✅ ±0  15 💤 ±0  0 ❌ ±0 
114 runs  ±0  99 ✅ ±0  15 💤 ±0  0 ❌ ±0 

Results for commit 772f575. ± Comparison against base commit 0010340.

♻️ This comment has been updated with latest results.

Comment on lines 11 to 17
usage() {
cat <<EOF
Usage: ${BASE_COMMAND:-$(basename "${BASH_SOURCE[0]}")} [ADDITIONAL_ARGS]

Produces a validation report for a file set designated in the pipeline \"${XPROC_FILE}\"
EOF
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is usage even callable when you want to know how to use it? I presume this is based in our previous conversation about scripts that may likely runt without arguments perhaps?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, it sure is! good spotting, what do you recommend?

Copy link
Contributor

@aj-stein-nist aj-stein-nist left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry it took me a while to review this today and not yesterday. Looks good to me.

Copy link
Collaborator

@galtm galtm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't had a chance to dig into these files, but two things caught my eye that I commented on. If you're ready to merge this PR, please don't wait for me.


For which capabilities can we write XSpec or meaningfully test and confirm (expose and document) in other ways, including

- `xsl:message` behavior including terminate="yes"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case you're not already aware of it, XSpec can test terminating messages
https://github.com/xspec/xspec/wiki/Testing-Dynamic-Errors

I don't know a way to make XSpec test non-terminating xsl:message instructions, though. That's why I gave the xslt3-functions message handler an option to emit a processing instruction instead of xsl:message. The idea is to use the message in production and the PI (a "mock" message) when testing.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This suggests that one step toward testing those requirements would be to use xslt3-functions message handling for all the messages. That is an awesome idea and not out of line with other plans or concepts.

For which capabilities can we write XSpec or meaningfully test and confirm (expose and document) in other ways, including

- `xsl:message` behavior including terminate="yes"
- side effects such as `xsl:result-document`?
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You probably already know that external transformations can handle XSLT that includes xsl:result-document. Beyond just not erroring out, I guess a test scenario can check availability of, or read the contents of, the file you expect to have been created.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not know that but it makes sense, thanks @galtm, very helpful indeed.

@wendellpiez wendellpiez merged commit 7637dd2 into usnistgov:develop Mar 15, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants