Field testing the JSON Schemas?

[wendellpiez]

User Story:

To validate the correct functionality of auto-generated schemas, field tests are more or less indispensable, at least as things become complex.

The term 'field test' in this case refers to the testing of validation runtimes with generated schemas. We have XProc performing testing of XSDs, running in XML Calabash.

What we don't have is the analogous functionality for JSON. Ideally it would

• run in Java under Maven (the present stack)
• support a CL invocation or from script, no Java to write or compile (a config file is okay)
• support JSON Schema v07

note: one alternative would be an XProc processor that supports https://spec.xproc.org/master/head/validation/#c.validate-with-json-schema

Goals:

• Determine what software package (subject to the requirements above) can be used to automate field testing of JSON Schemas vis-a-vis sets of 'working and broken' documents.
• Implement such a pipeline

Dependencies:

There is related work underway in a fork here https://github.com/wendellpiez/metaschema-xslt/tree/issue105-jsonschemaA addressing #105

Acceptance Criteria

• All website and readme documentation affected by the changes in this issue have been updated. Changes to the website can be made in the docs/content directory of your branch.
• A Pull Request (PR) is submitted that fully addresses the goals of this User Story. This issue is referenced in the PR.
• The CI-CD build process runs without any reported errors on the PR. This can be confirmed by reviewing that all checks have passed in the PR.

{The items above are general acceptance criteria for all User Stories. Please describe anything else that must be completed for this issue to be considered resolved.}

[wendellpiez]

See PR #108 for work in progress, including XProc3-based JSON Schema validation over batches of expected-valid and expected-invalid test sets.

But this work could also be migrated into another repository for testing the OSCAL metaschema library, specifically.

[wendellpiez]

Closing, as this is now happening over here:

https://github.com/usnistgov/oscal-xproc3