Global parameters per scenario

[AirQuick]

#892 (comment)

When implementing this feature, consider accompanying it with a test similar to #1309.

[ohsh6o]

Hi @AirQuick, is there anything I can do to help with this or pick it up?

[AirQuick]

@ohsh6o
From the point of view of testing Schematron with XSpec, there are two kinds of global parameters:

• (A) One is passed to the Schematron implementation at compile time as mentioned in Wiki Writing Tests section. Typically, phase, allow-foreign, visit-text and only-child-elements.
• (B) The other is passed to your compiled Schematron schema at run time.

Which one do you want to make scenario-local?
(B) will be in the scope of this issue, but (A) is out of its scope.

[ohsh6o]

I am most certainly interested in B, @AirQuick!

[AirQuick]

I've implemented a POC of this feature and it's working on XSLT and Schematron.

@ohsh6o
Before going further, I would like to better understand your use case.
Is there a working Schematron schema that has a global xsl:param you want to set various values in XSpec x:scenario? Is it this global xsl:param?: https://github.com/18F/fedramp-automation/blob/6c986baf6f3de1d1a46b2f93280b862d460fd9a5/resources/validations/src/ssp.sch#L15

[ohsh6o]

Excellent question, and yes your assumption was correct. If this context or the info below does not help, I can try to rewrite examples in a Gist to update it to my now much larger test suite.

In short, there are two major external dependencies, at a high level, I need to think about: resolved catalog profiles and value registries. This commit is around the time where you might sense I wanted to expand it further. I would like to pass in file paths that are collect() => doc()ed but allow another developer to just pass catalog profiles and tailored registries directly if they need to, as global parameters at XSLT runtime (after Schematron is compiled, at runtime). (NOTE: I know we are stretching, from my understanding, what this tooling ought to do and I have been reading up on XML Catalogs, but I think there are a few reasons I have not wanted to introduce this complexity.)

So I was unable to get the relevant tests at that time to work, I stopped trying and reported the issue to you.

Resolved catalog profiles defined required nodes and attributes for security documentation this Schematron/XSLT will check, and the values registry define the values to which many attributes and embedded nodes must be bound. When unable to test on a case by case basis, I need to test against the whole set of conditions, so my tests balloon way outside a "unit" of encapsulation. Newer subsequent tests show this is obvious: I have to test globally across the resolved catalog profile, and I cannot pass in a custom one focused on the unit of testing, we get a lot of this because I cannot make resolved catalog profiles for a specific test with one control requirement, for example.

This would allow me to test an approach for which I wish to bypass the whole XML Catalog issue: I am using this with a shell harness, but I need to be able to allow those to programmatically pass in global context variables wth XML/XSLT libraries in Java, .NET, and Python environments without adding more complexity unless I have to (XML runtimes feature sets are not super consistent across the board, as I have learned and you likely know!). I could keep going down that road, but testing that will be very difficult.

[ohsh6o]

Is there any additional help I can contribute at this point, @AirQuick or you just want me to test it as is by reviving my old branch of code?

[AirQuick]

@ohsh6o
Sorry for not responding sooner. I've been busy this month. Now I'm back to normal. In a few days, I'll open a pull request with which you can test. I'd like to ask @galtm to review it then.

[AirQuick]

@ohsh6o
It took longer than I expected but I finally opened the pull request #1354. Can you try it on to see if it fits your use case?

• You can set x:param within a particular Schematron x:scenario, like this:
  

  xspec/test/external_scenario-param_schematron.xspec

  Lines 38 to 50 in 91cb4aa

  	<x:scenario label="Scenario-level $phase">
  	<!-- This $phase (//x:scenario/x:param) takes effect only at run time -->
  	<x:param name="phase">C</x:param>
  	<!-- x:context[preceding-sibling::x:param] -->
  	<x:context>
  	<context-child />
  	</x:context>
  	<x:expect-report id="compile-time-phase-param-is-A" />
  	<x:expect-report id="run-time-phase-param-is-C" />
  	<x:expect-report count="2" />
  	</x:scenario>

• Note that x:param within a particular Schematron scenario takes effect only when running the compiled XSLT. At the time of compiling a Schematron schema into XSLT, //x:scenario/x:param is ignored and only /x:description/x:param takes effect. I hope this limitation doesn't affect your use case.
• To enable //x:scenario/x:param, you need to set /x:description/@run-as=external like this:
  

  xspec/test/external_scenario-param_schematron.xspec

  Line 2 in 91cb4aa

  <x:description run-as="external" schematron="external_scenario-param.sch"

Demo

$ git clone https://github.com/xspec/xspec.git
Cloning into 'xspec'...
...
$ cd xspec
$ git fetch origin pull/1354/head:pull_1354
...
 * [new ref]         refs/pull/1354/head -> pull_1354
$ git checkout pull_1354
Switched to branch 'pull_1354'
$ mkdir -p /tmp/saxon
$ # Use Saxon 10.3. Saxon 10.0, 10.1 and 10.2 have critical bugs in terms of XSpec v2.x and Schematron.
$ export SAXON_CP=/tmp/saxon/Saxon-HE-10.3.jar
$ wget -O "${SAXON_CP}" https://repo1.maven.org/maven2/net/sf/saxon/Saxon-HE/10.3/Saxon-HE-10.3.jar
...
$ bin/xspec.sh -s test/external_scenario-param_schematron.xspec 
...
Converting Schematron into XSLT...

Converting Schematron XSpec into XSLT XSpec...

Creating Test Stylesheet...
 

Running Tests...
Testing with SAXON HE 10.3
Base context
..Scenario-level $phase
report compile-time-phase-param-is-A
report run-time-phase-param-is-A
report count: 2
..Scenario-level $phase
report compile-time-phase-param-is-A
report run-time-phase-param-is-B
report count: 2
Scenario-level $phase
report compile-time-phase-param-is-A
report run-time-phase-param-is-C
report count: 2

Formatting Report...
passed: 9 / pending: 0 / failed: 0 / total: 9
Report available at test/xspec/external_scenario-param_schematron-result.html
Done.

[ohsh6o]

That's excellent news @AirQuick, thank you! I will take a look later this week.

[AirQuick]

@ohsh6o
The feature is now available in the master branch. Hope it helps.