Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

I don't think we should call out skeleton implementation in p:validate-with-schematron #195

Open
xml-project opened this issue Aug 6, 2019 · 20 comments

Comments

@xml-project
Copy link
Contributor

commented Aug 6, 2019

I do not see any reason why we state:

An XProc processor that implements this step should at least support the [Schematron Skeleton] implementation.

What is special that we call out this implementation here, as the only implementation we mention in the whole spec?

I think we should remove the whole paragraph.

@ndw

This comment has been minimized.

Copy link
Collaborator

commented Aug 6, 2019

Fine by me.

@gimsieke

This comment has been minimized.

Copy link
Contributor

commented Aug 6, 2019

It is important that there is an implementation that can include and execute XSLT. This behavior can be triggered by setting allow-foreign to true in the set of compile parameters. There are other params that users have come to rely on and that won't probably be supported by any random XPath-only implementation. I will respond later tonight in greater detail since I'm mobile-only atm.

@gimsieke

This comment has been minimized.

Copy link
Contributor

commented Aug 7, 2019

Suggestion:

There are multiple Schematron implementations. An XProc processor that implements this step must be able to use an implementation that generates XSLT code and that offers the xslt2 query binding. In addition, this Schematron implementation should be able to pass non-Schematron markup, including XSLT code, to the compiled XSLT stylesheet verbatim. The Schematron implementation should also be able to include machine-readable, namespace-aware XPath location expressions in its reports.

We can move the Skeleton citation to here:

The parameters map may contain two special entries, c:compile and c:xvrl, both are maps. If a code-generating implementation such as [Schematron Skeleton] is used, the entries of the c:compile map, for example allow-foreign, will be passed to the code generator.

@xml-project

This comment has been minimized.

Copy link
Contributor Author

commented Aug 7, 2019

@gimsieke I see your point and that you want all this features. But frankly: I think this should be a quality of implementation and we should not raise the bar for implementations beyond standard compatibility. Think of somebody coming up and demanding that any implementation of p:xslt must be streaming. Hopefully we would not accept it.
So still: I propose to delete the whole paragraph and not demanding anything beyond the respective standard.

@dmj

This comment has been minimized.

Copy link
Member

commented Aug 7, 2019

I don't understand why there are any requirements in the specification that go beyond simple conformance as defined in ISO 2016 at all. It is up to the implementors to provide an implementation that suits their customers and/or users, isn't it?

@gimsieke

This comment has been minimized.

Copy link
Contributor

commented Aug 7, 2019

The Schematron spec is fuzzy in some regards.

We already decided that we’ll have XVRL as the default reporting format, so in this regard this spec already goes beyond minimal conformance. Generating XVRL from SVRL is facilitated by machine-readable XPath locations with proper namespaces in the SVRL, therefore I suggested this additional requirement.

Another point that I think should be better specified in this validation steps spec is XSLT parameter passing, in particular for users that rely on the Skeleton implementation (that both XProc processors currently use).

The Schematron spec mentions “external variables” only once without saying anything about what they are, how they ought to be passed to a schema, or how they can be referred to in a schema. I just presume that “external variables” are what we call parameters. There is demand for passing parameters to a Schematron implementation from XProc. Allowing foreign markup and using xsl:param was one way of making these parameters available to the compilation output of Skeleton. I submitted a patch for Calabash that makes all parameters available to the generated XSLT.

Now we decided that we want to have special maps within the parameters map that influence compilation and XVRL generation. All other parameters should be passed to the compiled stylesheet.

If a Schematron implementation doesn’t have the notion of a compilation process or if it doesn’t accept parameters, it may ignore these parameters.

We could come up with a different parameter passing mechanism, but this seemed to be achievable most easily with Skeleton since Skeleton didn’t have to be changed for it. It is also achievable quite easily with the other Schematron implementation that I’m aware of that supports the xslt2 binding, which is your schxslt. Currently it doesn’t have the option to pass any foreign markup to the compiled XSLT, but I think you can easily add such an option.

Anyway, in order not to make an XProc processor non-conforming that supports Schematron validation and uses schxslt as its xslt2 binding implementation, I suggested that we only say it should (not: must) pass XSLT and parameters to the generated stylesheet.

But overall I regard this parameter passing as an important and underspecified feature that I’d like to regulate in this spec since this will foster interoperability.

@xml-project

This comment has been minimized.

Copy link
Contributor Author

commented Aug 7, 2019

We already decided that we’ll have XVRL as the default reporting format, so in this regard this spec already goes beyond minimal conformance. Generating XVRL from SVRL is facilitated by machine-readable XPath locations with proper namespaces in the SVRL, therefore I suggested this additional requirement.

Sorry, I am a bit confused about your intro:

  1. The specs say the step should output SVRL. So to my reading producing SVRL is an optional feature. When did we decide otherwise?

  2. Could you explain what "machine-readable XPath locations with proper namespaces in the SVRL" are relevant with this matter.

@xml-project

This comment has been minimized.

Copy link
Contributor Author

commented Aug 7, 2019

Sorry, hit too fast.

I completely agree with you on the parameter-passing issue. We have to say state that parameters might go to the "building phase" of the schema (if the used schematron implementation knows such a thing) and how we sort which parameters are used.

Bit we should do this in an abstract way (as we did for xslt and xquery) and not mention a specific implementation.

@gimsieke

This comment has been minimized.

Copy link
Contributor

commented Aug 7, 2019

We decided that the primary and mandatory reporting output is XVRL. Since all Schematron implementations (to my knowledge) create SVRL in the first place, I thought we may specify that the step also return the SVRL on the report port.

I want to have a parameter for XVRL generation that lets a user say that they want to have fully namespace-qualified location paths. This will facilitate patching back error messages into the source files. In Skeleton, you can opt for several path notations, only one of it includes the namespace URIs. So I’d like to specify that a Schematron implementation be able to generate location paths with all namespace URIs, just like Skeleton provides it.

@xml-project

This comment has been minimized.

Copy link
Contributor Author

commented Aug 7, 2019

We decided that the primary and mandatory reporting output is XVRL.

The specs say should, that is not mandatory, is it? I can not remember we decided otherwise.

I want to have a parameter for XVRL generation that lets a user say that they want to have fully namespace-qualified location paths.

May be that's nice. But we have never agree about this and I am strongly opposed to this. This would completely break our tools box because the java validation api report line numbers and columns. So neither RNG nor XML schema validation would work.

@ndw

This comment has been minimized.

Copy link
Collaborator

commented Aug 7, 2019

I see a rat hole here. I'm inclined to agree with @dmj that we should assert conformance to the spec and leave the rest to implementors. I'm fine with non-normative descriptions of desirable behavior.

@gimsieke

This comment has been minimized.

Copy link
Contributor

commented Aug 7, 2019

All these compilation and XVRL generation options are subject to support of the underlying validation libraries. If they don’t emit XPath locations, then there may be no XPath locations in the XVRL. If they cannot include namespace URIs in the XPath locations, so be it.

All I want to have is to standardize the parameter vocabulary that lets a user influence XPath generation beyond Skeleton’s full-path-notation = 1|2|3. So if a Schematron or Relax NG validator is able to report paths, and if there is the choice whether the paths are more like /TEI or /*:TEI[namespace-uri()='http://www.tei-c.org/ns/1.0'][1], users may use a standard parameter vocabulary to tell the validation step which paths to include.

@dmj

This comment has been minimized.

Copy link
Member

commented Aug 8, 2019

Since all Schematron implementations (to my knowledge) create SVRL in the first place, I thought we may specify that the step also return the SVRL on the report port.

That's actually not the case. The Skeleton as well as SchXslt use callback functions to generate a validation report. For convenience both provide an implementation of those callbacks that generates SVRL. It is the XProc implementor's choice to use the SVRL generating template.

All I want to have is to standardize the parameter vocabulary that lets a user influence XPath generation beyond Skeleton’s full-path-notation = 1|2|3. So if a Schematron or Relax NG validator is able to report paths, and if there is the choice whether the paths are more like /TEI or /*:TEI[namespace-uri()='http://www.tei-c.org/ns/1.0'][1], users may use a standard parameter vocabulary to tell the validation step which paths to include.

I think you are asking for a standardization across Schematron implementations. This should better be discussed with the respective implementors Rick (@rjelliffe), Philip Helger (@phax), Nico Kutscherauer (@nkutsche) and me (@dmj),

@gimsieke

This comment has been minimized.

Copy link
Contributor

commented Aug 8, 2019

In order to save us from waiting for this and other underspecified things to be specified, I proposed that we stipulate that the Skeleton implementation must be supported, besides possibly other implementations.

But Schematron validation is different than the other validation steps, as it can be written in XProc and XSLT entirely. So if, as I perceive, there is no urge by the other participants in this discussion to foster interoperability by stating that a specific implementation be supported, then let’s do without and people can roll their own steps, just like we’ve been doing for years now.

Still, as Achim said, parameter passing to Schematron compilation (if the Schema is compiled) and execution is probably a thing that we should specify in XProc. Whether the selected (in XProc-implementation-defined ways) Schematron processor is able to make use of these parameters is a different story.

And I also think that we can demand that an XProc processor must be able to use a Schematron implementation that supports the xslt2 query binding.

@rjelliffe

This comment has been minimized.

Copy link

commented Aug 8, 2019

@dmj

This comment has been minimized.

Copy link
Member

commented Aug 9, 2019

In order to save us from waiting for this and other underspecified things to be specified, I proposed that we stipulate that the Skeleton implementation must be supported, besides possibly other implementations.
...
So if, as I perceive, there is no urge by the other participants in this discussion to foster interoperability by stating that a specific implementation be supported, then let’s do without

Are you arguing for or against requiring a specific implementation?

@dmj

This comment has been minimized.

Copy link
Member

commented Aug 9, 2019

@rjelliffe I for one chose to use URIQualifiedNames [1] starting from the upcoming release 1.3 of SchXslt.

[1] https://www.w3.org/TR/2017/REC-xpath-31-20170321/#doc-xpath31-URIQualifiedName

@gimsieke

This comment has been minimized.

Copy link
Contributor

commented Aug 11, 2019

Are you arguing for or against requiring a specific implementation?

The text that I drafted contained this requirement: An XProc processor must be able to provide the Skeleton implementation. I put this in in order to ensure interoperability, since the Schematron spec itself is too loose about it. There can be conformant Schematron implementations that will choke on features that Schematron users have come to rely on, such as using XSLT in order to pass parameters to the schema, among other things.

But there doesn’t seem to be consensus about it. Therefore, and because XProc processors will probably use Skeleton anyway, and because it is possible to create a user-defined XProc step if they don’t, I’m not going to fight for it, and it’s likely that we remove this requirement after the next editorial conference call, or even before.

@gimsieke

This comment has been minimized.

Copy link
Contributor

commented Aug 11, 2019

@rjelliffe I for one chose to use URIQualifiedNames [1] starting from the upcoming release 1.3 of SchXslt.

[1] https://www.w3.org/TR/2017/REC-xpath-31-20170321/#doc-xpath31-URIQualifiedName

@rjelliffe,

I’m aware of 3 XPath notations, and ideally people would be able to choose any of them in any Schematron implementation:

  1. EQNames, or more specifically the Q{}-name notation that @dmj is going to use (/Q{http://www.tei-c.org/ns/1.0}TEI[1])
  2. expressions like /*:TEI[namespace-uri()='http://www.tei-c.org/ns/1.0'][1] (full-path-notation=1 in Skeleton)
  3. expressions constructed with name() like /tei:TEI[1] or /TEI[1], depending on whether the name contains a namespace prefix or not (full-path-notation=2 in Skeleton and the way schxslt constructed paths prior to v1.3)

The positional predicate [1] is of course optional in all cases if there is only one element of that name at the given path.

There should be names for the notations that users can select in the compile map of the XProc parameter map, for ex. parameters="map { 'compile': map { 'xpath-notation': 'Q' } }", with possible values Q, namespace-uri, and name. This is just a suggestion for the xpath-notation values. They are not completely self-explanatory, but at least they’re better than 1|2|3.

@rjelliffe

This comment has been minimized.

Copy link

commented Aug 12, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.