Minutes

Norman Walsh edited this page Jul 2, 2018 · 22 revisions

Minutes

  • AB – Achim Berndzen
  • AN – Ari Nordström
  • DM – David Maus (Mo)
  • ES – Erik Siegel
  • GB – Geert Bormans
  • GI – Gerrit Imsieke
  • JK – Jirka Kosek (Mo)
  • JS – Jorge Sanchez
  • MR – Matthieu Ricaud-Dussarget (Mo)
  • NW – Norman Walsh

Kindly hosted by MarkLogic’s UK office. Special thanks to Lorraine Bylett.

Monday June 11

(with additions made on Tuesday June 12)

Status

  • Some progress on the spec/issues
  • Not much progress in Calabash / 70-ish % implemented in Morgana
    • Import mechanisms, variables in the header roadblocks for AB (among others)

Review Minutes from Prague 2018

  • superficially skimming them

Working group procedures

  • AB is missing timely replies
  • NW: make rule that people (at least the editors) reply within 48 hrs
  • Consensus: Point to important issues also on the mailing list (xproc-dev@w3.org)

Maps

Keys

AB: Keys must be QNames all the time (a union type of QName and string is not supported by Saxon HE)

ES, NW: It should be possible to use strings.

See discussion in https://github.com/xproc/3.0-specification/issues/268 (related: https://github.com/xproc/3.0-specification/issues/371 for p:document-property)

Example map:

properties="map { 5: 'bar', 'a': 1, 'x:a': 2, $qname: 3,
                  q{uri}test: 4, '{uri}test': 5,
                  xs:QName('uri','test'): 6 }”

Use map(*), silently ignore any keys but string (with or without colon, no bindings apply) or QName.

NW asks about XSLT where QNames are expected:

xslt parameters="map { 5: 'bar', 'a': 1, 'x:a': 2 }"

GI: Introduce a function p:simple-map() that will convert colonized strings into QNames and pass non-colonized strings unaltered

NW: This should be done by the processor automatically. There will be an official function, JK proposes p:infer-qnames(), NW proposes p:force-qname-keys(). This function will drop any non-string/non-QName key and make QNames of strings, using in-scope bindings. It will raise errors on colonized strings if there is no binding in scope for a thing that looks like a prefix.

If a step, as per its signature, expects a map from QName to anything, this function will be applied to any supplied option.

Consensus on p:force-qname-keys() as described here.

content-type attribute

https://github.com/xproc/3.0-specification/issues/385

Consensus

JSON

https://github.com/xproc/3.0-specification/issues/392

p:load returns a document so it can’t return the JSON XDM item

$json-doc/item() = ?

Consider doc.json containing {"key": true}:

<p:load href="doc.json"/>
<p:variable name="json" select="p:json(.)"/>
<p:choose>
  <p:when test="p:json(.) ? key">…
  <p:when test="$json ? key">…

The tests above are what people will want to be able to do.

p:get-representation($doc)
   $doc is xml -> xdm node
   $doc is json -> xdm item
   $doc is text -> xdm node
   $doc is binary -> base64 encoded text node

<p:json-document json="{p:json()}"/>

<p:declare-step name="p:json-document">
  <p:option name="json" as="item()"/>
  <p:option name="parameters" as="map(xs:QName, item()*)"/>
  <p:output port="result" content-type="application/json"/>
</p:declare-step>

Can be stored into a variable when fn:parse-json is applied.

But returns empty node when accessed as a document. Use p:json($doc) to access them (AB’s proposal), or let maps (and other items) flow through a pipeline (NW’s preference, for better ease of use).

Name it p:json(). Preliminary consensus, dissenting opinion by AB who’d prefer p:get-json().

In addition, p:cast-content-type should be able to turn JSON to the XML representation that fn:json-to-xml would produce.

AB: Do we specify for p:inline: If content type is text/json, then an inlined {p:json(.)} item will be returned as a JSON document if . is a JSON document.

Or create a step p:json-document that accepts an item as the json option and outputs a JSON document.

The latter proposal is consensus.

RDF Graph as a document type

Should be possible. Consider:

<p:declare-step name="p:sparql">
  <p:input port="source" content-type="*/*"/>
  <p:input port="query" content-type="text/plain application/sparql …"/>
  <p:input port="result" content-type="*/*"/>
  <p:option name="parameters" as="map(xs:QName, item()*)"/>
</p:declare-step>

But note:

<p:when test="???"/> nothing you can write here to test a graph

Use p:cast-content-type to normalize any RDF input to application/rdf+xml (encourage implementors to normalize to Jena’s normalized XML format).

SPARQL steps should be specified in a separate spec as optional steps, but with the p namespace prefix.

Character Maps

Notes apparently lost.

Variables everywhere

Variables in the “header” section:

<p:pipeline>
  <p:variable name="a" select="map { 'a': 1 }"/>
  <p:option name="opt1" select="$a"/>

  <p:variable name="b" select="3 + 4"/>

A variable with a following sibling that is in the “header” section is in the header section.

Import mechanism

Consider:

<p:import href="mylib.xpl"/>
<p:declare-step>
  <p:input port="source"/>
  <p:variable name="foo" visibility="private"/>
  <p:declare-step name="sub">
     you CANNOT refer to $foo
  </p:declare-step>
  CAN refer to it here

NW: This seems really confusing. What about:

<p:library>
  <p:variable name="foo" select="3" visibility="public"/>
  <p:declare-step name="sub" visiblity="private">
     you CAN refer to $foo

  <p:declare-step>
    <p:variable name="foo" select="2"/>
    <p:import href="mylib.xpl"/>
    <p:declare-step name="mystep">
       you CAN refer to $foo

Unclear what consensus was reached.

Test suite

Good progress overall, even if there’s still a long way to go.

We need a story for optional features; an implementation should know what tests it can skip.

  • Come up with some way of naming optional features.
  • Add markup for identifying which optional feature is being tested

Review issues list

See individual status/label changes and comments

Tunneling

https://github.com/xproc/3.0-specification/issues/291

Do we really need to disallow required="true" when tunnel="true"? Yes, in order to be able to carry out static analysis.

Given the complexity and the fact that you can stuff the many options into a map, you can (and have to) pass the map to every invocation on the stack. Additions in between can be done by map:merge().

Tunneling deferred to 3.1 if it’s still desirable.

Tuesday June 12

Globbing

See discussion on https://github.com/xproc/3.0-specification/issues/325

Globbing on p:load, p:document and p:http-request? Globbing incompatible with URLs:

href="file:///path/to/root/**/*.docx" XXX not a valid URI

What about:

href="file:///path/to/root/"
include="('**/*.docx')"
exclude="('**/trash/.*')"

p.directory-listing

<p:directory-list path="/path/to/root/"
    depth="17"
    include-filter="{('^/final', '\.docx$')}"
    exclude-filter="{('/_[^/]+$', '/trash/', '/\.git/')}"
/>

include-filter, exclude-filter match URIs relative to the base dir.

Sequences of xs:strings (no parsing of space-separated strings necessary)

All URIs that are matched against will start with a slash, all directories end with a slash

Include first, then exclude

Add boolean recursive option.

Have another step <p:load-directory-list> that returns the matching documents as a sequence.

Do away with globbing.

Same approach for p:unarchive (or unzip)?

See https://github.com/xproc/steps/issues/4

<p:declare-step type="p:uncompress">
  <p:input port="source" content-types="*/*" />
  <p:output port="manifest" sequence="true" content-type="application/xml"/>
  <p:output port="result" content-types="*/*" sequence="true" primary="true"/>
  <p:option name="list-only" as="xs:boolean"/>
  <p:option name="parameters" as="map(xs:QName, item()*)"/>
  <p:option name="include-filter" as="xs:string*" />
  <p:option name="exclude-filter" as="xs:string*" />
  <p:option name="method" as="xs:token" select="'auto'" />
</p:declare-step>

Harmonize Martin’s proposal with the filtering options described above and with Achim’s proposal in http://markupuk.org/webhelp/#ar10s03.html

Some of the proposed parameters should become options (list-only for ex.)

Zip is the only required format.

list-only: result port will be empty, manifest is a c:directory-list that can be used as a manifest for archiving the data again.

XPath expressions in options

For options declared as XPathExpression, or XSLTMatchPattern, and for options of type map() or array()

Step Library

RDF Steps

Add a parameter to cast-content-type for application/rdf+xml to canonicalize it.

Validation

Look at MR’s unified report format later.

All should have a report port.

GI: Dual output formats, SVRL and c:errors, served us well so far. Ideally attach a @xpath attribute to c:errors/c:error (le-tex to submit their Jing patch to jing-trang).

No generic/universal validation step yet. Possibly write one in XProc, also for validating according to the xml-model PIs.

Standard Library

Eventually agree on standard library steps written in XProc (not in the p namespace) and ship it with the processors.

Text value templates

Missing: Create maps from TVTs that result in JSON: https://github.com/xproc/3.0-specification/issues/398

Reading HTML

Normally as binary, but maybe provide a parameter to p:load. Otherwise, p:cast-content-type to application/xhtml+xml.

Next meeting(s)

Sept. 5/6, probably Leipzig

Goals/Timeline for delivery

Core spec finished by then. List of open core step issues

AB tries to implement the core spec in Morgana by then

Phone conferences every 2 weeks. Next on June 28. Hangout link

Discuss step library (including validation) in Sept.

Clone this wiki locally
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.