Skip to content

Writing Scenarios

AirQuick edited this page Sep 17, 2019 · 25 revisions

Contents

Introduction

The description of the behaviour of a stylesheet lives within an XSpec document, which should adhere to the XSpec RELAX NG schema. All elements are in the http://www.jenitennison.com/xslt/xspec namespace, which is bound to x in these examples.

The document element is a x:description element, whose stylesheet attribute holds a relative URI pointing to the stylesheet that the XSpec document describes. You can also specify:

  • a xslt-version attribute that gives the version of XSLT the stylesheet uses; if you don't specify it, this defaults to 2.0

The x:description element contains a number of x:scenario elements, each of which describes a particular scenario that's being tested. Each x:scenario element has a label attribute that describes the scenario in human language. For example:

<x:scenario label="when processing a para element">
   ...
</x:scenario>

Scenarios fall into four main types:

Matching Scenarios

Matching scenarios hold a x:context element that describes a node to apply templates to. The context can be supplied in two main ways:

  • you can point to a node in an existing document by giving the document URI in the href attribute and, if you want, selecting a particular node by putting a path in the select attribute
  • you can embed XML within the x:context element; the content becomes the context node, although you can also select a node within that XML using the select attribute

The first method is useful if you already have example XML documents that you want to use as the basis of your testing. For example:

<x:scenario label="when processing a para element">
   <x:context href="source/test.xml" select="/doc/body/p[1]"/>
   ...
</x:scenario>

The second method is related to the concept of a mock object: it is an example of some XML which you have created simply for testing purposes. The XML might not be legal; it only needs to have the attributes or content necessary for the particular behaviour that needs to be tested. For example:

<x:scenario label="when processing a para element">
   <x:context>
      <para>...</para>
   </x:context>
   ...
</x:scenario>

The x:context element can also have a mode attribute that supplies the mode to apply templates in.

Function Scenarios

Function scenarios hold a x:call element with a function attribute whose content is a qualified name that is the same as the qualified name of the function you want to call. The x:call element should hold a number of x:param elements, one for each of the arguments to the function.

The x:param elements can specify node values in the same way as the x:context element gets set (described above), or simply by giving a select attribute which holds an XPath that specifies the value. You can specify a name or position attribute for each of the x:param elements; if you don't, the order in which they're specified will determine the order in which they're given in the function call. For example:

<x:scenario label="when capitalising a string">
   <x:call function="eg:capital-case">
      <x:param select="'an example string'"/>
      <x:param select="true()"/>
   </x:call>
   ...
</x:scenario>

will result in the call eg:capital-case('an example string', false()), as will the following:

<x:scenario label="when capitalising a string">
   <x:call function="eg:capital-case">
      <x:param select="true()" position="2"/>
      <x:param select="'an example string'" position="1"/>
   </x:call>
   ...
</x:scenario>

You can also use the @select attribute on <x:param> and nest the content of the context inside to provide a specific data context to the function:

<x:scenario label="When passing a node to function get-child-element-name">
    <x:call function="eg:get-child-element-name">
        <x:param select="/a/b">
            <a>
                <b>
                    <c>Text</c>
                </b>
            </a>
        </x:param>
    </x:call>

    <x:expect label="it should return the name of the child element" select="'c'"/>
</x:scenario>

In this case, the function parameter is <b><c>Text</c></b> (type element(b)). Also note that the whitespace-only text nodes are discarded.

If you specify no @select

<x:scenario label="When passing a node to function get-child-element-name">
    <x:call function="eg:get-child-element-name">
        <x:param>
            <a>
                <b>
                    <c>Text</c>
                </b>
            </a>
        </x:param>
    </x:call>

    <x:expect label="it should return the name of the child element" select="'b'"/>
</x:scenario>

the parameter is <a><b><c>Text</c></b></a> (type element(a)).

If you want to specify the document node, specify so in @select

<x:scenario label="When passing a node to function get-child-element-name">
    <x:call function="eg:get-child-element-name">
        <x:param select="/">
            <a>
                <b>
                    <c>Text</c>
                </b>
            </a>
        </x:param>
    </x:call>

    <x:expect label="it should return the name of the child element" select="'a'"/>
</x:scenario>

and then the parameter is a document node containing <a><b><c>Text</c></b></a> (type document-node(element(a))). select="self::document-node()" also results in the same parameter.

Named Scenarios

Named template scenarios are similar to function scenarios except that the x:call element takes a template attribute rather than a function attribute, and the x:param elements within it must have a name attribute that supplies the name of the parameter. These parameters can also have a tunnel attribute to indicate a tunnel parameter. For example:

<x:scenario label="when creating a table with two columns containing three values">
   <x:call template="createTable">
      <x:param name="nodes">
         <value>A</value>
         <value>B</value>
         <value>C</value>
      </x:param>
      <x:param name="cols" select="2"/>
   </x:call>
   ...
</x:scenario>

In fact, you can use x:param in the same way within the x:context element in matching scenarios.

Expectations

Each scenario can have one or more "expectations": things that should be true of the result of the function or template invocation described by the scenario. Each expectation is specified with an x:expect element. The label attribute on the x:expect element gives a human-readable description of the expectation.

There are two main kinds of expectations:

  • a value that the result should match, which may be
    • an atomic value
    • an XML snippet
  • an arbitrary XPath test that should be true of the result

To specify an atomic value, use the select attribute on the x:expect element. For example:

<x:scenario label="when capitalising a string">
   <x:call function="eg:capital-case">
      <x:param select="'an example string'"/>
      <x:param select="true()"/>
   </x:call>
   <x:expect label="it should capitalise every word in the string" select="'An Example String'"/>
</x:scenario>

To specify some XML, put it within the x:expect element. For example:

<x:scenario label="when processing a para element">
   <x:context>
      <para>...</para>
   </x:context>
   <x:expect label="it should produce a p element">
      <p>...</p>
   </x:expect>
</x:scenario>

One thing to note here is that when comparing the actual result with the expected result, three dots in an element or attribute value within the expected XML means that the values aren't compared. If the actual result is:

<p>A sample para</p>

and the expected result is given as:

<p>...</p>

then these match. If the expected result is:

<p>Some other para</p>

then they don't. For more examples of the three dot feature, see the test scenarios and their results.

To specify an arbitrary XPath test, use the test attribute on x:expect. For example:

<x:scenario label="when creating a table with two columns containing three values">
   <x:call template="createTable">
      <x:param name="nodes">
         <value>A</value>
         <value>B</value>
         <value>C</value>
      </x:param>
      <x:param name="cols" select="2"/>
   </x:call>
   <x:expect label="the resulting table should have two columns"
             test="count(/table/colspec/col) = 2"/>
</x:scenario>

Within the XPath expression, you can use the variable $x:result to access the result of the test (i.e. the result of calling the function or the template, or of applying the template rule). In addition, if the result is a sequence of nodes (except attribute and namespace nodes), it is wrapped in a document node and this document is set as the context node of the expression. (This context node is not available on XQuery.)

You can also combine the test attribute with the content of the x:expect element if you want to just test a portion of the result. For example:

<x:scenario label="when creating a table with two columns containing three values">
   <x:call template="createTable">
      <x:param name="nodes">
         <value>A</value>
         <value>B</value>
         <value>C</value>
      </x:param>
      <x:param name="cols" select="2" />
   </x:call>
   <x:expect label="the resulting table should have two columns"
             test="count(/table/colspec/col) = 2" />
   <x:expect label="the first row should contain the first two values"
             test="/table/tbody/tr[1]">
      <tr>
         <td>A</td><td>B</td>
      </tr>
   </x:expect>
</x:scenario>

Depending on the result, the context item (.) in the test attribute may not always exist and may be different from what you expect. In the test attribute, if you like to inspect the result as is, use $x:result instead of the context item (.).

Nesting Scenarios

You can nest scenarios inside each other. The nested scenarios inherit the context or call from its ancestor scenarios. All the scenarios in a particular tree have to be of the same type (matching, function or named). Usually only the lowest level of the scenarios will contain any expectations. Here's an example:

<x:scenario label="when creating a table">
   <x:call template="createTable" />
   <x:scenario label="holding three values">
      <x:call>
         <x:param name="nodes">
            <value>A</value>
            <value>B</value>
            <value>C</value>
         </x:param>
      </x:call>
      <x:scenario label="in two columns">
         <x:call>
            <x:param name="cols" select="2"/>
         </x:call>
         <x:expect label="the resulting table should have two columns"
                   test="count(/table/colspec/col) = 2"/>
         <x:expect label="the first row should contain the first two values"
                   test="/table/tbody/tr[1]">
            <tr>
               <td>A</td><td>B</td>
            </tr>
         </x:expect>
      </x:scenario>
      ... other scenarios around creating tables with three values (with different numbers of columns) ...
   </x:scenario>
   ... other scenarios around creating tables ...
</x:scenario>

When you create scenarios like this, the labels of the nested scenarios are concatenated to create the label for the scenario. In the above example, the third scenario has the label "when creating a table holding three values in two columns".

Focusing Your Efforts

XSpec descriptions can get quite large, which can mean that running the tests takes some time. There are three ways of dealing with this.

First, you can import other XSpec description documents into your main one using x:import. The href attribute holds the location of the imported document. All the scenarios from the referenced document are imported into this one, and will be run when you execute it. For example:

<x:import href="other_xspec.xml"/>

It helps if the imported XSpec description documents can stand alone; this enables you to perform a subset of the tests. To work effectively, you'll want the imported XSpec description documents to cover the same stylesheet as the main one, or a stylesheet module that's included or imported into that stylesheet.

Second, you can mark any scenario or expectation as "pending" by wrapping them within a x:pending element or adding a pending attribute to the x:scenario element. When the tests are run, any pending scenarios or expectations aren't tested (though they still appear, greyed out, in the test report). The x:pending element can have a label attribute to describe why the particular description is pending; for example it might hold "TODO". If you use the pending attribute, its value should give the reason the tests are pending. For example:

<x:pending label="no support for block elements yet">
   <x:scenario label="when processing a para element">
      <x:context>
         <para>...</para>
      </x:context>
      <x:expect label="it should produce a p element">
         <p>...</p>
      </x:expect>
   </x:scenario>
</x:pending>

or:

<x:scenario pending="no support for block elements yet" label="when processing a para element">
   <x:context>
      <para>...</para>
   </x:context>
   <x:expect label="it should produce a p element">
      <p>...</p>
   </x:expect>
</x:scenario>

Third, you can mark any scenario as having the current "focus" by adding a focus attribute to a <x:scenario> element. Effectively, this marks every other scenario as "pending", with the label given as the value of the focus attribute. For example:

<x:scenario focus="getting capitalisation working" label="when capitalising a string">
   <x:call function="eg:capital-case">
      <x:param select="'an example string'"/>
      <x:param select="true()"/>
   </x:call>
   <x:expect label="it should capitalise every word in the string" select="'An Example String'"/>
</x:scenario>

Using focus is a good way of working through one particular scenario, but once your code is working with that scenario, you should always test all the others again, just in case you've broken something else.

Global Parameters

You can put x:param elements at the top level of the XSpec description document (as a child of the x:description element). These effectively override any global parameters or variables that you have declared in your stylesheet. They are set in just the same way as setting parameters when testing named templates or functions.

With the current implementation, it isn't possible to have different scenarios use different values for global parameters. Testing is made easier if you declare local parameters on any templates or functions that use global parameters; these can default to the value of the global parameter, but be set explicitly when testing. For example, if $tableClass is a global parameter, you might do the following to enable the full testing of the createTable template:

<xsl:template name="createTable">
   <xsl:param name="nodes" as="node()+" required="yes"/>
   <xsl:param name="cols" as="xs:integer" required="yes"/>
   <xsl:param name="tableClass" as="xs:string" select="$tableClass"/>
   ...
</xsl:template>

Note that global parameters and global variables in your stylesheet are evaluated without context item. (. is absent.) For example:

<xsl:stylesheet ...>
   <xsl:param name="p" select="foo" />
   <xsl:variable name="v" select="base-uri()" />

$p and $v rely on . to exist. Testing this stylesheet will fail, unless you supply <x:param name="p"> and <x:param name="v"> in XSpec.

XSpec Variables

You can define variables that are specific to your XSpec description document, as opposed to overrides of variables in the code you are testing. Reasons for using XSpec variables can include reusing data within the XSpec file and naming intermediate results for clarity.

To define an XSpec variable, use the x:variable element as a child of x:description or x:scenario. Each variable has a required name attribute. Consider putting XSpec variable names in your own namespace, although doing so is not required. To specify a data type, use the optional as attribute.

Define the value of the variable using one of these approaches:

  • Use the select attribute alone to provide an XPath expression that specifies the value
  • Use attributes href and, optionally, select to point to a node in an existing document
  • Embed XML within the x:variable element
  • Embed XML within the x:variable element and select a node within that XML using the select attribute

Here are some examples of variable definitions, assuming you have bound a namespace to the prefix myv.

<x:variable name="myv:mystring" select="'text'" as="xs:string"/>
<x:variable name="myv:mydoc" href="mydoc.xml" as="document-node()?"/>
<x:variable name="myv:mysections" href="mydoc.xml" select="//section" as="element()*"/>
<x:variable name="myv:mypara" as="element(p)">
  <p><span>text</span></p>
</x:variable>
<x:variable name="myv:myspan" select="//span" as="element(span)">
  <p><span>text</span></p>
</x:variable>

After defining a variable, you can refer to it by name (prefixed with $) in XPath expressions in the same XSpec file. Valid locations where you can refer to XSpec variables include:

  • select attribute in x:context, x:param, x:expect, and x:variable
  • test attribute in x:expect
  • An attribute value template in user content, such as <x:context><mycontext role="{$myv:myvariable}"/></x:context>

Here is an example showing several references to XSpec variables. The example assumes you have declared namespaces (with prefixes my for code, myv for XSpec variables, and db for elements in the XML content), defined the my:select-figure function, and created a suitable mydoc.xml XML document with a section containing several child figure elements.

<x:scenario label="select-figure function">
  <x:variable name="myv:topic" as="element(db:section)"
    href="mydoc.xml" select="//db:section[@xml:id = 'topicwithimages']"/>
  <x:scenario label="with one nonempty argument">
    <x:call function="my:select-figure">
      <x:param select="$myv:topic"/>
      <x:param select="''"/>
    </x:call>
    <x:variable name="myv:last-figure" as="element(db:figure)"
      select="($myv:topic/db:figure)[last()]"/>
    <x:expect label="selects the last figure in the section"
      test="deep-equal($x:result, $myv:last-figure)"/>
  </x:scenario>
  <x:scenario label="with two nonempty arguments">
    <x:variable name="myv:sample-id" select="'scatterplot'" as="xs:string"/>
    <x:call function="my:select-figure">
      <x:param select="$myv:topic"/>
      <x:param select="$myv:sample-id"/>
    </x:call>
    <x:expect label="selects the figure with specified ID"
      select="$myv:topic/db:figure[@xml:id = $myv:sample-id]"/>
  </x:scenario>
</x:scenario>

Due to inheritance, you can refer to any XSpec variables that are defined in the same scenario, an ancestor scenario, or at the top level.

Attributes in Content

Attribute values in user content (such as x:param and x:context) work as AVT.

Depending on your purpose, you can use an expression:

<x:param name="base-uri" select="'http://www.example.com/'" />

<x:scenario label="when using expression in attribute">
  <x:call function="my:func">
    <!-- Will be resolved as <option file="http://www.example.com/dir/file.txt" /> -->
    <x:param>
      <option file="{$base-uri}dir/file.txt" />
    </x:param>
  </x:call>
  ...
</x:scenario>

or you may need to escape (i.e. double) the curly braces:

<x:scenario label="when escaping curly braces in attribute">
  <!-- Will be resolved as <element path="/Q{http://www.example.com}foo" /> -->
  <x:context>
    <element path="/Q{{http://www.example.com}}foo" />
  </x:context>
  ...
</x:scenario>

Whitespace-only Text Nodes

By default, XSpec discards whitespace-only text nodes when loading user-specified content such as x:param.

In this example

<x:param name="p">
  <span>&#x09;&#x0A;&#x0D;&#x20;</span>
</x:param>

$p is not &#x0A;&#x20;&#x20;<span>&#x09;&#x0A;&#x0D;&#x20;</span>&#x0A; but <span/>.

XSpec keeps a whitespace-only text node intact only when one of the following conditions is met:

  • Its nearest ancestor element with @xml:space has @xml:space="preserve". For example,
    <x:context xml:space="preserve"><span>&#x09;&#x0A;&#x0D;&#x20;</span></x:context>
  • Its parent element name is specified in /x:description/@preserve-space. For example,
    <x:description preserve-space="code pre">
    ...
      <x:param>
        <pre>&#x09;&#x0A;&#x0D;&#x20;</pre>
      </x:param>
  • Its parent element is x:text. For example,
    <x:expect label="Expects a whitespace-only text node">
      <x:text>&#x09;&#x0A;&#x0D;&#x20;</x:text>
    </x:expect>
  • It's in a document loaded via @href. For example,
    <x:param href="space.xml" />

Testing Dynamic Errors

Coming soon. Not available yet.

With <x:scenario catch="yes">, its descendant-or-self scenarios will catch dynamic errors when invoking the tested module. The standard error variables including $err:code and $err:description are collected in $x:result?err as a map. For example, $x:result?err?code stores $err:code. If the tested module didn't throw an error, $x:result contains its return items intact as usual.

Using @catch requires XSLT 3.0 or XQuery 3.1 (only when you set catch="yes"). For Schematron, @catch will work just like XSLT, although it won't make sense.

Stylesheet:

<xsl:template name="my-template-with-error">
  <xsl:sequence
    select="error(
      xs:QName('error-code-of-my-template'),
      'error description of my template',
      ('error', 'object', 'of', 'my', 'template')
      )" />
</xsl:template>

XSpec:

<x:description ... xslt-version="3.0">
  <x:scenario catch="yes" label="Testing an error">
    <x:call template="my-template-with-error" />
    <x:expect label="err:code" test="$x:result?err?code"
      select="xs:QName('error-code-of-my-template')" />
    <x:expect label="err:description" test="$x:result?err?description"
      select="'error description of my template'" />
    <x:expect label="err:value" test="$x:result?err?value treat as xs:string+"
       select="'error', 'object', 'of', 'my', 'template'" />
  </x:scenario>
</x:scenario>

@catch is a rather new feature and its details are subject to change. Any feedback is welcome.

Clone this wiki locally
You can’t perform that action at this time.