Skip to content

Commit

Permalink
Add XSLT transformations for iia-hash element
Browse files Browse the repository at this point in the history
  • Loading branch information
mkurzydlowski committed Aug 1, 2023
1 parent 326ff0d commit dd0f8a5
Show file tree
Hide file tree
Showing 14 changed files with 527 additions and 25 deletions.
18 changes: 18 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,24 @@ document clarifies the requirements for the technical solutions
developed under EWP and in the local implementation that should adequately support
the business processes related to the approval of IIAs at Higher Education Institutions.


### IIA hash calculation


As of IIA version 7 each agreement contains an `iia-hash` element that replaces the `conditions-hash` element
used in previous versions of this API.

To calculate the new hash an IIA get response XML has to be transformed using the appropriate XSLT template provided:
* [XSLT template for IIA version 6](resources/xsltKit/transform_version_6.xsl),
* [XSLT template for IIA version 7](resources/xsltKit/transform_version_7.xsl).

You can test these transformations using the provided [Java class](resources/xsltKit/XsltTest.java)

You may need to find the right XSLT processor for these templates to work.
For Java [Saxon-HE-9.5.1-8.jar](http://www.java2s.com/example/jar/s/download-saxonhe9518jar-file.html) processor works,
the previous versions fail. For more details please go to [XSLT kit resources](resources/xsltKit).


Security
--------

Expand Down
2 changes: 1 addition & 1 deletion endpoints/get-response-example.xml
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,7 @@
<total-days-per-year>14</total-days-per-year>
</staff-training-mobility-spec>
</cooperation-conditions>
<conditions-hash>7c045bc4ca23b3b9953adb27374aa27dcd41cfdda74fff9d2240a813a80443ae</conditions-hash>
<iia-hash>5071f1be5a672650fabc4d8a82887f8a75e0aaa784a61482c3c684abb2045487</iia-hash>
<pdf-file>4a483112-d306-4e77-87ff-05b823fb0e0f</pdf-file>
</iia>

Expand Down
22 changes: 5 additions & 17 deletions endpoints/get-response.xsd
Original file line number Diff line number Diff line change
Expand Up @@ -192,10 +192,8 @@
<xs:documentation>
List of all cooperation conditions defined in this agreement.

If you are sending conditions-hash, be consistent in ordering cooperation conditions.
The hash SHOULD NOT change if the cooperation conditions are not really changing.
An exception to this are the `sending-contact` and `receiving-contact` subelements
that are not taken into account when calculating hash.
You MUST be consistent in ordering cooperation conditions as it is used to
produce the `iia-hash` value.
</xs:documentation>
</xs:annotation>
<xs:complexType>
Expand Down Expand Up @@ -241,21 +239,11 @@
</xs:attribute>
</xs:complexType>
</xs:element>
<xs:element name="conditions-hash" type="ewp:Sha256Hex" minOccurs="0" maxOccurs="1">
<xs:element name="iia-hash" type="ewp:Sha256Hex" minOccurs="1" maxOccurs="1">
<xs:annotation>
<xs:documentation>
The SHA-256 digest of the cooperation-conditions element but *excluding*
`sending-contact` and `receiving-contact` subelements. Before
calculating the hash, the cooperation-conditions element MUST be normalized
using Exclusive XML Canonicalization.

Please be aware that XML Canonicalization does not imply that spaces are removed:
https://www.w3.org/TR/xml-c14n2/#sec-Requirements-Robustness

This element is not required. However, if it is not present, your partner
will not be able to approve your version of the agreement using EWP IIAs Approval API.
If you want to get approval of your agreements, then you should always send
the conditions hash.
The SHA-256 digest of the `text-to-hash` element content that is obtained
by using the appropriate XSLT transformation mentioned in the API specification.
</xs:documentation>
</xs:annotation>
</xs:element>
Expand Down
2 changes: 1 addition & 1 deletion example-scenario/01-A-get-response.part.xml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
<eqf-level>8</eqf-level>
</student-studies-mobility-spec>
</cooperation-conditions>
<conditions-hash>6bab23655105e47adffa15ef5409046bad7d3bc6c9b449f23ee1865bc55ee15e</conditions-hash>
<iia-hash>160b06656cb621eba6b86d0a5c3d9e37a38f782296be28242d82335726dcbde6</iia-hash>
<!-- This is a PDF generated and signed by A. -->
<pdf-file>f0f4934c-d384-41c3-a455-9bb9a689a5a3</pdf-file>
</iia>
Expand Down
2 changes: 1 addition & 1 deletion example-scenario/02-B-get-response.part.xml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
<eqf-level>8</eqf-level>
</student-studies-mobility-spec>
</cooperation-conditions>
<conditions-hash>cdcdf8bb7ce3f028a0cc6f7e848199a592a74b5014107ed4f4f6d4103421fa0a</conditions-hash>
<iia-hash>81aed817c4941cd4e8e424619de69578883b288ef4c00fd356133bc93d236780</iia-hash>
<!-- B wants to have on the PDF his logo and some additional information, so he generates his own PDF. If both -->
<!-- partners use the same PDF format, B could send here PDF fetched from A and signed by B (the same as in B's -->
<!-- approval response). -->
Expand Down
2 changes: 1 addition & 1 deletion example-scenario/03-B-approval-response.part.xml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
<iias-approval-response>
<approval>
<iia-id>0f7a5682-faf7-49a7-9cc7-ec486c49a281</iia-id>
<conditions-hash>6bab23655105e47adffa15ef5409046bad7d3bc6c9b449f23ee1865bc55ee15e</conditions-hash>
<iia-hash>160b06656cb621eba6b86d0a5c3d9e37a38f782296be28242d82335726dcbde6</iia-hash>
<!-- This is a PDF that B has fetched from A via IIAs API and signed. So it is generated by A and signed by both
partners. -->
<pdf-file>85ba5392-fd35-4baa-9afa-220537d2ebbe</pdf-file>
Expand Down
2 changes: 1 addition & 1 deletion example-scenario/04-A-approval-response.part.xml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
<iias-approval-response>
<approval>
<iia-id>79364</iia-id>
<conditions-hash>cdcdf8bb7ce3f028a0cc6f7e848199a592a74b5014107ed4f4f6d4103421fa0a</conditions-hash>
<iia-hash>81aed817c4941cd4e8e424619de69578883b288ef4c00fd356133bc93d236780</iia-hash>
<!-- This is a PDF that A has fetched from B via IIAs API and signed. So it is generated by B and signed by both
partners. -->
<pdf-file>6f09ac08-454a-4278-a290-16f55663237d</pdf-file>
Expand Down
7 changes: 4 additions & 3 deletions resources/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,15 @@ IIA templates mentioned below are previous versions of the template.

### Additional resources

This folder contains some additional documents related to IIAs API.
This folder contains some additional documents related to IIA API:

* [XSLT kit](xsltKit) contains XSLT templates and sample implementations for IIA hash calculation,
* [`template_EuropeanCommission_IIA_14-21.pdf`](template_EuropeanCommission_IIA_14-21.pdf)
contains the IIA template prepared by the European Commission, defining
requirements for Inter-Institutional Agreements for years 2014-20[21].
requirements for Inter-Institutional Agreements for years 2014-20[21],
* [`template_EuropeanCommission_IIA_21-29.pdf`](template_EuropeanCommission_IIA_21-29.pdf)
contains the IIA template prepared by the European Commission, defining
requirements for Inter-Institutional Agreements for years 2021-20[29].
requirements for Inter-Institutional Agreements for years 2021-20[29],
* [`mandatory_business_requirements_IIA.pdf`](mandatory_business_requirements_IIA.pdf)
contains the business requirements and processes for IIAs set up in the context of EWP.

Expand Down
93 changes: 93 additions & 0 deletions resources/xsltKit/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
The XSLT templates, Java class example and this explanation are based on
[this comment](https://github.com/erasmus-without-paper/ewp-specs-api-iias/issues/109#issuecomment-1593875245)
and inspired by [this comment](https://github.com/erasmus-without-paper/ewp-specs-api-iias/issues/109#issuecomment-1569982153).

### Short explanation

The result of the [XSLT transformation](transform_version_6.xsl) for
[IIA version v6 example](https://raw.githubusercontent.com/erasmus-without-paper/ewp-specs-api-iias/stable-v7/resources/xsltKit/get-response-v6.xml)
is something similar to this:

```
<?xml version="1.0" encoding="UTF-8"?>
<iia>
<iia-id>0f7a5682-faf7-49a7-9cc7-ec486c49a281</iia-id>
<text-to-hash>_0f7a5682-faf7-49a7-9cc7-ec486c49a281__1954991__uw.edu.pl__140__hibo.no__031__Social and behavioural sciences__5__false__7__8__2014/2015__2020/2021__uw.edu.pl__140__hibo.no__2__en__C1__0314__8__2016/2017__2017/2018_</text-to-hash>
</iia>
```

And it is the same result we obtain if we apply the [XSLT transformation](transform_version_7.xsl) for
[IIA version v7 example](https://raw.githubusercontent.com/erasmus-without-paper/ewp-specs-api-iias/stable-v7/resources/xsltKit/get-response-v7.xml).

We should next hash the value of the `text-to-hash` element with an SHA-256 algorithm to obtain the new `iia-hash` element value.

The transformation can manage changes to IIA introduced in version 7 of the API. It also reflects changes to IIA mapping.
Thanks to the `not-yet-defined` attribute it manages elements that are required by the new version but cannot be provided automatically for the old IIAs.
Also `v6-value` attribute of the `isced-f-code` element is used to acquire the old ISCED code value as defined in IIA version 6.

When an IIA has at least one element with the attribute `not-yet-defined`,
the transformation adds a `<valid-for-approval>false</valid-for-approval>` sub-element as the last child of the `<iia>` element.

To transform an XML you may need to find the right processor,
e.g. for Java [Saxon-HE-9.5.1-8.jar](http://www.java2s.com/example/jar/s/download-saxonhe9518jar-file.html) works while the previous versions fail.

We provide an example in Java that transforms an IIA XML (as a byte array) via the XSLT (as a byte array too) and obtain the above-mentioned XML:

```
public String getXmlTransformed(byte xmlBytes[], byte xsltBytes[]) throws Exception
{
DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
dbf.setNamespaceAware(true);
DocumentBuilder db = dbf.newDocumentBuilder();
Document document = db.parse(new ByteArrayInputStream(xmlBytes));
TransformerFactory transformerFactory = TransformerFactory.newInstance();
Transformer transformer = transformerFactory.newTransformer(
new StreamSource(new ByteArrayInputStream(xsltBytes)));
ByteArrayOutputStream output = new ByteArrayOutputStream();
transformer.transform(new DOMSource(document), new StreamResult(output));
return new String(output.toByteArray());
}
```

The solution described here consists of the following resources:
- [transform_version_6.xsl](transform_version_6.xsl) to be used with your IIA v6,
- [transform_version_7.xsl](transform_version_7.xsl) to be used with your IIA v7,
- [XsltTest.java](XsltTest.java) to transform an IIA XML using one of the above-mentioned XSLTs.


### Further explanation


The XSLT produces one `<iia>` element for every `<iia>` element present in the IIA Get response.
The `<iia-id>` sub-element can be used to identify each IIA.

Every `<iia>` element has two sub-elements and optionally a third one:
- `<iia-id>` to indicate the associated IIA ID,
- `<text-to-hash>` to expose the string built as a concatenation of all the leaf elements (elements without children)
separated by an underscore (e.g.: `_myValue_`). Before every element we have the values of its attributes (if they exist);
they are represented inside two markers` _@ `and `@_ `(e.g. `_@attributeValue@_`). There are some additional rules:
- the `<text-to-hash>` element contains both IIA IDs (sending and receiving); so it changes when IIA mapping changes,
- the `<text-to-hash>` element does not contain:
- the sending/receiving contacts (for both of v6 and v7 XSLT)
- the `receiving-academic-year-id` (only for v6 XSLT)
- the `receiving-first-academic-year-id` and the `receiving-last-academic-year-id` (only for v7 XSLT)
- the `<text-to-hash>` element contains, at the end of every mobility:
- for XSLT v6: the values of the first and last occurrence of the receiving-academic-year-id element
- for XSLT v7: the values of the receiving-first-academic-year-id and receiving-last-academic-year-id
- the `<text-to-hash>` element does not contain: (only for v7)
- all the elements having the attribute `not-yet-defined`; this attribute is never printed in the output,
- the `isced-f-code` element value is replaced with the `v6-value` attribute value if defined (only for v7),
- `<valid-for-approval>` is present and set to false if the IIA v7 contains:
- an element with the `not-yet-defined` attribute set to true,
- an element with the `v6-value` attribute set to a non-empty value.

To test the XSLT against your IIA Get response XMLs you can use those online tools:
- https://linangdata.com/xslt-tester/
- http://xsltransform.net/

To compute the hash code you can use:
- https://www.fileformat.info/tool/hash.htm
76 changes: 76 additions & 0 deletions resources/xsltKit/XsltTest.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;
import javax.xml.transform.stream.StreamSource;
import org.w3c.dom.Document;

/**
* Class to test the XSLT.
* The first parameter is the IIA xml file name, the second is the xlst stylesheet file name.
* They both must be in the directory from where you invoke this class
*
* @author Francesco De Milato
*/
public class XsltTest
{
public XsltTest (){}

public static void main(String args[]) throws Exception
{
XsltTest xml = new XsltTest();
System.setProperty(
"javax.xml.transform.TransformerFactory","net.sf.saxon.TransformerFactoryImpl");

if (args.length!=2) System.out.println("ERROR!\nSintax: java it.unige.ewp.tools.XmlTest <iiaFileName> <xsltFileName>");
else
{
String iiaFileName = args[0];
String xsltFileName = args[1];

Path xsltPath = Paths.get(xsltFileName);
Path iiaPath = Paths.get(iiaFileName);

byte[] xsltData = Files.readAllBytes(xsltPath);
byte[] iiaData = Files.readAllBytes(iiaPath);

System.out.println("\n\nXSLT Result: \n"+xml.getXmlTransformed(iiaData, xsltData));
System.out.println("\n\nBe careful to use the right xslt for your IIA Version!");
System.out.println("Now you can get the element text-to-hash and compute its SHA-256 hash code\n\n");
}
}

/**
* Transform an xml file by means of an xslt file
*
* @param xmlBytes The content of the xml file
* @param xsltBytes The content of the xslt file
* @return An xml useful to compute the hash code
* @throws Exception
*/
public String getXmlTransformed(byte xmlBytes[], byte xsltBytes[]) throws Exception
{
DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
dbf.setNamespaceAware(true);

DocumentBuilder db = dbf.newDocumentBuilder();
Document document = db.parse(new ByteArrayInputStream(xmlBytes));

TransformerFactory transformerFactory = TransformerFactory.newInstance();
Transformer transformer = transformerFactory.newTransformer(
new StreamSource(new ByteArrayInputStream(xsltBytes)));
ByteArrayOutputStream output = new ByteArrayOutputStream();

transformer.transform(new DOMSource(document), new StreamResult(output));

return new String( output.toByteArray());
}
}

Loading

3 comments on commit dd0f8a5

@jiripetrzelka
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is the terminated-as-a-whole attribute not part of the hashed string? According to the voting results from 7 June 2023, termination as a whole should be treated like any other modification/termination, which, I suppose, implies that it should require a re-approval. Or am I missing something?

@mkurzydlowski
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right that it should be part of the hashed string to meet the requirements. The issue now is how to modify the XSLT without changing the major version of IIA.

@demilatof
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right that it should be part of the hashed string to meet the requirements. The issue now is how to modify the XSLT without changing the major version of IIA.

There is no need to change the IIA version; bugs happen, improvements too...
Just replace the XSLT with a new one, the XSD remains the same.
And no one should trust in the XSLT until deeply tested

The new XSLT (v7) could start from here

Please sign in to comment.