DMN16-130/DMN16-130 Use fixed Namespace URIs and xsi:schemaLocation

[falko]

Currently, we identify new versions of DMN by changing the XML namespace with every revision. This breaks backwards compatibility, DMN files using older versions become invalid and have to be migrated to the new namespace to be validated with the new XML schema.

However, the DMN Revision Task Force makes only backwards-compatible changes to the language. So there is no need for such an aggressive versioning approach

The work on SCE made me review the OMG Policy for Versioning of Specification URIs, File URIs, and XML Namespaces and I discovered some interesting options...

Proposal

From DMN 1.6 and all future 1.x revisions, DMN uses fixed XML namespace URIs:

• https://www.omg.org/spec/DMN/ (this is also OMG's public homepage for the DMN specification)
• https://www.omg.org/spec/DMN/DI/ (will redirect to the latest DMNDI.xsd)

In addition to that, a version attribute on the root element indicates which concrete DMN version is used in a file:
<dmn:definitions version="1.6" ...

To indicates which exact DMN version is used and to make it easier for tools to find the right schema an xsi:schemaLocation attribute is added which points to the correct schema file for that version which is hosted by OMG at a fixed location:

omg-dmn-spec/examples/Diagram Interchange/diagram-interchange-dish-example.dmn

Lines 8 to 18 in 20def18

	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="
	https://www.omg.org/spec/DMN/
	https://www.omg.org/spec/DMN/20240322/DMN16.xsd
	https://www.omg.org/spec/DMN/DI/
	https://www.omg.org/spec/DMN/20240322/DMNDI16.xsd
	http://www.omg.org/spec/DMN/20180521/DI/
	https://www.omg.org/spec/DMN/20180521/DI.xsd
	http://www.omg.org/spec/DMN/20180521/DC/
	https://www.omg.org/spec/DMN/20180521/DC.xsd
	">

Tools like VS Code will immediately use that xsi:schemaLocation to download the XML schema and perform validation, which would be a great service to implementers and users who look at the XML. With xsi:schemaLocation in every DMN file there is no need to manually download the schema files and configure an XML catalog or otherwise instruct the XML validator, where to find the schema.

The OMG policy does not apply for the FEEL language URI because is not a real XML namespace, i.e. there is no XML schema for it and it is basically just a single string value. I propose to use the following URI including a semantic version number that would still be updated with every DMN revision:

• https://www.omg.org/spec/DMN/FEEL/1.6/ (could potentially redirect to a machine-readable FEEL grammar)

omg-dmn-spec/xsd/DMN16.xsd

Lines 59 to 60 in 20def18

	<xsd:attribute name="expressionLanguage" type="xsd:anyURI" use="optional" default="https://www.omg.org/spec/DMN/FEEL/1.6"/>
	<xsd:attribute name="typeLanguage" type="xsd:anyURI" use="optional" default="https://www.omg.org/spec/DMN/FEEL/1.6"/>

References

https://www.xfront.com/Versioning.pdf?page=3
https://xml.coverpages.org/HP-StephensonSchemaBestPractices.pdf?page=8

[falko]

@sringuette @BPMAdvantage I'd be interested in your feedback on this idea. If it works for DMN, I'd also make that the approach for SCE and SDMN. This approach could also be a gentle way to improve BPMN's XML schema without immediately fragmenting the tool market.

[SimonRinguette]

I have no problem with that, it would be a marginal improvements from my point of view on code generation.

[BPMAdvantage]

I have no problem with this if it doesn't delay any other issue resolutions

[barmac]

So this means that there is a shared namespace for DMN 1.6 and future versions, but the dmn:Definitions#version attribute defines the specification version. How should a tool handle mismatch between the version attribute and the given schema location? My understanding would be that the version attribute has priority, and schema location is optional.

[nikku]

From our (Camunda Modeling) perspective it would not make a big difference as we're forward migrating diagrams to the latest version we support anyway.

The good thing about the old (existing) approach is that it prevents issues like #33 (comment).

To me it feels a little bit as we work around the intention of XML schemata + namespaces with the new approach.

---

Also, schema location is optional and we should not use it to derive the used DMN version (but instead use xmlns + version attribute as the source of truth).

So to your point @barmac: Tools that play nicely with their users may update the schema location, as they migrate diagrams to a newer version.

[barmac]

Potentially we could fix the version attribute for given schema, but then we end up with the previous situation of "new schema shows old file as invalid":

- <xsd:attribute name="version" type="xsd:decimal" use="required"/>
+ <xsd:attribute name="version" type="xsd:decimal" use="required" fixed="1.6"/>

[falko]

Notes from today's SDMN meeting:

• The spec text will enforce xsi:schemaLocation to be mandatory for vendors to put in every DMN file
• We should make sure that this doesn't cause issues for vendors, e.g. their parsers insisting on using the online version over a locally provided one
• version can be left out and xsi:schemaLocation is enough to identify the version of all dependencies
• Does this technically prevent BC breaks if necessary?

[falko]

• I could re-run my tests for previous DMN versions to validate if this approach would have worked say since DMN 1.1 or to see how BC breaks if any would have affected this
• A PR to add xsi:schemaLocation to the models of the DMN TCK would show if it affects the schema validation there
• the BPMN MIWG tools could also be tried for impacts of xsi:schemaLocation

[nikku]

Will you/we guarantee the following:

• An editor A openes a DMN v1.8 diagram as an editor that can only read DMN v1.6
• A makes changes to the diagram (knowing only 1.6 semantics)
• A saves the diagram
• Another editor B knowing v1.8 semantics will find a completely consistent v1.8 diagram

Otherwise I don't understand how this approach improves the existing version scheme if all editors and engines have to do (right now) is to:

• Check if the DMN diagram they want to open is in a known version they support (for reading)
• Transform it (one way operation) to a DMN version they support for opening (and writing for editors)
• Open the diagram.
• As an editor, make changes + write in the new diagram version

[falko]

@nikku thank you for your feedback and questions. To protect the investments of vendors and users, DMN has been following the OMG's policy of backwards compatibility since version 1.1 (1.0 was unimplementably broken):

A machine readable file is backwards compatible with an earlier version if:

• Any implementation designed to work with the earlier version of the file will continue to interoperate with implementations designed to work with the newer file version.
• In general, this implies that no productions specified in the file will be removed in a backwards compatible version with that file alias
• Additional productions defining optional elements and types may be specified in the backwards compatible file, as long as an implementations not using these new optional features will interoperate with implementations designed to
  use the new features.
• Names of elements and types in the earlier version of the file MUST NOT be
  changed in a backwards compatible revision of that file

We even have a tool to test for BC breaks before a new release.

So in summary, DMN files created with newer versions are perfectly fine to open and edit with older tools unless new language features are used. I estimate that only small percentage of models actually use new features.

So the majority of models, could still be opened and edited in older tools if it weren't for the namespace to completely prevent that. With the new mechanism tool vendors have more options to handle older & newer versions more gracefully:

• It is still possible to read the mandatory schemaLocation to identify the DMN version indicated by the exporting tool
  • This can be used to display a warning if an older tool opens a newer file
• In addition, a validation using the XML schema of the version that the tool supports can be used to check if any unsupported features are used
  • Similarly, the FEEL parser can check for unsupported syntax from newer FEEL versions

In hindsight, it was foolish to just rely on a namespace URI to assert if a file is valid DMN. XML schema validation should be performed as often as possible to check files before reading and after writing. Over the years, we have seen a lot of interchange problems in the BPMN Model Interchange Working Group (MIWG) simply because vendors have not performed schema validations. So this proposal would further incentivize validation and could lead to better interchange in the long-run.

[falko]

The habit to always change the namespace has somewhat accidentally grown:

• DMN 1.2 had to change the namespace because the one from DMN 1.1 did not conform to OMG's policy due to last-minute bad advice from the technical director at the time.
• DMN 1.3 jumped on that to abuse it as a version indicator
• DMN 1.4 and DMN 1.5 followed to be "consistent with the scheme"

With DMN shipping bugfixes on a yearly pace, the fragmentation of the tool market increases, while the number of changes to the XML schema decreases. DMN 1.6 has no changes to the XML schema at all but it if we follow the "tradition" would declare itself totally incompatible with DMN 1.5 and all prior versions.

There is an even more specific OMG definition for backwards compatibility of XML schemas:

The following are examples of backwards compatible changes that
would not require assignment of a new namespace URI:

• addition of new global element, attribute, complexType or simpleType
  definitions
• modifications to the cardinality of elements for which the value-space of possible instance documents conformant to the previous revision of the schema would still be valid with regards to the revised cardinality rule

By that definition DMN 1.6 should not change its namespace URI at all, which shows that is was a bad design choice to use it as a version marker.

[nikku]

Thanks for elaborating in depth on this @falko! If there is clear indication from other tool vendors and users that this is an issue I have no objections to change course. It should not cause many issues on our end (other than adapting our current migration strategy).

The only thing I'd as is to define a clear mechanism to identify which DMN version a diagram adheres to so that I as an editor can handle the miss-match in one way or the other.

[falko]

We wouldn't be the first ones to do xsi:schemaLocation sniffing

[falko]

@opatrascoiu @baldimir @tarilabs @etirelli @agilepro @brsilver @garyhallmark do you have any opinion on this new DMN namespace & versioning scheme?

[SimonRinguette]

Personal 2 cents,

I would be ok with keeping the xsd:schema to the same URI but using the xsi:schemaLocation would not be my preferred way of handling the version. I would prefer a version attribute on the root element.

[falko]

@SimonRinguette the version property was criticized for being redundant with the xsi:schemaLocation, which would force tools to deal with inconsistencies. Also the URL in the xsi:schemaLocation would be quite similar to the current namespace URIs.

[SimonRinguette]

@falko Not going to die on that hill. xsi:schemaLocation could also be used to refer to shemas defined locally on your disk, just giving you my opinion.

[falko]

@SimonRinguette no worries. Your feedback is welcome. What also made me lean towards schemaLocation-only is its capability to identify exact versions for all dependencies, e.g.

<?xml version="1.0" encoding="UTF-8"?>
<sdmn:sharedDataModel id="HelloWorldDataModel"
   targetNamespace="https://example.org/hello-world/shared-data-model"
   xmlns="https://example.org/hello-world/shared-data-model"
   xmlns:sdmn="https://www.omg.org/spec/SDMN/"
   xmlns:sce="https://www.omg.org/spec/SCE/"
   xmlns:scedi="https://www.omg.org/spec/SCE/SCEDI/"
   xmlns:di="https://www.omg.org/spec/SCE/DI/"
   xmlns:dc="https://www.omg.org/spec/SCE/DC/"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xsi:schemaLocation="
     https://www.omg.org/spec/SDMN/
     https://www.omg.org/spec/SDMN/20240210/SDMN.xsd
     https://www.omg.org/spec/SCE/
     https://www.omg.org/spec/SCE/20240210/SCE.xsd
     https://www.omg.org/spec/SCE/SCEDI/
     https://www.omg.org/spec/SCE/20240210/SCEDI.xsd
     https://www.omg.org/spec/SCE/DI/
     https://www.omg.org/spec/SCE/20240210/DI.xsd
     https://www.omg.org/spec/SCE/DC/
     https://www.omg.org/spec/SCE/20240210/DC.xsd
   ">
 <!-- ... -->
</sdmn:sharedDataModel>

I'm not saying that I want to open the door for arbitrary combinations of those but the backwards compatibility would in theory even allow it.

[baldimir]

+1 from me to this.

[falko]

I hate to say it, but this proposal will have to wait until DMN 1.7 because it would otherwise conflict with the adoption of some other SCE features, which I would like to propose in DMN 1.7. So, for this release, I'll prepared yet another switch to a dated namespace URI in #36