XML attached to email is always in edge data format

[ebruchez]

We implement the POST of the data to the email service as a mode change, which always passes the data in DataFormatVersion.Edge format.

[ebruchez]

However, because DataFormatVersion.Edge was used, by default the format sent would have been the 4.8.0 format since Orbeon Forms 4.8, and 2019.1.0 format since Orbeon Forms 2019.1.

The current problem is probably occurring from a user upgrading from Orbeon Forms 2018.2 to Orbeon Forms 2022.1. So the data format used was probably 4.8.0.

Not changing the format is not great, because of this issue. Changing it to 4.0.0 could break things for more users.

• We should at least make the format controllable with a data-format-version action parameter to the email action.
• We should consider whether to change the default to 4.0.0 or 4.8.0 (which might be more compatible but more confusing)

[ebruchez]

There is a problem sending internally the XML in the 4.0.0 format or 4.8.0 format.

The data is used by email-form.xsl. The data is passed to searchHolders() and other functions, which calls findBindPathHoldersInDocument()`, which uses the bind hierarchy to find data holders.

The bind hierarchy comes from the XHTML of the form definition. This means the form definition could follow the 4.0.0, 4.8.0, or 2019.1.0 format.

This should be the case only if the form has been republished/upgraded, but that is a common scenario.

This means that things can break and we need a better solution.

[ebruchez]

The solution is to migrate the form data to the desired format separately.

[ebruchez]

• migrate data
• handle overriding of resources, see Allow form authors to specify success and error messages in Form Builder #5833
• support data-format-version param
• doc

[ebruchez]

When we call the email service, se use tryChangeMode(), which calls trySend().

tryChangeMode() is used by:

• trySendEmail()
• tryNavigateToReview()
• tryNavigateToEdit()
• tryOpenRenderedFormat() (for PDF or TIFF, which run the form, and for Excel/XML export, which may or may not)

It's "interesting" to see what happens with the data format in these cases. For the uses that run the form, there needs to be agreement between the XML data format sent and received. Right now, the "edge" format is hardcoded and the intent is that the form data is converted to "edge", sent, received on the other side (persistence-model.xml), and decoded from "edge".

Now, "edge" means the latest format supported by Orbeon Forms, which is right now 2019.1.0. A published form definition, as is the case here, contains, in the metadata instance, information about how to perform the data migration.

But a form definition can be, even with the latest Orbeon Forms version, still in 4.8.0 format, because form definitions are migrated to "edge" only when loaded in Form Builder or explicitly upgraded from the Admin page.

If, for a form definition in 4.8.0 format, you ask to migrate to the "edge" (2019.1.0) format, the migration information will be missing from the metadata, and so the data will in fact not be migrated to the 2019.1.0 format. But the XML data will contain an fr:data-format-version="2019.1.0", which is incorrect. Also, a data-format-version URL parameter will be present, but in fact incorrect.

For correctness and sanity purposes, we should fix that and use the version from getOrGuessFormDataFormatVersion() instead.

[ebruchez]

Documented the new data-format-version parameter