add ament_(cmake_)xmllint packages

[dirk-thomas]

https://ci.ros2.org/job/ci_linux/4876/testReport/ as an example of failed linting.

If xmllint is not available (as it is the case on the non-Linux Jenkins nodes) the tests are being skipped: https://ci.ros2.org/job/ci_linux/4875/testReport/junit/(root)/projectroot/

With other PRs merged this passes CI:

[nuclearsandwich]

@ros-pull-request-builder retest this please

[mikaelarguedas]

Does this check the validity of the xml against the schema ? or just xml validity in general (e.g. mismatching/missing tags)?

Edit: scratch this: I just saw https://github.com/ament/ament_lint/pull/104/files#diff-2b64320c40b951cc04ac6618030de634R89

[mikaelarguedas]

code change LGTM, with one request for additional doc.

Could you add to the documentation what xml syntaxes are supported for defining schemas?
The DDS-Security xml files are currently not being validated against the schema.
If it's useful:
DDS-Security permission xsd: https://www.omg.org/spec/DDS-SECURITY/20170901/omg_shared_ca_permissions.xsd
DDS-Security permission xml example: https://www.omg.org/spec/DDS-SECURITY/20170901/omg_shared_ca_permissions_example.xml
Issue tracking the fact that the generated XMLs are not valid: ros2/sros2#49

[mikaelarguedas]

what is the cmake 3.8 feature being used here ? From @dirk-thomas DISABLED used L66 is the answer

[dirk-thomas]

The commit d874409 adds support for specifying the schema in the xsi:noNamespaceSchemaLocation attribute of the root tag.

[mikaelarguedas]

error : Unknown IO error
warning: failed to load external entity "http://www.omg.org/spec/DDS-SECURITY/20160303/omg_shared_ca_governance.xsd"
Schemas parser error : Failed to locate the main schema resource at 'http://www.omg.org/spec/DDS-SECURITY/20160303/omg_shared_ca_governance.xsd'.
WXS schema http://www.omg.org/spec/DDS-SECURITY/20160303/omg_shared_ca_governance.xsd failed to compile

Arg that doesn't look good. I suggest to revert / remove the last commit and merge this as is. A follow-up ticket / PR can be opened with a reference to the commit so that we can look into reenabling it when we find time to look into it.

[dirk-thomas]

The url http://www.omg.org/spec/DDS-SECURITY/20160303/omg_shared_ca_governance.xsd seems to redirect to https. xmllint might not be able to follow that. Please try it with the non-redirected URL.

[mikaelarguedas]

This was just copied from the CI job output.

Just tried locally and the error is the same with the https URL:

$ ament_xmllint ./test/test_security_files/governance.xml 
File 'test/test_security_files/governance.xml' is invalid:
warning: failed to load external entity "https://www.omg.org/spec/DDS-SECURITY/20160303/omg_shared_ca_governance.xsd"
Schemas parser error : Failed to locate the main schema resource at 'https://www.omg.org/spec/DDS-SECURITY/20160303/omg_shared_ca_governance.xsd'.
WXS schema https://www.omg.org/spec/DDS-SECURITY/20160303/omg_shared_ca_governance.xsd failed to compile

1 files are invalid

[dirk-thomas]

Can you try it with xmllint directly.

[dirk-thomas]

With the downloaded schema it works for me: xmllint ./src/ros2/system_tests/test_security/test/test_security_files/governance.xml --schema omg_shared_ca_governance.xsd

[mikaelarguedas]

$ xmllint -schema https://www.omg.org/spec/DDS-SECURITY/20160303/omg_shared_ca_governance.xsd test/test_security_files/governance.xml

Fails: with

Schemas parser error : Failed to locate the main schema resource at 'https://www.omg.org/spec/DDS-SECURITY/20160303/omg_shared_ca_governance.xsd'.
WXS schema https://www.omg.org/spec/DDS-SECURITY/20160303/omg_shared_ca_governance.xsd failed to compile

But this works:

$ wget https://www.omg.org/spec/DDS-SECURITY/20160303/omg_shared_ca_governance.xsd
$ xmllint -schema ./omg_shared_ca_governance.xsd test/test_security_files/governance.xml 

I observe the same behavior with their new schema: https://www.omg.org/spec/DDS-SECURITY/20170901/omg_shared_ca_governance.xsd

[dirk-thomas]

The problem is actually not the redirect. xmllint doesn't support https URLs. See e.g. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=679867

Do you want to find the same schema on a http url and use that, or host the schema on http://download.ros.org/schema/ or should we disable the linter for that package?

[mikaelarguedas]

Let's disable that package for now as it would fail the test even if it was finding the schema properly.

We can revisit later, either use a linter that supports https or a non-secured hosting solution for these xsds

[dirk-thomas]

Let's disable that package for now as it would fail the test even if it was finding the schema properly.

Done in ros2/system_tests#287.

[mikaelarguedas]

👍

[dirk-thomas]

Follow up commit 0ad4e0f to avoid stderr output.

[mikaelarguedas]

@dirk-thomas These linter tests ran on MacOS last night (I was not expecting it based on the PR description).
Do we know what system dependency needs to be installed to get the xmllint binary ?

[dirk-thomas]

Do we know what system dependency needs to be installed to get the xmllint binary ?

xmllint - on Ubuntu it is part of the Debian package libxml2-utils (see ros2/ci#207).

[mikaelarguedas]

Does this mean that the package xmllint has been installed from brew on all MacOS nodes? (if so can you please update the buildfarmer logbook)
But not on Windows because no choco packages for it?

I'm asking because we will need this information to update the node setup instructions and eventually user install instructions

[dirk-thomas]

Does this mean that the package xmllint has been installed from brew on all MacOS nodes? (if so can you please update the buildfarmer logbook)

I have not touched any macOS node. It seems the executable was already available.

But not on Windows because no choco packages for it?

I didn't see a choco package but didn't look for long since it should be sufficient if one platform runs the linter.