Use XSD to lint tools and repositories. #130
Conversation
- Implement validation abstraction that will use either lxml (Python lib) or xmllint (command line app) dependending on what is available - with test cases. - Add a ``--xsd`` flag to the lint command that lints against the experimental XSD from https://github.com/JeanFred/Galaxy-XSD. - Implement ``repository_dependencies.xsd`` to describe Tool Shed ``repository_dependencies.xml`` files (fairly complete). - Implement ``tool_dependencies.xsd`` to describe Tool Shed ``tool_dependencies.xml`` files. - Validates attributes and elements down to the ``action`` elements and then largely gives up (sticking ``any`` and ``anyAttribute`` tags on that element). - Registers everything in tools-devteam as valid, and detects one invalid XML file in tools-iuc. - Implement new ``shed_lint`` command that: - Validates tool_dependencies.xml against schema - Validates repository_dependencies.xml against schema - Bare minimum to lint .shed.yml files. - Optionally also lints tools in repsitories with the ``--tools`` argument. - Can recursively lint many repositories at ont time ``-r``. - Refactoring of existing stuff to support this and make room for XSD validation of tool XML files and generalizing applying actions over many repositories and many tools.
| @@ -192,6 +194,24 @@ def build_tarball(tool_path): | |||
| return temp_path | |||
|
|
|||
|
|
|||
| def walk_repositories(path): | |||
There was a problem hiding this comment.
oh, nice, will reuse this in recursive shed uploads.
There was a problem hiding this comment.
That is where I got it, it has already be refactored out of shed_upload.
There was a problem hiding this comment.
Oh, excellent, thanks!
fre. 17. apr. 2015 kl. 09.51 skrev John Chilton notifications@github.com:
In planemo/shed.py
#130 (comment):@@ -192,6 +194,24 @@ def build_tarball(tool_path):
return temp_path+def walk_repositories(path):
That is where I got it, it has already be refactored out of shed_upload.
—
Reply to this email directly or view it on GitHub
https://github.com/galaxyproject/planemo/pull/130/files#r28600556.
|
This is looking great. Very useful. |
| lint_ctx.warn("Failed to parse .shed.yml file [%s]" % str(e)) | ||
|
|
||
| warned = False | ||
| for required_key in ["owner", "name"]: |
There was a problem hiding this comment.
any reason for not using the expanded key set? You might add description, as that's the only other thing that's strictly required for a create-repo to succeed.
There was a problem hiding this comment.
It is required for creation, but not for vanilla upload - even these two fields are not strictly required... hmm.... I am not against requiring everything in .shed.yml but this was a conservative first attempt.
| @@ -0,0 +1,15 @@ | |||
| <?xml version="1.0" encoding="UTF-8"?> | |||
There was a problem hiding this comment.
I know how much fun copy and pasting scripted import of files from other projects is ;)
There was a problem hiding this comment.
Hmm... maybe... pypa/pip#577 ... yeah I should do that.
There was a problem hiding this comment.
Done - thanks for the suggestion.
|
+1 from me, can't wait to add this to the jenkins testing bot! :D |
Use XSD to lint tools and repositories.
--xsdflag to thelintcommand (for tools) that lints against the experimental XSD from https://github.com/JeanFred/Galaxy-XSD.repository_dependencies.xsdto describe Tool Shedrepository_dependencies.xmlfiles (fairly complete).tool_dependencies.xsdto describe Tool Shedtool_dependencies.xmlfiles.actionelements and then largely gives up (stickinganyandanyAttributetags on that element).shed_lintcommand that:tool_dependencies.xmlagainst schemarepository_dependencies.xmlagainst schema.shed.ymlfiles.--toolsargument. Likeshed_uploaduses presence of.shed.ymlto detect repositories, this should be generalized.-r.