Add DTD to Galaxy

[ericenns]

This merges DTD upstream into Galaxy.

https://github.com/phac-nml/dynamic-tool-destination

WIP until all tests have been moved over and reviewed fully.

[bgruening]

Great poster, great idea, I like it much and I think it is a nice addition to the dynamic tool destinations.
Thanks @ericenns!

[ericenns]

I think WIP label can be removed, I have added all our tests in and they pass!

[bgruening]

Should this be commented out?

[ericenns]

Do you mean should the whole file be commented out? If so yes, is that what is done with out sample config files?

[bgruening]

@ericenns do you think we should keep the configuration under config or is this something really specific to the dynamic-job-destination and should be next to the job-destination to not pollute the config dir?

[ericenns]

The config file is specific to dtd, one benefit to keeping it in config is that it gives visibility to it. I guess I could add it as an example to job_conf.xml.sample_advanced

[ericenns]

Guess I have to remove WIP from the title or galaxybot will keep adding the label back.

[jmchilton]

@galaxybot test this

[jmchilton]

Really nifty - I ❤️ the tests and focus on validation - this great.

I created galaxyproject/starforge#100 so the wheel for testfixtures can be placed on wheels.galaxyproject.org.

I'm going to open a counter-PR to make the config handling more Galaxy-ish.

[jmchilton]

The upstream project has a different copyright statement and license than Galaxy and it is featured prominently in README - given that I just wanted to make sure that you are explicitly granting the Galaxy project permission to redistribute this code with our license and copyright.

[ericenns]

Yes I am granting the Galaxy project permission to redistribute this code with their license and copyright.

[jmchilton]

Thanks @ericenns!

I have two commits I'd like to discuss but Github isn't letting me open a PR against your repo and I don't understand why. Any chance you can review them, test them, and cherry-pick them into your branch if they look good?

They are from my branch here - https://github.com/jmchilton/galaxy/tree/add-dtd-to-galaxy.

jmchilton@ed48c6c Makes the config handling more like traditional Galaxy - navigating the Galaxy config object to find that YAML file.
jmchilton@d87957e Re-imagines how this would be used a bit - it makes this a special "type" of dynamic destination (there were already some others) so details such as the function name don't need to be exposed. It also adds some documentation to job_conf.xml.sample_advanced about how to use it.

Any problems with these changes?

[ericenns]

Thanks @jmchilton, the changes look good to me. I will merge them and retest.

[ericenns]

@jmchilton everything looks good, a revised test location and updated the sample config with a little more information on how to validate your config.

[ericenns]

@galaxybot test this

[ericenns]

Hmm looks like there is a problem with running the following command:

 bash -c '. .venv/bin/activate; python lib/galaxy/jobs/dynamic_tool_destination.py -c config/tool_destinations.yml.sample'

I am looking into it right now

[ericenns]

Ok everything should be good to go now

[ericenns]

@galaxybot test this

[jmchilton]

Very nice @ericenns - thanks a bunch for this contribution!

[jmchilton]

And of course I merged this without waiting for my own starforge PR (galaxyproject/starforge#100) 😠.

[bgruening]

🎉 This is awesome! Next GCC presentation in Galaxy main! Thanks a lot @ericenns!