Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix certain combos of dynamic deeply nested collections in workflows. #2634

Merged
merged 4 commits into from Jul 20, 2016

Conversation

Projects
None yet
3 participants
@jmchilton
Copy link
Member

commented Jul 19, 2016

xref #2581

This is a very naive approach - which might cause Galaxy to schedule these workflows very slowly under some scenarios - but this is I guess better than not scheduling them at all.

(If merged and confirmed to fix #2581 - I'll open a PR to backport just 85d7784 I think.)

jmchilton added some commits Jul 19, 2016

Test case for dynamically creating `list:list:list`s.
This could check the outputs ideally, but it was failing to simply execute the workflow so this does serve as check that some bugs are fixed.
Improved logging and error message when mapping over an empty* collec…
…tion.

*actually empty or accidentally empty.
@gregvonkuster

This comment has been minimized.

Copy link
Contributor

commented Jul 19, 2016

@jmchilton I have confirmed that this fixes #2581. Thanks a bunch for the fix - this is a very big deal for the CEGR! ;)

@gregvonkuster

This comment has been minimized.

Copy link
Contributor

commented Jul 20, 2016

With this fix I have an automated processing environment that takes raw sequenced data, runs it through a pre-processing pipeline which uses a fairly good swath of the Galaxy/bioblend API, imports the data into Galaxy data libraries and executes the soon-to-be-released Galaxy ChIP-exo workflow for each sample in the run. This environment is running with Galaxy 16.04, and the workflow includes tools that use many of the recent features that have been added to the Galaxy framework (e.g., dataset collections, etc).

I think this environment could be used as an additional test framework for future Galaxy releases, including 16.07. If @davebx has some time to help me get my test environment working (on brubeck), I would be willing to use it to test upcoming releases as soon as the new branch is created and provide feedback to the core team with any issues that may be uncovered. Let me know if you feel this would be beneficial.

@blankenberg

This comment has been minimized.

Copy link
Member

commented Jul 20, 2016

@galaxybot test this

@blankenberg

This comment has been minimized.

Copy link
Member

commented Jul 20, 2016

@jmchilton possibly a legit failure?

Rework test case that fails occasionally on Jenkins.
Should be a little more robust now.
@jmchilton

This comment has been minimized.

Copy link
Member Author

commented Jul 20, 2016

@blankenberg Yeah - I bet it is a timing issue and it takes more time because of this change. I can't reproduce the failure locally, but I've updated that test case to be a bit more robust I think. Lets see if Jenkins approves.

@blankenberg blankenberg merged commit 5fe4bea into galaxyproject:dev Jul 20, 2016

4 checks passed

api test Build finished. 224 tests run, 0 skipped, 0 failed.
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
framework test Build finished. 110 tests run, 0 skipped, 0 failed.
Details
toolshed test Build finished. 582 tests run, 0 skipped, 0 failed.
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.