Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes for workflow extraction of mapping collection jobs. #2625

Merged
merged 2 commits into from Jul 22, 2016

Conversation

Projects
None yet
4 participants
@jmchilton
Copy link
Member

commented Jul 18, 2016

Extraction might not work on older histories thanks to incorrect flushing fixed with 43523da being persisted incorrectly.

To test:

  • GALAXY_RUN_WITH_TEST_TOOLS=1 bash run.sh
  • Create a collection.
  • Run the "Create 10" tool over the collection.
  • Extract a workflow.
  • Note the duplicated "Create 10" job - one of those fails.
  • Apply the patch.
  • Run a new "Create 10" tool over the collection.
  • Second run should not be duplicated and workflow extraction should succeed.

Might fix #1465?

jmchilton added some commits Jul 18, 2016

Fix duplicated entries when using collections in history extract.
This would "sometimes" correctly pick the first job to represent a collection creation step but in many cases not. Explicitly grab the "first".

@galaxybot galaxybot added this to the 16.07 milestone Jul 18, 2016

@dannon

This comment has been minimized.

Copy link
Member

commented Jul 22, 2016

@jmchilton Is this ready for review?

@jmchilton

This comment has been minimized.

Copy link
Member Author

commented Jul 22, 2016

@dannon Sure.

@dannon

This comment has been minimized.

Copy link
Member

commented Jul 22, 2016

Fixes "failed to extract job" on collection for me (at least in the scenario described above).

@dannon dannon merged commit 1839e81 into galaxyproject:dev Jul 22, 2016

4 checks passed

api test Build finished. 223 tests run, 0 skipped, 0 failed.
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
framework test Build finished. 110 tests run, 0 skipped, 0 failed.
Details
toolshed test Build finished. 582 tests run, 0 skipped, 0 failed.
Details
@nsoranzo

This comment has been minimized.

Copy link
Member

commented on 9ede8e5 Aug 4, 2016

@jmchilton This commit consistently breaks the API tests for me locally:

$ ./run_tests.sh -api test/api/test_workflow_extraction.py
...
galaxy.workflow.extract: WARNING: job_id 88 not found in jobs_by_id {'fake_40': <galaxy.workflow.extract.DatasetCollectionCreationJob object at 0x7f26d0989650>, 86: <galaxy.model.Job object at 0x7f26d0bddc90>, 87: <galaxy.model.Job object at 0x7f26d0de6850>, 89: <galaxy.model.Job object at 0x7f26d0c862d0>, 90: <galaxy.model.Job object at 0x7f26d0ee5550>, 'fake_39': <galaxy.workflow.extract.DatasetCollectionCreationJob object at 0x7f26d0989850>}
galaxy.web.framework.decorators: ERROR: Uncaught exception in exposed API method:
Traceback (most recent call last):
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/lib/galaxy/web/framework/decorators.py", line 261, in decorator
    rval = func( self, trans, *args, **kwargs)
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/lib/galaxy/webapps/galaxy/api/workflows.py", line 238, in create
    workflow_name=workflow_name,
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/lib/galaxy/workflow/extract.py", line 29, in extract_workflow
    steps = extract_steps( trans, history=history, job_ids=job_ids, dataset_ids=dataset_ids, dataset_collection_ids=dataset_collection_ids )
  File "/usr/users/ga002/soranzon/software/nsoranzo_galaxy/lib/galaxy/workflow/extract.py", line 101, in extract_steps
    raise AssertionError( "Attempt to create workflow with job not connected to current history" )
AssertionError: Attempt to create workflow with job not connected to current history
requests.packages.urllib3.connectionpool: DEBUG: "POST /api/workflows HTTP/1.1" 500 None
--------------------- >> end captured logging << ---------------------

----------------------------------------------------------------------
Ran 9 tests in 121.323s

FAILED (failures=5)

Reverting it makes the tests pass again.

This comment has been minimized.

Copy link
Member Author

replied Aug 5, 2016

@nsoranzo I just re-ran the tests locally here on dev and they seem to work for me. Do you have the logs a little above this? Are you sure these problems are caused by this commit and are not sqlite locking errors?

This comment has been minimized.

Copy link
Member

replied Aug 5, 2016

I always use (an empty) PostgreSQL db for the API tests.

I tried twice and they failed both times, reverted this commit and passed, removed the revert and failed again. You'll have to wait until Monday for the complete logs, I'm out of office.

This comment has been minimized.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.