Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scottx611x/workflow launching and monitoring #1949

Merged
merged 54 commits into from
Aug 1, 2017
Merged
Show file tree
Hide file tree
Changes from 48 commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
8c95f6e
Merge remote-tracking branch 'origin/develop' into scottx611x/dataset…
scottx611x Jul 25, 2017
f070f70
Add galaxy_workflow_task_group_id field to AnalysisStatus
scottx611x Jul 26, 2017
614371b
Add field `galaxy_import_state` to AnalysisStatus
scottx611x Jul 26, 2017
c3fb945
Update docstring
scottx611x Jul 26, 2017
01d1af5
Organize analysis_manager.tasks
scottx611x Jul 26, 2017
ffcf7a2
Add some constants to Tool model
scottx611x Jul 26, 2017
f4e39b1
Add methods to Tool model
scottx611x Jul 26, 2017
2af4305
Update docstring
scottx611x Jul 26, 2017
91a759b
Add test coverage
scottx611x Jul 26, 2017
7b13b47
Add decorators to designate methods for specific Tool types
scottx611x Jul 26, 2017
d196cca
Update tests
scottx611x Jul 26, 2017
ee1d98f
Merge remote-tracking branch 'origin/develop' into scottx611x/workflo…
scottx611x Jul 26, 2017
45bab3e
Fix Test
scottx611x Jul 26, 2017
2c1ae4c
Add test coverage for `_get_nesting_string`
scottx611x Jul 26, 2017
0a79c1a
Organizing these methods in a prior commit made for a difficult to re…
scottx611x Jul 26, 2017
93f7203
Remove redundant decorator
scottx611x Jul 26, 2017
4a340ed
Remove redundant tests
scottx611x Jul 26, 2017
0640454
Remove unutilized function
scottx611x Jul 26, 2017
b9e4756
Designate `_launch_workflow` to only work for workflow-based tools
scottx611x Jul 26, 2017
7fceb60
Add test for update_galaxy_data
scottx611x Jul 26, 2017
d3bc59b
Add test coverage for `_invoke_tool_based_galaxy_workflow`
scottx611x Jul 26, 2017
fda063f
Rename method and adhere to style guide
scottx611x Jul 26, 2017
bbb0153
Fix naming
scottx611x Jul 26, 2017
278b854
Consolidate migrations
scottx611x Jul 26, 2017
97fe922
Just use a placeholder method for now
scottx611x Jul 26, 2017
b5a7e11
DRY
scottx611x Jul 26, 2017
ca6f766
Add appropriate error msg and fail task
scottx611x Jul 26, 2017
1ad9494
Fix typo
scottx611x Jul 26, 2017
b8b8336
Update docdtring
scottx611x Jul 26, 2017
2bb4746
Fix test
scottx611x Jul 26, 2017
20b4d44
Add test coverage for `_check_galaxy_history_state`
scottx611x Jul 27, 2017
7feada6
Polymorphic approach for Tools since Tools were becoming too heavily …
scottx611x Jul 28, 2017
2169041
Add test coverage for `_run_tool_based_galaxy_file_import`
scottx611x Jul 28, 2017
93b9334
Consolidate migrations
scottx611x Jul 28, 2017
6cd9479
Add `galaxy_connection` property and more class methods to WorkflowTool
scottx611x Jul 28, 2017
bc638c5
Add `set_galaxy_workflow_task_group_id` to AnalysisStatus
scottx611x Jul 28, 2017
a1571b3
Utilize `set_galaxy_workflow_task_group_id()`
scottx611x Jul 28, 2017
39f8eaf
Update test mock arg order, add test coverage for `_run_tool_based_ga…
scottx611x Jul 28, 2017
2ad9396
Add test coverage
scottx611x Jul 28, 2017
f39b634
Fix test
scottx611x Jul 28, 2017
04e4893
Add api test to check that polymorphism behaves appropriately
scottx611x Jul 28, 2017
9c0d342
DRY
scottx611x Jul 28, 2017
cd43f1f
Fix Tests
scottx611x Jul 28, 2017
b674493
Fix ordering of arguments
scottx611x Jul 31, 2017
7a6decb
Method renaming and Docstring updates
scottx611x Jul 31, 2017
7243289
Rename method to: `import_library_dataset_to_history`
scottx611x Jul 31, 2017
7d57a11
We already have access to the analysis through Tool here
scottx611x Jul 31, 2017
c412c89
Add abstract `launch()` to Tool and test it
scottx611x Jul 31, 2017
6fa2a98
Refactor `_get_nesting_string` into another step: `_flatten_file_rela…
scottx611x Aug 1, 2017
37fab7a
Pull out parameters common to both Tools and ToolDefinitions and use …
scottx611x Aug 1, 2017
33e2ef0
Set this value upon a Tool's creation
scottx611x Aug 1, 2017
6841eb1
DRY and style fix
scottx611x Aug 1, 2017
706f7ce
Add test coverage for `_flatten_file_relationships_nesting()`
scottx611x Aug 1, 2017
a47ddfc
Update `_get_file_relationships_galaxy` to access value at new location
scottx611x Aug 1, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
27 changes: 27 additions & 0 deletions refinery/analysis_manager/migrations/0003_auto_20170726_1601.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# -*- coding: utf-8 -*-
from __future__ import unicode_literals

from django.db import models, migrations
import django_extensions.db.fields


class Migration(migrations.Migration):

dependencies = [
('analysis_manager', '0002_analysisstatus_galaxy_import_progress'),
]

operations = [
migrations.AddField(
model_name='analysisstatus',
name='galaxy_import_state',
field=models.CharField(blank=True, max_length=10, choices=[(b'SUCCESS', b'OK'), (b'FAILURE', b'Error'), (b'PROGRESS', b'Running'), (b'PENDING', b'Unknown')]),
preserve_default=True,
),
migrations.AddField(
model_name='analysisstatus',
name='galaxy_workflow_task_group_id',
field=django_extensions.db.fields.UUIDField(max_length=36, null=True, editable=False, blank=True),
preserve_default=True,
),
]
33 changes: 31 additions & 2 deletions refinery/analysis_manager/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,12 @@ class AnalysisStatus(models.Model):
auto=False)
galaxy_import_task_group_id = UUIDField(blank=True, null=True, auto=False)
galaxy_export_task_group_id = UUIDField(blank=True, null=True, auto=False)
galaxy_workflow_task_group_id = UUIDField(blank=True,
null=True,
auto=False)
#: state of Galaxy file imports
galaxy_import_state = CharField(max_length=10, blank=True,
choices=GALAXY_HISTORY_STATES)
#: state of Galaxy history
galaxy_history_state = CharField(max_length=10, blank=True,
choices=GALAXY_HISTORY_STATES)
Expand All @@ -42,22 +48,37 @@ def __unicode__(self):
return self.analysis.name

def set_galaxy_history_state(self, state):
"""
Set the `galaxy_history_state` of an analysis
:param state: a valid GALAXY_HISTORY_STATE
"""
if state in dict(self.GALAXY_HISTORY_STATES).keys():
self.galaxy_history_state = state
self.save()
else:
raise ValueError("Invalid Galaxy history state given")

def set_galaxy_import_state(self, state):
"""
Set the `galaxy_import_state` of an analysis
:param state: a valid GALAXY_HISTORY_STATE
"""
if state in dict(self.GALAXY_HISTORY_STATES).keys():
self.galaxy_import_state = state
self.save()
else:
raise ValueError("Invalid Galaxy history state given")

def refinery_import_state(self):
return get_task_group_state(self.refinery_import_task_group_id)

def galaxy_file_import_state(self):
return get_task_group_state(self.galaxy_import_task_group_id)

def tool_based_galaxy_file_import_state(self):
if self.galaxy_history_state and self.galaxy_import_progress != 0:
if self.galaxy_import_state and self.galaxy_import_progress != 0:
galaxy_file_import_state = [{
'state': self.galaxy_history_state,
'state': self.galaxy_import_state,
'percent_done': self.galaxy_import_progress
}]
else:
Expand All @@ -77,6 +98,14 @@ def galaxy_analysis_state(self):
def galaxy_export_state(self):
return get_task_group_state(self.galaxy_export_task_group_id)

def set_galaxy_import_task_group_id(self, galaxy_import_task_group_id):
self.galaxy_import_task_group_id = galaxy_import_task_group_id
self.save()

def set_galaxy_workflow_task_group_id(self, galaxy_workflow_task_group_id):
self.galaxy_workflow_task_group_id = galaxy_workflow_task_group_id
self.save()


def get_task_group_state(task_group_id):
"""return a list containing states of all tasks given a task set ID"""
Expand Down