Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Workflow convergence #2389

Merged
merged 99 commits into from
Nov 27, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
99 commits
Select commit Hold shift + click to select a range
f5c10f9
support workflow convergence nodes
chrismeyersfsu Sep 27, 2018
a9365a3
code cleanup
chrismeyersfsu Oct 11, 2018
447dfbb
only visit nodes once for dnr
chrismeyersfsu Oct 11, 2018
779e1a3
remove dnr field from jt wf node
chrismeyersfsu Oct 11, 2018
ad56a27
mark dnr field read only
chrismeyersfsu Oct 11, 2018
cc374ca
update debug dot graph to output dnr data
chrismeyersfsu Oct 11, 2018
3506b9a
Revert "mark dnr field read only"
chrismeyersfsu Oct 11, 2018
ebabec0
always find and mark dnr nodes
chrismeyersfsu Oct 11, 2018
1a064bd
satisfy flake8
chrismeyersfsu Oct 12, 2018
ff6db37
correct stop DNR propogation
chrismeyersfsu Oct 15, 2018
b4fc585
stop DNR propogation on always path
chrismeyersfsu Oct 15, 2018
77661c6
short circuit performance optimization
chrismeyersfsu Oct 16, 2018
914892c
all parents should finish before start child
chrismeyersfsu Oct 17, 2018
9bf2a49
save state
chrismeyersfsu Oct 17, 2018
6ef6b64
cleaner code
chrismeyersfsu Oct 17, 2018
ea29e66
fix workflow finish state detector
chrismeyersfsu Oct 18, 2018
2742b00
flake8
chrismeyersfsu Oct 18, 2018
475c90f
prevent job launching twice
chrismeyersfsu Oct 19, 2018
02df0c2
merge artifacts deterministically
chrismeyersfsu Oct 22, 2018
4111e53
correctly name migration to align with 3.4.0
chrismeyersfsu Oct 22, 2018
1e10d43
update docs
chrismeyersfsu Oct 22, 2018
87d6253
Decouple editing a wf node with editing a node link
mabashian Oct 5, 2018
29b4979
Completed work necessary to support editing workflow links and nodes …
mabashian Oct 24, 2018
7b22d1b
cycle detection when multiple parents
chrismeyersfsu Oct 25, 2018
dfccc9e
rework wf cycle detection for convergence
chrismeyersfsu Oct 31, 2018
9afc38b
fixup migrations
chrismeyersfsu Oct 31, 2018
2f9dc4d
remove relationship in view if cycle detected
chrismeyersfsu Nov 1, 2018
6e40e9c
handle edge case ring cycle
chrismeyersfsu Nov 1, 2018
f8c53f4
handle job error state in convergence
chrismeyersfsu Nov 1, 2018
584b3f4
remove workflow test
chrismeyersfsu Nov 2, 2018
17b3996
fix flake8 anyway I can
chrismeyersfsu Nov 2, 2018
1120f8b
try2 at the devil flake8
chrismeyersfsu Nov 2, 2018
07db7a4
more flake8
chrismeyersfsu Nov 2, 2018
7b95d21
Implements workflow convergence without proper layout
mabashian Nov 6, 2018
61fb3eb
First pass at implementing better node placement in the workflow graph
mabashian Nov 8, 2018
05f4d94
Fixed serveral bugs including credential prompting. Added logic to b…
mabashian Nov 9, 2018
e1e8d3b
bump migration
chrismeyersfsu Nov 9, 2018
b84fc3b
Fixes for post-rebase bugs
mabashian Nov 9, 2018
9f3e272
optimize cycle detection
chrismeyersfsu Nov 8, 2018
16a6041
optimization fix
chrismeyersfsu Nov 9, 2018
3dadeb3
remove print statements
chrismeyersfsu Nov 9, 2018
700860e
Fix long name tooltip. Fixed bug adding new node before finishing ad…
mabashian Nov 12, 2018
0499d41
more efficient graph processing
chrismeyersfsu Nov 12, 2018
3f4d14e
crawl entire graph when marking DNR
chrismeyersfsu Nov 12, 2018
a176a4b
remove unused code
chrismeyersfsu Nov 13, 2018
8bb9cfd
add dag tests
chrismeyersfsu Nov 13, 2018
1b87e11
flake8
chrismeyersfsu Nov 13, 2018
b81d795
fix up dot graph generator
chrismeyersfsu Nov 13, 2018
ae0d0db
Added dagre to handle our workflow graph layout. Fixed various workf…
mabashian Nov 13, 2018
6529c1b
update done and fail detection for workflow
chrismeyersfsu Nov 13, 2018
a6e20ee
update wf done and failed tests
chrismeyersfsu Nov 13, 2018
266831e
add cycle unit test
chrismeyersfsu Nov 14, 2018
4a6a3b2
Fixed a number of workflow visualizer bugs. Added loading spinners w…
mabashian Nov 14, 2018
d6a8ad0
treat canceled jobs in wf the same as failed jobs
chrismeyersfsu Nov 15, 2018
c1171fe
treat canceled nodes as failed when processing wf
chrismeyersfsu Nov 15, 2018
1a85fcd
update docs to include workflow failure semantic
chrismeyersfsu Nov 15, 2018
281345d
flake8 fix
chrismeyersfsu Nov 16, 2018
72263c5
Addresses a number of workflow related bugs
mabashian Nov 16, 2018
00d71ce
detect workflow nodes without job templates
chrismeyersfsu Nov 16, 2018
676c068
add job_description to failed workflow node
chrismeyersfsu Nov 16, 2018
5b459e3
Code cleanup. Fixed bugs with workflow results page including detail…
mabashian Nov 16, 2018
f30f52a
handle missing unified job template in workflow
chrismeyersfsu Nov 18, 2018
d1aa52a
fix up mark dnr logic
chrismeyersfsu Nov 18, 2018
4c9a1d6
optimize mark dnr nodes algorithm
chrismeyersfsu Nov 18, 2018
3c510e6
Fixed bug where root link became clickable. Fix workflow key on resul…
mabashian Nov 19, 2018
cfa0984
Revert "optimize mark dnr nodes algorithm"
chrismeyersfsu Nov 19, 2018
7b087d4
loop over dnr nodes by topological sort
chrismeyersfsu Nov 19, 2018
a804c85
Fix test failures and jshint errors
mabashian Nov 19, 2018
623cf54
Added dagre and graphlib licenses
mabashian Nov 19, 2018
56885a5
Remove reference to isStartNode and just check the id of the node to …
mabashian Nov 19, 2018
febf051
do not mark ujt None nodes dnr
chrismeyersfsu Nov 19, 2018
0c8dde9
fix dfs_run_nodes()
chrismeyersfsu Nov 19, 2018
4c14727
bump migration number
AlanCoding Nov 20, 2018
1cfcaa7
Fixed editNodeHelpMessage logic that was broken during merge conflict
mabashian Nov 20, 2018
28a4bbb
Fixed jshint errors that fell out of merge conflict
mabashian Nov 20, 2018
2eeca3c
add example workflow run to docs
chrismeyersfsu Nov 20, 2018
d8bf82a
add help_text to do_not_run workflow field
chrismeyersfsu Nov 20, 2018
f8f2e00
better comment for deciding parent's status
chrismeyersfsu Nov 20, 2018
228e412
simplify workflow job failure reason
chrismeyersfsu Nov 20, 2018
625c6c3
Fixed edge dropdown id
mabashian Nov 20, 2018
1477bba
Fixed error Cannot read property 'type' of undefined in console when …
mabashian Nov 20, 2018
54d56f2
Fix node jobs column sorting. Adds arrows to potential workflow node …
mabashian Nov 20, 2018
ed40ba6
Fix searching on related fields
mabashian Nov 20, 2018
38dc0b8
fix workflow total jobs header to total nodes
Nov 21, 2018
343639d
fix workflow maker total templates header to total nodes
Nov 21, 2018
762c882
consume workflow maker total nodes label change
Nov 21, 2018
3762ba7
add back in workflow_nodes in order to be able to use it for count of…
Nov 21, 2018
7b4521f
workflow node prompt fixup
jakemcdermott Nov 23, 2018
65ec1d1
skip missing inventory prompt value check when selecting workflow node
jakemcdermott Nov 23, 2018
d5f07a9
hide inventory help message when not on jobs tab
jakemcdermott Nov 23, 2018
bfa361c
hide prompt button when not on jobs tab
jakemcdermott Nov 23, 2018
3975a2e
fix linkpath class
mabashian Nov 26, 2018
62a1f10
Fix node pagination for project/inv
mabashian Nov 26, 2018
7bad01e
Fixes e2e workflow visualizer tests
mabashian Nov 26, 2018
6fc2ba3
Fixes delete node shifting e2e test
mabashian Nov 26, 2018
55dc27f
Set active tab to jobs when initially clicking a workflow_job_templat…
mabashian Nov 26, 2018
10de5b6
Fixes clicking on a wf in wf node. Also fixes editing wf in wf node …
mabashian Nov 26, 2018
17cc82d
Ensure that selected row is cleared when adding new node after editin…
mabashian Nov 27, 2018
51669c9
Fixes hint/lint errors in workflow viz test
mabashian Nov 27, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
3 changes: 2 additions & 1 deletion awx/api/serializers.py
Original file line number Diff line number Diff line change
Expand Up @@ -3916,7 +3916,8 @@ class WorkflowJobNodeSerializer(LaunchConfigurationBaseSerializer):
class Meta:
model = WorkflowJobNode
fields = ('*', 'credential', 'job', 'workflow_job', '-name', '-description', 'id', 'url', 'related',
'unified_job_template', 'success_nodes', 'failure_nodes', 'always_nodes',)
'unified_job_template', 'success_nodes', 'failure_nodes', 'always_nodes',
'do_not_run',)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this worth exposing in the API, or is it an internal field we only care about in the context of the task scheduler graph? I wonder if API users will know (or care) what this is.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's worth showing to clients, because it's fundamentally core to how our form of convergence works. I can picture the questions now "the first node was successful, but the next node refuses to start!", okay, well it's waiting for the other branch to converge. You know if the branch is finished if there is a finished job or the DNR flag is true. Otherwise, what do you tell them?

That said, this needs help text. The help text should be added in the field on the model.


def get_related(self, obj):
res = super(WorkflowJobNodeSerializer, self).get_related(obj)
Expand Down
42 changes: 14 additions & 28 deletions awx/api/views/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@
from awx.api.serializers import * # noqa
from awx.api.metadata import RoleMetadata, JobTypeMetadata
from awx.main.constants import ACTIVE_STATES
from awx.main.scheduler.dag_workflow import WorkflowDAG
from awx.api.views.mixin import (
ActivityStreamEnforcementMixin,
SystemTrackingEnforcementMixin,
Expand Down Expand Up @@ -143,6 +144,9 @@
)


logger = logging.getLogger('awx.api.views')


def api_exception_handler(exc, context):
'''
Override default API exception handler to catch IntegrityError exceptions.
Expand Down Expand Up @@ -2950,35 +2954,17 @@ def is_valid_relation(self, parent, sub, created=False):
if created:
return None

workflow_nodes = parent.workflow_job_template.workflow_job_template_nodes.all().\
prefetch_related('success_nodes', 'failure_nodes', 'always_nodes')
graph = {}
for workflow_node in workflow_nodes:
graph[workflow_node.pk] = dict(node_object=workflow_node, metadata={'parent': None, 'traversed': False})

find = False
for node_type in ['success_nodes', 'failure_nodes', 'always_nodes']:
for workflow_node in workflow_nodes:
parent_node = graph[workflow_node.pk]
related_nodes = getattr(parent_node['node_object'], node_type).all()
for related_node in related_nodes:
sub_node = graph[related_node.pk]
sub_node['metadata']['parent'] = parent_node
if not find and parent == workflow_node and sub == related_node and self.relationship == node_type:
find = True
if not find:
sub_node = graph[sub.pk]
parent_node = graph[parent.pk]
if sub_node['metadata']['parent'] is not None:
return {"Error": _("Multiple parent relationship not allowed.")}
sub_node['metadata']['parent'] = parent_node
iter_node = sub_node
while iter_node is not None:
if iter_node['metadata']['traversed']:
return {"Error": _("Cycle detected.")}
iter_node['metadata']['traversed'] = True
iter_node = iter_node['metadata']['parent']
if parent.id == sub.id:
return {"Error": _("Cycle detected.")}

parent_node_type_relationship = getattr(parent, self.relationship)
parent_node_type_relationship.add(sub)

graph = WorkflowDAG(parent.workflow_job_template)
if graph.has_cycle():
parent_node_type_relationship.remove(sub)
return {"Error": _("Cycle detected.")}
parent_node_type_relationship.remove(sub)
return None


Expand Down
20 changes: 20 additions & 0 deletions awx/main/migrations/0054_v340_workflow_convergence.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# -*- coding: utf-8 -*-
# Generated by Django 1.11.11 on 2018-09-28 14:23
from __future__ import unicode_literals

from django.db import migrations, models


class Migration(migrations.Migration):

dependencies = [
('main', '0053_v340_workflow_inventory'),
]

operations = [
migrations.AddField(
model_name='workflowjobnode',
name='do_not_run',
field=models.BooleanField(default=False, help_text='Indidcates that a job will not be created when True. Workflow runtime semantics will mark this True if the node is in a path that will decidedly not be ran. A value of False means the node may not run.'),
),
]
8 changes: 7 additions & 1 deletion awx/main/models/workflow.py
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ def get_parent_nodes(self):
success_parents = getattr(self, '%ss_success' % self.__class__.__name__.lower()).all()
failure_parents = getattr(self, '%ss_failure' % self.__class__.__name__.lower()).all()
always_parents = getattr(self, '%ss_always' % self.__class__.__name__.lower()).all()
return success_parents | failure_parents | always_parents
return (success_parents | failure_parents | always_parents).order_by('id')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

iiiinteresting, it surprises me that this works with this syntax, but why not? I guess this is what enforces the parental ordering that we've mentioned several times.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

boolean logic operations on querysets always produce a new queryset object.


@classmethod
def _get_workflow_job_field_names(cls):
Expand Down Expand Up @@ -184,6 +184,12 @@ class WorkflowJobNode(WorkflowNodeBase):
default={},
editable=False,
)
do_not_run = models.BooleanField(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's worth having a description on this field at least to outline that it is part of the workflow execution resolution graph and the the information here should outline its meaning on said graph.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this conversation is resolved by the help_text that @chrismeyersfsu just added?

default=False,
help_text=_("Indidcates that a job will not be created when True. Workflow runtime "
"semantics will mark this True if the node is in a path that will "
"decidedly not be ran. A value of False means the node may not run."),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hate to be making suggestions at this point but

A value of False means the node may not run.

"A value of False means this node may or may not run."?

Perhaps it would be best to delete that sentence. I don't think we need it.

)

def get_absolute_url(self, request=None):
return reverse('api:workflow_job_node_detail', kwargs={'pk': self.pk}, request=request)
Expand Down