Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement conditional workflow steps #14846

Merged
merged 36 commits into from
Jan 19, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
0a79d23
Implement conditionals via boolean step parameter
jmchilton Nov 8, 2021
20f9b6f
Attempt to skip mapped-over steps
mvdbeek Oct 24, 2022
1f44e44
Into map-over
mvdbeek Oct 24, 2022
fbdc45f
Add when_expression migration
mvdbeek Oct 24, 2022
b06444f
WIP: test map over and pick_value
mvdbeek Oct 24, 2022
a7e392d
Fix up migration
mvdbeek Oct 25, 2022
b194db6
WIP: Produce json null on skip, load expression.json null as null in …
mvdbeek Oct 25, 2022
f087608
Fix up map_over of conditional output
mvdbeek Oct 26, 2022
c146f17
Assert skipped state in tests
mvdbeek Oct 26, 2022
d396743
Drop unreachable SkipWorkflowStepEvaluation except statement
mvdbeek Oct 26, 2022
0fb4118
Drop unused function
mvdbeek Oct 26, 2022
bfc4801
Fix type annotation for temp_input_connections
mvdbeek Oct 26, 2022
20d92ad
Make new tests robust to job and step order
mvdbeek Oct 26, 2022
a633f1e
Make sure we're not changing datatypes on skipped outputs
mvdbeek Oct 26, 2022
4ac429c
Add skipped to job state summary
mvdbeek Nov 10, 2022
bbfc689
Implement conditional subwokflow steps
mvdbeek Nov 10, 2022
63549fe
Stick with CWL syntax and CWL tooling to implement step skipping
mvdbeek Nov 13, 2022
9cdcde5
Add subworkflow skip test when tool structure changes
mvdbeek Nov 13, 2022
22a64fa
Fix mapping over subworkflow steps without explicit connection to out…
mvdbeek Nov 17, 2022
ca55ad8
Mark skipped outputs as populated
mvdbeek Nov 17, 2022
a2c6627
Test and fix conditional subworkflow steps where subworkflow step pro…
mvdbeek Nov 17, 2022
f911a54
Fix migration down_revision
mvdbeek Jan 13, 2023
961d64e
Update typescript schema
mvdbeek Jan 13, 2023
e59ff11
Backend for showing invocation failure/cancellation/warning reasons
mvdbeek Jan 15, 2023
b9b5f52
Record execution_tracker.execution_errors on invocation
mvdbeek Jan 15, 2023
7d659bc
Test invalid expressions and expression results
mvdbeek Jan 15, 2023
23cc0c4
Avoid Union of GenericModel
mvdbeek Jan 15, 2023
ecb1604
Add migration for updated HistoryDatasetCollectionJobStateSummary
mvdbeek Jan 16, 2023
7705038
Move InvocationMessage models to own file
mvdbeek Jan 16, 2023
bc337ce
Invocation schema import fixes
mvdbeek Jan 16, 2023
7dc0bf4
Fix dependent workflow step id
mvdbeek Jan 16, 2023
acf7a18
Make workflow_step_id required for workflow_output_not_found warning
mvdbeek Jan 16, 2023
d40cd7b
Generate model for InvocationMessage
mvdbeek Jan 16, 2023
4fe5874
Create and use useWorkflowInstance composable
mvdbeek Jan 16, 2023
b10cd45
Fix regular workflow file upload in UI
mvdbeek Jan 16, 2023
9a47b5e
Add InvocationMessage component and mount in invocation summary tab
mvdbeek Jan 16, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
192 changes: 192 additions & 0 deletions client/src/components/WorkflowInvocationState/InvocationMessage.vue
Original file line number Diff line number Diff line change
@@ -0,0 +1,192 @@
<script setup lang="ts">
import { computed } from "vue";
import { useWorkflowInstance } from "@/composables/useWorkflowInstance";
import type { InvocationMessageResponseModel } from "./invocationMessageModel";
import GenericHistoryItem from "@/components/History/Content/GenericItem.vue";
import WorkflowInvocationStep from "@/components/WorkflowInvocationState/WorkflowInvocationStep.vue";
import JobInformation from "@/components/JobInformation/JobInformation.vue";

type ReasonToLevel = {
history_deleted: "cancel";
user_request: "cancel";
cancelled_on_review: "cancel";
dataset_failed: "error";
collection_failed: "error";
job_failed: "error";
output_not_found: "error";
expression_evaluation_failed: "error";
when_not_boolean: "error";
unexpected_failure: "error";
workflow_output_not_found: "warning";
};

const level: ReasonToLevel = {
history_deleted: "cancel",
user_request: "cancel",
cancelled_on_review: "cancel",
dataset_failed: "error",
collection_failed: "error",
job_failed: "error",
output_not_found: "error",
expression_evaluation_failed: "error",
when_not_boolean: "error",
unexpected_failure: "error",
workflow_output_not_found: "warning",
};

const levelClasses = {
warning: "warningmessage",
cancel: "infomessage",
error: "errormessage",
};

interface Invocation {
workflow_id: string;
state: string;
}

interface InvocationMessageProps {
invocationMessage: InvocationMessageResponseModel;
invocation: Invocation;
}

const props = defineProps<InvocationMessageProps>();
const levelClass = computed(() => levelClasses[level[props.invocationMessage.reason]]);

const workflow = computed(() => {
if ("workflow_step_id" in props.invocationMessage) {
const { workflow } = useWorkflowInstance(props.invocation.workflow_id);
return workflow.value;
}
return undefined;
});

const workflowStep = computed(() => {
if ("workflow_step_id" in props.invocationMessage && workflow.value) {
return workflow.value.steps[props.invocationMessage.workflow_step_id];
}
return undefined;
});

const dependentWorkflowStep = computed(() => {
if ("dependent_workflow_step_id" in props.invocationMessage && workflow.value) {
const stepId = props.invocationMessage["dependent_workflow_step_id"];
if (stepId !== undefined) {
return workflow.value.steps[stepId];
}
}
return undefined;
});

const jobId = computed(() => "job_id" in props.invocationMessage && props.invocationMessage.job_id);
const HdaId = computed(() => "hda_id" in props.invocationMessage && props.invocationMessage.hda_id);
const HdcaId = computed(() => "hdca_id" in props.invocationMessage && props.invocationMessage.hdca_id);

const cancelFragment = "Invocation scheduling cancelled because";
const failFragment = "Invocation scheduling failed because ";
const stepDescription = computed(() => {
const messageLevel = level[props.invocationMessage.reason];
if (messageLevel === "warning") {
return "This step caused a warning";
} else if (messageLevel === "cancel") {
return "This step canceled the invocation";
} else if (messageLevel === "error") {
return "This step failed the invocation";
} else {
throw "Unknown message level";
}
});

const infoString = computed(() => {
const invocationMessage = props.invocationMessage;
const reason = invocationMessage.reason;
if (reason === "user_request") {
return `${cancelFragment} user requested cancellation.`;
} else if (reason === "history_deleted") {
return `${cancelFragment} the history of the invocation was deleted.`;
} else if (reason === "cancelled_on_review") {
return `${cancelFragment} the invocation was paused at step ${
invocationMessage.workflow_step_id + 1
} and not approved.`;
} else if (reason === "collection_failed") {
return `${failFragment} step ${
invocationMessage.workflow_step_id + 1
} requires a dataset collection created by step ${
invocationMessage.dependent_workflow_step_id + 1
}, but dataset collection entered a failed state.`;
} else if (reason === "dataset_failed") {
if (
invocationMessage.dependent_workflow_step_id !== null &&
invocationMessage.dependent_workflow_step_id !== undefined
) {
return `${failFragment} step ${invocationMessage.workflow_step_id + 1} requires a dataset created by step ${
invocationMessage.dependent_workflow_step_id + 1
}, but dataset entered a failed state.`;
} else {
return `${failFragment} step ${
invocationMessage.workflow_step_id + 1
} requires a dataset, but dataset entered a failed state.`;
}
} else if (reason === "job_failed") {
return `${failFragment} step ${invocationMessage.workflow_step_id + 1} depends on job(s) created in step ${
invocationMessage.dependent_workflow_step_id + 1
}, but a job for that step failed.`;
} else if (reason === "output_not_found") {
return `${failFragment} step ${invocationMessage.workflow_step_id + 1} depends on output '${
invocationMessage.output_name
}' of step ${
invocationMessage.dependent_workflow_step_id + 1
}, but this step did not produce an output of that name.`;
} else if (reason === "expression_evaluation_failed") {
return `${failFragment} step ${
invocationMessage.workflow_step_id + 1
} contains an expression that could not be evaluated.`;
} else if (reason === "when_not_boolean") {
return `${failFragment} step ${
invocationMessage.workflow_step_id + 1
} is a conditional step and the result of the when expression is not a boolean type.`;
} else if (reason === "unexpected_failure") {
return `${failFragment} an unexpected failure occurred.`;
} else if (reason === "workflow_output_not_found") {
return `Defined workflow output '${invocationMessage.output_name}' was not found in step ${
invocationMessage.workflow_step_id + 1
}.`;
} else {
return reason;
}
});
</script>

<template>
<div>
<div :class="levelClass" style="text-align: center">
{{ infoString }}
</div>
<div v-if="dependentWorkflowStep">
Problem occurred at this step:
<workflow-invocation-step
:invocation="invocation"
:workflow="workflow"
:workflow-step="dependentWorkflowStep"></workflow-invocation-step>
</div>
<div v-if="workflowStep">
{{ stepDescription }}
<workflow-invocation-step
:invocation="invocation"
:workflow="workflow"
:workflow-step="workflowStep"></workflow-invocation-step>
</div>
<div v-if="HdaId">
This dataset failed:
<generic-history-item :item-id="HdaId" item-src="hda" />
</div>
<div v-if="HdcaId">
This dataset collection failed:
<generic-history-item :item-id="HdcaId" item-src="hdca" />
</div>
<div v-if="jobId">
This job failed:
<job-information :job_id="jobId" />
</div>
</div>
</template>
Original file line number Diff line number Diff line change
@@ -1,8 +1,5 @@
<script setup>
import { computed, onMounted } from "vue";

import { useWorkflowStore } from "stores/workflowStore";

import { useWorkflowInstance } from "@/composables/useWorkflowInstance";
import ParameterStep from "./ParameterStep";
import GenericHistoryItem from "components/History/Content/GenericItem";
import WorkflowInvocationStep from "./WorkflowInvocationStep";
Expand All @@ -14,11 +11,7 @@ const props = defineProps({
},
});

const workflowStore = useWorkflowStore();

const workflow = computed(() => {
return workflowStore.workflowsByInstanceId[props.invocation.workflow_id];
});
const { workflow } = useWorkflowInstance(props.invocation.workflow_id);

function dataInputStepLabel(key, input) {
const invocationStep = props.invocation.steps[key];
Expand All @@ -32,12 +25,6 @@ function dataInputStepLabel(key, input) {
}
return label;
}

onMounted(async () => {
if (!workflowStore.workflowsByInstanceId[props.invocation.workflow_id]) {
workflowStore.fetchWorkflowForInstanceId(props.invocation.workflow_id);
}
});
</script>
<template>
<div v-if="invocation">
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,18 +12,26 @@
title="Download PDF" />
</span>
</div>
<div v-else>
<div v-else-if="!invocationSchedulingTerminal">
<b-alert variant="info" show>
<LoadingSpan :message="`Waiting to complete invocation ${indexStr}`" />
</b-alert>
<span
v-if="!invocationSchedulingTerminal"
v-b-tooltip.hover
class="fa fa-times cancel-workflow-scheduling"
title="Cancel scheduling of workflow invocation"
@click="onCancel"></span>
</div>
<progress-bar v-if="!stepCount" note="Loading step state summary..." :loading="true" class="steps-progress" />
<template v-if="invocation.messages?.length">
<invocation-message
v-for="message in invocation.messages"
:key="message.reason"
class="steps-progress my-1"
:invocation-message="message"
:invocation="invocation">
</invocation-message>
</template>
<progress-bar
v-else-if="invocationState == 'cancelled'"
note="Invocation scheduling cancelled - expected jobs and outputs may not be generated."
Expand Down Expand Up @@ -57,13 +65,15 @@ import { getRootFromIndexLink } from "onload";
import mixin from "components/JobStates/mixin";
import ProgressBar from "components/ProgressBar";
import LoadingSpan from "components/LoadingSpan";
import InvocationMessage from "@/components/WorkflowInvocationState/InvocationMessage.vue";

import { mapGetters } from "vuex";

const getUrl = (path) => getRootFromIndexLink() + path;

export default {
components: {
InvocationMessage,
ProgressBar,
LoadingSpan,
},
Expand Down