Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

can't list workflow runs in case of PVC failures #221

Closed
stefannica opened this issue Sep 8, 2021 · 0 comments · Fixed by fuseml/fuseml-core#105
Closed

can't list workflow runs in case of PVC failures #221

stefannica opened this issue Sep 8, 2021 · 0 comments · Fixed by fuseml/fuseml-core#105
Labels
area/cli CLI features and behavior bug Something isn't working
Projects
FuseML Project Board

Comments

@stefannica
Copy link
Member

This is taken from a FuseML installation in a cluster that accidentally had two default storage classes configured. Upon trying to run the mlflow-e2e workflow, an error was triggered:

Failed to create PVC for PipelineRun fuseml-workloads/fuseml-mlflow-project-01-mlflow-test-xkx4j Workspaces correctly: failed to create PVC pvc-df9cec9a83: persistentvolumeclaims "pvc-df9cec9a83" is forbidden: Internal error occurred: 2 default StorageClasses were found

The underlying cause was fixed, but then listing workflow runs with FuseML stopped working properly because:

error: [workflow listRuns]: invalid response: value of body.status must be one of "Started", "Running", "Cancelled", "Succeeded", "Failed", "Completed", "Timeout", "Unknown" but got value "CouldntCreateWorkspacePVC"
@stefannica stefannica added bug Something isn't working area/cli CLI features and behavior labels Sep 8, 2021
@stefannica stefannica added this to Backlog in FuseML Project Board via automation Sep 8, 2021
@stefannica stefannica moved this from Backlog to V2 in FuseML Project Board Sep 8, 2021
@stefannica stefannica moved this from V3 to Current Iteration in FuseML Project Board Sep 9, 2021
flaviodsr added a commit to flaviodsr/fuseml-core that referenced this issue Sep 10, 2021
@flaviodsr
Fix workflow list runs when status is a failed reason
d27f659 
- design: Removes the `Enum` filter as the status can also returns
a workflow failure reason.
- when a workflow run fails report its status as `Failed (reason)`.

Fixes: fuseml/fuseml#221
@flaviodsr flaviodsr mentioned this issue Sep 10, 2021
Merged
flaviodsr added a commit to flaviodsr/fuseml-core that referenced this issue Sep 10, 2021
@flaviodsr
Fix workflow list runs when status is a failed reason
31ecd11 
- design: Removes the `Enum` filter as the status can also returns
a workflow failure reason.
- when a workflow run fails report its status as `Failed (reason)`.

Fixes: fuseml/fuseml#221
flaviodsr added a commit to flaviodsr/fuseml-core that referenced this issue Sep 10, 2021
@flaviodsr
Fix workflow list runs when status is a failed reason
1f6746d 
- design: Removes the `Enum` filter as the status can also returns
a workflow failure reason.
- when a workflow run fails report its status as `Failed (reason)`.

Fixes: fuseml/fuseml#221
FuseML Project Board automation moved this from Current Iteration to Done Sep 10, 2021
flaviodsr added a commit to flaviodsr/fuseml-core that referenced this issue Sep 10, 2021
@flaviodsr
[0.2] Fix workflow list runs when status is a failed reason
95edbe6 
- design: Removes the `Enum` filter as the status can also returns
a workflow failure reason.
- when a workflow run fails report its status as `Failed (reason)`.

Fixes: fuseml/fuseml#221
(cherry picked from commit 1f6746d)
flaviodsr added a commit to flaviodsr/fuseml-core that referenced this issue Sep 10, 2021
@flaviodsr
Fix workflow list runs when status is a failed reason
3ffce55 
- design: Removes the `Enum` filter as the status can also returns
a workflow failure reason.
- when a workflow run fails report its status as `Failed (reason)`.

Fixes: fuseml/fuseml#221
(cherry picked from commit 1f6746d)
@flaviodsr flaviodsr mentioned this issue Sep 10, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/cli CLI features and behavior bug Something isn't working
Projects
FuseML Project Board
  
Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix workflow list runs when status is a failed reason flaviodsr/fuseml-core
1 participant
@stefannica