Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support to cleanup task executions #3902

Closed
sabbyanandan opened this issue Apr 23, 2020 · 6 comments · Fixed by #4394
Closed

Add support to cleanup task executions #3902

sabbyanandan opened this issue Apr 23, 2020 · 6 comments · Fixed by #4394
Assignees
Labels
type/enhancement Is an enhancement request type/help-needed Calling help
Milestone

Comments

@sabbyanandan
Copy link
Contributor

As a user, I'd like the ability to clean up all the task executions, so when I'm in an active dev mode on CF, I have the ability to clean slate the task footprint quickly.

Currently, we have the support to clean up the execution by a particular task execution ID.

dataflow:>task execution cleanup --id

It would be useful to expose the following option:

dataflow:>task execution cleanup --all

Acceptance:

  • Issuing --all would clean up all the task executions that aren't actively running or in use by some other state in SCDF.
  • Attempt to clean up any and all and not terminate if any of the executions are not in an inconsistent state or if it is not deletable.
  • Show the resulting cleanup count in the response.
  • If something fails, show the stacktrace as part of the resulting response.
@sabbyanandan sabbyanandan added type/enhancement Is an enhancement request type/help-needed Calling help and removed type/enhancement Is an enhancement request labels Apr 23, 2020
@sabbyanandan
Copy link
Contributor Author

This option would be a highly risky command to run in a higher-level environment like production. Especially, with the tasks as scheduled, and when you attempt to do this across all executions, things could get really bad, very fast.

@sabbyanandan sabbyanandan changed the title Add support to cleanup "all" task executions from Shell Add support to cleanup task executions Sep 21, 2020
@sabbyanandan sabbyanandan added the type/enhancement Is an enhancement request label Sep 21, 2020
@sabbyanandan
Copy link
Contributor Author

We already support optionally also deleting task executions when destroying the task definition via #3903 — available since v2.6.1.

@mminella
Copy link
Contributor

@sabbyanandan Do we have a confirmation step for these deletes? I think if we did it, running a count query and presenting the user with how many task executions are about to be deleted would be a good compromise. Something like:

dataflow:>task execution cleanup --id
dataflow:> About to delete 1 task executions and related records, are you sure (Y/n)? Y
dataflow:> 1 task executions deleted
...
dataflow:> task execution cleanup --all
dataflow:> About to delete 14293 task executions and related records, are you sure (Y/n)? Y
dataflow:> 14293 task executions deleted.

@sabbyanandan
Copy link
Contributor Author

Oh, that sounds like a great idea. I believe there's that option in Shell to be able to ask for user's confirmation. We could derive the count and build the experience around it. Thanks, @mminella!

@sabbyanandan
Copy link
Contributor Author

cc: @cppwfs @ilayaperumalg — let us know if you have any additional recommendations / concerns.

@cppwfs
Copy link
Contributor

cppwfs commented Sep 22, 2020

I think this is a great addition for those folks in development and testing. IIIIII like it!

ilayaperumalg added a commit to ilayaperumalg/spring-cloud-dataflow that referenced this issue Feb 17, 2021
 - This `all` option would remove all the task execution resources along with the task execution records
 - Allow option to remove only the `completed` task executions
 - Add controller endpoint to retrieve total task executions for performance reasons
 - Add new interface methods into TaskExecutionService to retrieve the task executions using the filter option
 - Add/update tests

Resolves spring-cloud#3902
ilayaperumalg added a commit to ilayaperumalg/spring-cloud-dataflow that referenced this issue Mar 1, 2021
 - This `all` option would remove all the task execution resources along with the task execution records
 - Allow option to remove only the `completed` task executions
 - Add controller endpoint to retrieve total task executions for performance reasons
 - Add new interface methods into TaskExecutionService to retrieve the task executions using the filter option
 - Add/update tests

Resolves spring-cloud#3902

Fix deletion for composed task apps

Add support for deletion by task name

Add a separate REST endpoint /tasks/executions/info

Move Tasks executions info to a separate controller
ilayaperumalg added a commit to ilayaperumalg/spring-cloud-dataflow that referenced this issue Mar 2, 2021
 - This `all` option would remove all the task execution resources along with the task execution records
 - Allow option to remove only the `completed` task executions
 - Add controller endpoint to retrieve total task executions for performance reasons
 - Add new interface methods into TaskExecutionService to retrieve the task executions using the filter option
 - Add/update tests

Resolves spring-cloud#3902

Fix deletion for composed task apps

Add support for deletion by task name

Add a separate REST endpoint /tasks/executions/info

Move Tasks executions info to a separate controller
jvalkeal pushed a commit that referenced this issue Mar 2, 2021
 - This `all` option would remove all the task execution resources along with the task execution records
 - Allow option to remove only the `completed` task executions
 - Add controller endpoint to retrieve total task executions for performance reasons
 - Add new interface methods into TaskExecutionService to retrieve the task executions using the filter option
 - Add/update tests

Resolves #3902

Fix deletion for composed task apps

Add support for deletion by task name

Add a separate REST endpoint /tasks/executions/info

Move Tasks executions info to a separate controller
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/enhancement Is an enhancement request type/help-needed Calling help
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants