Add daily job to cleanup stale resources #1700
Conversation
Co-authored-by: Adrien Mannocci <adrien.mannocci@gmail.com>
Pending to remove cloud cleanup step.
|
/test |
1 similar comment
|
/test |
|
/test |
| # TODO: List and delete the required resources using aws cli | ||
| echo "--- Cleaning up other AWS resources older than ${DELETE_RESOURCES_BEFORE_DATE}" | ||
| echo "--- Installing awscli" | ||
| with_aws_cli |
There was a problem hiding this comment.
What is the deadline? So we can add support for other resources in cloud-reaper?
There was a problem hiding this comment.
There is no hard deadline for this. We were planning to merge this PR to start checking (with plan command) if there is any stale resource with the current resources supported by cloud-reaper.
We were thinking to add in following PRs, at least listing and filtering by tags for the other resources using aws CLI directly starting with redshift clusters. But, we would prefer to do so also using cloud-reaper to keep all the cleanup process using the same tool.
Should we wait for the cloud-reaper support instead for the other resources ? Or, should we add the redshift listing/filtering with aws CLI for now and some notification ? Those redshift clusters were the resources that were left in some builds @jsoriano @kpollich
There was a problem hiding this comment.
If it can wait for some time, I should be able to add support for those resources.
There was a problem hiding this comment.
I should be able to add support for those resources.
Thanks @amannocci !
I think probably we could wait for some time, WDYT @jsoriano @kpollich ?
I've just created a PR to list if there is any AWS Redshift cluster stale due to CI builds. That would allow us to receive email notifications if there is any left cluster until that support is added into cloud-reaper:
#1710
Once this support is added into cloud-reaper, this logic could be moved to the cloud-reaper for sure, so the cleanup is carried out with the same tool for all resources.
In any case, this PR could be merged to start checking there is no resource like instances or queues.
There was a problem hiding this comment.
💚 Build Succeeded
History
cc @mrodm |
4 participants
This PR adds a new daily pipeline to delete all stale resources created in both AWS and GCP cloud providers.
Example build: https://buildkite.com/elastic/elastic-package/builds/2495
Currently, to confirm how it works the commands executed are just plan.
If there is any resource to be deleted, a mail notification will be sent.
To be done in following PRs:
Based on this example:
https://github.com/elastic/elastic-agent/blob/5e17bc222dfa1475bbdf8cade30d67a44dceb436/.buildkite/scripts/steps/gce-cleanup.sh#L15
Tasks to be done before merging:
pre-command