unable to delete velero restore objects from the cloud storage

[sawansharma005]

What steps did you take and what happened:
[A clear and concise description of what the bug is, and what commands you ran.)

I was trying to delete the restore objects using the below command but it is not deleting the restore objects from cloud storage.

velero restore delete ${restore_name}

What did you expect to happen:
It should delete the restore from velero as well as from the cloud storage

The output of the following commands will help us better understand what's going on:
(Pasting long output into a GitHub gist or other pastebin is fine.)

Server Side logs

root@testvm:~# velero restore get
NAME                        BACKUP                        STATUS      WARNINGS   ERRORS   CREATED                         SELECTOR
wordpress-mysql-service     velero-daily-20200708054623   Completed   6          0        2020-07-08 05:48:20 +0000 UTC   <none>
wordpress-service-restore   velero-daily-20200708054623   Completed   6          0        2020-07-08 05:49:48 +0000 UTC   <none>
root@testvm:~# velero restore delete --all
Are you sure you want to continue (Y/N)? Y
Restore "wordpress-mysql-service" deleted
Restore "wordpress-service-restore" deleted
root@testvm:~# velero restore get
root@testvm:~#

Cloud side restore components are not deleting

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

Environment:

• Velero version (use velero version): 1.4.0
• Velero features (use velero client config get features):
• Kubernetes version (use kubectl version):
• Kubernetes installer & version:
• Cloud provider or hardware configuration:
• OS (e.g. from /etc/os-release): ubuntu

Vote on this issue!

This is an invitation to the Velero community to vote on issues, you can see the project's top voted issues listed here.
Use the "reaction smiley face" up to the right of this comment to vote.

• 👍 for "I would like to see this bug fixed as soon as possible"
• 👎 for "There are more important bugs to focus on right now"

[carlisia]

This needs testing to see if it's a bug.

[ashish-amarnath]

A quick look at the code for this.
https://github.com/vmware-tanzu/velero/blob/main/pkg/cmd/cli/restore/delete.go#L108-L115
Also, restore_controller does not reconcile the restore object deletion.

We don't delete the restore data from the backup storage location. The only time we delete restores from the BSL is when the associated backup is deleted.
Not deleting the restore from the BSL on velero restore delete maybe by design and not a bug.

I think we can make a case to add a flag that will delete the restore from the BSL.

Going to let other @vmware-tanzu/velero-maintainers chime in on this.

[ashish-amarnath]

I think I am going to recant myself and call a bug.
Scenario:

1. Create a backup b1
2. Restore from b1 as r1
3. Restore from b1 as r2
4. velero restore delete r1 will delete the kubernetes object, and not the restore in the BSL.
5. Now, velero delete backup b1 --confirm will only discover r2 as the restore corresponding to b1 and therefore will delete only r2 from the BSL. This will leak r1 in the BSL that will have to be manually deleted.

Ran this on a cluster to confirm the behavior described in this issue.

Thank you @sawansharma005 for reporting this.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[farcaller]

not stale.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[farcaller]

not stale

[reasonerjt]

Restore CRs are less sensitive than Backup CRs so we may not want to implement "RestoreDeletionController"

One option can be leveraging the finalizer mechanism to we can read the spec of the restore and handle to clean up work in the restore controller.