[OpenStack] Introduce bulk deletion of Swift objects #3784
Conversation
|
/test e2e-openstack |
|
/retest |
Fedosin
force-pushed
the
swift_bulkdelete
branch
from
dc18d58
to
921a996
Compare
|
/approve |
openshift-ci-robot
added
the
do-not-merge/hold
|
/test e2e-aws |
|
/hold cancel |
openshift-ci-robot
removed
the
do-not-merge/hold
|
/assign @abhinavdahiya |
| return true, nil | ||
| }) | ||
| if err != nil { | ||
| if _, ok := err.(gophercloud.ErrDefault404); !ok { |
There was a problem hiding this comment.
use errors.As or errors.Is https://godoc.org/github.com/pkg/errors#Is https://godoc.org/github.com/pkg/errors#As more info https://golang.org/pkg/errors/
this is better then simple type coersion because it unwraps and checks.
There was a problem hiding this comment.
I agree with you, but we use this pattern in too many places in the code. I'd prefer to change this in a separate commit.
pkg/destroy/openstack/openstack.go
Outdated
| return false, err | ||
| } | ||
| if len(resp.Errors) > 0 { | ||
| return false, errors.Errorf("Errors occured during bulk deleting of container %q objects", container) |
There was a problem hiding this comment.
errors should not start with capital letters as it makes it weird when wrapped
There was a problem hiding this comment.
This is hiding the actaul errorrs in the returned error. You can try and use https://godoc.org/k8s.io/apimachinery/pkg/util/errors#NewAggregate if the list of resp.Errors implements error iterface
|
/approve |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: abhinavdahiya, mandre The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
openshift-ci-robot
added
the
approved
|
/hold for openstack CI and for the required changes |
openshift-ci-robot
added
the
do-not-merge/hold
Gophercloud 0.10.0 contains a breaking change, where all IDFromName functions were moved to a standalone gophercloud/utils repo: https://github.com/gophercloud/gophercloud/blob/master/CHANGELOG.md#0100-april-12-2020 This PR also addresses this change.
To incorporate latest changes from gophercloud, we have to bump the terraform plugin as well.
Now we delete Swift objects sequentially, Which, in the case of a large number of objects, can take a very long time. Gophercloud 0.11.0 added the possibility of objects bulk deletion: https://github.com/gophercloud/gophercloud/blob/master/CHANGELOG.md#0110-may-14-2020 This commit starts using this feature to delete multiple objects with one request. This should significantly improve cluster removal time. Implements: https://issues.redhat.com/browse/OSASINFRA-1059
Fedosin
force-pushed
the
swift_bulkdelete
branch
from
fbe30ea
to
013c689
Compare
|
/retest |
1 similar comment
|
/retest |
|
/test e2e-openstack |
1 similar comment
|
/test e2e-openstack |
|
/retest |
|
/test e2e-aws |
|
/hold cancel |
openshift-ci-robot
removed
the
do-not-merge/hold
|
/lgtm |
|
/retest Please review the full test history for this PR and help us cut down flakes. |
1 similar comment
|
/retest Please review the full test history for this PR and help us cut down flakes. |
|
@Fedosin: The following tests failed, say
Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
|
/retest Please review the full test history for this PR and help us cut down flakes. |
Now we delete Swift objects sequentially, Which, in the case of a large number of objects, can take a very long time.
Gophercloud 0.11.0 added the possibility of objects bulk deletion:
https://github.com/gophercloud/gophercloud/blob/master/CHANGELOG.md#0110-may-14-2020
This commit starts using this feature to delete multiple objects with one request. This should significantly improve cluster removal time.
Implements: https://issues.redhat.com/browse/OSASINFRA-1059
Implements: https://issues.redhat.com/browse/OSASINFRA-1710