Skip to content

Commit

Permalink
AWS: Ensure that the order returned by ListObjects is consistent
Browse files Browse the repository at this point in the history
When a backup is deleted, the delete method uses ListObjects to get a list of
files it needs to delete in s3. Different s3 implementations may return
the object lists in different, even non-deterministic orders, which can
result in the deletion not working because ark tries to delete a non empty folder
before it tries to delete the files in the folder.

Signed-off-by: Bastian Hofmann <bashofmann@gmail.com>
  • Loading branch information
bashofmann committed Oct 30, 2018
1 parent 9165d51 commit 8bbfc53
Showing 1 changed file with 6 additions and 0 deletions.
6 changes: 6 additions & 0 deletions pkg/cloudprovider/aws/object_store.go
Expand Up @@ -18,6 +18,7 @@ package aws

import (
"io"
"sort"
"strconv"
"time"

Expand Down Expand Up @@ -187,6 +188,11 @@ func (o *objectStore) ListObjects(bucket, prefix string) ([]string, error) {
return nil, errors.WithStack(err)
}

// ensure that returned objects are in a consistent order so that the deletion logic deletes the objects before
// the pseudo-folder prefix object for s3 providers (such as Quobyte) that return the pseudo-folder as an object.
// See https://github.com/heptio/ark/pull/999
sort.Sort(sort.Reverse(sort.StringSlice(ret)))

return ret, nil
}

Expand Down

0 comments on commit 8bbfc53

Please sign in to comment.