Motivation

As reported in #10881, we hit a deadlock in S3 when copying in place from a never versioned bucket into a Suspended bucket. This was due to a missing check in the copy in-place detection. We only ever checked if the object did not have a version_id, but version_id = None give the same result as version_id = "null".

This part of the code was then responsible for the deadlock:

if should_copy_in_place(src_bucket, src_object, dest_bucket, dest_object):
    return self.open(dest_bucket, dest_object, mode="r")

with self.open(src_bucket, src_object, mode="r") as src_stored_object:
    dest_stored_object = self.open(dest_bucket, dest_object, mode="w")
    dest_stored_object.write(src_stored_object)

As the copy was not flagged as "in-place" when it should, we would not return early and try to open the same underlying object both in write and read mode 😅
(We had an S3 object with version_id = None and the other was version_id = "null", effectively being the same thing).

Changes

• added more tests around copy in place behavior in AWS regarding Suspended buckets
• fixed the condition by adding a small helper method to verify that the object is non-versioned

fixes #10881