ListObjectVersions() separates delete markers from versions making it impossible to restore order in which they happened

[crusader-mike]

Confirm by changing [ ] to [x] below to ensure that it's a bug:

• I've gone though Developer Guide and API reference
• I've searched for previous similar issues and didn't find any solution

Describe the bug
ListObjectVersionsResult stores delete markers in a separate array. Because last-modified timestamp has limited resolution (1 second) -- it is impossible to figure out original order of versions and delete markers. If you select all versions and markers with the same key and (stable-)sort them by that timestamp -- if some entries have same timestamp, you will end up with uncertainty.

Expected behavior
ListObjectVersionsResult should store markers and versions in the same array (e.g. std::vector<std::variant<ObjectVersion, DeleteMarkerEntry>>) in correct order (as returned by server).

[KaibaLopez]

@crusader-mike ,
Hmmm, unless I am misunderstanding something, I would say this is a feature request and not a bug though.

It seems that the current behavior (having them separate) is what was intended by the service model. I need to investigate more to be sure but as far as I've seen in other SDKs implementations at least, it looks like that's the case.
That being said, the service response does come with delete markers in the right order so it should be possible.
So yea, don't think we'd replace the current behavior because then we'd be inconsistent with other SDKs, but maybe just adding a new function should be relatively straightforward.

[crusader-mike]

No idea what "service model" means here, but my point is that splitting data (from incoming xml) into two different arrays causes loss of important information and there is no way to recover it without hacking my own version of ListObjectVersions().

[KaibaLopez]

@crusader-mike ,
Sorry, I meant to say it's basically what the SDK is expected to do, and behavior shared across SDKs (check CLI for example) and that is why I don't consider this a bug... that being said, I guess you can argue that this is bad design on the s3 service but we are kind of limited by them in the SDK.

Anyways, yea you are right and it makes sense to have it your way, I think this was even added in the Java SDK at some point ... so...I'll see if I can add it, I would just not replace the current call, keep everybody kinda happy?

[crusader-mike]

@KaibaLopez It is your call... :-)

[crusader-mike]

@KaibaLopez Can you address another minor issue I discovered in ListObjectVersions() while working on this?

Basically, if iteration (with delimiter specified) stops at common prefix field NextVersionIdMarker is not simply empty in response -- it is completely absent. This means simply grabbing GetNextVersionIdMarker() value and passing it to next ListObjectVersions() calls will fail with "NextVersionIdMarker can't be empty" error. Nothing in documentation states that "" has special meaning wrt version id.

As of now workaround is simple: if this string is empty -- do not call SetVersionIdMarker() on request object. I hope other S3 implementation follow suit and never use empty string for version id.

Alternative way to address this is to update ListObjectVersionsRequest::SetVersionIdMarker() to explicitly ignore empty strings and update documentation with "version id is NEVER empty string" (thus forcing all S3 implementations to behave in same way).

[github-actions]

Greetings! Sorry to say but this is a very old issue that is probably not getting as much attention as it deservers. We encourage you to check if this is still an issue in the latest release and if you find that this is still a problem, please feel free to open a new one.