fixed marker being saved under wrong key in marker cache

[Decard6]

Currently pagination is broken for backends that use opaque markers. It is so because the key (last key from the set) under which a real marker is saved is different than the key (encoded marker) that S3Proxy is trying to use to read a real marker.

It results with S3Proxy being able to fetch only 1 page:

2023-07-04 14:02:31         13 hw
2023-07-04 14:37:11         13 hw-a-0
2023-07-04 14:43:21         13 hw-aa-0
2023-07-04 14:43:22         13 hw-aa-1
2023-07-04 14:43:29         13 hw-aa-10
2023-07-04 14:43:59         13 hw-aa-100
2023-07-04 14:48:29         13 hw-aa-1000
2023-07-04 14:48:30         13 hw-aa-1001
2023-07-04 14:48:30         13 hw-aa-1002
2023-07-04 14:48:30         13 hw-aa-1003

An error occurred (BadDigest) when calling the ListObjectsV2 operation (reached max retries: 4): Bad Request

This could be solved by either using last key from the set as a marker in response or using encoded nextMarker as part of key in cache. In this PR I decided to implement the second solution.

[gaul]

Thank you for your contribution @Decard6!

[Decard6]

@gaul You're welcome. Do you plan to release 2.0.1?

[larshagencognite]

@Decard6 @gaul This change broke clients that use list blobs v1, and use the last name in the response as the next marker. I don't think this is a great way to iterate over objects, but we are using a third party client that iterates in this way. What is the point of the cache after this change?

[larshagencognite]

For V1 we should keep the old code, and for V2 we should probably handle the encoding/decoding in a different way, as it is probably not related to whether the provider uses opaque markers in V1.

From S3 list blobs v2 docs:

NextContinuationToken
NextContinuationToken is sent when isTruncated is true, which means there are more keys in the bucket that can be listed. The next list requests to Amazon S3 can be continued with this NextContinuationToken. NextContinuationToken is obfuscated and is not a real key
Type: String