modernize s3/aws interface

[chassell]

This is to handle the "filter=" URL attribute (in place of "id=" and others) as well as to pass on a 'content-md5' header with a prepared MD5 hash for a lifecycle query. (? limited knowledge here ?).

This was proven to solve our issues with AWS in several ways.

[chassell]

If you mean md5_hash_b64 .. it’s cleaned up at line 2764 along with many other things. From: Dustin J. Mitchell <notifications@github.com> Sent: Monday, December 2, 2019 6:57 PM @djmitche approved this pull request. In device-src/s3.c<#137 (comment)>:

@@ -2491,11 +2503,19 @@ perform_request(S3Handle *hdl,

... does this need to be freed?

[djmitche]

Ah, got it.

[jamshid]

I'm seeing newer amanda backups to S3-compatible object storage send a Content-MD5 header and Content-length: 44 on GETs. Why would it send a body on a GET to S3? Is it possible that is from this change?

The requests fail with 403 SignatureDoesNotMatch for me but maybe it works against AWS S3.
Amanda does not seem to include the Content-MD5 header in the V2 signature of a GET nor does it validate it matches the body of the GET or the requested object. So I don't see why it's being included on GETs.

BEGIN REQUEST: GET '/amanda-backups/nightly1/slot-foo-tapestart' ...
Authorization: AWS 8b65b6X:ionZiFuzyW4T8j2nVJ56R78ASGI=
Content-Length: 44
Date: Sun, 21 May 2023 14:00:09 GMT
Content-MD5: 3tjrjT2tg/51Vteuk7ohIQ==

Or maybe I'm seeing the libcurl-related bug #213.