jewel: rgw: aws4: add STREAMING-AWS4-HMAC-SHA256-PAYLOAD support

[jmunhoz]

When authenticating requests using the Authorization header in AWS4, you have
the option of uploading the payload in chunks. You can send data in fixed size
or variable size chunks.

This patch enables streaming mode and signed headers support with chunked
uploads.

Fixes: http://tracker.ceph.com/issues/16311

Signed-off-by: Javier M. Mellid jmunhoz@igalia.com
(cherry picked from commit 5de5876)

[smithfarm]

@jmunhoz Thanks for the backport. If @yehudasa agrees for this feature to be backported (ordinarily, only bugfixes are eligible for backport), then you could cherry-pick the commit from #9689 into this PR and we could include it in the next round of jewel integration testing.

[smithfarm]

@jmunhoz Could you cherry-pick this again and:

1. use "git cherry-pick -x"
2. leave the original commit message unchanged
3. after the "(cherry picked from ...)" line, in the "Conflicts" section, describe how you resolved the conflict(s)

This is very helpful in case something unexpected happens and a forensic investigation becomes necessary.

[jmunhoz]

@smithfarm sure! I would like to run some more manual testing on the backport so I will work on it for the next days.

[smithfarm]

@jmunhoz Thanks! Marking DNM in the meantime.

[jmunhoz]

@smithfarm I would suggest backporting two commits related to memory management in AWS4. With those commits in place and simple manual fixing all things are working ok on my side.

[smithfarm]

@jmunhoz If you think this needs #9689 I would suggest to cherry-pick that into this PR. Also the memory-management commits. As long as @oritwas and @yehudasa agree for this whole feature to be backported into jewel (the "default policy" is to backport only bugfixes, but exceptions can be made), then we might as well include everything, right?

[jmunhoz]

@smithfarm I was considering adding #9689 as a new PR but now it is part of this batch as suggested. The memory-management commits are in place too. Thanks!

[smithfarm]

@jmunhoz Good. I think including all relevant patches in a single PR makes it easier to test and review.

[ghost]

Hey jenkins, test this please ! (bluefs crash)

[ghost]

@mattbenjamin does this backport look good to merge ? It passed the rgw suite http://tracker.ceph.com/issues/16344#note-19 (the failure was due to another pull request).