Streaming resources fail if "Authorization" header provided in request

[donaldgray]

Note that this is only relevant when using presignedUrls, see S3ProxyPathGenerator for details.

Files and AV transcodes are streamed from s3 through YARP, using the HTTP format s3 key. There are 2 mechanisms that can allow this to happen:

• A bucket policy that allows s3:GetObject for incoming requests, scoped to VpcEndpoint. This allows for direct S3 bucket access without additional headers, the fact that the request comes from a known vpce means that it's allowed. This can be handy but isn't as secure.
• Generating a presignedUrl. This all happens on the client, a query parameter is then added to the outgoing request - this query param allows access.

If a request for a /file/ or /iiif-av/ contains Authorization header, whether it has a value or not, and uses a presignedUrl the request will fail with a 400 error and body like

<?xml version="1.0" encoding="UTF-8"?>
<Error>
    <Code>InvalidArgument</Code>
    <Message>Only one auth mechanism allowed; only the X-Amz-Algorithm query parameter, Signature query string parameter or the Authorization header should be specified</Message>
    <ArgumentName>Authorization</ArgumentName>
    <ArgumentValue></ArgumentValue>
    <RequestId>ABC1DEFG234</RequestId>
    <HostId>abcdefghijklm/903493240923490243=</HostId>
</Error>

We should strip out any "Authorization" headers before forwarding the request. Optionally handle this to not expose the full s3 error.

[donaldgray]

Note that as part of fixing this I made a slight tweak to proxy logic, there was an un-awaited call that returned a ValueTask. The implementation of this is currently synchronous so there'll be no behavioural changes.

[donaldgray]

Confirmed on dev that the stripping of this header has no affect on access-controlled assets. The header is stripped from the outgoing request only, Protagonist auth has run by then.