Unable to add tags, metadata, parameters or headers with GetPreSignedURL (S3, C#)

[elsamilagros]

When generating a pre-signed URL for a client to upload an object to S3, I'm unable to add tags, metadata, headers or parameters without resulting in a SignatureDoesNotMatch code when using the generated URL. Extra information needs to be added to each object when it's uploaded. There's a PUT event in the folder where the file is uploaded that process the file further. This information is needed to know how to process it. At this point, the work-around I'm thinking in doing is to save a JSON file with this extra information in another folder. It would be ideal to attach it to the object in S3 at the point it's created by using tags, metadata, parameters or headers.

Expected Behavior

This is the basic request construction:

        string key = "Uploads/" + fileName;
        GetPreSignedUrlRequest request = new GetPreSignedUrlRequest
        {
            BucketName = _bucketName,
            Key = key,
            Verb = HttpVerb.PUT,
            Expires = DateTime.Now.AddMinutes(60),
            ContentType = contentType
        };

I've tried to add tags:

string tags = $"Description={description}&EmailSubject={requestSubject}";
request.Headers["x-amz-tagging"] = tags;

I've tried to add parameters:

request.Parameters.Add("Description", description);
request.Parameters.Add("EmailSubject", requestSubject);

I've tried to add metadata:

request.Metadata["Description"] = description;
request.Metadata["EmailSubject"] = requestSubject;

All of these individual efforts returns a URL. When each one is used from the client (browser or Postman), I receive a 403 Forbidden error:

<Code>SignatureDoesNotMatch</Code>
<Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>

Our company's AWS devops team indicates that the policy is correct from the lambda that creates the pre-signed URL and the bucket (bucket policy "Allow" to the lambda with actions s3:GetObject, s3:GetObjectTagging, s3:DeleteObject, s3:DeleteObjectTagging, s3:PutObject, s3:PuObjectTagging).

Current Behavior

The upload is successful if I remove anything beyond generating the pre-signed URL (i.e. I don't add tags, parameters, headers or metadata).

I receive a SignatureDoesNotMatch if I try to add any of these and use the pre-signed URL from a client (browser or Postman).

I would like to be able to add extra information to the object when generating the pre-signed URL or even from the client, if this is possible. It has to occur when creating the object, since there's an immediate event (PUT) when it reaches the bucket.

Your Environment

• AWSSDK.Lambda.AspNetCoreServer version used: 3.1.0
• AWSSDK.S3 version used: 3.3.104.3
• Visual Studio version: 2019
• Targeted .NET platform: Core 2.1

.NET Core Info

• .NET Core version used for development: 2.1
• .NET Core version installed in the environment where application runs: 2.1
• Output of dotnet --info:
  .NET Core SDK (reflecting any global.json):
  Version: 2.1.700
  Commit: c2ef055a0f

Runtime Environment:
OS Name: Windows
OS Version: 10.0.17763
OS Platform: Windows
RID: win10-x64
Base Path: C:\Program Files\dotnet\sdk\2.1.700\

Host (useful for support):
Version: 2.1.11
Commit: d6a5616240

.NET Core SDKs installed:
2.1.700 [C:\Program Files\dotnet\sdk]

.NET Core runtimes installed:
Microsoft.AspNetCore.All 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.App 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]

[klaytaybai]

Hi @elsamilagros, I'm not able to reproduce your problem. I suspect that the error you are seeing is due to the request you are sending. The request from your client (including browser/Postman) needs to match the details of the presigned request. My guess is that your request is missing the necessary headers, which you will need to enter manually into Postman.

[elsamilagros]

Thanks for your time, but the problem that I'm describing is that I'm not able to add tags, parameters, headers or metadata with the AWS S3 SDK for .NET. Your answer does not answer the problem I posted. Please do not close.

[klaytaybai]

Thanks for your time, but the problem that I'm describing is that I'm not able to add tags, parameters, headers or metadata with the AWS S3 SDK for .NET. Your answer does not answer the problem I posted. Please do not close.

Thanks for the response @elsamilagros. I'm sorry I didn't seem to meet your needs. Could you please try rephrasing what you are hoping to do? As I'm currently understanding your issue, I think my answer is suitable. Maybe I need clarification around the post processing step you mention and your level of control over it.

[elsamilagros]

You didn't use the variable that holds pre-signed URL generated (url) in . My request is clear. Please allow others to answer my request. Thanks!

[klaytaybai]

Right you were. Sorry for the mistake. Can you try the below code? The same concept I was discussing earlier still applies. You need to set the headers of the added data onto your upload request.

         AmazonS3Client client = new AmazonS3Client();
            GetPreSignedUrlRequest request = new GetPreSignedUrlRequest
            {
                BucketName = _bucketName,
                Key = _key,
                Verb = HttpVerb.PUT,
                Expires = DateTime.Now.AddMinutes(60),
                ContentType  = "text/plain"
            };
            request.Metadata.Add("Description", "This is my description");
            string tags = $"EmailSubject=This is my subject";
            request.Headers["x-amz-tagging"] = tags;
            string url = client.GetPreSignedURL(request);
            Console.WriteLine(url);

            UploadObject(url, request.Headers, request.Metadata);
        }

        private static void UploadObject(string url, HeadersCollection headers, MetadataCollection metadata)
        {
            byte[] data = Encoding.UTF8.GetBytes("This is my sample body");

            // Configure request
            HttpWebRequest httpRequest = WebRequest.Create(url) as HttpWebRequest;
            foreach (var header in headers.Keys)
            {
                httpRequest.Headers.Add(header, headers[header]);
            }
            foreach (var item in metadata.Keys)
            {
                httpRequest.Headers.Add(item, metadata[item]);
            }
            httpRequest.Method = "PUT";
            // Write data to stream
            Stream requestStream = httpRequest.GetRequestStream();
            requestStream.Write(data, 0, data.Length);
            requestStream.Close();

            // Issue request
            HttpWebResponse response = httpRequest.GetResponse() as HttpWebResponse;
        }

[moedeveloper]

@elsamilagros how did you solve your issue ? because I have the same issue

[elsamilagros]

@moedeveloper I worked around this problem by creating a json file with the metadata I need, storing it in S3 and deleting it when I was done. Very clunky, I know. I wish other people would have chimed in to help out.

[surajg101]

@elsamilagros can you please elaborate , on how you achieved it , well even i am getting the same exception.

[elsamilagros]

@elsamilagros can you please elaborate , on how you achieved it , well even i am getting the same exception.

@surajg101 - Thanks for confirming that this is a real issue and that it shouldn’t have been closed. The work-around I chose is described above your comment. #1359 (comment)