Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PresignPostObject requires a PutObjectInput with a non-nil Key #3011

Closed
2 of 3 tasks
bytefluxio opened this issue Feb 14, 2025 · 3 comments
Closed
2 of 3 tasks

PresignPostObject requires a PutObjectInput with a non-nil Key #3011

bytefluxio opened this issue Feb 14, 2025 · 3 comments
Assignees
Labels
bug This issue is a bug. closed-for-staleness response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days.

Comments

@bytefluxio
Copy link

Acknowledgements

Describe the bug

The following code should in my understanding work to upload files with the presigned url with a file name that starts with key:

	req, err := presignedClient.PresignPostObject(
		goContext.Background(),
		nil,
		func(opts *s3.PresignPostOptions) {
			opts.Expires = time.Duration(fileHandler.config.PresignURLExpiry) * time.Minute
			opts.Conditions = append(
				opts.Conditions,
				map[string]interface{}{"bucket": fileHandler.config.Bucket},
				[]string{"starts-with", "$key", *key},
			)
		},
	)

This results in the following error:

{
  "code": 500,
  "error": "operation error S3: $type:L, 2 validation error(s) found.\n- missing required field, PutObjectInput.Bucket.\n- missing required field, PutObjectInput.Key.\n"
}

However if I were to add the PutObjectInput

		&s3.PutObjectInput{
			Bucket: aws.String(fileHandler.config.Bucket),
			Key:    key,
		},

The key would have priority over starts-with resulting in a response on upload using the presigned URL with the following message:
Access Denied. (Invalid according to Policy: Policy Condition failed)

The decoded base64-encoded JSON policy would look like this:

{
    "conditions": [
        {
            "X-Amz-Algorithm": "AWS4-HMAC-SHA256"
        },
        {
            "bucket": "runnersbnb"
        },
        {
            "X-Amz-Credential": "REDACTED"
        },
        {
            "X-Amz-Date": "20250214T190554Z"
        },
        [
            "starts-with",
            "$key",
            "5ef7f39d-976c-4d77-ae02-d1a47922a490"
        ],
        {
            "key": "5ef7f39d-976c-4d77-ae02-d1a47922a490"
        }
    ],
    "expiration": "2025-02-14T19:15:54Z"
}

Regression Issue

  • Select this option if this issue appears to be a regression.

Expected Behavior

If you check, it looks like the PresignPostObject expects PutObjectInput instead of a PostObjectInput.

The PostObjectInput's Key member is commented as follows:

	// Object key for which the PUT action was initiated.
	//
	// This member is required.
	Key *string

But maybe I'm missing something?

Current Behavior

Not allowed/able to a omit the key

Reproduction Steps

See above.

Possible Solution

No response

Additional Information/Context

No response

AWS Go SDK V2 Module Versions Used

v1.32.7

Compiler and Version used

1.23.4

Operating System and version

macOs sequoia 15.1.1 (24B91)

@bytefluxio bytefluxio added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Feb 14, 2025
@cajund
Copy link

cajund commented Feb 17, 2025

My situation is similar - and I have also noted the variance with PutObjectInput and the non-existing PostObjectInput type.

It may be required for you to use a variable in the key. See: https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-HTTPPOSTConstructPolicy.html#sigv4-PolicyConditions

My issue is with the generated policy from this function call. I'm getting a Invalid JSON error (note the variable that may help in your case):

  "conditions": [
    {
      "X-Amz-Algorithm": "AWS4-HMAC-SHA256"
    },
    {
      "bucket": "<bucket-name>"
    },
    {
      "X-Amz-Credential": "ASIA5C37OGSDGJTNU***/20250217/us-west-2/s3/aws4_request"
    },
    {
      "X-Amz-Date": "20250217T213809Z"
    },
    {
      "X-Amz-Security-Token": "<token>"
    },
    null,
    null,
    null,
    "starts-with",
    "$key",
    "manual/incoming/de2f8af3-3848-4bf7-9b8f-2471dfb27f9d",
    {
      "key": "manual/incoming/de2f8af3-3848-4bf7-9b8f-2471dfb27f9d/${filename}"
    }
  ],
  "expiration": "2025-02-17T22:38:09Z"

Going to try and roll my own policy.

@cajund
Copy link

cajund commented Feb 17, 2025

This code works for me. Your append() on the PresignPostOptions.Conditions was what I was missing:

func (this *S3Client) GenerateSignedPostURL(txId string) (*s3.PresignedPostRequest, error) {

	presignClient := s3.NewPresignClient(this.client)

	request, err := presignClient.PresignPostObject(this.ctx, &s3.PutObjectInput{
		Bucket: aws.String(os.Getenv("S3_BUCKET")),
		Key:    aws.String("manual/incoming/" + txId + "/${filename}"),
	}, func(options *s3.PresignPostOptions) {
		options.Expires = 3600 * time.Second
		options.Conditions = append(options.Conditions, []interface{}{"starts-with", "$key", "manual/incoming/" + txId + "/"})
	})
	if err != nil {
		return nil, fmt.Errorf("Couldn't get a presigned post request to put %v:%v. Here's why: %v\n", 
			os.Getenv("S3_BUCKET"), 
			"manual/incoming/"+txId, 
			err,
		)
	}
	return request, nil

}

and the generated policy:

{
  "conditions": [
    {
      "X-Amz-Algorithm": "AWS4-HMAC-SHA256"
    },
    {
      "bucket": "<bucket-name>"
    },
    {
      "X-Amz-Credential": "ASIA5C37OGSDGJTNU***/20250217/us-west-2/s3/aws4_request"
    },
    {
      "X-Amz-Date": "20250217T230210Z"
    },
    {
      "X-Amz-Security-Token": "<token>"
    },
    [
      "starts-with",
      "$key",
      "manual/incoming/23b17539-7a5c-4fe6-9b3a-a56419919cb1/"
    ]
  ],
  "expiration": "2025-02-18T00:02:10Z"
}

Best of luck!

@adev-code adev-code self-assigned this Feb 19, 2025
@adev-code adev-code added investigating This issue is being investigated and/or work is in progress to resolve the issue. response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. and removed needs-triage This issue or PR still needs to be triaged. investigating This issue is being investigated and/or work is in progress to resolve the issue. labels Feb 19, 2025
Copy link

This issue has not received a response in 1 week. If you want to keep this issue open, please just leave a comment below and auto-close will be canceled.

@github-actions github-actions bot added closing-soon This issue will automatically close in 4 days unless further comments are made. closed-for-staleness and removed closing-soon This issue will automatically close in 4 days unless further comments are made. labels Feb 28, 2025
@github-actions github-actions bot closed this as completed Mar 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue is a bug. closed-for-staleness response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days.
Projects
None yet
Development

No branches or pull requests

3 participants