Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

forward message content length and checksum to s3 backend uploads when provided from frontend #345

Open
benmcclelland opened this issue Dec 18, 2023 · 0 comments
Labels
backend issue relates to backend enhancement New feature or request s3proxy

Comments

@benmcclelland
Copy link
Member

Describe the solution you'd like
We are currently limited to unsigned payloads for the backend S3 service for put object and put part because the content body is set to streaming mode. In streaming mode, the body is not seekable, and the s3 sdk is unable to compute the checksums correctly.

So currently we have put object and put part set to use unsigned payloads to the backend to workaround the streaming body:
https://github.com/versity/versitygw/blob/main/backend/s3proxy/s3.go#L312

	output, err := client.PutObject(ctx, input, s3.WithAPIOptions(
		v4.SwapComputePayloadSHA256ForUnsignedPayloadMiddleware,
	))

and
https://github.com/versity/versitygw/blob/main/backend/s3proxy/s3.go#L275

	output, err := client.UploadPart(ctx, input, s3.WithAPIOptions(
		v4.SwapComputePayloadSHA256ForUnsignedPayloadMiddleware,
	))

Since we have the content length and content checksum from the incoming request, we should be able to pass this along to the backend request. We need to investigate if the s3 sdk allows us to provide these values to the backend instead of setting the request to be unsigned.

This might either be some other middleware that allows providing these, or maybe we have to write that middleware?

@benmcclelland benmcclelland added enhancement New feature or request backend issue relates to backend s3proxy labels Dec 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend issue relates to backend enhancement New feature or request s3proxy
Projects
None yet
Development

No branches or pull requests

1 participant