v3 support for S3.getSignedUrl behavior

[zfoster]

Describe the issue with documentation
I see the s3 request presigner package is in v3 but I don't quite see a way to replicate the behavior of s3.getSignedUrl('getObject'). I am curious if there could be documentation or an answer on how to replicate the behavior following:

To Reproduce (observed behavior)

const params: S3.Types.PutObjectRequest = {
      Bucket: this.bucket,
      Key: `${key}`,
      ContentType: 'text/csv',
      Body: new Buffer(data, 'binary'),
    };

    await this.s3.putObject(params).promise();

    return this.s3.getSignedUrl('getObject', {
      Bucket: this.bucket,
      Key: `${key}`,
      Expires: 60 * 10, // 10 minutes
    });

Expected behavior
I am able to use the v3 SDK to replicate the putObject call, but I am unable to find source code to return a string URL for an s3 action. The presigner requires an http request object and returns a new one, but I'd like to get a signedUrl like this behavior.

No screenshots or additional context

[simonbuchan]

Looks like you're looking for @aws-sdk/util-create-request

[KeithGillette]

I have the same question as @zfoster. I can use the @aws-sdk/util-create-request and @aws-sdk/s3-request-presigner to create a pre-signed HttpRequest, but how does that map onto the aws-sdk s3.getSignedUrl() functionality? I have attempted to manually build the URL from the HttpRequest object returned by S3RequestPresigner but have not been successful in creating a working URL. Any pointers appreciated!

[zfoster]

Yeah, same thing as @KeithGillette which prompted the original issue. The util-create-request API is very different. I don't have an issue using a request object to create the presigned request, but I just want to return a url.

[KeithGillette]

OK, I finally figured out the correct syntax! Here's an excerpt from the service class that includes a getSignedURL method which will return an AWS Signature v4 for a GetObject command:

import { GetObjectCommand, S3, S3Configuration } from '@aws-sdk/client-s3-browser';
import { S3RequestPresigner } from '@aws-sdk/s3-request-presigner';
import { Sha256 } from '@aws-crypto/sha256-browser';
import { createRequest } from '@aws-sdk/util-create-request';

export class FileStorageService {
	private s3Configuration: S3Configuration = {
		credentials: {
			accessKeyId: environment.ContentBucketKey,
			secretAccessKey: environment.ContentBucketSecret
		},
		region: environment.ContentBucketRegion
	};
	private s3Client: S3 = new S3(this.s3Configuration);
	private s3RequestPresigner: S3RequestPresigner = new S3RequestPresigner({
		...this.s3Configuration,
		sha256: Sha256
	});

	private createS3ObjectKey(s3URL: string): string {
		return payload.slice(s3URL.lastIndexOf('/') + 1);
	}

	public async getSignedURL(s3URL: string, expirationSeconds: number = 15 * 60): Promise<string> {
		const expiration = new Date(Date.now() + expirationSeconds * 1000);
		const request = await createRequest(
			this.s3Client,
			new GetObjectCommand({Bucket: environment.ContentBucketName, Key: this.createS3ObjectKey(s3URL)})
		);
		const presignedRequest = await this.s3RequestPresigner.presignRequest(request, expiration);
		const queryParameters = Object.entries(presignedRequest.query)
			.map(([key, value]: [string, string]) => {
				return `${key}=${encodeURIComponent(value)}`;
			})
			.join('&');
		return `${presignedRequest.protocol}//${presignedRequest.hostname}${presignedRequest.path}?${queryParameters}`;
	}
}

[trivikr]

Sample code and output for getSignedUrl behavior in beta version is shared below:

Code

const AWS = require("aws-sdk");
const { S3, GetObjectCommand } = require("@aws-sdk/client-s3");
const { S3RequestPresigner } = require("@aws-sdk/s3-request-presigner");
const { createRequest } = require("@aws-sdk/util-create-request");
const { formatUrl } = require("@aws-sdk/util-format-url");
const fetch = require("node-fetch");

(async () => {
  let signedUrl;
  let response;

  const region = "us-west-2";
  const Bucket = `test-bucket-${Math.ceil(Math.random() * 10 ** 10)}`;
  const Key = `test-object`;
  const Body = "Body";

  const v2Client = new AWS.S3({ region });
  const v3Client = new S3({ region });

  console.log(`Creating bucket ${Bucket}`);
  await v3Client.createBucket({ Bucket });

  console.log(`Waiting for "${Bucket}" bucket creation...\n`);
  await v2Client.waitFor("bucketExists", { Bucket }).promise();

  console.log(`Putting object "${Key}" in bucket`);
  await v3Client.putObject({ Bucket, Key, Body });

  signedUrl = await v2Client.getSignedUrlPromise("getObject", {
    Bucket,
    Key
  });
  console.log(signedUrl);
  response = await fetch(signedUrl);
  console.log(
    `\nResponse returned by signed URL from v2: ${await response.text()}\n`
  );

  const signer = new S3RequestPresigner({ ...v3Client.config });
  const request = await createRequest(
    v3Client,
    new GetObjectCommand({ Key, Bucket })
  );

  signedUrl = formatUrl(await signer.presign(request));
  console.log(signedUrl);
  response = await fetch(signedUrl);
  console.log(
    `\nResponse returned by signed URL from v3: ${await response.text()}`
  );

  console.log(`\nDeleting object "${Key}" from bucket`);
  await v3Client.deleteObject({ Bucket, Key });

  console.log(`\nDeleting bucket ${Bucket}`);
  await v3Client.deleteBucket({ Bucket });
})();

Output

Creating bucket test-bucket-453738966
Waiting for "test-bucket-453738966" bucket creation...

Putting object "test-object" in bucket
https://test-bucket-453738966.s3.us-west-2.amazonaws.com/test-object?AWSAccessKeyId=ASIA5ROEOH3WC2WAJIQ5&
Expires=1585788810&Signature=TE2YVt5PfuKfyZaQxZDaZFSaLXs%3D&x-amz-security-token=IQoJb3JpZ2luX2VjEDEaCXVzLWVhc3QtMSJHMEUCIFh0pupnvQLdJ%2BnVs2BYnyMLJsfXDYG%2BottMdgrWb06VAiEAxaiFtbIhFy11F2y9CZCh2bVKdEFCIncO%2BqSK4PD0MBUq2AEIOhABGgw5MzA4MDkyNjU5MDAiDIzG0ONVo4rGhaZOqyq1ATeDjWdbHz0cxVr5vRAJVl3mwVvuW2YZpV3bbjLhECQxe6UTz3KF0dBANTCO9rXFH%2B1fjwL0RCwcJhGwShUf0zkZCu5jKuvmVN%2FKz5uyord2bTJiFzsvGwQ%2FZjlJGVRpwFmVDwHruUv4XMD66u5WLcXEHkwvlo%2BX6LiCkT1m%2FWKbivFizzzNka%2FAzMQZWm0HDGaWwGqNZdiNc7cM1pqjDN275ybPvYxqERTFtTTymEMzWCCruLIwzueU9AU64wGika6jhhF1DAsMUJaxrhiIjWVZRg%2BpRyHqtGoex5yzO8XsBnxOr%2BgMwOFRMAOnpKAm4aK3L0QFWEkuLPEvCkTzk%2FmGhvExZb8gFdLIygWpQvOFTWhBiEBupY6zN60B5d3xfwzl%2BWD6%2BjmhArQ71KrDsfion7EgCct5YXuNZhj8LFtnhgaW0xrun%2Fp1OAovZXWWjrM1AojGY6ZUKIhVovxp2XJmnYKABfqCkvYEQO%2FyjVcTQsTiGcG3X9S9RF%2B8uieY3WVg%2Fu%2FUL8n7HiJMtsSkHl6UuPYySJGGvgs%2Fuk2JTSdLRw%3D%3D

Response returned by signed URL from v2: Body

https://test-bucket-453738966.s3.us-west-2.amazonaws.com/test-object?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=ASIA5ROEOH3WC2WAJIQ5%2F20200402%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20200402T003830Z&X-Amz-Expires=900&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEDEaCXVzLWVhc3QtMSJHMEUCIFh0pupnvQLdJ%2BnVs2BYnyMLJsfXDYG%2BottMdgrWb06VAiEAxaiFtbIhFy11F2y9CZCh2bVKdEFCIncO%2BqSK4PD0MBUq2AEIOhABGgw5MzA4MDkyNjU5MDAiDIzG0ONVo4rGhaZOqyq1ATeDjWdbHz0cxVr5vRAJVl3mwVvuW2YZpV3bbjLhECQxe6UTz3KF0dBANTCO9rXFH%2B1fjwL0RCwcJhGwShUf0zkZCu5jKuvmVN%2FKz5uyord2bTJiFzsvGwQ%2FZjlJGVRpwFmVDwHruUv4XMD66u5WLcXEHkwvlo%2BX6LiCkT1m%2FWKbivFizzzNka%2FAzMQZWm0HDGaWwGqNZdiNc7cM1pqjDN275ybPvYxqERTFtTTymEMzWCCruLIwzueU9AU64wGika6jhhF1DAsMUJaxrhiIjWVZRg%2BpRyHqtGoex5yzO8XsBnxOr%2BgMwOFRMAOnpKAm4aK3L0QFWEkuLPEvCkTzk%2FmGhvExZb8gFdLIygWpQvOFTWhBiEBupY6zN60B5d3xfwzl%2BWD6%2BjmhArQ71KrDsfion7EgCct5YXuNZhj8LFtnhgaW0xrun%2Fp1OAovZXWWjrM1AojGY6ZUKIhVovxp2XJmnYKABfqCkvYEQO%2FyjVcTQsTiGcG3X9S9RF%2B8uieY3WVg%2Fu%2FUL8n7HiJMtsSkHl6UuPYySJGGvgs%2Fuk2JTSdLRw%3D%3D&X-Amz-Signature=23a1ccc9baa451ee1e52be8214179af5bc6585f67ee0868bd8c2a92b080eff0a&X-Amz-SignedHeaders=host&x-id=GetObject

Response returned by signed URL from v3: Body

Deleting object "test-object" from bucket

Deleting bucket test-bucket-453738966

Versions:

    "@aws-sdk/client-s3": "^1.0.0-beta.3",
    "@aws-sdk/s3-request-presigner": "^1.0.0-beta.3",
    "@aws-sdk/util-create-request": "^1.0.0-beta.3",
    "@aws-sdk/util-format-url": "^1.0.0-beta.2",

[studds]

Is there any plan to make this less verbose?

[waleedshkt]

@trivikr, Great answer by you.

I guess for restricted access to S3 commands, one would need to enter access credentials at service-level in v3. So in this case, adding them as constructor arguments along with region when instantiating S3 client - just clarifying things here for others viewing this resolved issue

[antstanley]

Hey folks. You really really need to put this into the docs somewhere. I've spent 10 minutes searching for how to do this with v3 and this Github issue is all I could find. You've moved a method from one class to a whole other library, and not documented it. This is incredibly poor.

In general you need to improve the documentation for v3. It's in a very poor state. Literally the worst docs site I've seen from AWS.

[brmur]

@antstanley Thanks for you feedback. Sorry to hear you've experienced frustration with our docs. We're working hard to document the significant changes due to modularisation in V3, but occasionally there are gaps. If you would like to share more information, it will help us provide you with a better service. For example, are you referring primarily to the API docs, the dev guide, or both?

Just to note, an example covering similar functionality for putting an object using a presigned URL is available here in our code repository, and documented here. I've added an example for getting an object with a presigned url, which will be available shortly.

[mfogel]

I find the example from the @aws-sdk/s3-request-presigner README really helpful. It's working for me.

Reproducing it here for clarity:

import { getSignedUrl } from "@aws-sdk/s3-request-presigner";
import { S3Client, GetObjectCommand } from "@aws-sdk/client-s3";
const client = new S3Client(clientParams);
const command = new GetObjectCommand(getObjectParams);
const url = await getSignedUrl(client, command, { expiresIn: 3600 });

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.