Non cognito related v4 signature #1

ctippur · 2019-12-12T02:13:02Z

Hello,

I need some help with v4 signature.
I am generating pre signed url using generate_presigned_post call - https://boto3.amazonaws.com/v1/documentation/api/latest/guide/s3-presigned-urls.html

This comes with a output like this:

{'url': 'https://mybucket.s3.amazonaws.com
'fields': {'acl': 'public-read',
'key': 'mykey', 'signature': 'mysignature', 'policy': 'mybase64 encoded policy'}

}

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Client.generate_presigned_post

Here is my dart code

    File vfile=File(filepath);
    var stream = new http.ByteStream(DelegatingStream.typed(vfile.openRead()));
    var length = await vfile.length();
    var uri = Uri.parse(signed_url['url']);
    var request = new http.MultipartRequest("POST", uri);

    String _secretKeyId = signed_url['fields']['signature'];
    String accessKeyId = signed_url['fields']['accessKeyId'];
    String sessionToken = signed_url['fields']['policy'];
    const _region = 'ap-south-1';
    String _s3Endpoint = signed_url['url'];
    String _policy=signed_url['fields']['policy'];

    dynamic policy_json=jsonDecode(utf8.decode(base64.decode(_policy)));

    final multipartFile = http.MultipartFile('file', stream, length,
      filename: path.basename(vfile.path));

    final presigned_policy = Policy.fromS3PresignedPost(
      'square-cinnamon.mp4',
      'my-s3-bucket',
      accessKeyId,
      15,
      length,
      region: _region);
    
    
    final key = SigV4.calculateSigningKey(
      _secretKeyId, presigned_policy.datetime, _region, 's3');
    final signature = SigV4.calculateSignature(key, presigned_policy.encode());
    final req = http.MultipartRequest("POST", uri);

    req.files.add(multipartFile);
    req.fields['key'] = presigned_policy.key;
    req.fields['acl'] = 'public-read';
    req.fields['X-Amz-Credential'] = presigned_policy.credential;
    req.fields['X-Amz-Algorithm'] = 'AWS4-HMAC-SHA256';
    req.fields['X-Amz-Date'] = presigned_policy.datetime;
    req.fields['Policy'] = presigned_policy.encode();
    req.fields['X-Amz-Signature'] = signature;
    req.fields['x-amz-security-token'] = "null"; // As we dont get this via generate_presigned_port call

    try {
      final res = await req.send();
      await for (var value in res.stream.transform(utf8.decoder)) {
        print(value);
      }
    } catch (e) {
      print("***** Exception in sending file *****" + e.toString());
    }

I get a error

<Error><Code>InvalidAccessKeyId</Code><Message>The AWS Access Key Id you provided does not exist in our records.</Message><AWSAccessKeyId>null</AWSAccessKeyId><RequestId>C7315716981A0E20</RequestId><HostId>R/Xkg4evb/oZR1EleAm+zHahUb5R9fFjL+wv5mcENwTG1EB95o0A5KojKq2RdhJyhHkLyxrrNFA=</HostId></Error>

Appreciate a response.

Thanks,
Shekar

The text was updated successfully, but these errors were encountered:

furaiev · 2019-12-12T05:55:02Z

@ctippur have you checked your AWS credentials?
https://aws.amazon.com/premiumsupport/knowledge-center/access-key-does-not-exist/

ctippur · 2019-12-12T05:56:50Z

Also, I tried using the values from signed url directly (instead of policy)


    File videofile=File(filepath);
    var stream = new http.ByteStream(DelegatingStream.typed(videofile.openRead()));

    var length = await videofile.length();
    var uri = Uri.parse(signed_url['url']);
    var request = new http.MultipartRequest("POST", uri);


    String _secretKeyId = signed_url['fields']['signature'];
    String accessKeyId = signed_url['fields']['accessKeyId'];
    String policy = signed_url['fields']['policy'];
    const _region = 'ap-south-1';
    String _s3Endpoint = signed_url['url'];
    String _policy=signed_url['fields']['policy'];


    dynamic policy_json=jsonDecode(utf8.decode(base64.decode(_policy)));


    final multipartFile = http.MultipartFile('file', stream, length,
      filename: path.basename(videofile.path));


    final presigned_policy = Policy.fromS3PresignedPost(
      'square-cinnamon.mp4',
      'my-s3-bucket',
      accessKeyId,
      15,
      length,
      region: _region);
    
    


    final key = SigV4.calculateSigningKey(
      _secretKeyId, presigned_policy.datetime, _region, 's3');

    final signature = SigV4.calculateSignature(key, presigned_policy.encode());
    final req = http.MultipartRequest("POST", uri);

    req.files.add(multipartFile);
    req.fields['key'] = accessKeyId;
    req.fields['acl'] = 'public-read';
    req.fields['X-Amz-Credential'] = presigned_policy.credential;
    req.fields['X-Amz-Algorithm'] = 'AWS4-HMAC-SHA256';
    req.fields['X-Amz-Date'] = presigned_policy.datetime;
    req.fields['Policy'] = policy;
    req.fields['X-Amz-Signature'] = signature;
    req.fields['x-amz-security-token'] = "null";


    try {
      final res = await req.send();
      await for (var value in res.stream.transform(utf8.decoder)) {
        print(value);
      }
    } catch (e) {
      print("***** Exception in sending file *****" + e.toString());
    }

ctippur · 2019-12-12T06:04:44Z

The above code is giving a different error Invalid argument(s)

@ctippur have you checked your AWS credentials?
https://aws.amazon.com/premiumsupport/knowledge-center/access-key-does-not-exist/

Yes. I am using my credentials. I have access to my bucket. I can copy objects to the bucket via cli.

ctippur · 2019-12-12T06:05:37Z

also I tried to print policy.credentials I get null/20191212/ap-south-1/s3/aws4_request
Not sure if this is ok.

furaiev · 2019-12-12T06:08:41Z

Pls print all presigned_policy fields

ctippur · 2019-12-12T06:23:01Z

flutter: bucket <my bucket>
flutter:  credentials null/20191212/ap-south-1/s3/aws4_request
flutter:  datetime 20191212T062124Z
flutter: Presigned policy encoded eyAiZXhwaXJhdGlvbiI6ICIyMDE5LTfdsfsdDA2OjM2OjI0Ljc3MDA0MVoiLAogICJjb25kaXRpb25zIjogWwogICAgeyJidWNrZXQiOiAiZ3BlcnQtZGVwZW5kZW50LXRlc3QifSwKICAgIFsic3RhcnRzLXdpdGgiLCAiJGtleSIsICJzcXVhcmUtY2fdfsdf24ubXA0Il0sCiAgICB7ImFjbCI6ICJwdWJsaWMtcmVhZCJ9LAogICAgWyJjb250ZW50LWxlbmd0aC1yYW5nZSIsIDEsIDk2ODIyXSwKICAgIHsieC1hbXotY3JlZGVudGlhbCI6ICJudWxsLzIwMTkxMjEyL2FwLXNvdXRoLTEvczMvYXdzNF9yZXF1ZXN0In0sCiAgICB7IngtYW16LWFsZ29yaXRobSI6ICJBV1M0LUhNQUMtU0hBMjU2In0sCiAgICB7IngtYW16LWRhdGUiOiAiMjAxOTEyMTJUMDYyMTI0WiIgfQogIF0KfQo=
flutter: Presigned policy expiration 2019-12-12T06:36:24.770041Z
flutter: Presigned policy region ap-south-1
flutter: Presigned policy key square-cinnamon.mp4

Pls let me know if I have missed anything

furaiev · 2019-12-12T06:35:06Z

flutter: credentials null/20191212/ap-south-1/s3/aws4_request
null - is your accessKeyId

ctippur · 2019-12-12T06:39:39Z

flutter: credentials null/20191212/ap-south-1/s3/aws4_request
null - is your accessKeyId

I just printed accessKeyId. It does print the right one. Not sure how null is creeping in.

furaiev · 2019-12-12T06:42:39Z

Check accessKeyId in the factory Policy.fromS3PresignedPost
final cred = '$accessKeyId/${SigV4.buildCredentialScope(datetime, region, 's3')}';

ctippur · 2019-12-12T06:53:24Z

fromS3PresignedPost

That was a good find. I fixed that. I still get the same error. Invalid argument(s)

Can you pls give me a second look at

final multipartFile = http.MultipartFile('file', stream, length,
      filename: path.basename(videofile.path));

If I get this right, the filename is taken from the basename. For example if the path is a/b/d/e.mp4, path.basename(videofile.path) would just give e.mp4

Where in the code do we specify the complete file location :/ that it picks the file from?

furaiev · 2019-12-12T06:55:16Z

I don't see this error in this thread, pls add details where do you catch it.

ctippur · 2019-12-12T06:58:49Z

fromS3PresignedPost

That was a good find. I fixed that. I still get the same error. Invalid argument(s)

The above code is giving a different error Invalid argument(s)

@ctippur have you checked your AWS credentials?
https://aws.amazon.com/premiumsupport/knowledge-center/access-key-does-not-exist/

Yes. I am using my credentials. I have access to my bucket. I can copy objects to the bucket via cli.

It has been referenced here.

it is coming in the exception code

catch (e){
  print("***** Exception in sending file *****" + e.toString());
}

furaiev · 2019-12-12T07:04:28Z

Try to remove req.fields['x-amz-security-token'] = "null";

Do you get an error ***** Exception in sending file ***** Invalid argument(s) ?

ctippur · 2019-12-12T07:07:36Z

Did that. Same error.

furaiev · 2019-12-12T07:08:09Z

Do you get an error ***** Exception in sending file ***** Invalid argument(s) ?

ctippur · 2019-12-12T07:08:39Z

Do you get an error ***** Exception in sending file ***** Invalid argument(s) ?
Yes

flutter: ***** Exception in sending file *****Invalid argument(s)

Doe sit mean I need the token?

furaiev · 2019-12-12T07:16:05Z

Pls try to use presigned url from response response['url'] for file loading.

ctippur · 2019-12-12T07:22:20Z

Pls try to use presigned url from response response['url'] for file loading.

I dont think I got that. can you please explain?

You mean here? instead of bucket, use the url?

final presigned_policy = Policy.fromS3PresignedPost(
      'square-cinnamon.mp4',
      'my-s3-bucket',
      accessKeyId,
      15,
      length,
      region: _region);

BTW .. Thanks for handholding.

furaiev · 2019-12-12T07:35:04Z

pls split this into 2 different try / catch to specify the error place

var res;
try {
    res = await req.send();
 } catch (e) {
    print('request ${e.toString()}');
  }
try {
    await for (var value in res.stream.transform(utf8.decoder)) {
      print(value);
    }
  } catch (e) {
    print('transform ${e.toString()}');
  }

ctippur · 2019-12-12T16:57:24Z

Just tried that. Apologies for the delay.

flutter: request Invalid argument(s) from the first try .. catch

ctippur · 2019-12-13T06:41:14Z

@furaiev - any other thing you can think of?

Is there a way to print the complete res object?

var res;
try {
    res = await req.send();
 } catch (e) {
    print('request ${e.toString()}');
  }
try {
    await for (var value in res.stream.transform(utf8.decoder)) {
      print(value);
    }
  } catch (e) {
    print('transform ${e.toString()}');
  }

furaiev · 2019-12-13T07:45:46Z

Pls take a look at this working code - https://github.com/conghua2411/FlutterDemoEverything/blob/e07cd5e1c9f751395a050a9a37ed09e821ee607b/lib/aws_cognito_dev_upload_s3/aws_cognito_dev_upload_s3.dart

I believe you need req.fields['x-amz-security-token'] but maybe you will find another differences.
Pls let me know if something should be changed in this repo Readme.

ctippur · 2019-12-13T14:11:47Z

I will take a look. Thx.
BTW, I dont think it is a aws issue. The error see to be coming from dart. I looked up. Looks like Invalid Argument(s) is a generic error thrown by dart. I am not sure if it is even hitting aws. I can try tcpdump and closely watch the network traffic. I will let you know.

as an alternative, I tried using dio

var formData = FormData.fromMap({
        "key": signed_url['fields']['key'],
        "AWSAccessKeyId": accessKeyId,
        "policy": policy,
        "signature": signed_url['fields']['signature'],
        "file": await MultipartFile.fromFile(path.dirname(filepath),filename: path.basename(filepath),contentType: new MediaType('application', 'x-tar'))
        
    });
    try {
      var response = await Dio().post('<my bucket>', data: formData);

      print (response.statusCode);
    }catch (e){
      print (e.toString());
    }

I dont see an exception but nothing seem to happen :/. There are no errors

furaiev · 2019-12-16T06:16:42Z

Hi @ctippur,
Have you managed to adjust presigned post call ?

furaiev added the question Further information is requested label Dec 12, 2019

furaiev closed this as completed Jan 15, 2020

magicmass mentioned this issue Aug 21, 2020

Unhandled Exception: MissingPluginException #87

Closed

zelliot mentioned this issue Apr 30, 2021

Breaking change in v1.0.0 (Unhandled Exception) #146

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Non cognito related v4 signature #1

Non cognito related v4 signature #1

ctippur commented Dec 12, 2019 •

edited

Loading

furaiev commented Dec 12, 2019

ctippur commented Dec 12, 2019

ctippur commented Dec 12, 2019 •

edited

Loading

ctippur commented Dec 12, 2019 •

edited

Loading

furaiev commented Dec 12, 2019

ctippur commented Dec 12, 2019 •

edited

Loading

furaiev commented Dec 12, 2019

ctippur commented Dec 12, 2019

furaiev commented Dec 12, 2019

ctippur commented Dec 12, 2019 •

edited

Loading

furaiev commented Dec 12, 2019

ctippur commented Dec 12, 2019

furaiev commented Dec 12, 2019 •

edited

Loading

ctippur commented Dec 12, 2019

furaiev commented Dec 12, 2019

ctippur commented Dec 12, 2019 •

edited

Loading

furaiev commented Dec 12, 2019

ctippur commented Dec 12, 2019 •

edited

Loading

furaiev commented Dec 12, 2019

ctippur commented Dec 12, 2019 •

edited

Loading

ctippur commented Dec 13, 2019 •

edited

Loading

furaiev commented Dec 13, 2019

ctippur commented Dec 13, 2019 •

edited

Loading

furaiev commented Dec 16, 2019

Non cognito related v4 signature #1

Non cognito related v4 signature #1

Comments

ctippur commented Dec 12, 2019 • edited Loading

furaiev commented Dec 12, 2019

ctippur commented Dec 12, 2019

ctippur commented Dec 12, 2019 • edited Loading

ctippur commented Dec 12, 2019 • edited Loading

furaiev commented Dec 12, 2019

ctippur commented Dec 12, 2019 • edited Loading

furaiev commented Dec 12, 2019

ctippur commented Dec 12, 2019

furaiev commented Dec 12, 2019

ctippur commented Dec 12, 2019 • edited Loading

furaiev commented Dec 12, 2019

ctippur commented Dec 12, 2019

furaiev commented Dec 12, 2019 • edited Loading

ctippur commented Dec 12, 2019

furaiev commented Dec 12, 2019

ctippur commented Dec 12, 2019 • edited Loading

furaiev commented Dec 12, 2019

ctippur commented Dec 12, 2019 • edited Loading

furaiev commented Dec 12, 2019

ctippur commented Dec 12, 2019 • edited Loading

ctippur commented Dec 13, 2019 • edited Loading

furaiev commented Dec 13, 2019

ctippur commented Dec 13, 2019 • edited Loading

furaiev commented Dec 16, 2019

ctippur commented Dec 12, 2019 •

edited

Loading

ctippur commented Dec 12, 2019 •

edited

Loading

ctippur commented Dec 12, 2019 •

edited

Loading

ctippur commented Dec 12, 2019 •

edited

Loading

ctippur commented Dec 12, 2019 •

edited

Loading

furaiev commented Dec 12, 2019 •

edited

Loading

ctippur commented Dec 12, 2019 •

edited

Loading

ctippur commented Dec 12, 2019 •

edited

Loading

ctippur commented Dec 12, 2019 •

edited

Loading

ctippur commented Dec 13, 2019 •

edited

Loading

ctippur commented Dec 13, 2019 •

edited

Loading