minio return SignatureDoesNotMatch error when trying to upload large file using iOS AWS SDK 2.5.8
#4731
Comments
Update:I followed #4039, and run minio at port 80, still, it does not work. minio server is running at http://192.168.1.198, port should be 80. full charles log: as a compare, I tested a small file, which is 4M, it can upload successfully without issue. request:response |
|
I upload a test demo app for iOS (Xcode 8.3) https://pan.baidu.com/s/1hsGc0rM |
|
have you tried this @liuxuan30 - https://github.com/minio/cookbook/blob/master/docs/aws-sdk-for-iOS-with-minio.md ? |
|
@harshavardhana No, it does not work. Actually, your mentioned function is not getting called at all. Please check my full charles log you can see the uploading requests are fine. But after uploading requests finished, there are more requests sent and failed. Instead, it's calling:
not
why it's if ([hostArray firstObject] && [[hostArray firstObject] rangeOfString:@"s3"].location != NSNotFound) {
//If it is a S3 Request
authorization = [self signS3RequestV4:request
credentials:credentials];
} else {
authorization = [self signRequestV4:request
credentials:credentials];
}The This is how I specify s3, you can see I aleady specify // AWS
AWSStaticCredentialsProvider *s3CredentialsProvider =
[[AWSStaticCredentialsProvider alloc] initWithAccessKey:UploaderAccessKey
secretKey:UploaderSecKey];
NSURL *s3Url = [NSURL URLWithString:uploaderURLString];
AWSEndpoint *s3Endpoint = [[AWSEndpoint alloc] initWithRegion:AWSRegionUSEast1 service:AWSServiceS3 URL:s3Url];
AWSServiceConfiguration *configuration = [[AWSServiceConfiguration alloc] initWithRegion:AWSRegionUSEast1
endpoint:s3Endpoint
credentialsProvider:s3CredentialsProvider];
// regsiter TransferManager with custom configuration
[AWSS3TransferManager registerS3TransferManagerWithConfiguration:configuration forKey:@"StaticAnalysis-S3"];
// get our TransferManager instance
AWSS3TransferManager *s3TransferManager = [AWSS3TransferManager S3TransferManagerForKey:@"StaticAnalysis-S3"];
AWSS3TransferManagerUploadRequest *s3UploadRequest = [AWSS3TransferManagerUploadRequest new];
s3UploadRequest.bucket = bucketName;
s3UploadRequest.key = [NSString stringWithFormat:@"%@.%@", key, @"decrypted"];
s3UploadRequest.body = uploadingFileURL;I think it's a new bug? Because what I search and found previously does not involves multi part upload (the file must be > 5M) You can try my demo project if you have Xcode installed, I already put AWSCore and AWSS3 in the app, so you can directly break at where you want. Here's the string to sign which is failing: and and final authorization field: |
|
Another thing to heads up is, I tried both server 192.168.100.8:9000 and 192.168.100.8 without port (sorry I am at home right now so the IP changed), it both fails for the same reason. So I think it's irrelevant to port issue? I also tried to modify S3 SDK to append port after host like below but does not help as well Right now minio server is black box to me as I don't know which part goes off when calculating the signature. Ideally I would like to start a debugger server of minio so I know where it fails. But I don't know how to run minio from source code and add breakpoints. |
Unless port is included in the canonicalRequest its not a problem with or without.
This is not a problem we will work on it and fix it if there is an interaction issue. Thanks for putting up the project and doing the hard work here - really appreciate your help.
This seems a new interaction bug perhaps same type of fix is required. Can you use simple PutObject instead of multipart operation for now? |
|
Also |
The previous PR we sent is still not accepted and there are perhaps more issues.. @krishnasrinivas |
|
Another weird thing to note is: This is not the sha256 of the body of Also in canonical request: Instead of 83eb01962d21e7183915f8632ce87f858f8b9ee2d44f0c708c0fabbea4e381a7 it should use e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 as the request does not contain the http header X-Amz-Content-Sha256 |
|
@krishnasrinivas Are you able to run the iOS app demo? Or any chance you are familiar with Objective-C? You can then add break points to print anything you want. NSString *contentSha256 = [AWSSignatureSignerUtility hexEncode:[[NSString alloc] initWithData:[AWSSignatureSignerUtility hash:request.HTTPBody] encoding:NSASCIIStringEncoding]];it's doing sha256 hash of the Have you looked at my charles logs? The first several requests that uploading each part of the file is success, it's the last 4 requests failed. I am confused why some requests work and some fails as the signing method should be consistent... |
For small files I could change I wish you have hands on experience for Xcode or Objective-C, so you can directly add breakpoints and point out all values that are used to hash or sha256? I'm not sure if it's iOS SDK issue not following S3 doc. But if you are not, I could offer help, just tell me what data you need so I can input the data and generate the signature along with the string to sign and other useful output I can take that to modify iOS SDK code to be compatible with minio, since we only use minio. But I need to know how to make it work, currently it's blocking issue. Thank you! The file multi part upload already succeeded.. Just the following rquests failed 😂 So I think we are really closing to solve it |
New updateI managed to run minio from source code and add some logs to print out the strings to sign. It turned out that the On iOS, it's using HTTP body to generate: NSString *contentSha256 = [AWSSignatureSignerUtility hexEncode:[[NSString alloc] initWithData:[AWSSignatureSignerUtility hash:request.HTTPBody] encoding:NSASCIIStringEncoding]];These requests have body emptySHA256 = "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"I don't know why but contentCkSum := r.Header.Get("X-Amz-Content-Sha256")
if contentCkSum == "" {
// If not set content checksum is defaulted to sha256([]byte("")).
contentCkSum = emptySHA256
}its calling sequence is
If I manually add
I also notice, for requests have The HTTP Body at that time is empty, but later minio will report "Unable to create object part", I guess I can't force to set computed sha256 into I wish to file a PR for you guys but right now I don't have the context which is the correct way, but this should leave you enough to fix it. Thanks. |
|
@liuxuan30 can you check if this works with ios-sdk? https://github.com/krishnasrinivas/minio/tree/ios-signature-check |
|
@krishnasrinivas looks like it's working, cheers! |
|
@liuxuan30 this is a bug in IOS SDK, this AWS Signature V4 spec http://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-header-based-auth.html says: i.e x-amz-content-sha256 is mandatory. Your earlier observation is also correct regarding: i.e it is calling signRequestV4 instead of signS3RequestV4. Can you raise an issue with IOS SDK saying that x-amz-content-sha256 is missing and giving a reference to the spec doc? |
|
Filed one. However, what confuses me is that, when use multi upload to AWS S3, it works without problem. Why Amazon S3 server works though.. |
Amazon S3 is notorious in not following their own spec @liuxuan30 |
|
All other SDKs other than iOS work fine with Minio. Closing this bug as non issue for Minio is following AWS s3 spec properly. |
|
This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
minio return
SignatureDoesNotMatcherror when trying to upload large file using iOS AWS SDK 2.5.8 in Objective-CExpected Behavior
upload success and signature verify pass
Current Behavior
The file upload requests returns status 200, for each part number.
However, when start verifying, it return SignatureDoesNotMatch
Request header for signature verify:
response:
Possible Solution
No idea
Steps to Reproduce (for bugs)
http://192.168.1.198:9000{ "version": "18", "credential": { "accessKey": "M29O5VEJHNUJILWR1IJA", "secretKey": "xn7sETYYHMJtqHpT2K91llGljBuwmQg2cY1GH0C7" }, "region": "us-east-1", "browser": "on", "logger": { "console": { "enable": true }, "file": { "enable": false, "filename": "" } }, "notify": { "redis": { "1": { "enable": true, "format": "namespace", "address": "192.168.65.1:6379", "key": "bucket_events" } } } }Minio reports:
Context
full charles trace:
s3fail_github.chls.zip
minio server launch info:
Your Environment
minio version): 2017-07-24T18:27:35Zuname -a):Darwin 16.7.0 Darwin Kernel Version 16.7.0: Thu Jun 15 17:36:27 PDT 2017; root:xnu-3789.70.16~2/RELEASE_X86_64 x86_64The text was updated successfully, but these errors were encountered: