S3 Provider fails to upload #1618

brandonhilkert · 2017-06-05T21:04:49Z

Version: 18.3.0

Electron-Updater: 2.0.0

Target: OSX/Windows

I'm using the S3 provider. I can confirm my ENV vars are set properly by successfully uploading via the command line using the aws tool. However, when I go to publish I get:

Error: Cannot cleanup:

Error #1 --------------------------------------------------------------------------------
AccessDenied: Access Denied
    at Request.extractError (/Users/bhilkert/Dropbox/code/bark-desktop/node_modules/electron-publisher-s3/node_modules/aws-sdk/lib/services/s3.js:539:35)
    at Request.callListeners (/Users/bhilkert/Dropbox/code/bark-desktop/node_modules/electron-publisher-s3/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
    at Request.emit (/Users/bhilkert/Dropbox/code/bark-desktop/node_modules/electron-publisher-s3/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
    at Request.emit (/Users/bhilkert/Dropbox/code/bark-desktop/node_modules/electron-publisher-s3/node_modules/aws-sdk/lib/request.js:682:14)
    at Request.transition (/Users/bhilkert/Dropbox/code/bark-desktop/node_modules/electron-publisher-s3/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/Users/bhilkert/Dropbox/code/bark-desktop/node_modules/electron-publisher-s3/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /Users/bhilkert/Dropbox/code/bark-desktop/node_modules/electron-publisher-s3/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/Users/bhilkert/Dropbox/code/bark-desktop/node_modules/electron-publisher-s3/node_modules/aws-sdk/lib/request.js:38:9)
    at Request.<anonymous> (/Users/bhilkert/Dropbox/code/bark-desktop/node_modules/electron-publisher-s3/node_modules/aws-sdk/lib/request.js:684:12)
    at Request.callListeners (/Users/bhilkert/Dropbox/code/bark-desktop/node_modules/electron-publisher-s3/node_modules/aws-sdk/lib/sequential_executor.js:115:18)
    at Request.emit (/Users/bhilkert/Dropbox/code/bark-desktop/node_modules/electron-publisher-s3/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
    at Request.emit (/Users/bhilkert/Dropbox/code/bark-desktop/node_modules/electron-publisher-s3/node_modules/aws-sdk/lib/request.js:682:14)
    at Request.transition (/Users/bhilkert/Dropbox/code/bark-desktop/node_modules/electron-publisher-s3/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/Users/bhilkert/Dropbox/code/bark-desktop/node_modules/electron-publisher-s3/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /Users/bhilkert/Dropbox/code/bark-desktop/node_modules/electron-publisher-s3/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/Users/bhilkert/Dropbox/code/bark-desktop/node_modules/electron-publisher-s3/node_modules/aws-sdk/lib/request.js:38:9)
    at Request.<anonymous> (/Users/bhilkert/Dropbox/code/bark-desktop/node_modules/electron-publisher-s3/node_modules/aws-sdk/lib/request.js:684:12)
    at Request.callListeners (/Users/bhilkert/Dropbox/code/bark-desktop/node_modules/electron-publisher-s3/node_modules/aws-sdk/lib/sequential_executor.js:115:18)
    at callNextListener (/Users/bhilkert/Dropbox/code/bark-desktop/node_modules/electron-publisher-s3/node_modules/aws-sdk/lib/sequential_executor.js:95:12)
    at IncomingMessage.onEnd (/Users/bhilkert/Dropbox/code/bark-desktop/node_modules/electron-publisher-s3/node_modules/aws-sdk/lib/event_listeners.js:256:13)
    at emitNone (events.js:91:20)
    at IncomingMessage.emit (events.js:188:7)
From previous event:
    at Request.promise (/Users/bhilkert/Dropbox/code/bark-desktop/node_modules/electron-publisher-s3/node_modules/aws-sdk/lib/request.js:776:12)
    at /Users/bhilkert/Dropbox/code/bark-desktop/node_modules/electron-publisher-s3/src/uploader.ts:86:153
    at Timeout.tryRun [as _onTimeout] (/Users/bhilkert/Dropbox/code/bark-desktop/node_modules/electron-publisher-s3/src/uploader.ts:198:9)
    at ontimeout (timers.js:386:14)
    at tryOnTimeout (timers.js:250:5)
    at Timer.listOnTimeout (timers.js:214:5)

The text was updated successfully, but these errors were encountered:

develar · 2017-06-05T21:13:39Z

Could you please specify version of electron-publisher-s3? And try to use latest electron-builder?

brandonhilkert · 2017-06-05T21:14:43Z

I originally missed the Cannot cleanup line, which mean me realize it's possible it needed more than just PutObject permissions. I allowed all actions and it worked. Can you confirm the minimal permissions for the S3 provider?

electron-publisher-s3: 18.5.0

brandonhilkert · 2017-06-09T20:34:17Z

Closing b/c increased permissions seems to have resolved it.

develar · 2017-06-11T18:09:48Z

Reopened — question "Can you confirm the minimal permissions for the S3 provider?" is not answered, we must do something smart to save user's time.

kasprownik · 2017-07-11T08:24:07Z

Same problem here, what are the minimal working permissions?

dsagal · 2017-07-12T04:29:28Z

In case this helps anyone, I had a similar symptom that was caused by using a non-default AWS profile. The aws tool uses AWS_DEFAULT_PROFILE variable, but the SDK ignores it and uses only AWS_PROFILE. If you use a non-default profile, you need to set the latter one. (See also here.)

romanrev · 2017-07-12T07:25:20Z

@dsagal and @develar that's what we ended up with after trailing the CloudTrail logs for the requests issued by the electron-builder:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowAppS3Releases",
            "Effect": "Allow",
            "Action": [
                "s3:AbortMultipartUpload",
                "s3:GetObject",
                "s3:GetObjectAcl",
                "s3:GetObjectVersion",
                "s3:ListMultipartUploadParts",
                "s3:PutObject",
                "s3:PutObjectAcl"
            ],
            "Resource": [
                "arn:aws:s3:::release-bucket/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:ListBucketMultipartUploads"
            ],
            "Resource": [
                "arn:aws:s3:::release-bucket"
            ]
        }
    ]
}

notice the *ObjectAcl actions that we need to allow: that's because the electron-build adds a header "x-amz-acl": "public-read" with each upload - trying to mark every object as publicly readable. I am going to open another issue #1822 to ask the developers to make that optional, since one can also achieve the same effect with appropriately crafted S3 bucket policy.

develar · 2017-07-12T07:49:49Z

@romanrev Thanks (official docs: http://docs.aws.amazon.com/AmazonS3/latest/dev/mpuAndPermissions.html)

romanrev · 2017-07-12T07:54:11Z

thanks @develar yes, that's what we looked at after we saw the CreateMultiPartUpload call :-)

emilbruckner · 2018-01-17T17:30:17Z

To everyone coming from the docs who is as stupid as I am:

You have to change your bucket-name in these permissions …

jacob-qurika · 2018-04-27T01:57:50Z

I needed s3:GetBucketLocation as well.

PriscilaAlves · 2018-09-24T17:52:12Z

I also needed s3:GetBucketLocation. This should probably be referred to in the documentation.

JM-Mendez · 2018-09-29T20:04:20Z

@kalokiston @PriscilaAlves

Do you have your bucket in a region besides the default US East (N. Virginia)?

I didn't need to add the s3:GetBucketLocation permission. But I'm using the default region, so wondering if it's just falling back to that region.

mlynch · 2018-10-19T15:18:25Z

Thanks for the policy example! What do you all use for your Principal fields? Can't create a bucket policy without it and it seems like the two entries should have different values for that field.

JM-Mendez · 2018-10-19T16:39:34Z

@mlynch you have two options

use * and allow everyone access to your bucket (not recommended)
create an iam user and use it's AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY

Then add the user to the policy. I named my iam user ci_server

"Sid": "Stmt123456789",
"Effect": "Allow",
"Principal": {
    "AWS": "arn:aws:iam::1234567890:user/ci_server"
 },

walkthrough: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html

mlynch · 2018-10-19T16:48:45Z

Thanks, though there are two operations here. It seems you want to allow read access to everyone (so users can see and download new versions), but only write access to an IAM user, correct? That would mean two separate principal options?

JM-Mendez · 2018-10-19T16:54:30Z

@mlynch If you want to give users direct access to your bucket, then yeah, you'd need to set up those two levels of access. But I recommend using cloudfront.

The data transfer pricing is cheaper than having users download directly from s3. And when you set up cloudfront, it'll ask to add a policy to the bucket for you.

Here are some articles that led me to this conclusion:

https://medium.com/devopslinks/this-is-how-i-reduced-my-cloudfront-bills-by-80-a7b0dfb24128

https://www.expatsoftware.com/articles/2009/01/cloudfront-costs-compared-to-s3.html

mlynch · 2018-10-19T17:40:46Z

Awesome thanks so much! Also probably better download performance for users around the globe I’d imagine

JM-Mendez · 2018-10-19T17:48:38Z

You're welcome :-).

Yeah you'll get way better download performance since your app will be distributed across the edge network, and will be much closer to your users.

One thing to keep in mind -- distributing to cloudfront is not immediate. I've seen an average of 20-30 min for full availability. So if you want to test publishing, use minio server like the guide suggests. The cloudfront url is static, so it doesn't affect publishing.

dariocravero · 2018-11-27T20:31:46Z

Thanks for the policy guide! I wanted to add that apart from setting I also had to uncheck the Block new public ACLs and uploading public objects (Recommended) option in the Permissions tab for it to let me upload, otherwise I kept on getting Access Denied.

EDIT: I also had to uncheck the second option Remove public access granted through public ACLs (Recommended) otherwise the app wasn't able to look for updates.

erikjalevik · 2019-02-08T13:00:59Z

@JM-Mendez, if I use CloudFront for serving my updates, how do I configure the electron-updater inside my app to use the CloudFront URL instead of trying to access S3 directly?

erikjalevik · 2019-02-08T15:04:07Z

@JM-Mendez, if I use CloudFront for serving my updates, how do I configure the electron-updater inside my app to use the CloudFront URL instead of trying to access S3 directly?

Using autoUpdater.setFeedURL seems to do the trick, but then a deprecation message appears in the log: "Feed url Deprecated. Do not use it."

What is the correct way?

JM-Mendez · 2019-02-08T17:20:38Z

@erikjalevik this is how I did it. Are you sure you're calling setFeedUrl and not getFeedUrl? The latter is the one that's deprecated according to the source code

electron-builder/packages/electron-updater/src/AppUpdater.ts

Lines 183 to 185 in 2c674df

    
           getFeedURL(): string | null | undefined { 
        
             return "Deprecated. Do not use it." 
        
           }

electron-builder/packages/electron-updater/src/AppUpdater.ts

Lines 191 to 205 in 2c674df

    
           setFeedURL(options: PublishConfiguration | AllPublishOptions | string) { 
        
             const runtimeOptions = this.createProviderRuntimeOptions() 
        
             // https://github.com/electron-userland/electron-builder/issues/1105 
        
             let provider: Provider<any> 
        
             if (typeof options === "string") { 
        
               provider = new GenericProvider({provider: "generic", url: options}, this, { 
        
                 ...runtimeOptions, 
        
                 isUseMultipleRangeRequest: isUrlProbablySupportMultiRangeRequests(options), 
        
               }) 
        
             } 
        
             else { 
        
               provider = createClient(options, this, runtimeOptions) 
        
             } 
        
             this.clientPromise = Promise.resolve(provider) 
        
           }

kersten · 2019-09-16T19:33:48Z

Sorry to bother u again with this issue, but i can't upload to S3 I always get the error like the issue opener.

laurenschroeder · 2020-05-01T15:12:57Z

I was able to get this working for a private bucket after setting ACL to 'private':

 "publish": {    
      "provider": "s3",    
      "bucket": "bucket-name",    
      "region": "us-east-2",    
      "acl": "private"    
    }

navelDirt · 2023-01-12T02:10:52Z

I was able to get this working for a private bucket after setting ACL to 'private':
 "publish": {    
      "provider": "s3",    
      "bucket": "bucket-name",    
      "region": "us-east-2",    
      "acl": "private"    
    }

This worked for me, thanks!

brandonhilkert closed this as completed Jun 9, 2017

develar reopened this Jun 11, 2017

develar added the deployment label Jun 11, 2017

develar closed this as completed in 203c8c4 Jul 12, 2017

sProject mentioned this issue Nov 24, 2017

ERR unable to publish Linux target on s3 #2332

Closed

dariocravero mentioned this issue Nov 27, 2018

electron-auto-update not working with S3 #3498

Closed

kittaakos mentioned this issue May 20, 2020

Example for S3 provider instead of github iffy/electron-updater-example#41

Open

S3 Provider fails to upload #1618

S3 Provider fails to upload #1618

brandonhilkert commented Jun 5, 2017

develar commented Jun 5, 2017

brandonhilkert commented Jun 5, 2017

brandonhilkert commented Jun 9, 2017

develar commented Jun 11, 2017

kasprownik commented Jul 11, 2017

dsagal commented Jul 12, 2017

romanrev commented Jul 12, 2017 •

edited

develar commented Jul 12, 2017

romanrev commented Jul 12, 2017

emilbruckner commented Jan 17, 2018

jacob-qurika commented Apr 27, 2018

PriscilaAlves commented Sep 24, 2018

JM-Mendez commented Sep 29, 2018

mlynch commented Oct 19, 2018

JM-Mendez commented Oct 19, 2018 •

edited

mlynch commented Oct 19, 2018

JM-Mendez commented Oct 19, 2018

mlynch commented Oct 19, 2018 •

edited

JM-Mendez commented Oct 19, 2018

dariocravero commented Nov 27, 2018 •

edited

erikjalevik commented Feb 8, 2019

erikjalevik commented Feb 8, 2019 •

edited

JM-Mendez commented Feb 8, 2019

kersten commented Sep 16, 2019

laurenschroeder commented May 1, 2020

navelDirt commented Jan 12, 2023

S3 Provider fails to upload #1618

S3 Provider fails to upload #1618

Comments

brandonhilkert commented Jun 5, 2017

develar commented Jun 5, 2017

brandonhilkert commented Jun 5, 2017

brandonhilkert commented Jun 9, 2017

develar commented Jun 11, 2017

kasprownik commented Jul 11, 2017

dsagal commented Jul 12, 2017

romanrev commented Jul 12, 2017 • edited

develar commented Jul 12, 2017

romanrev commented Jul 12, 2017

emilbruckner commented Jan 17, 2018

jacob-qurika commented Apr 27, 2018

PriscilaAlves commented Sep 24, 2018

JM-Mendez commented Sep 29, 2018

mlynch commented Oct 19, 2018

JM-Mendez commented Oct 19, 2018 • edited

mlynch commented Oct 19, 2018

JM-Mendez commented Oct 19, 2018

mlynch commented Oct 19, 2018 • edited

JM-Mendez commented Oct 19, 2018

dariocravero commented Nov 27, 2018 • edited

erikjalevik commented Feb 8, 2019

erikjalevik commented Feb 8, 2019 • edited

JM-Mendez commented Feb 8, 2019

kersten commented Sep 16, 2019

laurenschroeder commented May 1, 2020

navelDirt commented Jan 12, 2023

romanrev commented Jul 12, 2017 •

edited

JM-Mendez commented Oct 19, 2018 •

edited

mlynch commented Oct 19, 2018 •

edited

dariocravero commented Nov 27, 2018 •

edited

erikjalevik commented Feb 8, 2019 •

edited