Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ERR] Auto-update raises access denied when acl is set to private for S3 #2355

Closed
sProject opened this issue Dec 4, 2017 · 20 comments
Closed

Comments

@sProject
Copy link

sProject commented Dec 4, 2017

  • 19.47.1:
  • 2.16.3
  • Linux, AppImage, x64, Fedora 27:

Hi! I'm working on getting publishing for linux + auto update for AppImage to work with private s3 bucket and private AppImage. I have a very simple app that print its version... and that's about it. When I use S3Options and I set acl : public-read everything works perfect. I'm also able to publish to the private s3 bucket and I'm able to set acl: private. However, I get an error access denied when I run the app and the app wants to check for updates. I have configured ~/.aws/credentials, I even tried setting env vars, nothing helps. Here is my log

[fedora@localhost electron-update-example]$ dist/simple-electron-0.0.1-x86_64.AppImage 
installed: X-AppImage-BuildId=eb49d8b0-d51e-11a7-213a-0b6d859a326c image: X-AppImage-BuildId=bda05140-d8da-11a7-1864-9feaf57d15d2
[10:22:32.809] [info] App starting...
[10:22:33.480] [info] Checking for update
[10:22:33.481] [info] Checking for update...
[10:22:34.684] [error] Error: HttpError: 403 Forbidden
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>CBC7496C38F832C2</RequestId><HostId>ss8I01KR+SqYTft6Rcq78XrSdLPwAqPrW4T9FU7UykqGk6a0Dvdj49nzv2a/zwcRsD8tyTNfM7A=</HostId></Error>"
Headers: {
  "content-type": [
    "application/xml"
  ],
  "date": [
    "Mon, 04 Dec 2017 10:34:29 GMT"
  ],
  "server": [
    "AmazonS3"
  ],
  "transfer-encoding": [
    "chunked"
  ],
  "x-amz-id-2": [
    "ss8I01KR+SqYTft6Rcq78XrSdLPwAqPrW4T9FU7UykqGk6a0Dvdj49nzv2a/zwcRsD8tyTNfM7A="
  ],
  "x-amz-request-id": [
    "CBC7496C38F832C2"
  ]
}
    at createHttpError (/tmp/.mount_simplejtlBEG/app/resources/app.asar/node_modules/builder-util-runtime/src/httpExecutor.ts:29:10)
    at IncomingMessage.response.on (/tmp/.mount_simplejtlBEG/app/resources/app.asar/node_modules/builder-util-runtime/src/httpExecutor.ts:146:18)
    at emitNone (events.js:86:13)
    at IncomingMessage.emit (events.js:188:7)
    at endReadableNT (_stream_readable.js:975:12)
    at _combinedTickCallback (internal/process/next_tick.js:80:11)
    at process._tickCallback (internal/process/next_tick.js:104:9)
From previous event:
    at CancellationToken.createPromise (/tmp/.mount_simplejtlBEG/app/resources/app.asar/node_modules/builder-util-runtime/src/CancellationToken.ts:51:5)
    at ElectronHttpExecutor.doApiRequest (/tmp/.mount_simplejtlBEG/app/resources/app.asar/node_modules/builder-util-runtime/src/httpExecutor.ts:79:30)
    at ElectronHttpExecutor.request (/tmp/.mount_simplejtlBEG/app/resources/app.asar/node_modules/builder-util-runtime/src/httpExecutor.ts:71:17)
    at /tmp/.mount_simplejtlBEG/app/resources/app.asar/node_modules/electron-updater/src/GenericProvider.ts:19:55
    at Generator.next (<anonymous>)
From previous event:
    at GenericProvider.getLatestVersion (/tmp/.mount_simplejtlBEG/app/resources/app.asar/node_modules/electron-updater/out/GenericProvider.js:72:11)
    at /tmp/.mount_simplejtlBEG/app/resources/app.asar/node_modules/electron-updater/src/AppUpdater.ts:251:37
From previous event:
    at AppImageUpdater.doCheckForUpdates (/tmp/.mount_simplejtlBEG/app/resources/app.asar/node_modules/electron-updater/out/AppUpdater.js:326:11)
    at /tmp/.mount_simplejtlBEG/app/resources/app.asar/node_modules/electron-updater/src/AppUpdater.ts:228:25
    at Generator.next (<anonymous>)
    at runCallback (timers.js:672:20)
    at tryOnImmediate (timers.js:645:5)
    at processImmediate [as _immediateCallback] (timers.js:617:5)
From previous event:
    at AppImageUpdater._checkForUpdates (/tmp/.mount_simplejtlBEG/app/resources/app.asar/node_modules/electron-updater/out/AppUpdater.js:280:11)
    at AppImageUpdater.checkForUpdates (/tmp/.mount_simplejtlBEG/app/resources/app.asar/node_modules/electron-updater/src/AppUpdater.ts:177:35)
    at App.<anonymous> (/tmp/.mount_simplejtlBEG/app/resources/app.asar/main.js:139:15)
    at emitTwo (events.js:111:20)
    at App.emit (events.js:194:7)
[10:22:34.698] [info] Error in auto-updater. HttpError: 403 Forbidden
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>CBC7496C38F832C2</RequestId><HostId>ss8I01KR+SqYTft6Rcq78XrSdLPwAqPrW4T9FU7UykqGk6a0Dvdj49nzv2a/zwcRsD8tyTNfM7A=</HostId></Error>"
Headers: {
  "content-type": [
    "application/xml"
  ],
  "date": [
    "Mon, 04 Dec 2017 10:34:29 GMT"
  ],
  "server": [
    "AmazonS3"
  ],
  "transfer-encoding": [
    "chunked"
  ],
  "x-amz-id-2": [
    "ss8I01KR+SqYTft6Rcq78XrSdLPwAqPrW4T9FU7UykqGk6a0Dvdj49nzv2a/zwcRsD8tyTNfM7A="
  ],
  "x-amz-request-id": [
    "CBC7496C38F832C2"
  ]
}
@sProject sProject changed the title [ERR] Auto-update raises access denied from a private S3 bucket [ERR] Auto-update raises access denied when acl is set to private for S3 Dec 4, 2017
@develar
Copy link
Member

develar commented Dec 8, 2017

Auto-update downloads file using public HTTP urls, not using API. Do you understand that your users will need to set credentials also?

@sProject
Copy link
Author

sProject commented Dec 8, 2017

Yes, I understand this. I have aws cli credentials set up on my machine but auto update ignores them. Can you give me an advise on how to use aws module for auto updater instead of normal HTTPs.

Thanks

@develar
Copy link
Member

develar commented Dec 8, 2017

As far I see, you need to set Authorization header. You can simply set it using autoUpdater.requestHeaders property.

See http://docs.aws.amazon.com/AmazonS3/latest/dev/RESTAuthentication.html#ConstructingTheAuthenticationHeader

https://www.npmjs.com/package/aws4 can be used to generate such header. Please try :)

autoUpdater.requestHeaders = {Authorization: "secret"}

@sProject
Copy link
Author

sProject commented Dec 8, 2017

Awesome man! Thanks for the help! I'll try it and report back

@sProject
Copy link
Author

OK here it is - the ultimate guide to having private s3 bucket, for anyone else who is interested in this.

HOWTO:

Dependencies
aws-sign
aws-sdk

Main.js
All modifications are done in the main.js electron file - the file where you implement auto-update.

Step 1 - Import libraries

const aws = require('aws-sdk');
const AwsSign = require('aws-sign');

Step 2 - Set constants
You'll need a copy of your package.json build section, the one that looks like this

{
  "build": {
    "publish": {
      "bucket": "BUCKET_NAME",
      "path": "PATH/TO/FILES"
    }
  }
}

Using it we define the following constants

const release_path = config.get('build').publish.path;
const latest_yml_path = `/${release_path}/latest-${process.platform}.yml`;
const s3_bucket = config.get('build').publish.bucket;

Step 3 - Create signer

const credentials = new aws.SharedIniFileCredentials();
const signer = new AwsSign({
  accessKeyId: credentials.accessKeyId,
  secretAccessKey: credentials.secretAccessKey
});

Step 4 - Update checking-for-update

autoUpdater.on('checking-for-update', () => {
  var opts = {
    method: 'GET',
    host: `${s3_bucket}.s3.amazonaws.com`,
    path: latest_yml_path
  };
  signer.sign(opts);
  autoUpdater.requestHeaders = opts.headers
})

Step 5 - Update update-available

autoUpdater.on('update-available', (info) => {
  let update_path = `/${release_path}/${info.path}`;
  let opts = {
    method: 'GET',
    host: `${s3_bucket}.s3.amazonaws.com`,
    path: update_path
  };
  signer.sign(opts);
  autoUpdater.requestHeaders = opts.headers
  autoUpdater.downloadUpdate()
  }
})

Hope this helps!

@marcusjwhelan
Copy link

@sProject I currently have a private S3 without all of this work. I simply have permissions set to getObject only for public but no write access. And have the aws credentials on my build laptop in my .aws folder. You could I presume have these credentials in CodeShip as well and use docker in there to build the project but I haven't done that yet.

@marcolink
Copy link

marcolink commented Feb 19, 2019

For us, hosting our tools (electron apps) on public accessible buckets is not an option. The given example didn't work with our region eu-central-1 which only allows V4 signing. Therefore we had to change the signing part a bit:

Solution:

We use aws4 for request signing.

and

autoUpdater.on('checking-for-update', () => {
  var opts = {
    method: 'GET',
    host: `${s3_bucket}.s3.amazonaws.com`,
    path: latest_yml_path
  };
  signer.sign(opts);
  autoUpdater.requestHeaders = opts.headers
})

becomes

import * as aws4 from 'aws4';
import * as path from 'path'
const pkg = require('../../package');

autoUpdater.on('checking-for-update', () => {
  const opts = {
    service: 's3',
      region: pkg.build.publish.region,
      method: 'GET',
      host: `s3-${pkg.build.publish.region}.amazonaws.com`,
      path: path.join('/', pkg.build.publish.bucket, latest_yml_path)
  };
  
  aws4.sign(opts, {
    accessKeyId: <AWS_ACCESS_KEY>,
    secretAccessKey: <AWS_SECRET_ACCESS_KEY>
  });
  signer.sign(opts);
  autoUpdater.requestHeaders = opts.headers
})

The same also works for the actual update.

Question:

Is there a way to write a custom Provider, or extends the existing GenericProvider to apply signing where it should actually happen

@stale
Copy link

stale bot commented Jul 1, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the backlog label Jul 1, 2019
@stale stale bot closed this as completed Jul 8, 2019
@stuartcusackie
Copy link

stuartcusackie commented Jul 9, 2019

@marcolink

How come you have signer.sign(opts) after aws4.sign? Seems like you are signing with two different aws sdk versions?

I'm having trouble with autoUpdater on my private repo:

at downloadBlockMap
status 403: Forbidden

Very reluctant to switch to a public repo...

I've opened an issue here:
#4030

@marcolink
Copy link

@stuartcusackie

sorry, my code had a bug, it's actually:

import * as aws4 from 'aws4';
import * as path from 'path'
const pkg = require('../../package');

autoUpdater.on('checking-for-update', () => {
  const opts = {
    service: 's3',
      region: pkg.build.publish.region,
      method: 'GET',
      host: `s3-${pkg.build.publish.region}.amazonaws.com`,
      path: path.join('/', pkg.build.publish.bucket, latest_yml_path)
  };
  
  aws4.sign(opts, {
    accessKeyId: <AWS_ACCESS_KEY>,
    secretAccessKey: <AWS_SECRET_ACCESS_KEY>
  });
  // signer.sign(opts); --remove this line --
  autoUpdater.requestHeaders = opts.headers
})

@stuartcusackie
Copy link

Thanks @marcolink !

Still having trouble with these blockmaps...

@stuartcusackie
Copy link

@marcolink @sProject

Did you ever have trouble with the blockmap requests? I seem to need to create two more signing requests for the old blockmap and the new blockmap?

Can I see your builder configs?

@JTInfinite
Copy link

Hi all, sorry to resurrect a closed issue (I can of course open a new one if preferred) but using the above configuration I receive the error: Error: net::ERR_TOO_MANY_REDIRECTS.

Any ideas on what I might be doing wrong here?
Thanks

@JTInfinite
Copy link

Referencing my above code - Error: net::ERR_TOO_MANY_REDIRECTS is the error in the development environment.

When I build the app I get the following:

"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>

This leads me to believe I am constructing the opts object wrong. Super annoying!

@scorring
Copy link

scorring commented Nov 10, 2020

@JTInfinite, I was having the same SignatureDoesNotMatch issue with the code provided by @marcolink.
Here is my working code sample:

autoUpdater.on("checking-for-update", async () => {
  let opts = {
    region: "eu-west-1",
    protocol: "https:",
    hostname: "BUCKET_NAME.s3.amazonaws.com",
    path: "/PATH/TO/latest.yml",
    host: "s3-eu-west-1.amazonaws.com"
  };

  await aws4.sign(opts, {
    accessKeyId: "XXX",
    secretAccessKey: "YYY"
  });

  autoUpdater.requestHeaders = opts.headers;
});

Change host and region following your values.

@Cyrillius
Copy link

@scorring

I am having this exact issue but I didn't find a working solution.

What is strange is that to find an existing version the Host is correctly set:

11:35:04.067 > Checking for update
11:35:04.118 > checking-for-update options: {
  service: 's3',
  region: 'eu-west-1',
  host: '{MYBUCKET}.s3-eu-west-1.amazonaws.com',
  path: '/4/latest.yml'
}
11:35:04.254 > checking-for-update header: {
  Host: '{MYBUCKET}.s3-eu-west-1.amazonaws.com',
  'X-Amz-Content-Sha256': 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
  'X-Amz-Date': '20201118T103504Z',
  Authorization: 'AWS4-HMAC-SHA256 Credential={MYKEYID}/20201118/eu-west-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=fcf7d3b39bae40e56f748f5a3a64f6b366e0f23941a5dd047783007cabe69ce8'
}
11:35:06.359 > Found version 0.1.5 (url: {MYAPP} 0.1.5.exe)

But to download the actual block map the host is not set anymore:

11:35:06.512 > Download block maps (old: "https://{MYBUCKET}.s3.amazonaws.com/4/Festina%20Support%20Setup%200.1.4.exe.blockmap", new: https://{MYBUCKET}.s3.amazonaws.com/4/Festina%20Support%20Setup%200.1.5.exe.blockmap)
header of the failed request : {
  accept: '*/*',
  'X-Amz-Content-Sha256': 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855',
  'X-Amz-Date': '20201118T103504Z',
  Authorization: 'AWS4-HMAC-SHA256 Credential={MYKEYID}/20201118/eu-west-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=fcf7d3b39bae40e56f748f5a3a64f6b366e0f23941a5dd047783007cabe69ce8',
  'User-Agent': 'electron-builder',
  'Cache-Control': 'no-cache'
}

It seems like electron is copying signature information but not the Host information.

Maybe this fix is not totally working: https://github.com/electron-userland/electron-builder/pull/4848/files

Still searching where is the issue....

@scorring
Copy link

scorring commented Dec 15, 2020

@Cyrillius : Did you try with exactly these options:

region: "eu-west-1",
protocol: "https:",
hostname: "BUCKET_NAME.s3.amazonaws.com",
path: "/PATH/TO/latest.yml",
host: "s3-eu-west-1.amazonaws.com"

I had to play around before finding these fields (less, more or distinct properties caused error).

@Cyrillius
Copy link

my options were:

  const opts = {
    service: 's3',
    region: api.amazon.s3Bucket.region,
    host: `${api.amazon.s3Bucket.name}.${api.amazon.s3Bucket.host}`,
    path:
      process.platform === 'win32'
        ? `${api.amazon.s3Bucket.releasePath}/latest.yml`
        : `${api.amazon.s3Bucket.releasePath}/latest-${process.platform}.yml`,
  };
  log.info('checking-for-update', opts);
  aws4.sign(opts, {
    accessKeyId: api.amazon.production.keyId,
    secretAccessKey: api.amazon.production.accessKey,
  });
  autoUpdater.requestHeaders = opts.headers;

with parameters:

    "s3Bucket": {
    "name": "kronabyflashstationupdatebucket",
    "host": "s3.amazonaws.com",
    "region": "eu-west-1",
    "releasePath": "/4"
  }

and still have this kind of errors:

[2020-11-19 00:15:52.727] [error] Cannot download differentially, fallback to full download: Error: Cannot download "https://{bucketname}.s3.amazonaws.com/4/xxxx-support-0.1.6.exe.blockmap", status 403: Forbidden
    at ClientRequest.<anonymous> (C:\Users\{user}\AppData\Local\Programs\xxxx-support\resources\app.asar\main.prod.js:2:36762)
    at ClientRequest.emit (events.js:210:5)
    at SimpleURLLoaderWrapper.<anonymous> (electron/js2c/browser_init.js:2559:12)
    at SimpleURLLoaderWrapper.emit (events.js:210:5)

I have given up as on Linux platform there is no error and on windows it will the app completely but will work at the end.

I tried your solution which is for me:

  const opts = {
    service: 's3',
    protocol: "https:",
    region: api.amazon.s3Bucket.region,
    hostname: `${api.amazon.s3Bucket.name}.${api.amazon.s3Bucket.host}`,
    host: "s3-eu-west-1.amazonaws.com",
    path:
      process.platform === 'win32'
        ? `${api.amazon.s3Bucket.releasePath}/latest.yml`
        : `${api.amazon.s3Bucket.releasePath}/latest-${process.platform}.yml`,
  };
  log.info('checking-for-update', opts);
  aws4.sign(opts, {
    accessKeyId: api.amazon.production.keyId,
    secretAccessKey: api.amazon.production.accessKey,
  });
  autoUpdater.requestHeaders = opts.headers;

and still have this issue:

[2020-12-31 14:24:19.213] [error] Cannot download differentially, fallback to full download: Error: Cannot download "https://{bucketname}.s3.amazonaws.com/4/xxxx-support-0.1.9.exe.blockmap", status 403: Forbidden
    at ClientRequest.<anonymous> (C:\Users\{user}\Documents\Git\xxx-support\release\win-unpacked\resources\app.asar\main.prod.js:2:36989)
    at ClientRequest.emit (events.js:210:5)
    at SimpleURLLoaderWrapper.<anonymous> (electron/js2c/browser_init.js:2559:12)
    at SimpleURLLoaderWrapper.emit (events.js:210:5)

@reddybhavanish
Copy link

Does anthing worked on this ?

@fabiobsantosprogrow
Copy link

fabiobsantosprogrow commented May 14, 2024

Nowadays we still facing the same problem with version 6.1.4.
I don't understand why this issue is closed when people still having issues...
We will try the download manually on private s3 approach but I would like to have some guidance how to this using the auto-update without extra work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

10 participants