Issue on the post-install script behind corporate proxy

[jmorille]

On Ubuntu 13.04, behing a corporate proxy I have this error (same as https://github.com/Polymer/wct-sauce/issues/3 ) :

~/someFolder$ npm install -g wct-sauce

> sauce-connect-launcher@0.9.3 postinstall /home/a000cqp/opt/nodejs/lib/node_modules/wct-sauce/node_modules/sauce-connect-launcher
> node scripts/install.js

/
> wct-sauce@1.2.0 postinstall /home/a000cqp/opt/nodejs/lib/node_modules/wct-sauce
> node scripts/postinstall.js

Prefetching the Sauce Connect binary.
Missing Sauce Connect local proxy, downloading dependency
This will only happen once.

events.js:72
        throw er; // Unhandled 'error' event
              ^
Error: connect ETIMEDOUT
    at errnoException (net.js:905:11)
    at Object.afterConnect [as oncomplete] (net.js:896:19)

npm ERR! wct-sauce@1.2.0 postinstall: `node scripts/postinstall.js`
npm ERR! Exit status 8
npm ERR! 
npm ERR! Failed at the wct-sauce@1.2.0 postinstall script.
npm ERR! This is most likely a problem with the wct-sauce package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node scripts/postinstall.js
npm ERR! You can get their info via:
npm ERR!     npm owner ls wct-sauce
npm ERR! There is likely additional logging output above.
npm ERR! System Linux 3.13.0-45-generic
npm ERR! command "/home/a000cqp/opt/nodejs/bin/node" "/home/a000cqp/opt/nodejs/bin/npm" "install" "-g" "wct-sauce"
npm ERR! cwd /home/a000cqp/project/tt-components/tt-crud
npm ERR! node -v v0.10.36
npm ERR! npm -v 1.4.28
npm ERR! code ELIFECYCLE
npm ERR! not ok code 0

They a way to activate a verbose mode, in order to check the correct reading of system env http_proxy, https_proxy, no_proxy?

[gabuzomeu]

It was the sauce-connect-launcher that doind a simple https request in
https://github.com/bermi/sauce-connect-launcher/blob/master/lib/sauce-connect-launcher.js

function fetchAndUnpack(options, callback) {
  var req = https.request({
      host: "saucelabs.com",
      port: 443,
      path: "/downloads/" + getArchiveName()
    });

and I don't if the standard https module honor the HTTP_PROXY , HTTPS_PROXY and NO_PROXY system env.
the documentation is not clear about this subject http://nodejs.org/api/https.html

[jmorille]

I found a ticket about this subject : bermi/sauce-connect-launcher#26

[sylvainemery]

I have the same problem with my proxy at work. After reading the ticket mentioned above, I've put up this workaround:

• Install the sauce-connect-launcher module with npm install sauce-connect-launcher
• Download the latest Sauce Connect binary for your platform from here: https://docs.saucelabs.com/reference/sauce-connect/#basic-setup
• Uncompress the download in node_modules/sauce-connect-launcher/sc/. You should have a path like node_modules/sauce-connect-launcher/sc/sc-X.Y.Z-platform/bin/ etc.
• Now you can npm install wct-sauce. Because the binary has been manually prefetched, the postinstall.js script won't throw an error.

[sbryan]

I am also hitting this same error, also behind a firewall but with working proxy settings for npm (meaning I have no problems installing other modules from the standard registry).

Tried the workaround @sylvainemery provided above, but it did not solve the problem for me.

One thing to double check... after installing sauce-connect-launcer, I did not have an /sc/ subdir in the module install dir. So I created it, and uncompressed the sc binary package into it... Should I have seen/done something different?

[sylvainemery]

Hi @sbryan,

You're right, the /sc/ subdir doesn't exist when you install sauce-connect-launcher, you have to create it.
After uncompressing, you have to be sure that the sc binary is in node_modules/sauce-connect-launcher/sc/sc-X.Y.Z-platform/bin/

[sbryan]

OK, so I did the right thing, and yet, it behaves as if it still can't find sauce-connect and tries to download it again... meaning it still does this:

Prefetching the Sauce Connect binary.
Missing Sauce Connect local proxy, downloading dependency
This will only happen once.

events.js:72
        throw er; // Unhandled 'error' event
              ^
Error: connect ETIMEDOUT
    at errnoException (net.js:905:11)
    at Object.afterConnect [as oncomplete] (net.js:896:19)

I did notice that when I grabbed the prebuilt sc module from the URL provided in an earlier comment, what I got was the latest version (4.3.6) but the sauce-connect-launcher seems to want an earlier version (4.3.5)... so, I found and unzipped that one instead (removing the 4.3.6 version) into the /sc subdir... 🎱 ...no love :(

There must be something I'm missing or doing wrong as it seems like wct-sauce (or sauce-connect-launcher?) is just not finding the pre-built sc binary... ideas?

As further data points, this morning I tried installing when not behind the proxy/firewall (actually, I tried installing web-component-tester, which is my real goal here). It got past the timeout (still pre-fetching sauce-connect, despite having it installed), but now web-component-tester install fails:

> wct-sauce@1.2.0 postinstall /Users/sbryan/dev/test/elements/node_modules/web-component-tester/node_modules/wct-sauce
> node scripts/postinstall.js

Prefetching the Sauce Connect binary.
Missing Sauce Connect local proxy, downloading dependency
This will only happen once.
Downloading 2.92MB
Unzipping sc-4.3.5-osx.zip
Removing sc-4.3.5-osx.zip
Sauce Connect downloaded correctly
Removing /Users/sbryan/dev/test/elements/node_modules/web-component-tester/node_modules/wct-sauce/node_modules/sauce-connect-launcher/sc/sc-4.3.5-osx.zip
npm ERR! cb() never called!

Note the npm ERR! cb() never called!...

This is probably now an issue I should bring up in web-component-tester repo, not here.

Never-the-less, installation of wct-sauce from behind a proxy still remains broken, or, more diagnosis and explicit instructions on exactly where (local, global for user, system global) to install sauce-connect-launcher (and pre-built sc module) so that wct-sauce postinstall.sh will not trigger an attempt to fetch it again.

I'm happy to provide any more details on my env or commands used if it will help, just let me know what info is needed.

One possibly relevant piece of data is that I am using nvm (node version manager) to juggle between versions of node, so when I do install modules globally, they actually end up installed in ~/.nvm/<node-version>/lib/node_modules/. nvm exports, among other things, a NODE_PATH environment variable in the running shell that points to this module path... maybe this is breaking the module lookup process? I don't know enough about now npm postinstall scripts have their environments set up to know if this could be a factor.

[sylvainemery]

@sbryan, have you tried installing sauce-connect-launcher and wct-sauce directly and not via web-component-tester?
This is what I did on my Ubuntu laptop at work, behind a proxy, and it worked fine.

Also, I've installed web-component-tester on my MBP at home, without proxy, and I have not encountered your error. Maybe it is related to nvm.
Sorry I can't help more...

[jreichenberg]

It would help if the postinstall.js had the sauce-connect-launcher download() call in a try/catch (or maybe wct-sauce becomes an optional dependency...not sure if that solves the issue). It seems unstable over proxies, at least, so If it errors out due to a connection issue npm can still proceed and it doesn't bomb the whole web-component-tester install. This appears to be the intention judging by the console log messages after the call to download(), but there are lots of ways the download() can blow up and not be handled.

[sbryan]

have you tried installing sauce-connect-launcher and wct-sauce directly and not via web-component-tester?

Yes. sauce-connect-launcher does not complain... installs fine, but does not attempt to build or fetch the native sauce-connect module. Direct install of just wct-sauce fails as I described above, same as when being installed as a dependency of web-component-tester.

When I can, I'll try to isolate nvm from the equation to see if it's the related.

[sbryan]

It would be even nicer if web-component-tester did not require wct-sauce :/ Seems odd that it is required to me, especially since it is a plug-in.

That's a feature request for web-component-tester though, not a bug of wct-sauce...

[jmorille]

I completely agree, and this plugins cause me a lot of trouble in order to validate for using it in my company.
In fact, this plugin don't use the npm download mechanism, and in consequence it don't use my Npm registry of my company ( that cache and proxify the internet one, I using Nexus for Npm Registry).

In assurance company, I could select a tool only if it guarantee the Business Continuity Plan (BCP), and the non installation in case of no internet connectivity is a blocking point for my company use case.

And this break this rules, because the install of dev environment of the project fail for an optional plugins.

In any case, the good way to solve this problem is not using the http.request and to create the npm dependencies this the various ArchiveName that wtc-saucer have to download.
Thank to that, it use the npm download mechanism and not a direct http request. That will permit to secure, cache and accelerate the installation procedure.

[jmorille]

I have open an issue on
https://github.com/Polymer/web-component-tester/issues/140

[jmalonzo]

This seems to be working now/atm? Can anyone please confirm?

[jmorille]

I found the phamtomjs that do download with proxy management
https://github.com/Medium/phantomjs/blob/master/install.js

function getRequestOptions(conf) {
  var strictSSL = conf.get('strict-ssl')
  if (process.version == 'v0.10.34') {
    console.log('Node v0.10.34 detected, turning off strict ssl due to https://github.com/joyent/node/issues/8894')
    strictSSL = false
  }


  var options = {
    uri: downloadUrl,
    encoding: null, // Get response as a buffer
    followRedirect: true, // The default download path redirects to a CDN URL.
    headers: {},
    strictSSL: strictSSL
  }

  var proxyUrl = conf.get('https-proxy') || conf.get('http-proxy') || conf.get('proxy')
  if (proxyUrl) {

    // Print using proxy
    var proxy = url.parse(proxyUrl)
    if (proxy.auth) {
      // Mask password
      proxy.auth = proxy.auth.replace(/:.*$/, ':******')
    }
    console.log('Using proxy ' + url.format(proxy))

    // Enable proxy
    options.proxy = proxyUrl

function requestBinary(requestOptions, filePath) {
  var deferred = kew.defer()

  var count = 0
  var notifiedCount = 0
  var writePath = filePath + '-download-' + Date.now()

  console.log('Receiving...')
  var bar = null
  requestProgress(request(requestOptions, function (error, response, body) {
    console.log('');
    if (!error && response.statusCode === 200) {

[davidmaxwaterman]

I'm seeing this today too, so it seems like it is still current.

[nevir]

It would be even nicer if web-component-tester did not require wct-sauce :/ Seems odd that it is required to me, especially since it is a plug-in.

Completely agree! I left it as a hard dependency so that it would not break compatibility with existing users who are expecting that. I hope to drop it as a dependency in 3.0 of WCT (or at least mark it as an optionalDependency)

[nevir]

In any case, the good way to solve this problem is not using the http.request and to create the npm dependencies this the various ArchiveName that wtc-saucer have to download.
Thank to that, it use the npm download mechanism and not a direct http request. That will permit to secure, cache and accelerate the installation procedure.

This would be a good feature to request over at https://github.com/bermi/sauce-connect-launcher

[davidmaxwaterman]

I'm hitting this again today. I ended up installing on the internet and copying it via usb flash drive.

It's a bit difficult to see if there's a workaround in this thread...could someone tell me if there is a way to make this work?

[aearly]

Workaround: bermi/sauce-connect-launcher#26 (comment)

[aearly]

Also, later versions of sauce-connect-launcher don't try to download sauce-connect on npm install anymore

[davidmaxwaterman]

Workaround: bermi/sauce-connect-launcher#26 (comment)

FWIW, in case it helps anyone else, to fix it for me, I had to unzip sauce-connect-launcher (master.zip) to :

node_modules/web-component-tester/node_modules/wct-sauce/node_modules/sauce-connect-launcher/sc

[davidmaxwaterman]

Is there any progress on this issue? It's a real pain to have to workaround this problem.

[ek]

+1

[sbley]

+1 please fix

[yashdoshi89]

+1 please fix it

[kwge]

+1 still have problem. please fix it

[PhilKearney]

Any update on this yet, The work around does not seem to work for me.

[zardoz]

+1 You seriously need to fix this, please

[dfreedm]

I've reviewed the sauce-connect-launcher plugin for what options I could use with wct-sauce, but there are few.

There are two workarounds right now for restricted environments:

1. Manual Download of sauce connect, and placement in the right spot (node_modules/wct-sauce/node_modules/sauce-connect-launcher/sc): Download fails in conjunction with corporate proxy bermi/sauce-connect-launcher#26 (comment)
2. Manually download sauce connect, and start the tunnel before launching WCT, and expose the tunnel identifier as environment variable SAUCE_TUNNEL_ID.
   This will make WCT use the available tunnel instead of trying to spin up a new one. You are then responsible for spinning up and tearing down the tunnel. (https://github.com/Polymer/wct-sauce/blob/b62e1d28a268e98c4e56a48ef843cb96b3f0f2c3/lib/plugin.js#L109)

I'm going to request that sauce-connect-launcher be able to take an arbitrary system path to the sauce connect executable, so that the same mechanisms for starting and stopping the tunnel can be used, but requires less arcane file placement than the first option.

[jonniespratley]

What no love for proxys?

npm info postinstall wct-sauce@1.8.4

> wct-sauce@1.8.4 postinstall /Users/predixgo/2016GithubEnterprise/predix-dev-components/px-page/node_modules/wct-sauce
> node scripts/postinstall.js

Prefetching the Sauce Connect binary.
Missing Sauce Connect local proxy, downloading dependency
This will only happen once.
events.js:154
      throw er; // Unhandled 'error' event
      ^

Error: getaddrinfo ENOTFOUND saucelabs.com saucelabs.com:443
    at errnoException (dns.js:26:10)
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:77:26)
npm info wct-sauce@1.8.4 Failed to exec postinstall script
npm WARN install:wct-sauce Couldn't install optional dependency: wct-sauce@1.8.4 postinstall: `node scripts/postinstall.js`
npm WARN install:wct-sauce Exit status 1

[bermi]

sauce-connect-launcher@0.15.1 has added support for http_proxy and https_proxy environment variables and could potentially solve this issue.

[zardoz]

awesome! Will give it a go.

[benmak11]

@bermi I have tried all the suggestions mentioned within this thread (even downloaded the sauce-connect-launcher manually to configure with the wct-sauce) but I still get no 🎲... Is there another workaround?

[Vinnl]

@azakus Are you still on this? According to this PR, a simple version bump of sauce-connect-launcher should solve it - but it cannot get merged because the author did not sign a CLA. Since it's just a version bump with no other way to do it, I'm guessing it's safe for someone who has signed it to make the same change without infringing on any one's copyright :)

[btmurrell]

hitting this too. come on guys, #17 seems to be a simple fix. thanks for the project! please accept the PR to help us move on.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[jmorille]

Dead project