Static build is critically broken (development blocker)

[thevoiceofzeke]

As of #335, running uw-frame static locally is broken to the point that it is a major blocker for development.

Build fails

When running npm run static:dev, the build will fail ~60% of the time and log the following error:

Error: Command failed: /bin/sh -c node_modules/.bin/lessc -x uw-frame-components/css/themes/uw-river-falls.less > uw-frame-static/target/css/themes/uw-river-falls.css
FileError: resource 'https://cdn.rawgit.com/twbs/bootstrap/v3.3.6/less/mixins/opacity.less' gave this Error:
  Error: getaddrinfo ENOTFOUND cdn.rawgit.com cdn.rawgit.com:443
 in https://cdn.rawgit.com/twbs/bootstrap/v3.3.6/less/mixins.less on line 6, column 1:
5 @import "mixins/hide-text.less";
6 @import "mixins/opacity.less";
7 @import "mixins/image.less";


    at ChildProcess.exithandler (child_process.js:213:12)
    at emitTwo (events.js:87:13)
    at ChildProcess.emit (events.js:172:7)
    at maybeClose (internal/child_process.js:821:16)
    at Socket.<anonymous> (internal/child_process.js:319:11)
    at emitOne (events.js:77:13)
    at Socket.emit (events.js:169:7)
    at Pipe._onclose (net.js:469:12)

npm ERR! Darwin 16.0.0
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "run" "static:dev"
npm ERR! node v5.4.0
npm ERR! npm  v3.3.12
npm ERR! code ELIFECYCLE
npm ERR! uw-frame@2.9.0 static:dev: `npm run build-static -s && concurrently -k "npm run static -s" "npm run static:watch -s"`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the uw-frame@2.9.0 static:dev script 'npm run build-static -s && concurrently -k "npm run static -s" "npm run static:watch -s"'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the uw-frame package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     npm run build-static -s && concurrently -k "npm run static -s" "npm run static:watch -s"
npm ERR! You can get their info via:
npm ERR!     npm owner ls uw-frame
npm ERR! There is likely additional logging output above.

When it fails, it always fails on an @import statement, but not always the same one.

Build succeeds, but watch fails

In the unlikely event that the app builds, it will crash 90% of the time any changes in any files are saved, resulting in the following error log (which varies in the same way as the build-failure log):

[1] Error: Command failed: /bin/sh -c node_modules/.bin/lessc -x uw-frame-components/css/themes/uw-superior.less > uw-frame-static/target/css/themes/uw-superior.css
[1] FileError: resource 'https://cdn.rawgit.com/twbs/bootstrap/v3.3.6/less/mixins/text-overflow.less' gave this Error:
[1]   Error: getaddrinfo ENOTFOUND cdn.rawgit.com cdn.rawgit.com:443
[1]  in https://cdn.rawgit.com/twbs/bootstrap/v3.3.6/less/mixins.less on line 16, column 1:
[1] 15 @import "mixins/text-emphasis.less";
[1] 16 @import "mixins/text-overflow.less";
[1] 17 @import "mixins/vendor-prefixes.less";
[1] 
[1] 
[1]     at ChildProcess.exithandler (child_process.js:213:12)
[1]     at emitTwo (events.js:87:13)
[1]     at ChildProcess.emit (events.js:172:7)
[1]     at maybeClose (internal/child_process.js:821:16)
[1]     at Socket.<anonymous> (internal/child_process.js:319:11)
[1]     at emitOne (events.js:77:13)
[1]     at Socket.emit (events.js:169:7)
[1]     at Pipe._onclose (net.js:469:12)

These problems combine for an untenable development environment. I think we should revert the changes made in #335 unless @jsamsa can come up with an working static build very quickly.

[jsamsa]

It appears that cdn.rawgit.com may be throttling our use of their resources. This throttling came up yesterday in a code review for star app. I'm not sure reverting #335 will entirely solve the problem as the change to CDN resources is likely the source of the issue. See commit:

00f2d09

[thevoiceofzeke]

For what it's worth, that commit was merged 7 days before the build changes, and I experienced no build issues during those 7 days.

[jsamsa]

I understand and I'm sorry this is blocking you, I'm sure it's frustrating. Based on the errors you've shared above, the lessc build is failing because a reference resource is responding with 404 NOT FOUND intermittently. This would align with throttling as they describe in their FAQ https://github.com/rgrove/rawgit/wiki/Frequently-Asked-Questions#can-i-use-a-rawgitcom-development-url-on-a-production-website-or-in-public-example-code .

I could be wrong, but I think you'll run into these failures if you revert the changes to the build process.

[paulerickson]

We're not getting throttled downloading https://cdn.rawgit.com/twbs/bootstrap/v3.3.6/less/mixins.less, but less is trying to resolve a relative path within that file. I did curl the path 1000 times just to make sure it's not throttling us.

I think what is happening is that @import "mixins/opacity.less" defined within https://cdn.rawgit.com/twbs/bootstrap/v3.3.6/less/mixins.less is resolving to just https://cdn.rawgit.com/opacity.less, rather than https://cdn.rawgit.com/twbs/bootstrap/v3.3.6/less/mixins/opacity.less, which would actually resolve.

[paulerickson]

But if so, then I think it should fail consistently and since my change.

[paulerickson]

@thevoiceofzeke it looks like you have a very old version of node... maybe this is an old (fixed) bug about too many open file handles causing DNS lookups to fail in node, in which case it only would have started when Jason moved this build step from a shell script to a node script.
Could you try upgrading node and see if it resolves?

nodejs/node#7931

[jsamsa]

I'm running v6.8.1 and getting this issue. It does appear to have something to do with DNS resolution, not getting a 404 NOT FOUND. Digging deeper now.

[paulerickson]

I've got v6.9.0 and I was able to run npm run build-static 20 times without issue

[jsamsa]

$ node --version
v6.8.1
$ npm --version
3.10.8

The npm version is probably irrelevant because the build is shelling out to run less which appears to be a node program.

[thevoiceofzeke]

@paulerickson I can run npm run build-static without issue as well, but that doesn't help me run the app :/. I have also updated my npm and node versions and still encounter the same errors.

[jsamsa]

I managed to cause a failure running less from bash as well, but it took 937 attempts at compiling all *.less files before the error showed its face again. I don't think upgrading to node v6.9.0 would solve the problem without "less" changing its code to force IP v4 as suggested in the node issue @paulerickson linked. In my opinion the appropriate solution is to go back to downloading the resources required for the build, at the very least this should be done with the files required by the less compilation step.

Can we all agree to this approach? I will attempt to patch this quickly.