Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed builds when running in CI Environments #4870

Closed
troyschneringer opened this issue Aug 9, 2018 · 11 comments

Comments

@troyschneringer
Copy link

commented Aug 9, 2018

I'm not sure if this is the right place for this or not, but I am curious if anyone else is having any issues running their build from within circleci?

I have tried using both yarn and npm as well as versions 1.1.4 and 2.0.0-next.3e165448 all of which produce the same error. In all cases I am just running the build target

Is there a way to get more verbose logging on the build so that I can troubleshoot this more?

The error I am seeing is

events.js:183
      throw er; // Unhandled 'error' event
      ^

Error: This socket has been ended by the other party
    at Socket.writeAfterFIN [as write] (net.js:364:12)
    at PoolWorker.writeJson (/home/circleci/project/node_modules/thread-loader/dist/WorkerPool.js:89:22)
    at PoolWorker.run (/home/circleci/project/node_modules/thread-loader/dist/WorkerPool.js:69:12)
    at WorkerPool.distributeJob (/home/circleci/project/node_modules/thread-loader/dist/WorkerPool.js:326:20)
    at /home/circleci/project/node_modules/thread-loader/node_modules/async/queue.js:10:5
    at Object.process (/home/circleci/project/node_modules/thread-loader/node_modules/async/internal/queue.js:175:17)
    at /home/circleci/project/node_modules/thread-loader/node_modules/async/internal/queue.js:115:15
    at /home/circleci/project/node_modules/thread-loader/node_modules/async/internal/onlyOnce.js:12:16
    at callback (/home/circleci/project/node_modules/thread-loader/dist/WorkerPool.js:163:21)
    at /home/circleci/project/node_modules/thread-loader/dist/WorkerPool.js:190:15
    at /home/circleci/project/node_modules/thread-loader/node_modules/async/internal/map.js:32:9
    at /home/circleci/project/node_modules/thread-loader/node_modules/async/internal/once.js:12:16
    at replenish (/home/circleci/project/node_modules/thread-loader/node_modules/async/internal/eachOfLimit.js:61:25)
    at iterateeCallback (/home/circleci/project/node_modules/thread-loader/node_modules/async/internal/eachOfLimit.js:50:17)
    at /home/circleci/project/node_modules/thread-loader/node_modules/async/internal/onlyOnce.js:12:16
    at /home/circleci/project/node_modules/thread-loader/node_modules/async/internal/map.js:29:13
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
{ Error: EPIPE: broken pipe, write errno: -32, code: 'EPIPE', syscall: 'write' }
{ Error: EPIPE: broken pipe, write errno: -32, code: 'EPIPE', syscall: 'write' }
ERROR: "build-js" exited with 1.
{ Error: EPIPE: broken pipe, write errno: -32, code: 'EPIPE', syscall: 'write' }
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
{ Error: EPIPE: broken pipe, write errno: -32, code: 'EPIPE', syscall: 'write' }
Exited with code 1
@alex-pex

This comment has been minimized.

Copy link

commented Aug 13, 2018

Same problem, on TravisCI

> react-scripts build
Creating an optimized production build...
events.js:183
      throw er; // Unhandled 'error' event
      ^
Error: This socket has been ended by the other party
    at Socket.writeAfterFIN [as write] (net.js:364:12)
    at PoolWorker.writeJson (/home/travis/build/stadline/project/node_modules/thread-loader/dist/WorkerPool.js:85:22)
    at PoolWorker.run (/home/travis/build/stadline/project/node_modules/thread-loader/dist/WorkerPool.js:65:12)
    at WorkerPool.distributeJob (/home/travis/build/stadline/project/node_modules/thread-loader/dist/WorkerPool.js:329:20)
    at /home/travis/build/stadline/project/node_modules/async/queue.js:10:5
    at Object.process (/home/travis/build/stadline/project/node_modules/async/internal/queue.js:175:17)
    at /home/travis/build/stadline/project/node_modules/async/internal/queue.js:115:15
    at /home/travis/build/stadline/project/node_modules/async/internal/onlyOnce.js:12:16
    at callback (/home/travis/build/stadline/project/node_modules/thread-loader/dist/WorkerPool.js:159:21)
    at /home/travis/build/stadline/project/node_modules/thread-loader/dist/WorkerPool.js:186:15
    at /home/travis/build/stadline/project/node_modules/async/internal/map.js:32:9
    at /home/travis/build/stadline/project/node_modules/async/internal/once.js:12:16
    at replenish (/home/travis/build/stadline/project/node_modules/async/internal/eachOfLimit.js:61:25)
    at iterateeCallback (/home/travis/build/stadline/project/node_modules/async/internal/eachOfLimit.js:50:17)
    at /home/travis/build/stadline/project/node_modules/async/internal/onlyOnce.js:12:16
    at /home/travis/build/stadline/project/node_modules/async/internal/map.js:29:13
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! @app/core@0.1.0 build: `react-scripts build`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the @app/core@0.1.0 build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR!     /home/travis/.npm/_logs/2018-08-13T14_47_08_752Z-debug.log
{ Error: EPIPE: broken pipe, write errno: -32, code: 'EPIPE', syscall: 'write' }
@alex-pex

This comment has been minimized.

Copy link

commented Aug 13, 2018

I removed thead-loader from webpack.config.dev.js (for src and node_modules) and the tests pass!

Is it possible not to use this loader when CI=true ?

Related topics:
#4562
#4582

@troyschneringer

This comment has been minimized.

Copy link
Author

commented Aug 13, 2018

I can confirm @alex-pex findings.

I ejected my app, removed the thread-loader from webpack.config.prod.js and now the build succeeds on CircleCI.

Is there a way to optionally configure use of this loader so that I don't have to eject?

@troyschneringer troyschneringer changed the title Failed builds when running in CircleCI Failed builds when running in CI Environments Aug 13, 2018

@Timer Timer added the issue: bug label Aug 13, 2018

@Timer Timer added this to the 2.0.0 milestone Aug 13, 2018

@troyschneringer

This comment has been minimized.

Copy link
Author

commented Aug 13, 2018

Working with CircleCI support we found this issue:

from thread-loader

options: {
      // the number of spawned workers, defaults to number of cpus
      workers: 2,

from CircleCI

The issue with this on Dockerized services such as CircleCI is that the container thinks it has access to all of the hosts cores, although its being limited down to 4 by another process.

Perhaps this value should get set when CI:true to something lower, or expose an environment variable that would allow overriding that value.

@alex-pex

This comment has been minimized.

Copy link

commented Aug 14, 2018

From TravisCI https://docs.travis-ci.com/user/common-build-problems/#my-build-script-is-killed-without-any-error

Sometimes, you’ll see a build script being causing an error [...]. This is usually caused by the script or one of the programs it runs exhausting the memory available in the build sandbox, which is currently 3GB. Plus, there are two cores available, bursted.

A suggested fix travis-ci/travis-ci#4696 (comment)

In Python, one way to obtain the number of cores available for a Travis job, other than hard-coding '2', is to use psutil's Process.cpu_affinity. On travis the following gives the correct answer

len(psutil.Process().cpu_affinity())
2
@sambegin

This comment has been minimized.

Copy link

commented Aug 15, 2018

thanks @alex-pex and @troyschneringer

I had slightly the same problem and fixed it by adding workers: 2 in the webpack.config.prod.js file. I'm on CircleCI and also ejected from CRA with new webpack configuration.

Creating an optimized production build...
events.js:165
      throw er; // Unhandled 'error' event
      ^

Error: write EPIPE
    at WriteWrap.afterWrite [as oncomplete] (net.js:844:14)
Emitted 'error' event at:
    at onwriteError (_stream_writable.js:431:12)
    at onwrite (_stream_writable.js:453:5)
    at _destroy (internal/streams/destroy.js:39:7)
    at Socket._destroy (net.js:548:3)
    at Socket.destroy (internal/streams/destroy.js:32:8)
    at WriteWrap.afterWrite [as oncomplete] (net.js:846:10)
npm { Error: EPIPE: broken pipe, write errno: -32, code: 'EPIPE', syscall: 'write' }
ERR! code ELIFECYCLE
npm ERR! errno 1
@troyschneringer

This comment has been minimized.

Copy link
Author

commented Aug 15, 2018

@sambegin based on feedback from Circle Support I used the following:

workers: process.env.CIRCLE_NODE_TOTAL

which is the effective number of CPUs available to your build. Minor change, but if you have more resources available it will use the max.

@kevinhughes27

This comment has been minimized.

Copy link

commented Aug 24, 2018

If anyone is running with create-react-app-rewired I added this config to fix my CI builds

 module.exports = {
   webpack: function(config, env) {
     if (process.env.CI) {
      const threadLoader = config.module.rules[3].oneOf[1].use[0];
      config.module.rules[3].oneOf[1].use[0] = {
        loader: threadLoader,
        options: {
          workers: process.env.CIRCLE_NODE_TOTAL
        }
      }
    }
     return config;
  }
 };

@Timer Timer modified the milestones: 2.0.x, 2.x Sep 26, 2018

@Timer Timer removed this from the 2.x milestone Nov 2, 2018

@jedmao

This comment has been minimized.

Copy link

commented Nov 7, 2018

Is this being reprioritized? I'm also having the same issues.

@troyschneringer

This comment has been minimized.

Copy link
Author

commented Nov 7, 2018

@jedmao checkout 2.0. I updated last week and it resolved this issue.

@bugzpodder

This comment has been minimized.

Copy link
Collaborator

commented Nov 7, 2018

threadLoader was removed in 2.0.x, please update if possible or file a new issue if you are still having difficulty.

@bugzpodder bugzpodder closed this Nov 7, 2018

@lock lock bot locked and limited conversation to collaborators Jan 9, 2019

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
7 participants
You can’t perform that action at this time.