New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Running ng-killall in plugin integration tests tearDown #138

Closed
wants to merge 7 commits into
base: master
from

Conversation

Projects
None yet
2 participants
@peiyuwang
Contributor

peiyuwang commented May 6, 2016

Abnormal build process termination in #133 was due to:

Now after running some number pants commands, the ng processes left on travis reaches a limit
and crashed later java process.

We saw this behavior at Twitter as well. Solution there is to turn on ng only for compile, plus
running ng-killall at the very beginning of remote execution (consists serveral pants runs).

Here added is to run ng-killall in tearDown as a cleanup, at a cost of increasing the overall
test time, but that hopefully we can address via #137

@wisechengyi

This comment has been minimized.

Show comment
Hide comment
@wisechengyi

wisechengyi May 6, 2016

Contributor

ah is it this basically from https://travis-ci.org/peiyuwang/intellij-pants-plugin/jobs/128213131?

1) testScalaWithJavaSources(com.twitter.intellij.pants.integration.OSSPantsScalaExamplesIntegrationTest)
                     junit.framework.AssertionFailedError: Import failed: Failed to update the project!
                     pants export examples/src/scala/org/pantsbuild/example/scala_with_java_sources/:: --export-output-file=/tmp/unitTest_scalaWithJavaSources/pants_depmap_run.out
                     Exit code: 1

                     FAILURE

                     INFO] Detected git repository at /home/travis/build/peiyuwang/intellij-pants-plugin/.cache/pants on branch master
                     Waiting on pants process 8152 (python /home/travis/build/peiyuwang/intellij-pants-plugin/.cache/pants/src/python/pants/bin/pants_exe.py ng-killall) to complete
                     Exception caught: (<class 'lockfile.AlreadyLocked'>)
                       File "/home/travis/build/peiyuwang/intellij-pants-plugin/.cache/pants/src/python/pants/bin/pants_exe.py", line 50, in <module>
                         main()
                       File "/home/travis/build/peiyuwang/intellij-pants-plugin/.cache/pants/src/python/pants/bin/pants_exe.py", line 44, in main
                         PantsRunner(exiter).run()
                       File "/home/travis/build/peiyuwang/intellij-pants-plugin/.cache/pants/src/python/pants/bin/pants_runner.py", line 53, in run
                         options_bootstrapper=options_bootstrapper)
                       File "/home/travis/build/peiyuwang/intellij-pants-plugin/.cache/pants/src/python/pants/bin/pants_runner.py", line 43, in _run
                         return LocalPantsRunner(exiter, args, env, options_bootstrapper=options_bootstrapper).run()
                       File "/home/travis/build/peiyuwang/intellij-pants-plugin/.cache/pants/src/python/pants/bin/local_pants_runner.py", line 49, in run
                         self._maybe_profiled(self._run)
                       File "/home/travis/build/peiyuwang/intellij-pants-plugin/.cache/pants/src/python/pants/bin/local_pants_runner.py", line 46, in _maybe_profiled
                         runner()
                       File "/home/travis/build/peiyuwang/intellij-pants-plugin/.cache/pants/src/python/pants/bin/local_pants_runner.py", line 84, in _run
                         result = goal_runner.run()
                       File "/home/travis/build/peiyuwang/intellij-pants-plugin/.cache/pants/src/python/pants/bin/goal_runner.py", line 335, in run
                         result = self._execute_engine()
                       File "/home/travis/build/peiyuwang/intellij-pants-plugin/.cache/pants/src/python/pants/bin/goal_runner.py", line 324, in _execute_engine
                         result = engine.execute(self._context, self._goals)
                       File "/home/travis/build/peiyuwang/intellij-pants-plugin/.cache/pants/src/python/pants/engine/engine.py", line 26, in execute
                         self.attempt(context, goals)
                       File "/home/travis/build/peiyuwang/intellij-pants-plugin/.cache/pants/src/python/pants/engine/round_engine.py", line 237, in attempt
                         context.acquire_lock()
                       File "/home/travis/build/peiyuwang/intellij-pants-plugin/.cache/pants/src/python/pants/goal/context.py", line 234, in acquire_lock
                         self._lock.acquire()
                       File "/home/travis/build/peiyuwang/intellij-pants-plugin/.cache/pants/src/python/pants/process/pidlock.py", line 34, in acquire
                         return pidlockfile.PIDLockFile.acquire(self, timeout)
                       File "/home/travis/build/peiyuwang/intellij-pants-plugin/.cache/pants/build-support/pants_dev_deps.venv/local/lib/python2.7/site-packages/lockfile/pidlockfile.py", line 88, in acquire
                         self.path)

                     Exception message: /home/travis/build/peiyuwang/intellij-pants-plugin/.cache/pants/.pants.run is already locked


                        at junit.framework.Assert.fail(Assert.java:57)
                        at junit.framework.TestCase.fail(TestCase.java:227)
                        at com.intellij.openapi.externalSystem.test.ExternalSystemImportingTestCase.doImportProject(ExternalSystemImportingTestCase.java:461)
                        at com.intellij.openapi.externalSystem.test.ExternalSystemImportingTestCase.importProject(ExternalSystemImportingTestCase.java:422)
                        at com.twitter.intellij.pants.testFramework.PantsIntegrationTestCase.doImport(PantsIntegrationTestCase.java:294)
                        at com.twitter.intellij.pants.integration.OSSPantsScalaExamplesIntegrationTest.testScalaWithJavaSources(OSSPantsScalaExamplesIntegrationTest.java:46)
Contributor

wisechengyi commented May 6, 2016

ah is it this basically from https://travis-ci.org/peiyuwang/intellij-pants-plugin/jobs/128213131?

1) testScalaWithJavaSources(com.twitter.intellij.pants.integration.OSSPantsScalaExamplesIntegrationTest)
                     junit.framework.AssertionFailedError: Import failed: Failed to update the project!
                     pants export examples/src/scala/org/pantsbuild/example/scala_with_java_sources/:: --export-output-file=/tmp/unitTest_scalaWithJavaSources/pants_depmap_run.out
                     Exit code: 1

                     FAILURE

                     INFO] Detected git repository at /home/travis/build/peiyuwang/intellij-pants-plugin/.cache/pants on branch master
                     Waiting on pants process 8152 (python /home/travis/build/peiyuwang/intellij-pants-plugin/.cache/pants/src/python/pants/bin/pants_exe.py ng-killall) to complete
                     Exception caught: (<class 'lockfile.AlreadyLocked'>)
                       File "/home/travis/build/peiyuwang/intellij-pants-plugin/.cache/pants/src/python/pants/bin/pants_exe.py", line 50, in <module>
                         main()
                       File "/home/travis/build/peiyuwang/intellij-pants-plugin/.cache/pants/src/python/pants/bin/pants_exe.py", line 44, in main
                         PantsRunner(exiter).run()
                       File "/home/travis/build/peiyuwang/intellij-pants-plugin/.cache/pants/src/python/pants/bin/pants_runner.py", line 53, in run
                         options_bootstrapper=options_bootstrapper)
                       File "/home/travis/build/peiyuwang/intellij-pants-plugin/.cache/pants/src/python/pants/bin/pants_runner.py", line 43, in _run
                         return LocalPantsRunner(exiter, args, env, options_bootstrapper=options_bootstrapper).run()
                       File "/home/travis/build/peiyuwang/intellij-pants-plugin/.cache/pants/src/python/pants/bin/local_pants_runner.py", line 49, in run
                         self._maybe_profiled(self._run)
                       File "/home/travis/build/peiyuwang/intellij-pants-plugin/.cache/pants/src/python/pants/bin/local_pants_runner.py", line 46, in _maybe_profiled
                         runner()
                       File "/home/travis/build/peiyuwang/intellij-pants-plugin/.cache/pants/src/python/pants/bin/local_pants_runner.py", line 84, in _run
                         result = goal_runner.run()
                       File "/home/travis/build/peiyuwang/intellij-pants-plugin/.cache/pants/src/python/pants/bin/goal_runner.py", line 335, in run
                         result = self._execute_engine()
                       File "/home/travis/build/peiyuwang/intellij-pants-plugin/.cache/pants/src/python/pants/bin/goal_runner.py", line 324, in _execute_engine
                         result = engine.execute(self._context, self._goals)
                       File "/home/travis/build/peiyuwang/intellij-pants-plugin/.cache/pants/src/python/pants/engine/engine.py", line 26, in execute
                         self.attempt(context, goals)
                       File "/home/travis/build/peiyuwang/intellij-pants-plugin/.cache/pants/src/python/pants/engine/round_engine.py", line 237, in attempt
                         context.acquire_lock()
                       File "/home/travis/build/peiyuwang/intellij-pants-plugin/.cache/pants/src/python/pants/goal/context.py", line 234, in acquire_lock
                         self._lock.acquire()
                       File "/home/travis/build/peiyuwang/intellij-pants-plugin/.cache/pants/src/python/pants/process/pidlock.py", line 34, in acquire
                         return pidlockfile.PIDLockFile.acquire(self, timeout)
                       File "/home/travis/build/peiyuwang/intellij-pants-plugin/.cache/pants/build-support/pants_dev_deps.venv/local/lib/python2.7/site-packages/lockfile/pidlockfile.py", line 88, in acquire
                         self.path)

                     Exception message: /home/travis/build/peiyuwang/intellij-pants-plugin/.cache/pants/.pants.run is already locked


                        at junit.framework.Assert.fail(Assert.java:57)
                        at junit.framework.TestCase.fail(TestCase.java:227)
                        at com.intellij.openapi.externalSystem.test.ExternalSystemImportingTestCase.doImportProject(ExternalSystemImportingTestCase.java:461)
                        at com.intellij.openapi.externalSystem.test.ExternalSystemImportingTestCase.importProject(ExternalSystemImportingTestCase.java:422)
                        at com.twitter.intellij.pants.testFramework.PantsIntegrationTestCase.doImport(PantsIntegrationTestCase.java:294)
                        at com.twitter.intellij.pants.integration.OSSPantsScalaExamplesIntegrationTest.testScalaWithJavaSources(OSSPantsScalaExamplesIntegrationTest.java:46)
@peiyuwang

This comment has been minimized.

Show comment
Hide comment
@peiyuwang

peiyuwang May 6, 2016

Contributor

That was my first try before I realize the 'ng-killall' call needs to be blocked.

As fixed in the next commit:

bd68564

Contributor

peiyuwang commented May 6, 2016

That was my first try before I realize the 'ng-killall' call needs to be blocked.

As fixed in the next commit:

bd68564

@wisechengyi

This comment has been minimized.

Show comment
Hide comment
@wisechengyi

wisechengyi May 6, 2016

Contributor

Could you paste the original error observed about the nailgun issue?

Also I understand rglobs is not good practice, but why is it related to this? Is it because in general more tests are running so more pants commands are piling up for the nailgun crash?

Contributor

wisechengyi commented May 6, 2016

Could you paste the original error observed about the nailgun issue?

Also I understand rglobs is not good practice, but why is it related to this? Is it because in general more tests are running so more pants commands are piling up for the nailgun crash?

@peiyuwang

This comment has been minimized.

Show comment
Hide comment
@peiyuwang

peiyuwang May 6, 2016

Contributor

Added comment in #133

Contributor

peiyuwang commented May 6, 2016

Added comment in #133

@peiyuwang

This comment has been minimized.

Show comment
Hide comment
@peiyuwang
Contributor

peiyuwang commented May 8, 2016

@peiyuwang peiyuwang closed this May 8, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment