Tunnel termination hanging when using Jenkins with ECS slaves

[justinbarry]

See process output below:

[root@d72757b430d8 src]# ps -aux
Warning: bad syntax, perhaps a bogus '-'? See /usr/share/doc/procps-3.2.8/FAQ
USER        PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root          1  1.5  1.0 12961964 331964 ?     Ssl  09:38   0:36 java -Duser.home=/home/jenkins -cp /usr/share/jenkins/slave.jar hudson.remoting.jnlp.Main -headless -url http://10.4.1.5 f34bb1a77887fbb790ee7baff1abb643f9344c84d0e16fa0ae48dd67ea11ecfd jenkins-44700c09073e
root        426  0.0  0.0  12892  2768 ?        S    09:42   0:00 sh -c echo $$ > '/home/jenkins/workspace/d-outcomes_HO-test-ie-in-cd-6GICDBXOWCXKWVWSICMEHJLKF4MHPTPJDJGILMJQHGBPNGY3VGQQ/jsclient@tmp/durable-f84387dd/pid'; jsc=durable-59651029b78f11e9a37f7b961cbdfc1d; JE
root        428  0.0  0.0  12892  2784 ?        S    09:42   0:00 /bin/sh -xe /home/jenkins/workspace/d-outcomes_HO-test-ie-in-cd-6GICDBXOWCXKWVWSICMEHJLKF4MHPTPJDJGILMJQHGBPNGY3VGQQ/jsclient@tmp/durable-f84387dd/script.sh
root        429  0.0  0.1 1063648 49904 ?       Sl   09:42   0:00 npm                                           
root        439  0.0  0.0  12896  2800 ?        S    09:42   0:00 sh -c set -e && node scripts/local.runner.js -c conf/parallel_local.conf.js -e chrome
root        440  0.0  0.1 1204000 42656 ?       Sl   09:42   0:01 node scripts/local.runner.js -c conf/parallel_local.conf.js -e chrome
root        465  0.0  0.0      0     0 ?        Zs   09:42   0:00 [BrowserStackLoc] <defunct>
root        487  0.0  0.1 938152 44372 ?        Sl   09:43   0:00 /root/.browserstack/BrowserStackLocal --daemon stop --log-file /home/jenkins/workspace/d-outcomes_HO-test-ie-in-cd-6GICDBXOWCXKWVWSICMEHJLKF4MHPTPJDJGILMJQHGBPNGY3VGQQ/jsclient/local.log --key xoWQ57MU6qEkK
root        496  0.0  0.1 906392 42508 ?        Sl   09:43   0:02 /root/.browserstack/BrowserStackLocal --daemon stop --log-file /home/jenkins/workspace/d-outcomes_HO-test-ie-in-cd-6GICDBXOWCXKWVWSICMEHJLKF4MHPTPJDJGILMJQHGBPNGY3VGQQ/jsclient/local.log --key xoWQ57MU6qEkK
root        519  0.4  0.0  13028  3076 ?        Ss   10:17   0:00 /bin/bash
root        533  0.0  0.0  14904  2268 ?        R+   10:17   0:00 ps -aux
[root@d72757b430d8 src]# 

Elastic Container Service](https://aws.amazon.com/ecs/) runs Docker containers leading me to think could be related to #25.

By default Jenkins uses set -xe per the docs . Is this enough to satisfy the intent of the workaround in #25?

Please let me know what further info I can provide to assist in debugging.

[xscheiner]

Also experiencing this. We run jenkins in a container with two slave containers. We recently built a local jenkins running on an osx machine with one slave container from the same dockerfile to try and reproduce the problem. We can't and we cannot see what is different. Manually killing the process works and the builds complete.

[SlavikCA]

We are having same issue.
More details:
here my config file:

[sfursov@qaauto01 webdriverio-browserstack2]$ cat conf/local.conf.js 
var browserstack = require('browserstack-local');

exports.config = {
  user: process.env.BROWSERSTACK_USERNAME || 'slavikfursov1',
  key: process.env.BROWSERSTACK_ACCESS_KEY || '??????',

  updateJob: true,
  specs: [
    './tests/specs/local_test.js'
  ],
  exclude: [],

  capabilities: [{
    browser: 'chrome',
    name: 'local_test',
    build: 'webdriver-browserstack',
    'browserstack.local': true,
    'browserstack.debug': true
  }],

  logLevel: 'debug',
  coloredLogs: true,
  screenshotPath: './errorShots/',
  baseUrl: '',
  waitforTimeout: 10000,
  connectionRetryTimeout: 90000,
  connectionRetryCount: 3,
  host: 'hub.browserstack.com',

  before: function () {
    var chai = require('chai');
    global.expect = chai.expect;
    chai.Should();
  },
  framework: 'mocha',
  mochaOpts: {
    ui: 'bdd',
    timeout: 60000
  },

  // Code to start browserstack local before start of test
  onPrepare: function (config, capabilities) {
    console.log("Connecting local");
    return new Promise(function (resolve, reject) {
      exports.bs_local = new browserstack.Local();
      exports.bs_local.start({ 'key': exports.config.key, 'force-proxy':'', 'local-proxy-host':'****', 'local-proxy-port':'8080' }, function (error) {
        if (error) return reject(error);

        console.log('Connected. Now testing...');
        resolve();
      });
    });
  },

  // Code to stop browserstack local after end of test
  onComplete: function (exitCode, config, capabilities, results) {
    exports.bs_local.stop();
  },
}

Tests runs fine 1st time,
But on 2nd run, I'm getting:

/home/sfursov/webdriverio-browserstack2/node_modules/@wdio/cli/build/run.js:94
    throw e;
    ^
LocalError: Either another browserstack local client is running on your machine or some server is listening on port 45691
    at /home/sfursov/webdriverio-browserstack2/node_modules/browserstack-local/lib/Local.js:58:20
    at ChildProcess.exithandler (child_process.js:285:7)
    at ChildProcess.emit (events.js:189:13)
    at ChildProcess.EventEmitter.emit (domain.js:441:20)
    at maybeClose (internal/child_process.js:970:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:259:5)

I found it caused by hanging processes:

[sfursov@qaauto01 webdriverio-browserstack2]$ sudo ps aux | grep rowser
sfursov  26407  3.3  0.2 1215364 57372 ?       Ssl  16:56   0:00 /home/sfursov/.browserstack/BrowserStackLocal --daemon start --log-file /home/sfursov/webdriverio-browserstack2/local.log --key **** --force-proxy  --local-proxy-host **** --local-proxy-port 8080 -daemonInstance
sfursov  26428  3.5  0.2 1214620 57160 ?       Sl   16:56   0:00 /home/sfursov/.browserstack/BrowserStackLocal --log-file /home/sfursov/webdriverio-browserstack2/local.log --key **** --force-proxy  --local-proxy-host **** --local-proxy-port 8080 -daemonInstance --local-identifier browserstack-fork-0
sfursov  26433  4.9  0.2 1214724 57924 ?       Sl   16:56   0:00 /home/sfursov/.browserstack/BrowserStackLocal --log-file /home/sfursov/webdriverio-browserstack2/local.log --key **** --force-proxy  --local-proxy-host **** --local-proxy-port 8080 -daemonInstance --local-identifier browserstack-fork-1
sfursov  26438  4.7  0.1 1214616 55256 ?       Sl   16:56   0:00 /home/sfursov/.browserstack/BrowserStackLocal --log-file /home/sfursov/webdriverio-browserstack2/local.log --key **** --force-proxy  --local-proxy-host **** --local-proxy-port 8080 -daemonInstance --local-identifier browserstack-fork-2
sfursov  26444  3.8  0.1 1214488 55392 ?       Sl   16:56   0:00 /home/sfursov/.browserstack/BrowserStackLocal --log-file /home/sfursov/webdriverio-browserstack2/local.log --key **** --force-proxy  --local-proxy-host **** --local-proxy-port 8080 -daemonInstance --local-identifier browserstack-fork-3
sfursov  26449  3.5  0.1 1214624 55628 ?       Sl   16:56   0:00 /home/sfursov/.browserstack/BrowserStackLocal --log-file /home/sfursov/webdriverio-browserstack2/local.log --key **** --force-proxy  --local-proxy-host **** --local-proxy-port 8080 -daemonInstance --local-identifier browserstack-fork-4

If I kill them - tests will run again