Proxy randomly dies

[bobbytables]

What were you trying to do?

Proxy via --docker-run

What did you expect to happen?

It to last longer than it did

What happened instead?

It died randomly

Automatically included information

Command line: ['/usr/local/bin/telepresence', '--swap-deployment', 'dev-workspacer-dummy-sre-123', '--expose', '80', '--docker-run', '-v', '/Users/bobbytables/Code/namely/platform/../workspacer-dummy/www:/usr/share/nginx/html', 'platform_workspace_workspacer-dummy']
Version: 0.71
Python version: 3.6.3 (default, Oct 4 2017, 06:09:15) [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.37)]
kubectl version: Client Version: v1.8.1
oc version: (error: [Errno 2] No such file or directory: 'oc': 'oc')
OS: Darwin Bobby-Tables 16.7.0 Darwin Kernel Version 16.7.0: Thu Jun 15 17:36:27 PDT 2017; root:xnu-3789.70.16~2/RELEASE_X86_64 x86_64
Traceback:

Traceback (most recent call last):
  File "/usr/local/bin/telepresence", line 259, in call_f
    return f(*args, **kwargs)
  File "/usr/local/bin/telepresence", line 2395, in go
    ssh,
  File "/usr/local/bin/telepresence", line 2054, in run_docker_command
    wait_for_exit(runner, p, subprocesses)
  File "/usr/local/bin/telepresence", line 1604, in wait_for_exit
    dead_process = processes.any_dead()
  File "/usr/local/bin/telepresence", line 960, in any_dead
    self.killall()
  File "/usr/local/bin/telepresence", line 946, in killall
    killer()
  File "/usr/local/bin/telepresence", line 1937, in kill
    runner.check_call(sudo + ["docker", "stop", "--time=1", name])
  File "/usr/local/bin/telepresence", line 386, in check_call
    raise CalledProcessError(retcode, args)
subprocess.CalledProcessError: Command '(['docker', 'stop', '--time=1', 'telepresence-1511217478-334368-69286'],)' returned non-zero exit status 1.

Logs:

629) to None: 35 bytes
  14.7 19 | 2017-11-20T22:30:30+0000 [stdout#info] Set DNS suffix we filter out to: []
  14.7 19 | 2017-11-20T22:30:30+0000 [stdout#info] Result for b'hellotelepresence' is ['127.0.0.1']
  15.7 39 | [INFO  tini (1)] Main child exited normally (with status '100')
  15.9 TL | [39] exit 100.
  15.9 TL | [40] Capturing: (['docker', 'run', '--help'],)...
  16.0 TL | [40] captured.
 499.1 38 |  485.6 TL | A subprocess (['ssh', '-N', '-oServerAliveInterval=1', '-oServerAliveCountMax=10', '-F', '/dev/null', '-q', '-oStrictHostKeyChecking=no', '-oUserKnownHostsFile=/dev/null', '-p', '63935', 'telepresence@198.18.0.254', '-R', '*:80:127.0.0.1:80']) died with code 255, killed all processes...
 499.1 38 | Proxy to Kubernetes exited. This is typically due to a lost connection.
 499.1 38 | [INFO  tini (1)] Main child exited normally (with status '3')
 499.7 TL | [41] Running: (['docker', 'stop', '--time=1', 'telepresence-1511217478-334368-69286'],)...
 499.8 TL | [41] exit 1.

[ark3]

See also https://gist.github.com/TD-4242/4fd4a7f567ff16ec95cae3e468fcc158 from @TD-4242. In this case, it may be an ELB or similar timing out after five minutes of no output; this can be resolved by having the proxy emit some output periodically. Another possibility is that the kubelet serving the kubectl logs is timing out at 5 minutes (unconditionally, regardless of whether there's output? See chain starting at kubernetes/kubernetes#4844). In that case, Telepresence needs to be smarter about what it does when subprocesses die.

[donnyyung]

I believe this is no longer an issue in Telepresence 2. Here are the docs on how to install Telepresence (https://www.telepresence.io/docs/latest/install/), please re-open if you still see this issue in our latest version!