Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Runner's get_output(...) elides the beginning of captured output that's too long #981

Closed
Chaitanya-Veluvolu-Randstad opened this issue Apr 5, 2019 · 6 comments

Comments

2 participants
@Chaitanya-Veluvolu-Randstad
Copy link

commented Apr 5, 2019

What were you trying to do?

Trying to telepresence into kube cluster

What did you expect to happen?

I get a kube deployment and my laptop is added into the namespace and i can access kube objects in that namespace as if i was in the cluster

What happened instead?

i get an error message saying there is a bug in your code.

(please tell us - the traceback is automatically included, see below.
use https://gist.github.com to pass along full telepresence.log)

Automatically included information

Command line: ['/usr/local/bin/telepresence', '--method', 'vpn-tcp', '--namespace', 'svc-edtest-stable-test', '--expose', '9090']
Version: 0.98
Python version: 3.7.3 (default, Apr 4 2019, 17:04:14) [Clang 9.0.0 (clang-900.0.39.2)]
kubectl version: Client Version: v1.9.6 // Server Version: v1.8.15-1+9f66f5c998b397
oc version: (error: [Errno 2] No such file or directory: 'oc': 'oc')
OS: Darwin Chaitanya.bose.com 16.6.0 Darwin Kernel Version 16.6.0: Fri Apr 14 16:13:31 PDT 2017; root:xnu-3789.60.24~4/RELEASE_X86_64 x86_64

Traceback (most recent call last):
  File "/usr/local/bin/telepresence/telepresence/cli.py", line 130, in crash_reporting
    yield
  File "/usr/local/bin/telepresence/telepresence/main.py", line 77, in main
    runner, remote_info, env, socks_port, ssh, mount_dir, pod_info
  File "/usr/local/bin/telepresence/telepresence/outbound/setup.py", line 73, in launch
    runner_, remote_info, command, args.also_proxy, env, ssh
  File "/usr/local/bin/telepresence/telepresence/outbound/local.py", line 121, in launch_vpn
    connect_sshuttle(runner, remote_info, also_proxy, ssh)
  File "/usr/local/bin/telepresence/telepresence/outbound/vpn.py", line 256, in connect_sshuttle
    ] + get_proxy_cidrs(runner, remote_info, hosts_or_ips),
  File "/usr/local/bin/telepresence/telepresence/outbound/vpn.py", line 82, in get_proxy_cidrs
    result.update(context_cache.lookup("podCIDRs", lambda: podCIDRs(runner)))
  File "/usr/local/bin/telepresence/telepresence/runner/cache.py", line 105, in lookup
    value = function()
  File "/usr/local/bin/telepresence/telepresence/outbound/vpn.py", line 82, in <lambda>
    result.update(context_cache.lookup("podCIDRs", lambda: podCIDRs(runner)))
  File "/usr/local/bin/telepresence/telepresence/outbound/vpn.py", line 156, in podCIDRs
    runner.get_output(runner.kubectl("get", "nodes", "-o", "json"))
  File "/Users/chaitanyaveluvolu/.pyenv/versions/3.7.3/lib/python3.7/json/__init__.py", line 348, in loads
    return _default_decoder.decode(s)
  File "/Users/chaitanyaveluvolu/.pyenv/versions/3.7.3/lib/python3.7/json/decoder.py", line 340, in decode
    raise JSONDecodeError("Extra data", s, end)
json.decoder.JSONDecodeError: Extra data: line 1 column 21 (char 20)

Logs:

 Listening...
  20.5  30 | 2019-04-05T14:26:20+0000 [-] Loading ./forwarder.py...
  20.5  30 | 2019-04-05T14:26:22+0000 [-] /etc/resolv.conf changed, reparsing
  20.5  30 | 2019-04-05T14:26:22+0000 [-] Resolver added ('100.64.0.10', 53) to server list
  20.5  30 | 2019-04-05T14:26:22+0000 [-] SOCKSv5Factory starting on 9050
  20.5  30 | 2019-04-05T14:26:22+0000 [socks.SOCKSv5Factory#info] Starting factory <socks.SOCKSv5Factory object at 0x7fed4ae65240>
  20.5  30 | 2019-04-05T14:26:22+0000 [-] DNSDatagramProtocol starting on 9053
  20.5  30 | 2019-04-05T14:26:22+0000 [-] Starting protocol <twisted.names.dns.DNSDatagramProtocol object at 0x7fed48fddd30>
  20.5  30 | 2019-04-05T14:26:22+0000 [-] Loaded.
  20.5  30 | 2019-04-05T14:26:22+0000 [twisted.scripts._twistd_unix.UnixAppLogger#info] twistd 18.9.0 (/usr/bin/python3.6 3.6.5) starting up.
  20.5  30 | 2019-04-05T14:26:22+0000 [twisted.scripts._twistd_unix.UnixAppLogger#info] reactor class: twisted.internet.epollreactor.EPollReactor.

@ark3

This comment has been minimized.

Copy link
Contributor

commented Apr 5, 2019

Sorry about the crash. Can you please pass along the full telepresence.log as a Gist? Thank you.

@Chaitanya-Veluvolu-Randstad

This comment has been minimized.

@ark3

This comment has been minimized.

Copy link
Contributor

commented Apr 5, 2019

Is that the right file? It says it's from Telepresence 0.92 from September. And it looks like a successful session that ends cleanly.

@Chaitanya-Veluvolu-Randstad

This comment has been minimized.

Copy link
Author

commented Apr 5, 2019

@ark3

This comment has been minimized.

Copy link
Contributor

commented Apr 5, 2019

Thanks. What does this return?

/usr/local/bin/kubectl --context live-aws-useast1-chaitanya_veluvolu@bose.com --namespace svc-edtest-stable-test get nodes -o json

It seems to be invalid JSON for some reason.

@ark3

This comment has been minimized.

Copy link
Contributor

commented Apr 12, 2019

Thanks to lots of help and patience from @Chaitanya-Veluvolu-Randstad, I discovered the source of this error:

        self.capture = deque(
            maxlen=max_capture
        )  # type: typing.MutableSequence[str]

where max_capture happens to be too small to hold the entire result. As such, the top of the output was being thrown away, causing the JSON decoder to blow up. I'll have this fixed for the next release. Sorry for the dumb bug!

@ark3 ark3 changed the title Unable to create a connection(can't telepresence into cluster) and errors out saying there is a bug Runner's get_output(...) elides the beginning of captured output that's too long Apr 12, 2019

@ark3 ark3 added the bug label Apr 12, 2019

@ark3 ark3 added this to To do in Tel Tracker via automation Apr 12, 2019

@ark3 ark3 moved this from To do to In progress in Tel Tracker Apr 12, 2019

@ark3 ark3 closed this in 74ab1c4 Apr 12, 2019

Tel Tracker automation moved this from In progress to Done Apr 12, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.