fleetctl request timeout too low

[bcwaldon]

When using fleetctl remotely, the default request timeout of 1s is typically too low. This default should be raised to something higher than a typical round-trip from a laptop on wifi into AWS.

[developerinlondon]

i still get timeout issues later on setting it to 5 like you suggested @bcwaldon
i get a whole bunch of go errors from fleet when deis tries to start the units.

[bcwaldon]

@developerinlondon I'm more than happy to help, but I need a bit more information than that.

[developerinlondon]

heres the gist: https://gist.github.com/developerinlondon/f1c3efd389d5bf0b2bfe

[bcwaldon]

@developerinlondon could you run that same command with --debug on, capturing the output in another gist? Are you running over an SSH tunnel? if so, how long does it take on average to establish an SSH connection?

[developerinlondon]

I actually dumped the cluster after i couldnt get it to work yesterday, but let me recreate it again and get you the --debug.

was using eval ssh-agent -s && /usr/bin/ssh-add ~/.ssh/<mykey>

normally ssh to any of the machines were pretty quick, but i noticed last time i had debug on it would make a whole bunch of etcd calls and the more units i have there the more etcd calls it would make.

anyways, debug output to follow shortly.

[bcwaldon]

@developerinlondon The high number of etcd requests was identified and mostly fixed in #735. Expect that to be available in v0.6.2 which should be cut tonight/tomorrow.

[developerinlondon]

for some reason fleetctl seems to be querying 127.0.0.1 always even if the FLEETCTL_TUNNEL is set elsewhere. is this expected?

https://gist.github.com/developerinlondon/468d4586687aaf27ba39

[bcwaldon]

@developerinlondon it's using your SSH tunnel to hit 127.0.0.1 from the other side. If you need to configure a custom etcd endpoint, use the --endpoint flag.