ProxyCommand performance issues #420

Closed
acdha opened this Issue Oct 15, 2014 · 2 comments

Projects

None yet

2 participants

@acdha
acdha commented Oct 15, 2014

I noticed Python chewing 100% CPU and transferring very slowly while using a host with ProxyCommand set.

Looking at https://github.com/paramiko/paramiko/blob/e27964254fbf5a55d5cc0c4db9268cc001827af1/paramiko/proxy.py#L83-L95 shows two performance issues:

  1. It hammers datetime.now() (17% of total time) to get the time in milliseconds, which is a LOT slower than calling time.time():
cadams@ganymede:~ $ python -m timeit -s 'from datetime import datetime' 'datetime.now()'
1000000 loops, best of 3: 1.08 usec per loop
cadams@ganymede:~ $ python -m timeit -s 'from time import time' 'time()'
10000000 loops, best of 3: 0.0801 usec per loop

(To avoid portability issues, from timeit import default_timer picks the best timer function on both Unix and Windows)

  1. The select call specifies 0.0 for the timeout, which causes the loop to endlessly churn reading only a few bytes in each call. It looks like simply calling select with self.timeout would match semantics with blocking behaviour by default.
@lndbrg
Contributor
lndbrg commented Oct 15, 2014

Could you create a patch/pull request of your findings? :)

@acdha
acdha commented Oct 15, 2014

Ack, forgot to close this earlier – this is the same issue in #413 / #414. I'm not entirely sure of the merits of io_sleep vs. select() with a timeout but either should avoid the massive performance hit.

@acdha acdha closed this Oct 15, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment