Slow performance when using compression #60 #1037
Looking at the performance of Paramiko when trying to copy a 1.3Gb CSV file, to be moved between two machines connected by a 100Mbps lan segment. The root cause of the slow performance with compression is in the choice of compression level in paramiko/paramiko/compress.py
The change here is to use the default level of zlib compression (6) rather than the max of 9 which is a bad trade off in CPU and time vs the actual gain on compression
Only gut reactions:
Thanks again for this.
Reproducing this locally:
I'm not seeing results anywhere near as dramatic as OP's "level 6 compression only takes 1/4 the time of level 9 compression", though there is still a noticeable difference.
With compression set to 9:
Set to 6:
For funsies, set to 3:
But anyways, it's basically moot since OpenSSH demonstrably sets it to 6 by default, and that's always our basic guideline (well, mine. clearly wasn't original author's. unless SSH changed its behavior sometime in the last 15 years…)