New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix ParamikoSSHClient.run() so that it works in Python 3.4.1 #347

Closed
wants to merge 1 commit into
base: trunk
from

Conversation

Projects
None yet
2 participants
@ereyes01
Contributor

ereyes01 commented Aug 14, 2014

I was trying to do the following using apache-libcloud==0.15.1 on Python 3.4.1:

  • Run deploy_node to create an EC2 node with a MultiStepDeployment.
  • The MultiStepDeployment consists of 2 FileDeployment's followed by 1 ScriptDeployment

The deploy_node would work fine in Python 2.7.6 but failed when I switched to Python 3,4,1 with the following:

DeploymentError                           Traceback (most recent call last)
<ipython-input-18-79924dc2c1dc> in <module>()
      2                           ex_security_groups=[SECURITY_GROUP_NAME],
      3                           ex_keyname=KEY_NAME, deploy=deployment, ssh_username=USERNAME,
----> 4                           ssh_key=PRIVATE_KEY_PATH)
      5 print(node)

/home/ereyes/.pyenv/versions/deploy3/lib/python3.4/site-packages/libcloud/compute/base.py in deploy_node(self, **kwargs)
    924         if deploy_error is not None:
    925             raise DeploymentError(node=node, original_exception=deploy_error,
--> 926                                   driver=self)
    927 
    928         return node

DeploymentError: <DeploymentError: node=i-1ff25335, error=<LibcloudError in <libcloud.compute.drivers.ec2.EC2NodeDriver object at 0x7f91a0782d68> "Failed after 3 tries: string argument expected, got 'bytes'">, driver=<libcloud.compute.drivers.ec2.EC2NodeDriver object at 0x7f91a0782d68>>

After instrumenting the code to print more informative stack traces, I located the source of the Exception, which was in ParamikoSSHClient.run(). The code where a data chunk is obtained from chan.recv() returns a bytes object in Python 3, where it presumably returns a descendant of basestring in Python 2. This object is then passed to stdout.write() (or stderr.write), which blows up because it's not a stringy value.

My change decodes the bytes into strings as they are passed into those write() methods.

This change made deploy_node() work in my code for Python 3.4.1. I'm not sure if the same type of bug exists elsewhere, but this fix is what I needed to unblock myself.

Thanks for the excellent library, and I hope this helps.

Fix ParamikoSSHClient.run() so that it works in Python 3.4.1 by decoding
incoming bytes into strings using the bytes decode method.
@Kami

This comment has been minimized.

Show comment
Hide comment
@Kami

Kami Aug 15, 2014

Member

Good catch, the same issue should exist in all the Python 3.x versions where bytes / unicode handling has been changed.

Member

Kami commented Aug 15, 2014

Good catch, the same issue should exist in all the Python 3.x versions where bytes / unicode handling has been changed.

@@ -364,7 +364,7 @@ def run(self, cmd, timeout=None):
data = chan.recv(CHUNK_SIZE)
while data:
stdout.write(data)
stdout.write(data.decode("utf-8"))

This comment has been minimized.

@Kami

Kami Aug 15, 2014

Member

Looks good.

Only thing I would do is wrap data with libcloud.utils.py3 with b function call to be on the safe side and make sure that type of data is in fact bytes in Python 3 (if for some reason, data is not bytes in Python 3, decode call will throw).

I'll go ahead and make this change.

@Kami

Kami Aug 15, 2014

Member

Looks good.

Only thing I would do is wrap data with libcloud.utils.py3 with b function call to be on the safe side and make sure that type of data is in fact bytes in Python 3 (if for some reason, data is not bytes in Python 3, decode call will throw).

I'll go ahead and make this change.

@asfgit asfgit closed this in 8b8b67c Aug 15, 2014

@Kami

This comment has been minimized.

Show comment
Hide comment
@Kami

Kami Aug 15, 2014

Member

Merged into trunk. Thanks.

Member

Kami commented Aug 15, 2014

Merged into trunk. Thanks.

@ereyes01 ereyes01 deleted the ereyes01:ssh_decode branch Aug 15, 2014

@ereyes01

This comment has been minimized.

Show comment
Hide comment
@ereyes01

ereyes01 Aug 15, 2014

Contributor

Thank you for merging, and for the quick response!

Contributor

ereyes01 commented Aug 15, 2014

Thank you for merging, and for the quick response!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment