New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

put error "unsupported operand type(s) for &: 'NoneType' and 'int'" #811

Closed
jaraco opened this Issue Jan 7, 2013 · 13 comments

Comments

Projects
None yet
5 participants
@jaraco

jaraco commented Jan 7, 2013

When I run a simple put with Fabric 1.5.1 using a mode parameter, I get an error. Here's the fabfile.py:

from __future__ import unicode_literals

import io
from fabric.api import task, put

@task
def test_put_empty_file():
    put(io.StringIO(''), '/tmp/foo', mode=0440)

When I run it, I get this error:

PS C:\Users\jaraco\G\support\yg.deploy> fab -H vdev-jaraco test_put_empty_file
[vdev-jaraco] Executing task 'test_put_empty_file'
[vdev-jaraco] put: <file obj> -> /tmp/foo

Fatal error: put() encountered an exception while uploading '<_io.StringIO object at 0x0000000002DAE1B0>'

Underlying exception:
    unsupported operand type(s) for &: 'NoneType' and 'int'

Aborting.

If I take the mode out of the script, it appears to work.

@jaraco

This comment has been minimized.

jaraco commented Jan 7, 2013

After tracing briefly through the code, I see that mode is passed through almost directly to SFTP.put (from paramiko 1.10.0, a pre-release built from my fork of paramiko).

@jaraco

This comment has been minimized.

jaraco commented Jan 7, 2013

I traced it a bit further, re-raising the original exception rather than masking it. When I did so, I get this traceback:

PS C:\Users\jaraco\G\support\yg.deploy> fab --show=DEBUG -H vdev-jaraco test_put_empty_file
[vdev-jaraco] Executing task 'test_put_empty_file'
[vdev-jaraco] put: <file obj> -> /tmp/foo
Traceback (most recent call last):
  File "C:\Users\jaraco\AppData\Roaming\Python\Python27\site-packages\fabric-1.5.1-py2.7.egg\fabric\main.py", line 739, in main
    *args, **kwargs
  File "C:\Users\jaraco\AppData\Roaming\Python\Python27\site-packages\fabric-1.5.1-py2.7.egg\fabric\tasks.py", line 314, in execute
    multiprocessing
  File "C:\Users\jaraco\AppData\Roaming\Python\Python27\site-packages\fabric-1.5.1-py2.7.egg\fabric\tasks.py", line 211, in _execute
    return task.run(*args, **kwargs)
  File "C:\Users\jaraco\AppData\Roaming\Python\Python27\site-packages\fabric-1.5.1-py2.7.egg\fabric\tasks.py", line 121, in run
    return self.wrapped(*args, **kwargs)
  File "C:\Users\jaraco\G\support\yg.deploy\fabfile.py", line 8, in test_put_empty_file
    put(io.StringIO(''), '/tmp/foo', mode=0440)
  File "C:\Users\jaraco\AppData\Roaming\Python\Python27\site-packages\fabric-1.5.1-py2.7.egg\fabric\network.py", line 523, in host_prompting_wrapper
    return func(*args, **kwargs)
  File "C:\Users\jaraco\AppData\Roaming\Python\Python27\site-packages\fabric-1.5.1-py2.7.egg\fabric\operations.py", line 391, in put
    mode, local_is_path)
  File "C:\Users\jaraco\AppData\Roaming\Python\Python27\site-packages\fabric-1.5.1-py2.7.egg\fabric\sftp.py", line 234, in put
    rmode = rattrs.st_mode & 07777
TypeError: unsupported operand type(s) for &: 'NoneType' and 'int'
Disconnecting from vdev-jaraco... done.

It seems the latest version of paramiko.sftp_client.SFTPClient.putfo is returning an rattrs where st_mode is None.

@jaraco

This comment has been minimized.

jaraco commented Jan 7, 2013

I note that the error doesn't occur if some content is added to the file. i.e.

put(io.StringIO('#'), '/tmp/foo', mode=0440)
@jaraco

This comment has been minimized.

jaraco commented Jan 7, 2013

The issue lies in the logic of putfo: it apparently only wants to return file attributes if at least one byte was written.

@jaraco

This comment has been minimized.

jaraco commented Jan 7, 2013

Should I open a separate ticket with paramiko?

@bitprophet

This comment has been minimized.

Member

bitprophet commented Jan 27, 2013

That part of Paramiko was modified mid last year to more gracefully handle file sizes and related things, which is what added the "only return a stat if the file has a size" angle.

So this issue is either an oversight in that change (meaning a Paramiko ticket is needed), or there's a conflict of interests going on (in which case it could be fixable there, or maybe worked around here).

If you can poke a bit farther to figure out whether the Paramiko logic is sound (cannot/should not stat if the file is empty) or can be modified, please do, and file a ticket there if needed (I'll go through my backlog there in a while). Thanks!

@dirkgomez

This comment has been minimized.

dirkgomez commented Mar 4, 2013

I've had a very similar problem with 1.6.0:

upload_template(filename=src, destination=dst, mode=0755) # fails
upload_template(filename=src, destination=dst) # works

Failure message is: unsupported operand type(s) for &: 'NoneType' and 'int'

Going back to 1.5.4 solved my problem.

@jaraco

This comment has been minimized.

jaraco commented Mar 4, 2013

I created paramiko 142.

@bitprophet

This comment has been minimized.

Member

bitprophet commented Mar 21, 2013

@dirkgomez That's actually unrelated, though same symptom. I just ran into it myself and think I've fixed it. Look for a new 1.6.x release real soon with the fix.

@jaraco I'll try to confirm your Paramiko fix at the same time, while I am at it. Thanks!

@bitprophet

This comment has been minimized.

Member

bitprophet commented Mar 22, 2013

ugh, burned a lot of time on other, related errors instead. @dirkgomez the fix I mentioned is in. @jaraco I haven't gotten to the paramiko thing yet. Leaving tab open though for later!

@bitprophet

This comment has been minimized.

Member

bitprophet commented Apr 5, 2013

OK, merging Paramiko 142/143 now. No Fabric level changes are needed so I am closing this now. Users who want this fix can just update to paramiko 1.10.1 when it is released (probably today because why not).

@bitprophet bitprophet closed this Apr 5, 2013

@lorrin

This comment has been minimized.

lorrin commented May 14, 2013

Upgrading paramiko to 1.10.1 fixed the issue for me. Thanks!

@umiyosh

This comment has been minimized.

umiyosh commented May 27, 2013

I also encountered the same issue, but the issue was resolved by upgrading the paramiko to 1.10.1, Thank you.

pip install Paramiko==1.10.1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment