put error "unsupported operand type(s) for &: 'NoneType' and 'int'" #811

Closed
jaraco opened this Issue Jan 7, 2013 · 13 comments

Projects

None yet

5 participants

@jaraco
Contributor
jaraco commented Jan 7, 2013

When I run a simple put with Fabric 1.5.1 using a mode parameter, I get an error. Here's the fabfile.py:

from __future__ import unicode_literals

import io
from fabric.api import task, put

@task
def test_put_empty_file():
    put(io.StringIO(''), '/tmp/foo', mode=0440)

When I run it, I get this error:

PS C:\Users\jaraco\G\support\yg.deploy> fab -H vdev-jaraco test_put_empty_file
[vdev-jaraco] Executing task 'test_put_empty_file'
[vdev-jaraco] put: <file obj> -> /tmp/foo

Fatal error: put() encountered an exception while uploading '<_io.StringIO object at 0x0000000002DAE1B0>'

Underlying exception:
    unsupported operand type(s) for &: 'NoneType' and 'int'

Aborting.

If I take the mode out of the script, it appears to work.

@jaraco
Contributor
jaraco commented Jan 7, 2013

After tracing briefly through the code, I see that mode is passed through almost directly to SFTP.put (from paramiko 1.10.0, a pre-release built from my fork of paramiko).

@jaraco
Contributor
jaraco commented Jan 7, 2013

I traced it a bit further, re-raising the original exception rather than masking it. When I did so, I get this traceback:

PS C:\Users\jaraco\G\support\yg.deploy> fab --show=DEBUG -H vdev-jaraco test_put_empty_file
[vdev-jaraco] Executing task 'test_put_empty_file'
[vdev-jaraco] put: <file obj> -> /tmp/foo
Traceback (most recent call last):
  File "C:\Users\jaraco\AppData\Roaming\Python\Python27\site-packages\fabric-1.5.1-py2.7.egg\fabric\main.py", line 739, in main
    *args, **kwargs
  File "C:\Users\jaraco\AppData\Roaming\Python\Python27\site-packages\fabric-1.5.1-py2.7.egg\fabric\tasks.py", line 314, in execute
    multiprocessing
  File "C:\Users\jaraco\AppData\Roaming\Python\Python27\site-packages\fabric-1.5.1-py2.7.egg\fabric\tasks.py", line 211, in _execute
    return task.run(*args, **kwargs)
  File "C:\Users\jaraco\AppData\Roaming\Python\Python27\site-packages\fabric-1.5.1-py2.7.egg\fabric\tasks.py", line 121, in run
    return self.wrapped(*args, **kwargs)
  File "C:\Users\jaraco\G\support\yg.deploy\fabfile.py", line 8, in test_put_empty_file
    put(io.StringIO(''), '/tmp/foo', mode=0440)
  File "C:\Users\jaraco\AppData\Roaming\Python\Python27\site-packages\fabric-1.5.1-py2.7.egg\fabric\network.py", line 523, in host_prompting_wrapper
    return func(*args, **kwargs)
  File "C:\Users\jaraco\AppData\Roaming\Python\Python27\site-packages\fabric-1.5.1-py2.7.egg\fabric\operations.py", line 391, in put
    mode, local_is_path)
  File "C:\Users\jaraco\AppData\Roaming\Python\Python27\site-packages\fabric-1.5.1-py2.7.egg\fabric\sftp.py", line 234, in put
    rmode = rattrs.st_mode & 07777
TypeError: unsupported operand type(s) for &: 'NoneType' and 'int'
Disconnecting from vdev-jaraco... done.

It seems the latest version of paramiko.sftp_client.SFTPClient.putfo is returning an rattrs where st_mode is None.

@jaraco
Contributor
jaraco commented Jan 7, 2013

I note that the error doesn't occur if some content is added to the file. i.e.

put(io.StringIO('#'), '/tmp/foo', mode=0440)
@jaraco
Contributor
jaraco commented Jan 7, 2013

The issue lies in the logic of putfo: it apparently only wants to return file attributes if at least one byte was written.

@jaraco
Contributor
jaraco commented Jan 7, 2013

Should I open a separate ticket with paramiko?

@bitprophet
Member

That part of Paramiko was modified mid last year to more gracefully handle file sizes and related things, which is what added the "only return a stat if the file has a size" angle.

So this issue is either an oversight in that change (meaning a Paramiko ticket is needed), or there's a conflict of interests going on (in which case it could be fixable there, or maybe worked around here).

If you can poke a bit farther to figure out whether the Paramiko logic is sound (cannot/should not stat if the file is empty) or can be modified, please do, and file a ticket there if needed (I'll go through my backlog there in a while). Thanks!

@dirkgomez

I've had a very similar problem with 1.6.0:

upload_template(filename=src, destination=dst, mode=0755) # fails
upload_template(filename=src, destination=dst) # works

Failure message is: unsupported operand type(s) for &: 'NoneType' and 'int'

Going back to 1.5.4 solved my problem.

@jaraco
Contributor
jaraco commented Mar 4, 2013

I created paramiko 142.

@bitprophet
Member

@dirkgomez That's actually unrelated, though same symptom. I just ran into it myself and think I've fixed it. Look for a new 1.6.x release real soon with the fix.

@jaraco I'll try to confirm your Paramiko fix at the same time, while I am at it. Thanks!

@bitprophet
Member

ugh, burned a lot of time on other, related errors instead. @dirkgomez the fix I mentioned is in. @jaraco I haven't gotten to the paramiko thing yet. Leaving tab open though for later!

@bitprophet
Member

OK, merging Paramiko 142/143 now. No Fabric level changes are needed so I am closing this now. Users who want this fix can just update to paramiko 1.10.1 when it is released (probably today because why not).

@bitprophet bitprophet closed this Apr 5, 2013
@lorrin
lorrin commented May 14, 2013

Upgrading paramiko to 1.10.1 fixed the issue for me. Thanks!

@umiyosh
umiyosh commented May 27, 2013

I also encountered the same issue, but the issue was resolved by upgrading the paramiko to 1.10.1, Thank you.

pip install Paramiko==1.10.1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment