fix an error if a stream encoding type is None #1180

Closed
wants to merge 4 commits into
from

Projects

None yet

3 participants

@joekiller
Contributor

With the encoding refactor an error was introduced where if the encoding type of the stream was None a TypeError exception was thrown saying encode() argument 1 must be string, not None.

This rare case happens if the stdout stream is being emulated or intercepted in a build system such as Jenkins and unicode characters were being fed through the put() command. The resulting sys.system.out stream.encoding would be None.

This fix ensures that if the stream encoding is unknown if allows the message to just decode it the best it can.

@bitprophet bitprophet modified the milestone: 1.7.5 et al, 1.8.6/1.9.2 Aug 26, 2014
@msabramo
Contributor

Doesn't merge cleanly

@msabramo
Contributor

I have a similar issue; not sure if it's the same yet though.

  File "/opt/doula/src/doula/doula/models/push.py", line 318, in _chown
    result = sudo(chown_command)
  File "/opt/doula/local/lib/python2.7/site-packages/fabric/network.py", line 647, in host_prompting_wrapper
    return func(*args, **kwargs)
  File "/opt/doula/local/lib/python2.7/site-packages/fabric/operations.py", line 1101, in sudo
    stderr=stderr, timeout=timeout, shell_escape=shell_escape,
  File "/opt/doula/local/lib/python2.7/site-packages/fabric/operations.py", line 938, in _run_command
    error(message=msg, stdout=out, stderr=err)
  File "/opt/doula/local/lib/python2.7/site-packages/fabric/utils.py", line 333, in error
    message += _format_error_output("Standard output", stdout)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 104: ordinal not in range(128)

So the chown command is failing and it uses funny Unicode quote characters in it.

chown: invalid user: ‘foo’

I am redirecting Fabric output to a stream so that it can be displayed in a web app.

@msabramo
Contributor

Ah, the above UnicodeDecodeError I got when my Fabric tasks were wrapped with hide('stdout'):

chown_command = 'chown foo bar'

with hide('stdout'):
    result = sudo(chown_command)

After removing that, now I get this error instead, which sounds like the one mentioned in this PR:

  File "/opt/doula/src/doula/doula/models/push.py", line 319, in _chown
    result = sudo(chown_command)
  File "/opt/doula/local/lib/python2.7/site-packages/fabric/network.py", line 647, in host_prompting_wrapper
    return func(*args, **kwargs)
  File "/opt/doula/local/lib/python2.7/site-packages/fabric/operations.py", line 1101, in sudo
    stderr=stderr, timeout=timeout, shell_escape=shell_escape,
  File "/opt/doula/local/lib/python2.7/site-packages/fabric/operations.py", line 938, in _run_command
    error(message=msg, stdout=out, stderr=err)
  File "/opt/doula/local/lib/python2.7/site-packages/fabric/utils.py", line 336, in error
    return func(message)
  File "/opt/doula/local/lib/python2.7/site-packages/fabric/utils.py", line 36, in abort
    sys.stderr.write(red("\nFatal error: %s\n" % _encode(msg, sys.stderr)))
  File "/opt/doula/local/lib/python2.7/site-packages/fabric/utils.py", line 13, in _encode
    return msg.encode(stream.encoding)
TypeError: encode() argument 1 must be string, not None

Will this go away if make sure that sys.stdout is UTF-8?

@msabramo msabramo referenced this pull request Oct 25, 2014
Closed

Fix stream encoding none #1214

@msabramo
Contributor

Hey @joekiller - thanks for your work on this -- I fixed the merge conflict with #1214, but couldn't figure out how to send the PR to you. You can apply it manually if you like so @bitprophet can accept this PR instead of mine, as you deserve the credit.

@joekiller
Contributor

Thanks @msabramo. I'll try to get this tonight.

@joekiller
Contributor

@bitprophet @msabramo should be good, thanks all.

@bitprophet
Member

I thought I merged this already but nope, still in the queue for next bugfix release. I'll try to get on that between now and early next week. Thanks for keeping it updated!

@bitprophet
Member

Cherry-picked back to the 1.9 release line. Thanks!

@bitprophet bitprophet closed this Dec 19, 2014
@joekiller
Contributor

Yay releases! I'll test this soon. Thanks @bitprophet

@bitprophet
Member

1.9 and 1.10 bugfixes are out on PyPI now too, so yay. Obviously let me know if something is still broke :) (I fixed my comment above - it said 0.9 - we haven't had an 0.9 release in about 4-5 years. hah!)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment