New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TypeError in svn_utils.py:decode_as_string to crash #105

Closed
bb-migration opened this Issue Nov 13, 2013 · 9 comments

Comments

Projects
None yet
1 participant
@bb-migration

bb-migration commented Nov 13, 2013

Originally reported by: guyroz (Bitbucket: guyroz, GitHub: Unknown)


Traceback (most recent call last):
  File "setup.py", line 204, in <module>
    dist = setuptools.setup(**setup_params)
  File "/Users/jenkins/python/lib/python2.7/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/Users/jenkins/python/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/Users/jenkins/python/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/private/var/folders/39/kgfk3gv137n9slxvv4_mdvcr0000gs/T/tmpLLUGaS/setuptools-1.3.2/setuptools/command/bdist_egg.py", line 177, in run
    self.run_command("egg_info")
  File "/Users/jenkins/python/lib/python2.7/distutils/cmd.py", line 326, in run_command
    self.distribution.run_command(command)
  File "/Users/jenkins/python/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/private/var/folders/39/kgfk3gv137n9slxvv4_mdvcr0000gs/T/tmpLLUGaS/setuptools-1.3.2/setuptools/command/egg_info.py", line 187, in run
    self.find_sources()
  File "/private/var/folders/39/kgfk3gv137n9slxvv4_mdvcr0000gs/T/tmpLLUGaS/setuptools-1.3.2/setuptools/command/egg_info.py", line 230, in find_sources
    mm.run()
  File "/private/var/folders/39/kgfk3gv137n9slxvv4_mdvcr0000gs/T/tmpLLUGaS/setuptools-1.3.2/setuptools/command/egg_info.py", line 296, in run
    self.add_defaults()
  File "/private/var/folders/39/kgfk3gv137n9slxvv4_mdvcr0000gs/T/tmpLLUGaS/setuptools-1.3.2/setuptools/command/egg_info.py", line 335, in add_defaults
    rcfiles = list(walk_revctrl())
  File "/private/var/folders/39/kgfk3gv137n9slxvv4_mdvcr0000gs/T/tmpLLUGaS/setuptools-1.3.2/setuptools/command/sdist.py", line 18, in walk_revctrl
    for item in ep.load()(dirname):
  File "/private/var/folders/39/kgfk3gv137n9slxvv4_mdvcr0000gs/T/tmpLLUGaS/setuptools-1.3.2/setuptools/command/sdist.py", line 58, in _default_revctrl
    for item in finder(dirname):
  File "/private/var/folders/39/kgfk3gv137n9slxvv4_mdvcr0000gs/T/tmpLLUGaS/setuptools-1.3.2/setuptools/svn_utils.py", line 414, in svn_finder
    info = SvnInfo.load(dirname)
  File "/private/var/folders/39/kgfk3gv137n9slxvv4_mdvcr0000gs/T/tmpLLUGaS/setuptools-1.3.2/setuptools/svn_utils.py", line 221, in load
    code, data = _run_command(['svn', 'info', normdir])
  File "/private/var/folders/39/kgfk3gv137n9slxvv4_mdvcr0000gs/T/tmpLLUGaS/setuptools-1.3.2/setuptools/svn_utils.py", line 41, in _run_command
    data = decode_as_string(data, encoding)
  File "/private/var/folders/39/kgfk3gv137n9slxvv4_mdvcr0000gs/T/tmpLLUGaS/setuptools-1.3.2/setuptools/svn_utils.py", line 93, in decode_as_string
    text = text.decode(encoding)
TypeError: decode() argument 1 must be string, not None

locale.getpreferredencoding may return None (if LC_ALL= on Mac), causes svn_utils.py:decode_as_string to crash


@bb-migration

This comment has been minimized.

bb-migration commented Nov 14, 2013

Original comment by philip_thiem (Bitbucket: philip_thiem, GitHub: Unknown):


Thx Guy. I knew that there was a reason I didn't want to use getprefferedencoding. Will have to have a think on this one. Are you on a mac? I'm probably going to have to get my hands on an environment to do some experimenting on.

@bb-migration

This comment has been minimized.

bb-migration commented Nov 15, 2013

Original comment by guyroz (Bitbucket: guyroz, GitHub: Unknown):


Yes Philip I am using a Mac
Let me know if I can help

@bb-migration

This comment has been minimized.

bb-migration commented Nov 19, 2013

Original comment by philip_thiem (Bitbucket: philip_thiem, GitHub: Unknown):


Thx, I think I will definitely take you up on that. It looks like between the various versions of pythons supported there are a number of "fun" nuisances with mac. Sufficiently old versions of python appear to bus error on certain calls even. I have a few ideas, but I figure that instead of guessing I might as well see how these things will behave. So I have an encoding test which I tossed together. Should be able to run it in a console with py26+ So if you would be so kind, ;), I would appreciate the output.

@bb-migration

This comment has been minimized.

bb-migration commented Nov 20, 2013

Original comment by guyroz (Bitbucket: guyroz, GitHub: Unknown):


here are the results:

ci-macpro:~ jenkins$ python /tmp/encodingtestv2.py
darwin
STDOUT Encoding: US-ASCII
Has locale.CODESET? False
PreferredEncoding: None
DefaultLocale: None, None
Initializing Local?
PreferredEncoding: None
DefaultLocale: None, None

Other interesting results:

ci-macpro:~ jenkins$ python -c "import locale; print locale.getpreferredencoding()"
None

ci-macpro:~ jenkins$ LC_ALL=en_US python -c "import locale; print locale.getpreferredencoding()"
ISO8859-1

ci-macpro:~ jenkins$ LC_ALL=en_US python /tmp/encodingtestv2.py
darwin
Traceback (most recent call last):
  File "/tmp/encodingtestv2.py", line 6, in <module>
    print("STDOUT Encoding: " + sys.stdout.encoding)
TypeError: cannot concatenate 'str' and 'NoneType' objects
ci-macpro:~ jenkins$ 
@bb-migration

This comment has been minimized.

bb-migration commented Nov 20, 2013

Original comment by philip_thiem (Bitbucket: philip_thiem, GitHub: Unknown):


Thank you! I find it interesting that the stdout.encoding is None when LC_ALL is given. That and I used the wrong category in my script. Here is a newer version which you can try, please, with and without LC_ALL=en_US. I think I've figured out how to possibly get a mac VM with travis-ci.org it feels like a bit of a hack to tell it that I need an objective-c environment, but should have access to at least py27 and py3 interpreter.

@bb-migration

This comment has been minimized.

bb-migration commented Nov 21, 2013

Original comment by philip_thiem (Bitbucket: philip_thiem, GitHub: Unknown):


Ok, got the vm's working. With 2.7 and 3.3 I get

darwin
STDOUT Encoding: US-ASCII
Has locale.CODESET? True
nl_langinfo returned US-ASCII
PreferredEncoding: US-ASCII
DefaultLocale: None, None
Initializing Local LC_ALL?
nl_langinfo returned US-ASCII
PreferredEncoding: US-ASCII
DefaultLocale: None, None
/usr/local/bin/python

darwin
STDOUT Encoding: US-ASCII
Has locale.CODESET? True
nl_langinfo returned US-ASCII
PreferredEncoding: US-ASCII
DefaultLocale: None, None
Initializing Local LC_ALL?
nl_langinfo returned US-ASCII
PreferredEncoding: US-ASCII
DefaultLocale: None, None

Which I can't envision is right, however, US-ASCII should decode as UTF-8 and I'm pretty sure that osX pretty much enforces UTF-8 for the filesystem and tends to prefer that, so it might be a more reasonable default... Also seems like trying the nudge initialization of the locale with setlocale doesn't work thus far on any mac setup I've seen. ISO8859-1 is latin-1. LC_ALL=en_US.utf8 will probably get default encoding to return (en_us, utf-8)..

@bb-migration

This comment has been minimized.

bb-migration commented Nov 21, 2013

Original comment by philip_thiem (Bitbucket: philip_thiem, GitHub: Unknown):


https://bitbucket.org/pypa/setuptools/pull-request/26/svn-encoding-issues/diff
Basically for determining the terminal encoding:

  1. Try getpreferredencoding
  2. If that is US_ASCII or None try the encoding from getdefaultlocale() if that was a "fallback" because python couldn't figure it out from the environment or OS, this wll result in None.
  3. If not none, then make sure python actually knows about the encoding.
  4. On the event of an error or unknown codec, revert to fallbacks
  5. That is, on darwin try utf-8 everything else ASCII. Also on the darwin and mac-roman, switch to utf-8 this was a bug on older pythons.
@bb-migration

This comment has been minimized.

bb-migration commented Nov 23, 2013

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


Update changelog. Fixes #105 and Fixes #113.

@bb-migration

This comment has been minimized.

bb-migration commented Nov 23, 2013

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


1.4.1 includes these fixes. Please test and confirm.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment