Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

'utf8' codec can't decode byte 0x81 in position 29: invalid start byte after upgrade to 2014.1.13 or 2014.7.0 #18053

Closed
danielchristianschroeter opened this issue Nov 13, 2014 · 3 comments
Closed

'utf8' codec can't decode byte 0x81 in position 29: invalid start byte after upgrade to 2014.1.13 or 2014.7.0 #18053

danielchristianschroeter opened this issue Nov 13, 2014 · 3 comments
Labels
Bug broken, incorrect, or confusing behavior fixed-pls-verify fix is linked, bug author to confirm fix severity-low 4th level, cosemtic problems, work around exists
Milestone
Approved

Comments

@danielchristianschroeter
Copy link

After upgrading from Salt stack version 2014.1.10 to 2014.1.13 (CentOS 6 master and Windows Server 2008 R2 minion) I receive the following error after the state.highstate (or salt 'win' state.sls test) is completed:

[root@localhost states]# salt 'win' state.highstate -t 999
[ERROR   ] An un-handled exception was caught by salt's global exception handler:
UnicodeDecodeError: 'utf8' codec can't decode byte 0x81 in position 29: invalid start byte
Traceback (most recent call last):
  File "/usr/bin/salt", line 10, in <module>
    salt_main()
  File "/usr/lib/python2.6/site-packages/salt/scripts.py", line 135, in salt_main
    client.run()
  File "/usr/lib/python2.6/site-packages/salt/cli/__init__.py", line 140, in run
    self._output_ret(ret, out)
  File "/usr/lib/python2.6/site-packages/salt/cli/__init__.py", line 155, in _output_ret
    salt.output.display_output(ret, out, self.config)
  File "/usr/lib/python2.6/site-packages/salt/output/__init__.py", line 35, in display_output
    display_data = get_printout(out, opts)(data).rstrip()
  File "/usr/lib/python2.6/site-packages/salt/output/highstate.py", line 44, in output
    return _format_host(host, hostdata)[0]
  File "/usr/lib/python2.6/site-packages/salt/output/highstate.py", line 84, in _format_host
    schanged, ctext = _format_changes(ret['changes'])
  File "/usr/lib/python2.6/site-packages/salt/output/highstate.py", line 235, in _format_changes
    __opts__)
  File "/usr/lib/python2.6/site-packages/salt/output/__init__.py", line 101, in out_format
    return get_printout(out, opts)(data).rstrip()
  File "/usr/lib/python2.6/site-packages/salt/output/nested.py", line 97, in output
    return nest.display(ret, __opts__.get('nested_indent', 0), '', '')
  File "/usr/lib/python2.6/site-packages/salt/output/nested.py", line 88, in display
    out = self.display(val, indent + 4, '', out)
  File "/usr/lib/python2.6/site-packages/salt/output/nested.py", line 63, in display
    prefix=prefix)
  File "/usr/lib/python2.6/site-packages/salt/output/nested.py", line 36, in ustring
    indent, color, prefix, msg.decode(encoding), endc, suffix)
  File "/usr/lib64/python2.6/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0x81 in position 29: invalid start byte
Traceback (most recent call last):
  File "/usr/bin/salt", line 10, in <module>
    salt_main()
  File "/usr/lib/python2.6/site-packages/salt/scripts.py", line 135, in salt_main
    client.run()
  File "/usr/lib/python2.6/site-packages/salt/cli/__init__.py", line 140, in run
    self._output_ret(ret, out)
  File "/usr/lib/python2.6/site-packages/salt/cli/__init__.py", line 155, in _output_ret
    salt.output.display_output(ret, out, self.config)
  File "/usr/lib/python2.6/site-packages/salt/output/__init__.py", line 35, in display_output
    display_data = get_printout(out, opts)(data).rstrip()
  File "/usr/lib/python2.6/site-packages/salt/output/highstate.py", line 44, in output
    return _format_host(host, hostdata)[0]
  File "/usr/lib/python2.6/site-packages/salt/output/highstate.py", line 84, in _format_host
    schanged, ctext = _format_changes(ret['changes'])
  File "/usr/lib/python2.6/site-packages/salt/output/highstate.py", line 235, in _format_changes
    __opts__)
  File "/usr/lib/python2.6/site-packages/salt/output/__init__.py", line 101, in out_format
    return get_printout(out, opts)(data).rstrip()
  File "/usr/lib/python2.6/site-packages/salt/output/nested.py", line 97, in output
    return nest.display(ret, __opts__.get('nested_indent', 0), '', '')
  File "/usr/lib/python2.6/site-packages/salt/output/nested.py", line 88, in display
    out = self.display(val, indent + 4, '', out)
  File "/usr/lib/python2.6/site-packages/salt/output/nested.py", line 63, in display
    prefix=prefix)
  File "/usr/lib/python2.6/site-packages/salt/output/nested.py", line 36, in ustring
    indent, color, prefix, msg.decode(encoding), endc, suffix)
  File "/usr/lib64/python2.6/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0x81 in position 29: invalid start byte

It is very strange because on the minion the executed states was all successfully executed.
No error in minion log.

UPDATE1
I upgraded the master to 2014.7.0 (2014.7.0-3.el6) and downgraded the Windows Server 2008 R2 minion to 2014.1.10 (default config). The error still exists, but it seems it is not in all state files. It seems it is an issue with the master version...
Maybe I should try to downgrade the master back to 2014.1.10, but I don't know where I can find older versions of salt?!

I found the old v2014.1.10 salt and salt-master here:
ftp://ftp.sunet.se/pub/Linux/distributions/fedora/epel/6/i386/salt-2014.1.10-4.el6.noarch.rpm
ftp://ftp.sunet.se/pub/Linux/distributions/fedora/epel/6/i386/salt-master-2014.1.10-4.el6.noarch.rpm

It seems in epel is only the latest version salt-master-2014.7.0-3.el6 available.

Here is the error I received in the master log:

[salt.log.setup   ][ERROR   ] An un-handled exception was caught by salt's global exception handler:
UnicodeDecodeError: 'utf8' codec can't decode byte 0x81 in position 29: invalid start byte
Traceback (most recent call last):
  File "/usr/bin/salt", line 10, in <module>
    salt_main()
  File "/usr/lib/python2.6/site-packages/salt/scripts.py", line 240, in salt_main
    client.run()
  File "/usr/lib/python2.6/site-packages/salt/cli/__init__.py", line 189, in run
    self._output_ret(ret_, out)
  File "/usr/lib/python2.6/site-packages/salt/cli/__init__.py", line 243, in _output_ret
    salt.output.display_output(ret, out, self.config)
  File "/usr/lib/python2.6/site-packages/salt/output/__init__.py", line 44, in display_output
    display_data = get_printout(out, opts)(data).rstrip()
  File "/usr/lib/python2.6/site-packages/salt/output/highstate.py", line 74, in output
    return _format_host(host, hostdata)[0]
  File "/usr/lib/python2.6/site-packages/salt/output/highstate.py", line 118, in _format_host
    schanged, ctext = _format_changes(ret['changes'])
  File "/usr/lib/python2.6/site-packages/salt/output/highstate.py", line 340, in _format_changes
    __opts__)
  File "/usr/lib/python2.6/site-packages/salt/output/__init__.py", line 121, in out_format
    return get_printout(out, opts)(data).rstrip()
  File "/usr/lib/python2.6/site-packages/salt/output/nested.py", line 123, in output
    return nest.display(ret, __opts__.get('nested_indent', 0), '', '')
  File "/usr/lib/python2.6/site-packages/salt/output/nested.py", line 114, in display
    out = self.display(val, indent + 4, '', out)
  File "/usr/lib/python2.6/site-packages/salt/output/nested.py", line 89, in display
    prefix=prefix)
  File "/usr/lib/python2.6/site-packages/salt/output/nested.py", line 58, in ustring
    indent, color, prefix, msg.decode(encoding), endc, suffix)
  File "/usr/lib64/python2.6/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0x81 in position 29: invalid start byte

Master version report:

           Salt: 2014.7.0
         Python: 2.6.6 (r266:84292, Jan 22 2014, 09:42:36)
         Jinja2: unknown
       M2Crypto: 0.20.2
 msgpack-python: 0.1.13
   msgpack-pure: Not Installed
       pycrypto: 2.0.1
        libnacl: Not Installed
         PyYAML: 3.10
          ioflo: Not Installed
          PyZMQ: 14.3.1
           RAET: Not Installed
            ZMQ: 3.2.4
           Mako: Not Installed

UPDATE2
Master downgraded to 2014.1.10, everything runs smoothly without errors.

 salt-master --versions-report
           Salt: 2014.1.10
         Python: 2.6.6 (r266:84292, Jan 22 2014, 09:42:36)
         Jinja2: unknown
       M2Crypto: 0.20.2
 msgpack-python: 0.1.13
   msgpack-pure: Not Installed
       pycrypto: 2.0.1
         PyYAML: 3.10
          PyZMQ: 14.3.1
            ZMQ: 3.2.4
@danielchristianschroeter danielchristianschroeter changed the title 'utf8' codec can't decode byte 0x81 in position 29: invalid start byte after upgrade to 2014.1.13 'utf8' codec can't decode byte 0x81 in position 29: invalid start byte after upgrade to 2014.1.13 or 2014.7.0 Nov 13, 2014
@rallytime rallytime added Bug broken, incorrect, or confusing behavior severity-low 4th level, cosemtic problems, work around exists labels Nov 13, 2014
@rallytime rallytime added this to the Approved milestone Nov 13, 2014
@rallytime
Copy link
Contributor

Thanks for the detailed report @myg0v! That will make it easier to track this one down. I am also wondering if this is related to #17227.

Closed
@fayetted
Copy link
Contributor

I've also experienced this problem and have enclosed the details.

salt-master --versions-report
           Salt: 2014.7.0
         Python: 2.6.6 (r266:84292, Jan 22 2014, 09:42:36)
         Jinja2: unknown
       M2Crypto: 0.20.2
 msgpack-python: 0.1.13
   msgpack-pure: Not Installed
       pycrypto: 2.0.1
        libnacl: Not Installed
         PyYAML: 3.10
          ioflo: Not Installed
          PyZMQ: 14.3.1
           RAET: Not Installed
            ZMQ: 3.2.4
           Mako: Not Installed

I edited my '''/usr/lib64/python2.6/encodings/utf_8.py''' file and added a 'print input' to the decode function in order to see the actual outpt which is listed below.

... snip ...
Microsoft Office Proofing Tools 2013 - Espa�ol
[ERROR   ] An un-handled exception was caught by salt's global exception handler:
UnicodeDecodeError: 'ascii' codec can't decode byte 0xa4 in position 43: ordinal not in range(128)
Traceback (most recent call last):
  File "/usr/bin/salt", line 10, in 
    salt_main()
  File "/usr/lib/python2.6/site-packages/salt/scripts.py", line 240, in salt_main
    client.run()
  File "/usr/lib/python2.6/site-packages/salt/cli/__init__.py", line 189, in run
    self._output_ret(ret_, out)
  File "/usr/lib/python2.6/site-packages/salt/cli/__init__.py", line 243, in _output_ret
    salt.output.display_output(ret, out, self.config)
  File "/usr/lib/python2.6/site-packages/salt/output/__init__.py", line 44, in display_output
    display_data = get_printout(out, opts)(data).rstrip()
  File "/usr/lib/python2.6/site-packages/salt/output/nested.py", line 123, in output
    return nest.display(ret, __opts__.get('nested_indent', 0), '', '')
  File "/usr/lib/python2.6/site-packages/salt/output/nested.py", line 114, in display
    out = self.display(val, indent + 4, '', out)
  File "/usr/lib/python2.6/site-packages/salt/output/nested.py", line 113, in display
    prefix=prefix)
  File "/usr/lib/python2.6/site-packages/salt/output/nested.py", line 58, in ustring
    indent, color, prefix, msg.decode(encoding), endc, suffix)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xa4 in position 43: ordinal not in range(128)
Traceback (most recent call last):
  File "/usr/bin/salt", line 10, in 
    salt_main()
  File "/usr/lib/python2.6/site-packages/salt/scripts.py", line 240, in salt_main
    client.run()
  File "/usr/lib/python2.6/site-packages/salt/cli/__init__.py", line 189, in run
    self._output_ret(ret_, out)
  File "/usr/lib/python2.6/site-packages/salt/cli/__init__.py", line 243, in _output_ret
    salt.output.display_output(ret, out, self.config)
  File "/usr/lib/python2.6/site-packages/salt/output/__init__.py", line 44, in display_output
    display_data = get_printout(out, opts)(data).rstrip()
  File "/usr/lib/python2.6/site-packages/salt/output/nested.py", line 123, in output
    return nest.display(ret, __opts__.get('nested_indent', 0), '', '')
  File "/usr/lib/python2.6/site-packages/salt/output/nested.py", line 114, in display
    out = self.display(val, indent + 4, '', out)
  File "/usr/lib/python2.6/site-packages/salt/output/nested.py", line 113, in display
    prefix=prefix)
  File "/usr/lib/python2.6/site-packages/salt/output/nested.py", line 58, in ustring
    indent, color, prefix, msg.decode(encoding), endc, suffix)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xa4 in position 43: ordinal not in range(128)

Another lines that result in same error:

Outils de v�rification linguistique 2013 de Microsoft Office�- Fran�ais

@fayetted
Copy link
Contributor

I know this is not an ideal solution but for me it's usable until somebody with a better understanding can provide a real fix and hopefully these details will save somebody the effort of tracing it from the ground up.

edit /usr/lib/python2.6/site-packages/salt/output/nested.py
At Line 56, at a try block that when it fails uses a different encoding. For me ISO-8859-1 works.

        except UnicodeDecodeError:
            try:
                return u'{0}{1}{2}{3}{4}{5}\n'.format(
                    indent, color, prefix, msg.decode(encoding), endc, suffix)
            except:
                return u'{0}{1}{2}{3}{4}{5}\n'.format(
                        indent, color, prefix, msg.decode("ISO-8859-1"), endc, suffix)

Here is the Diff:

diff /usr/lib/python2.6/site-packages/salt/output/nested.py /usr/lib/python2.6/site-packages/salt/output/nested.py.orig
57,62c57,58
<             try:
<                 return u'{0}{1}{2}{3}{4}{5}\n'.format(
<                     indent, color, prefix, msg.decode(encoding), endc, suffix)
<             except:
<                 return u'{0}{1}{2}{3}{4}{5}\n'.format(
<                         indent, color, prefix, msg.decode("ISO-8859-1"), endc, suffix)
---
>             return u'{0}{1}{2}{3}{4}{5}\n'.format(
>                 indent, color, prefix, msg.decode(encoding), endc, suffix)

Output now looks like this

...snip...
    Microsoft Office Proofing Tools 2013 - Espa¤ol:
        15.0.4569.1506
..snip..
    Outils de vrification linguistique 2013 de Microsoft Officeÿ- Franais:
        15.0.4569.1506

@thatch45 thatch45 mentioned this issue Nov 21, 2014
Merged
@rallytime rallytime added the fixed-pls-verify fix is linked, bug author to confirm fix label Nov 21, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug broken, incorrect, or confusing behavior fixed-pls-verify fix is linked, bug author to confirm fix severity-low 4th level, cosemtic problems, work around exists
Projects
None yet
Milestone
Approved
Development

No branches or pull requests
4 participants
@thatch45 @fayetted @danielchristianschroeter @rallytime