TypeError: startswith first arg must be bytes or a tuple of bytes, not str

[chaosx]

python 3, winrm version 0.2

code

command = 'mkdir C:/a/b/c'
s = winrm.Session(hostname, auth=(username, password))
r = s.run_ps(command)

If command is 'ls C:/a/b' ,then success.

Traceback

Traceback (most recent call last):
  File "/usr/lib64/python3.4/site-packages/tornado/web.py", line 1443, in _execute
    result = method(*self.path_args, **self.path_kwargs)
  File "/g/ops_card-master/handler/business/script.py", line 40, in post
    result = self.exec_script(pk)
  File "/g/ops_card-master/handler/decorator.py", line 4, in wrapper
    result = func(*args, **kwargs)
  File "/g/ops_card-master/handler/business/script.py", line 136, in exec_script
    result = self.pool.starmap(self.exe, li)
  File "/usr/lib64/python3.4/multiprocessing/pool.py", line 268, in starmap
    return self._map_async(func, iterable, starmapstar, chunksize).get()
  File "/usr/lib64/python3.4/multiprocessing/pool.py", line 599, in get
    raise self._value
  File "/usr/lib64/python3.4/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/usr/lib64/python3.4/multiprocessing/pool.py", line 47, in starmapstar
    return list(itertools.starmap(args[0], args[1]))
  File "/g/ops_card-master/handler/decorator.py", line 18, in wrapper
    result = func(*args, **kwargs)
  File "/g/ops_card-master/handler/business/script.py", line 172, in exe
    r = s.run_ps(command)
  File "/usr/lib/python3.4/site-packages/pywinrm-0.2.0-py3.4.egg/winrm/__init__.py", line 54, in run_ps
    rs.std_err = self._clean_error_msg(rs.std_err)
  File "/usr/lib/python3.4/site-packages/pywinrm-0.2.0-py3.4.egg/winrm/__init__.py", line 62, in _clean_error_msg
    if msg.startswith("#< CLIXML\r\n"):
TypeError: startswith first arg must be bytes or a tuple of bytes, not str

[programmerchad]

I was able to modify /usr/local/lib/python3.5/dist-packages/winrm/init.py and make the following adjustment:

Before:
if msg.startswith("#< CLIXML\r\n"):

After:
if msg.startswith(b"#< CLIXML\r\n"):

[bikerider262]

I hit the same issue, but my tweak was a little different:

I am on python3.4 and version 0.2.0

import winrm

s = winrm.Session('hostname', auth=('administrator', 'password'))

command = """(Get-NetIPAddress).IPAddress | % { if ($_ -match "^10.") {write-host $_} }"""

r = s.run_ps(command)

print(r.std_out.decode('utf-8'))

Output:

...
...py_projects/winrm_ps_basic.py", line 7, in
r = s.run_ps(command)
File "/usr/local/lib/python3.4/site-packages/winrm/init.py", line 55, in run_ps
rs.std_err = self._clean_error_msg(rs.std_err)
File "/usr/local/lib/python3.4/site-packages/winrm/init.py", line 63, in _clean_error_msg
if msg.startswith("#< CLIXML\r\n"):
TypeError: startswith first arg must be bytes or a tuple of bytes, not str

Changing code as suggested above gave me this message:

Warning: there was a problem converting the Powershell error message: can't use a string pattern on a bytes-like object
192.168.33.32

Process finished with exit code 0

So... warning looked suspicious..

I removed the b

I changed the message before it went into def _clean_error_msg(self, msg):

I changed rs.std_err in def run_ps(self, script):

from:
rs.std_err = self._clean_error_msg(rs.std_err)
to:
rs.std_err = self._clean_error_msg(rs.std_err.decode('utf-8'))

I now have a clean output.

Hope this helps.

[almightyfoon]

I had the same result as bikerider262 and the same fix worked for me.

[JPvRiel]

If anyone is looking for a quick monkey patch based on @bikerider262's solution, this worked for me

def fix_run_ps(self, script):
    from base64 import b64encode
    encoded_ps = b64encode(script.encode('utf_16_le')).decode('ascii')
    rs = self.run_cmd('powershell -encodedcommand {0}'.format(encoded_ps))
    if len(rs.std_err):
        rs.std_err = self._clean_error_msg(rs.std_err.decode('utf-8'))
    return rs

winrm.Session.run_ps = fix_run_ps

[du5t1n55]

The fix above did clean things up so it would run but then I found that if there was an error returned from PS, I just get a nice exception message. Appears there is an underlying issue with what data type is being used.

Warning: there was a problem converting the Powershell error message: cannot use a bytes pattern on a string-like object

I was able to remove the exception to (maybe??) find where the true error is however I am not knowledgeable enough to resolve it.

File "C:\Python\Python36\lib\site-packages\winrm_init_.py", line 54, in run_ps
rs.std_err = self.clean_error_msg(rs.std_err)
File "C:\Python\Python36\lib\site-packages\winrm_init.py", line 74, in _clean_error_msg
root = ET.fromstring(msg_xml)
File "C:\Python\Python36\lib\xml\etree\ElementTree.py", line 1314, in XML
parser.feed(text)
TypeError: a bytes-like object is required, not 'NoneType'

after also modifying

p = re.compile('xmlns=[""][^\"\"][""]')
to
p = re.compile(b'xmlns=[""][^\"\"][""]')

In attempts to resolve

Warning: there was a problem converting the Powershell error message: cannot use a string pattern on a bytes-like object

Hopefully someone else understands how to properly clean up the data type in this scenario.

[sovetov]

@bikerider262 @JPvRiel @almightyfoon Why do you think that output is UTF-8? I'm sure that even default PowerShell output encoding is UTF-8. It works in this particular case just because your output is ASCII and it's subset of UTF-8, i.e. ASCII string is UTF-8 string. But reverse is not true at all. Would it work with Chinese, Greek or Cyrillic output?

As @programmerchad pointed out, correct way is to handle output as bytes simply using bytes type explicitly: b"...". Maybe, same fixed have to be done somewhere else. That works on Python 2.7 and Python 3 and very trivial.

[luc-alquier]

Hi everybody,

The issue is relative to the codepage you choose at the openshell invocation.
If you have choosen codepage 65001 (utf-8) like this:

protocol.open_shell(codepage=65001)

The correct test is:

      if msg.startswith(codecs.BOM_UTF8+"#< CLIXML"):
         # for proper xml, we need to remove the CLIXML part
         # (the first line)
         msg_xml = msg[(len(codecs.BOM_UTF8+"#< CLIXML")):]

And may be the codepage 65001 should the default and the only choice because you're sure of what you expect. So it works whether your error results contains chineese or armenian characters.

If you choose the codepage 437 which is the default, the original test works.
But if there is some non ascii char in the message it's the:

root = ET.fromstring(msg_xml)

which failed.

Sorry i've no time to do more investigations, if you do have any questions, do not hesitate.

PS: Once fix there is another issue for pretty error printing, it's the default shell width (80 chars).

[nitzmahone]

addressed via #222