New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't run cmd.run with UTF-8 chars as arguments / parameters. With custom module. #48880

Open
damntoken opened this Issue Aug 2, 2018 · 6 comments

Comments

Projects
None yet
5 participants
@damntoken

damntoken commented Aug 2, 2018

Description of Issue/Question

I'm trying to create a custom module because standard ones are not sufficient enought for managaging certain Windows management tastks.

But I can't run custom execution module to execute powershell with any Õ Ä Ö Ü character.

First I googled pyton ascii unicode a lot and tried unicode() encode() decode(), because I used str(name) and other variable verifications, but this has been futile because the last stripped down simple example showed me that the error is somewhere further than in my custom sls file.

Setup

Steps to Reproduce Issue

Sample file /srv/salt/_modules/str_aduser.py

# -*- coding: utf-8 -*-
from __future__ import absolute_import
import logging
LOG = logging.getLogger(__name__)
__virtualname__ = 'str_aduser'

def __virtual__():
	# # Determine whether or not to load this module
	if __salt__['grains.equals']('osfinger','Windows-2016Server'):
		return __virtualname__
	else:
		LOG.error('osfinger: '+ __salt__['grains.get']('osfinger') +' not Windows-2016Server')
	return False

def test(name):
	runtest=__salt__['cmd.run_all'](name,shell='powershell')
	#runtest=__salt__['cmd.powershell_all'](name)
	ret = {
		'runtest retrun': str(runtest['retcode']) +'|'+ runtest['stderr'] +'|'+ runtest['stdout'],
		}
	return ret

Run with ASCII chars only..
salt 'abelia' saltutil.sync_all; salt 'abelia' str_aduser.test 'echo ABC > C:\salt\var\log\salt\test.txt'
Result: runtest retrun: 0|| , Result file content: ABC
Run with UTF chars also..
salt 'abelia' saltutil.sync_all; salt 'abelia' str_aduser.test 'echo ABCÄÖÜ > C:\salt\var\log\salt\test.txt'
Result:

    The minion function caused an exception: Traceback (most recent call last):
      File "c:\salt\bin\lib\site-packages\salt\minion.py", line 1606, in _thread_return
        return_data = minion_instance.executors[fname](opts, data, func, args, kwargs)
      File "c:\salt\bin\lib\site-packages\salt\executors\direct_call.py", line 12, in execute
        return func(*args, **kwargs)
      File "c:\salt\var\cache\salt\minion\extmods\modules\str_aduser.py", line 93, in test
        runtest=__salt__['cmd.powershell_all'](name)
      File "c:\salt\bin\lib\site-packages\salt\modules\cmdmod.py", line 3524, in powershell_all
        **kwargs)
      File "c:\salt\bin\lib\site-packages\salt\modules\cmdmod.py", line 1834, in run_all
        **kwargs)
      File "c:\salt\bin\lib\site-packages\salt\modules\cmdmod.py", line 582, in _run
        proc = salt.utils.timed_subprocess.TimedProc(cmd, **kwargs)
      File "c:\salt\bin\lib\site-packages\salt\utils\timed_subprocess.py", line 45, in __init__
        self.process = subprocess.Popen(args, **kwargs)
      File "c:\salt\bin\lib\subprocess.py", line 390, in __init__
        errread, errwrite)
      File "c:\salt\bin\lib\subprocess.py", line 610, in _execute_child
        args = '{} /c "{}"'.format (comspec, args)
    UnicodeEncodeError: 'ascii' codec can't encode characters in position 64-66: ordinal not in range(128)

Versions Report

Salt Version:
Salt: 2018.3.2 (Salt client is 2018.3.1 - Win2016 updated.)
Dependency Versions:
dateutil: 2.4.2
Jinja2: 2.8
msgpack-python: 0.4.6
pycrypto: 2.6.1
Python: 2.7.12 (default, Dec 4 2017, 14:50:18)
python-gnupg: 0.3.8
PyYAML: 3.11
PyZMQ: 15.2.0
Tornado: 4.2.1
ZMQ: 4.1.4
System Versions:
locale: UTF-8
machine: x86_64
version: Ubuntu 16.04 xenial

@Ch3LL

This comment has been minimized.

Contributor

Ch3LL commented Aug 3, 2018

ping @saltstack/team-windows any ideas here?

I can replicate with jsut this command: salt-call --local cmd.run 'echo ABCÄÖÜ'

This command works fine on linux but not windows

@Ch3LL Ch3LL added this to the Blocked milestone Aug 3, 2018

@twangboy

This comment has been minimized.

Contributor

twangboy commented Aug 3, 2018

This is an issue with python (especially py2) when shelling out to cmd or powershell. Python uses utf-8 where cmd is cp437 in English versions of Windows. We transitioned to using the PyWin32 modules for adding users in the win_useradd.py module. I have had no problems adding users with Unicode characters. Would you mind explaining exactly what you're trying to do and how you've been going about it?

@damon-atkins

This comment has been minimized.

Member

damon-atkins commented Aug 3, 2018

You need Salt-Minion to ship with Python 3.6 for Windows Unicode to work 100% correctly. It contains a lot of fixes which date back to python 2.

@twangboy

This comment has been minimized.

Contributor

twangboy commented Aug 3, 2018

@dwoz Didn't you do some testing with codepage stuff on Py3.6/3.7? Were there still some issues? Or are they fully resolved?

@damon-atkins

This comment has been minimized.

Member

damon-atkins commented Aug 3, 2018

Windows PY3 resolved issues outstanding for PY2, non-salt #36031

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment