Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

salt-minion broken after upgrade to 3000 #56121

Open
githubcdr opened this issue Feb 11, 2020 · 4 comments · May be fixed by #56143
Open

salt-minion broken after upgrade to 3000 #56121

githubcdr opened this issue Feb 11, 2020 · 4 comments · May be fixed by #56143

Comments

@githubcdr
Copy link
Contributor

@githubcdr githubcdr commented Feb 11, 2020

Description of Issue

Salt-minion is not functioning as expected after upgrading to Salt 3000

Setup

apt source:
deb http://repo.saltstack.com/py3/ubuntu/18.04/amd64/latest bionic main

Distributor ID: Ubuntu
Description: Ubuntu 18.04.4 LTS
Release: 18.04
Codename: bionic

Steps to Reproduce Issue

run salt-call test.version

Versions Report

/e/a/sources.list.d # ❯❯❯ salt-minion --versions
Salt Version:
Salt: 3000

Dependency Versions:
cffi: 1.12.3
cherrypy: Not Installed
dateutil: 2.6.1
docker-py: 4.0.2
gitdb: Not Installed
gitpython: Not Installed
Jinja2: 2.10
libgit2: Not Installed
M2Crypto: Not Installed
Mako: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.5.6
mysql-python: Not Installed
pycparser: 2.19
pycrypto: 2.6.1
pycryptodome: Not Installed
pygit2: Not Installed
Python: 3.6.9 (default, Nov 7 2019, 10:44:02)
python-gnupg: 0.4.1
PyYAML: 3.12
PyZMQ: 16.0.2
smmap: Not Installed
timelib: Not Installed
Tornado: 4.5.3
ZMQ: 4.2.5

System Versions:
dist: Ubuntu 18.04 bionic
locale: ANSI_X3.4-1968
machine: x86_64
release: 4.15.0-72-generic
system: Linux
version: Ubuntu 18.04 bionic

/e/a/sources.list.d # ❯❯❯ salt-call test.version
[ERROR   ] An un-handled exception was caught by salt's global exception handler:
TypeError: a bytes-like object is required, not 'str'
Traceback (most recent call last):
  File "/usr/bin/salt-call", line 11, in <module>
    load_entry_point('salt==3000', 'console_scripts', 'salt-call')()
  File "/usr/lib/python3/dist-packages/salt/scripts.py", line 445, in salt_call
    client.run()
  File "/usr/lib/python3/dist-packages/salt/cli/call.py", line 47, in run
    caller = salt.cli.caller.Caller.factory(self.config)
  File "/usr/lib/python3/dist-packages/salt/cli/caller.py", line 63, in factory
    return ZeroMQCaller(opts, **kwargs)
  File "/usr/lib/python3/dist-packages/salt/cli/caller.py", line 312, in __init__
    super(ZeroMQCaller, self).__init__(opts)
  File "/usr/lib/python3/dist-packages/salt/cli/caller.py", line 87, in __init__
    self.minion = salt.minion.SMinion(opts)
  File "/usr/lib/python3/dist-packages/salt/minion.py", line 868, in __init__
    salt.utils.yaml.safe_dump(cache_top, fp_)
  File "/usr/lib/python3/dist-packages/salt/utils/yamldumper.py", line 124, in safe_dump
    return yaml.dump(data, stream, Dumper=SafeOrderedDumper, **kwargs)
  File "/usr/lib/python3/dist-packages/yaml/__init__.py", line 200, in dump
    return dump_all([data], stream, Dumper=Dumper, **kwds)
  File "/usr/lib/python3/dist-packages/yaml/__init__.py", line 188, in dump_all
    dumper.represent(data)
  File "/usr/lib/python3/dist-packages/yaml/representer.py", line 27, in represent
    self.serialize(node)
  File "ext/_yaml.pyx", line 1350, in _yaml.CEmitter.serialize (ext/_yaml.c:19880)
  File "ext/_yaml.pyx", line 1525, in _yaml.output_handler (ext/_yaml.c:22762)
TypeError: a bytes-like object is required, not 'str'
Traceback (most recent call last):
  File "/usr/bin/salt-call", line 11, in <module>
    load_entry_point('salt==3000', 'console_scripts', 'salt-call')()
  File "/usr/lib/python3/dist-packages/salt/scripts.py", line 445, in salt_call
    client.run()
  File "/usr/lib/python3/dist-packages/salt/cli/call.py", line 47, in run
    caller = salt.cli.caller.Caller.factory(self.config)
  File "/usr/lib/python3/dist-packages/salt/cli/caller.py", line 63, in factory
    return ZeroMQCaller(opts, **kwargs)
  File "/usr/lib/python3/dist-packages/salt/cli/caller.py", line 312, in __init__
    super(ZeroMQCaller, self).__init__(opts)
  File "/usr/lib/python3/dist-packages/salt/cli/caller.py", line 87, in __init__
    self.minion = salt.minion.SMinion(opts)
  File "/usr/lib/python3/dist-packages/salt/minion.py", line 868, in __init__
    salt.utils.yaml.safe_dump(cache_top, fp_)
  File "/usr/lib/python3/dist-packages/salt/utils/yamldumper.py", line 124, in safe_dump
    return yaml.dump(data, stream, Dumper=SafeOrderedDumper, **kwargs)
  File "/usr/lib/python3/dist-packages/yaml/__init__.py", line 200, in dump
    return dump_all([data], stream, Dumper=Dumper, **kwds)
  File "/usr/lib/python3/dist-packages/yaml/__init__.py", line 188, in dump_all
    dumper.represent(data)
  File "/usr/lib/python3/dist-packages/yaml/representer.py", line 27, in represent
    self.serialize(node)
  File "ext/_yaml.pyx", line 1350, in _yaml.CEmitter.serialize (ext/_yaml.c:19880)
  File "ext/_yaml.pyx", line 1525, in _yaml.output_handler (ext/_yaml.c:22762)
TypeError: a bytes-like object is required, not 'str'

All dependencies are provided upstream.

@DmitryKuzmenko

This comment has been minimized.

Copy link
Contributor

@DmitryKuzmenko DmitryKuzmenko commented Feb 11, 2020

Reproduced on python 3.7. The issue is actual for all the currently supported salt branches.
Steps:

  • Use Python 3.
  • Set minion_pillar_cache: True in minion config file and ensure file_client is either unset or set to remote.
  • Start salt-master.
  • Run salt-call test.version.
    The reason is SMinion.__init__ tries to yaml-dump a dictionary with string data to a cache file opened in a binary mode.
@OrangeDog

This comment has been minimized.

Copy link
Contributor

@OrangeDog OrangeDog commented Feb 11, 2020

Looks like #51854

@dwoz

This comment has been minimized.

Copy link
Contributor

@dwoz dwoz commented Feb 11, 2020

Yes, looks like we need to port this to the master branch #52195

@max-arnold

This comment has been minimized.

Copy link
Contributor

@max-arnold max-arnold commented Feb 12, 2020

A quick & dirty fix:

patches/pillar_cache.sls:

patch:
  pkg.installed

pillar_cache_patch:
  file.patch:
    - name: '{{ grains.saltpath }}'
    - source: salt://patches/files/pillar-cache.diff
    - strip: 2
    - require:
        - pkg: patch

restart_salt_minion:
  cmd.run:
    - name: 'salt-call service.restart salt-minion'
    - bg: true
    - onchanges:
      - file: pillar_cache_patch

patches/files/pillar-cache.diff:

diff --git a/salt/minion.py b/salt/minion.py
index 6a77d90185..10d727aebe 100644
--- a/salt/minion.py
+++ b/salt/minion.py
@@ -27,6 +27,7 @@ from salt.ext import six
 from salt._compat import ipaddress
 from salt.utils.network import parse_host_port
 from salt.ext.six.moves import range
+from salt.template import SLS_ENCODING
 from salt.utils.zeromq import zmq, ZMQDefaultLoop, install_zmq, ZMQ_VERSION_INFO
 import salt.transport.client
 import salt.defaults.exitcodes
@@ -865,11 +866,11 @@ class SMinion(MinionBase):
                 penv = 'base'
             cache_top = {penv: {self.opts['id']: ['cache']}}
             with salt.utils.files.fopen(ptop, 'wb') as fp_:
-                salt.utils.yaml.safe_dump(cache_top, fp_)
+                salt.utils.yaml.safe_dump(cache_top, fp_, encoding=SLS_ENCODING)
                 os.chmod(ptop, 0o600)
             cache_sls = os.path.join(pdir, 'cache.sls')
             with salt.utils.files.fopen(cache_sls, 'wb') as fp_:
-                salt.utils.yaml.safe_dump(self.opts['pillar'], fp_)
+                salt.utils.yaml.safe_dump(self.opts['pillar'], fp_, encoding=SLS_ENCODING)
                 os.chmod(cache_sls, 0o600)
 
 

Run sudo salt MINION state.apply patches.pillar_cache to apply the fix.

@waynew waynew added the Confirmed label Feb 12, 2020
@waynew waynew linked a pull request that will close this issue Feb 12, 2020
@sagetherage sagetherage added the 3000.1 label Feb 20, 2020
@sagetherage sagetherage added this to Planning in 3000.1 Release Feb 20, 2020
@sagetherage sagetherage moved this from Planning to Commit in 3000.1 Release Feb 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Linked pull requests

Successfully merging a pull request may close this issue.

7 participants
You can’t perform that action at this time.