Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

salt.match.search_by resulting in 'None' since 2019.2 #51865

Closed
magenbrot opened this issue Feb 27, 2019 · 4 comments

Comments

@magenbrot
Copy link

commented Feb 27, 2019

Description of Issue/Question

I'm using a setup as described in #39457 (comment) to apply roles to my minions. This worked fine until I upgraded to 2019.02.

It seems like the match.search_by isn't working correctly, or maybe it's something I've missed in the release notes. Here's what I have (I've added log.debug for some variables):

Setup

pillar/roles.jinja

{% load_yaml as role_defs %}
salt-master:
  - balder.mydomain.com

cloud:
  - balder.mydomain.com
  - heimdall.mydomain.com
  - loki.mydomain.com

weechat:
  - balder.mydomain.com
{% endload %}

{% set roles = salt.match.search_by(role_defs) %}

# testing
{% set test_roles = role_defs | yaml() %}
{%- do salt.log.debug('MYDEBUG-role_defs: ' ~ role_defs) %}
{%- do salt.log.debug('MYDEBUG-test: ' ~ test_roles) %}
{%- do salt.log.debug('MYDEBUG-roles: ' ~ roles) %}

pillar/roles.sls

{% from "roles.jinja" import roles with context %}
roles: {{ roles | yaml() }}

pillar/top.sls

{% from "roles.jinja" import roles with context %}

base:
  '*':
    - roles
    - users

Steps to Reproduce Issue

Setup like above and run "salt 'balder*' pillar.items roles". roles pillar is empty, but should contain this minions roles instead:

balder.mydomain.com:
    ----------
    retcode:
        0
    roles:
        None
jid:
    20190227143040611412
retcode:
    0

This is the "extra" debug output:

2019-02-27 14:22:24,042 [salt.loaded.int.module.logmod:41  ][DEBUG   ][326] MYDEBUG-role_defs: {u'weechat': [u'balder.mydomain.com'], u'salt-master': [u'balder.mydomain.com'], u'cloud': [u'balder.mydomain.com', u'heimdall.mydomain.com', u'loki.mydomain.com']}
2019-02-27 14:22:24,042 [salt.loaded.int.module.logmod:41  ][DEBUG   ][326] MYDEBUG-test: {cloud: [balder.mydomain.com, heimdall.mydomain.com, loki.mydomain.com], salt-master: [balder.mydomain.com],
2019-02-27 14:22:24,042 [salt.loaded.int.module.logmod:41  ][DEBUG   ][326] MYDEBUG-roles: None

So role_defs is populated (MYDEBUG-role_defs) and it can be rendered as yaml (MYDEBUG-test). However roles get 'Null' as it seems that {% set roles = salt.match.search_by(role_defs) %} is not working.

Versions Report

Salt Version:
           Salt: 2019.2.0
 
Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.5.3
      docker-py: 3.2.1
          gitdb: 2.0.0
      gitpython: 2.1.1
          ioflo: Not Installed
         Jinja2: 2.9.4
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.8
   mysql-python: 1.3.7
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: 3.6.1
         pygit2: Not Installed
         Python: 2.7.13 (default, Sep 26 2018, 18:42:22)
   python-gnupg: Not Installed
         PyYAML: 3.12
          PyZMQ: 16.0.2
           RAET: Not Installed
          smmap: 2.0.1
        timelib: Not Installed
        Tornado: 4.4.3
            ZMQ: 4.2.1
 
System Versions:
           dist: debian 9.8 
         locale: UTF-8
        machine: x86_64
        release: 4.19.0-0.bpo.2-amd64
         system: Linux
        version: debian 9.8 

Master and minions have the same version.

@garethgreenaway

This comment has been minimized.

Copy link
Member

commented Mar 19, 2019

@magenbrot Apologies for the delay in responding. I think I've got this one figured out. Can you confirm that you're only seeing this behavior when targeting the instance that is your Salt master? Thanks!

@magenbrot

This comment has been minimized.

Copy link
Author

commented Mar 19, 2019

@garethgreenaway no problem, thanks for the feedback! However the problem also occurs when running state.apply on other targets than my salt master. The same when I use salt-call state.apply from one of my minions.

This is the error I get (roles would be normally populated by match.search_by):

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/salt/utils/templates.py", line 392, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "<template>", line 14, in top-level template code
TypeError: argument of type 'NoneType' is not iterable

[DEBUG   ] TraceStr: Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/salt/utils/templates.py", line 392, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "<template>", line 14, in top-level template code
TypeError: argument of type 'NoneType' is not iterable

[ERROR   ] Rendering exception occurred
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/salt/utils/templates.py", line 169, in render_tmpl
    output = render_str(tmplstr, context, tmplpath)
  File "/usr/lib/python2.7/dist-packages/salt/utils/templates.py", line 441, in render_jinja_tmpl
    trace=tracestr)
SaltRenderError: Jinja error: argument of type 'NoneType' is not iterable
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/salt/utils/templates.py", line 392, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "<template>", line 14, in top-level template code
TypeError: argument of type 'NoneType' is not iterable

; line 14

---
[...]
    - sudoers
    - sudoers.included
    - sysctl
    - users

    {% if 'container' not in salt['pillar.get']('roles', []) %}    <======================
    - chrony
    - chrony.config
    {% endif %}

    {% if 'server' in salt['pillar.get']('roles', []) %}
[...]
@garethgreenaway

This comment has been minimized.

Copy link
Member

commented Mar 19, 2019

@magenbrot Thanks for that additional failure, I was able to reproduce that as well. I tested both cases with the change from pull request #52234 and it's functioning as expected. Are you able to test that PR as well and see if it fixes the issues you're seeing?

@magenbrot

This comment has been minimized.

Copy link
Author

commented Mar 20, 2019

@garethgreenaway seems to work. my roles pillar gets populated again. thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Salt Open Workspace
  
In progress
3 participants
You can’t perform that action at this time.