New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mine.get not returning data in a state.orchestrate sls #48020

Closed
calvinhp opened this Issue Jun 7, 2018 · 5 comments

Comments

Projects
None yet
5 participants
@calvinhp

calvinhp commented Jun 7, 2018

Description of Issue/Question

mine.get is not working correctly from inside an orchestration.

Setup

Add a mine_function to a pillar to return network.ip_addrs

Create an orch.sls with the following contents

{% set app_servers = salt.saltutil.runner('mine.get',
        tgt='*',
        fun='network.ip_addrs') %}

{% for app_server in app_servers.keys() %}

test-{{ app_server }}:
  salt.function:
    - tgt: {{ app_server }}
    - fun: test.ping

{% endfor %}

Steps to Reproduce Issue

When running the following, app_servers will be an empty dictionary

$ sudo salt-run state.orchestrate orch

but if I run the same function via saltutil.runner it returns as I expect

$ sudo salt-call saltutil.runner mine.get tgt='*' fun='network.ip_addrs'

gives me this output

local:
    ----------
    minion01:
        - 10.192.2.58
    minion02:
        - 10.192.2.59

Versions Report

Salt Version:
           Salt: 2018.3.0

Dependency Versions:
           cffi: Not Installed
       cherrypy: 3.5.0
       dateutil: 2.4.2
      docker-py: Not Installed
          gitdb: 0.6.4
      gitpython: 1.0.1
          ioflo: Not Installed
         Jinja2: 2.8
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: 1.0.3
   msgpack-pure: Not Installed
 msgpack-python: 0.4.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.12 (default, Nov 20 2017, 18:23:56)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 15.2.0
           RAET: Not Installed
          smmap: 0.9.0
        timelib: 0.2.4
        Tornado: 4.2.1
            ZMQ: 4.1.4

System Versions:
           dist: Ubuntu 16.04 xenial
         locale: UTF-8
        machine: x86_64
        release: 4.4.0-97-generic
         system: Linux
        version: Ubuntu 16.04 xenial
@Ch3LL

This comment has been minimized.

Contributor

Ch3LL commented Jun 8, 2018

was this working previously? are you just seeing it in 2018.3.0?

@Ch3LL Ch3LL added the Info Needed label Jun 8, 2018

@Ch3LL Ch3LL added this to the Blocked milestone Jun 8, 2018

@robertpenberthy

This comment has been minimized.

Contributor

robertpenberthy commented Jun 18, 2018

@Ch3LL Since he hasn't responded and I have just found this same issue after upgrading to 2018.3.1 on Friday, I'll add some additional information.

I find that I cannot get mine data from the master with salt-run commands. This is a problem as any mine data you want to use to control the flow of your orchestration or pillar will fail because they're rendered on the master (which is why saltutil.runner is used).

I upgraded from 2017.7.0 to 2018.3.1. This functionality has worked for us since 2016.3.3 and now doesn't work in 2018.3.1. This currently works in our 2017.7.0 production environments, but doesn't work in my test environment that I just upgraded to 2018.3.1.

salt-run mine.get '*' <mine function>

@marceliq

This comment has been minimized.

marceliq commented Jun 19, 2018

It does not work for me either. Same behavior on 2018.3.1.

edit:
It looks like it works if I'm using tgt_type='pillar' instead of 'glob'

@Ch3LL

This comment has been minimized.

Contributor

Ch3LL commented Jun 21, 2018

thanks for clarifying the versions it was working in that helped me bisect the issue. The bisect showed this commit: bc30796 which is this pr: #44890

particularly i found its the diff of this file that created the problem:

diff --git a/salt/utils/minions.py b/salt/utils/minions.py
index 2049f5549a27..f0e18625016c 100644
--- a/salt/utils/minions.py
+++ b/salt/utils/minions.py
@@ -228,6 +228,10 @@ def _pki_minions(self):
         Retreive complete minion list from PKI dir.
         Respects cache if configured
         '''
+        if self.opts.get('__role') == 'master':
+            # Compiling pillar directly on the master, just return the master's
+            # ID as that is the only one that is available.
+            return [self.opts['id']]
         minions = []
         pki_cache_fn = os.path.join(self.opts['pki_dir'], self.acc, '.key_cache')
         try:
@@ -241,7 +245,10 @@ def _pki_minions(self):
                         minions.append(fn_)
             return minions
         except OSError as exc:
-            log.error('Encountered OSError while evaluating  minions in PKI dir: {0}'.format(exc))
+            log.error(
+                'Encountered OSError while evaluating minions in PKI dir: %s',
+                exc
+            )
             return minions
 
     def _check_cache_minions(self,

I created a docker container to quickly replicate this:

  1. docker run -it -v ~/git/salt/:/testing/ ch3ll/issues:48020 (where ~/git/salt is a local cloned git repo of salt)
  2. bash -x ~/run.sh

ping @terminalmage

@terminalmage

This comment has been minimized.

Member

terminalmage commented Aug 7, 2018

This was fixed in #48765.

kiemlicz added a commit to kiemlicz/envoy that referenced this issue Oct 24, 2018

porting fix for
saltstack/salt#48020
Otherwise mine.get from orchestrate will not work

kiemlicz added a commit to kiemlicz/envoy that referenced this issue Oct 24, 2018

Revert "porting fix for saltstack/salt#48020 Otherwise mine.get from …
…orchestrate will not work"

This reverts commit 30e03dc
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment