runners when executed in jinja templates should not print out to stdout #28354

arthurzenika · 2015-10-28T15:04:56Z

I have some jinja templates that run some salt runners like this :

[snip]
{% set known_hosts = salt['saltutil.runner']('manage.status')['up'] %}
{%- for host,status in salt['saltutil.runner']('virt.list').items() %}
[snip]

This ends up spewing out a bunch of output when being rendered. It would be nice if was quiet.

# salt-call --versions-report
                  Salt: 2015.5.3
                Python: 2.7.9 (default, Mar  1 2015, 12:57:24)
                Jinja2: 2.7.3
              M2Crypto: 0.21.1
        msgpack-python: 0.4.2
          msgpack-pure: Not Installed
              pycrypto: 2.6.1
               libnacl: Not Installed
                PyYAML: 3.11
                 ioflo: Not Installed
                 PyZMQ: 14.4.0
                  RAET: Not Installed
                   ZMQ: 4.0.5
                  Mako: 1.0.0
               Tornado: Not Installed
 Debian source package: 2015.5.3+ds-1~bpo8+1

jfindlay · 2015-10-29T22:55:29Z

@arthurlogilab, thanks for the report.

gmolight · 2015-11-19T22:46:23Z

👍 This is happening to me when using saltutil.runner under orchestrator. Can you hint where this can be potentially happening. I am will to take a look and see if I can propose a PR

jfindlay · 2015-11-23T19:54:20Z

@gmolight, I am not sure where to look, but I would probably start with salt/runner.py.

gmolight · 2015-11-23T20:22:33Z

thanks, I will take a look if I find something I will post here and if a solution I will send PR

multani · 2015-12-16T16:54:51Z

FYI, here is the stacktrace after I traced the problem:

(Pdb) bt full
  /usr/bin/salt-run(10)<module>()
-> salt_run()
  /usr/lib/python2.7/dist-packages/salt/scripts.py(354)salt_run()
-> client.run()
  /usr/lib/python2.7/dist-packages/salt/cli/run.py(41)run()
-> runner.run()
  /usr/lib/python2.7/dist-packages/salt/runner.py(181)run()
-> False)  # Don't daemonize
  /usr/lib/python2.7/dist-packages/salt/client/mixins.py(408)_proc_function()
-> return self.low(fun, low)
  /usr/lib/python2.7/dist-packages/salt/client/mixins.py(289)low()
-> _functions = copy.deepcopy(self.functions)
  /usr/lib/python2.7/dist-packages/salt/runners/state.py(59)orchestrate()
-> pillar=pillar)
  /usr/lib/python2.7/dist-packages/salt/modules/state.py(699)sls()
-> st_ = salt.state.HighState(opts, pillar, kwargs.get('__pub_jid'), proxy=__proxy__)
  /usr/lib/python2.7/dist-packages/salt/state.py(3170)__init__()
-> self.state = State(self.opts, pillar, jid, proxy=proxy)
  /usr/lib/python2.7/dist-packages/salt/state.py(604)__init__()
-> self.opts['pillar'] = self._gather_pillar()
  /usr/lib/python2.7/dist-packages/salt/state.py(626)_gather_pillar()
-> ret = pillar.compile_pillar()
  /usr/lib/python2.7/dist-packages/salt/pillar/__init__.py(678)compile_pillar()
-> pillar, errors = self.render_pillar(matches)
  /usr/lib/python2.7/dist-packages/salt/pillar/__init__.py(551)render_pillar()
-> pstate, mods, err = self.render_pstate(sls, saltenv, mods)
  /usr/lib/python2.7/dist-packages/salt/pillar/__init__.py(486)render_pstate()
-> fn_, self.rend, self.opts['renderer'], saltenv, sls, _pillar_rend=True, **defaults)
  /usr/lib/python2.7/dist-packages/salt/template.py(95)compile_template()
-> ret = render(input_data, saltenv, sls, **render_kwargs)
  /usr/lib/python2.7/dist-packages/salt/renderers/jinja.py(333)render()
-> **kws)
  /usr/lib/python2.7/dist-packages/salt/utils/templates.py(180)render_tmpl()
-> output = render_str(tmplstr, context, tmplpath)
  /usr/lib/python2.7/dist-packages/salt/utils/templates.py(368)render_jinja_tmpl()
-> output = template.render(**decoded_context)
  /usr/lib/python2.7/dist-packages/jinja2/environment.py(966)render()
-> return concat(self.root_render_func(self.new_context(vars)))
  <template>(9)root()
  /usr/lib/python2.7/dist-packages/jinja2/runtime.py(193)call()
-> return __obj(*args, **kwargs)
  /usr/lib/python2.7/dist-packages/salt/modules/saltutil.py(952)runner()
-> return rclient.cmd(_fun, kwarg=kwargs)
  /usr/lib/python2.7/dist-packages/salt/client/mixins.py(223)cmd()
-> return self.low(fun, low)
  /usr/lib/python2.7/dist-packages/salt/client/mixins.py(289)low()
-> _functions = copy.deepcopy(self.functions)
  /usr/lib/python2.7/dist-packages/salt/utils/event.py(700)fire_event()
-> self.print_func(tag, data)
  /usr/lib/python2.7/dist-packages/salt/client/mixins.py(483)print_async_event()
-> salt.output.display_output(event_data, outputter, self.opts)
  /usr/lib/python2.7/dist-packages/salt/output/__init__.py(106)display_output()
-> print_cli(display_data)
> /usr/lib/python2.7/dist-packages/salt/utils/__init__.py(1919)print_cli()
-> try:

I'm not sure I understand everything yet but it's seems there's an event triggered after the completion of the saltutil.runner function with the event content set to the function result, and this content gets displayed during the print_async_event, probably because self.opts.get('quiet') of the AsyncClientMixin returns False.

Another stacktrace, maybe more useful:

> /usr/lib/python2.7/dist-packages/salt/client/mixins.py(458)print_async_event()
-> if self.opts.get('quiet', False):
(Pdb) bt
  /usr/bin/salt-run(10)<module>()
-> salt_run()
  /usr/lib/python2.7/dist-packages/salt/scripts.py(354)salt_run()
-> client.run()
  /usr/lib/python2.7/dist-packages/salt/cli/run.py(41)run()
-> runner.run()
  /usr/lib/python2.7/dist-packages/salt/runner.py(181)run()
-> False)  # Don't daemonize
  /usr/lib/python2.7/dist-packages/salt/client/mixins.py(407)_proc_function()
-> return self.low(fun, low)
  /usr/lib/python2.7/dist-packages/salt/client/mixins.py(337)low()
-> data['return'] = self.functions[fun](*args, **kwargs)
  /usr/lib/python2.7/dist-packages/salt/runners/state.py(59)orchestrate()
-> pillar=pillar)
  /usr/lib/python2.7/dist-packages/salt/modules/state.py(699)sls()
-> st_ = salt.state.HighState(opts, pillar, kwargs.get('__pub_jid'), proxy=__proxy__)
  /usr/lib/python2.7/dist-packages/salt/state.py(3170)__init__()
-> self.state = State(self.opts, pillar, jid, proxy=proxy)
  /usr/lib/python2.7/dist-packages/salt/state.py(604)__init__()
-> self.opts['pillar'] = self._gather_pillar()
  /usr/lib/python2.7/dist-packages/salt/state.py(626)_gather_pillar()
-> ret = pillar.compile_pillar()
  /usr/lib/python2.7/dist-packages/salt/pillar/__init__.py(678)compile_pillar()
-> pillar, errors = self.render_pillar(matches)
  /usr/lib/python2.7/dist-packages/salt/pillar/__init__.py(551)render_pillar()
-> pstate, mods, err = self.render_pstate(sls, saltenv, mods)
  /usr/lib/python2.7/dist-packages/salt/pillar/__init__.py(486)render_pstate()
-> fn_, self.rend, self.opts['renderer'], saltenv, sls, _pillar_rend=True, **defaults)
  /usr/lib/python2.7/dist-packages/salt/template.py(95)compile_template()
-> ret = render(input_data, saltenv, sls, **render_kwargs)
  /usr/lib/python2.7/dist-packages/salt/renderers/jinja.py(333)render()
-> **kws)
  /usr/lib/python2.7/dist-packages/salt/utils/templates.py(180)render_tmpl()
-> output = render_str(tmplstr, context, tmplpath)
  /usr/lib/python2.7/dist-packages/salt/utils/templates.py(368)render_jinja_tmpl()
-> output = template.render(**decoded_context)
  /usr/lib/python2.7/dist-packages/jinja2/environment.py(966)render()
-> return concat(self.root_render_func(self.new_context(vars)))
  <template>(9)root()
  /usr/lib/python2.7/dist-packages/jinja2/runtime.py(193)call()
-> return __obj(*args, **kwargs)
  /usr/lib/python2.7/dist-packages/salt/modules/saltutil.py(952)runner()
-> return rclient.cmd(_fun, kwarg=kwargs)
  /usr/lib/python2.7/dist-packages/salt/client/mixins.py(223)cmd()
-> return self.low(fun, low)
  /usr/lib/python2.7/dist-packages/salt/client/mixins.py(350)low()
-> namespaced_event.fire_event(data, 'ret')
  /usr/lib/python2.7/dist-packages/salt/utils/event.py(700)fire_event()
-> self.print_func(tag, data)
> /usr/lib/python2.7/dist-packages/salt/client/mixins.py(458)print_async_event()
-> if self.opts.get('quiet', False):
(Pdb) self.opts.get('quiet')
False
(Pdb) c

I'm using a configuration where I have a pillar filled with the result of a salt['saltutil.runner'], and I got this result during a orchestration run.

If I set quiet: True in the master's configuration file, I don't get the output of salt['saltutil.runner'], but then I don't get any output (at all).

multani · 2015-12-21T16:34:44Z

Another comment: the extra output I get when running an orchestration seems to come from the fact that I'm using salt['saltutil.runner']('mine.get', ...) which produces an event (the above function print_async_event() in salt/client/mixins.py gets called with an event with event['fun'] == 'runner.mine.get') which results get printed on stdout.

I still don't understand how everything fits together...

multani · 2016-02-08T10:03:43Z

I'm still facing this issue on a daily basis.

More info though:

the salt['saltutil.runner']('mine.get', ...) call that I'm using and which is producing the undesirable output is inside a pillar file;
this pillar file is sourced by base's '*' in my pillar top file:

base:
  '*':
    - file_that_calls_saltutil_dot_runner

if I move this include statement out of the '*' scope, I don't get the undesirable output anymore.

stale · 2018-03-19T10:31:01Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

jfindlay added Bug broken, incorrect, or confusing behavior severity-low 4th level, cosemtic problems, work around exists P3 Priority 3 Runners Renderers labels Oct 29, 2015

jfindlay added this to the Approved milestone Oct 29, 2015

jfindlay added severity-medium 3rd level, incorrect or bad functionality, confusing and lacks a work around Core relates to code central or existential to Salt and removed severity-low 4th level, cosemtic problems, work around exists labels Nov 23, 2015

jfindlay added Renderers and removed Renderers labels Nov 23, 2015

cro mentioned this issue Mar 1, 2016

Change the undocumented 'quiet' master opt to 'debug_async_events_to_console' #31571

Closed

2 tasks

stale bot added the stale label Mar 19, 2018

stale bot closed this as completed Mar 26, 2018

nicholasmhughes mentioned this issue Oct 25, 2023

[BUG] salt-master wastes time writing event data to standard out when executing runner functions through the api #58203

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

runners when executed in jinja templates should not print out to stdout #28354

runners when executed in jinja templates should not print out to stdout #28354

arthurzenika commented Oct 28, 2015

jfindlay commented Oct 29, 2015

gmolight commented Nov 19, 2015

jfindlay commented Nov 23, 2015

gmolight commented Nov 23, 2015

multani commented Dec 16, 2015

multani commented Dec 21, 2015

multani commented Feb 8, 2016

stale bot commented Mar 19, 2018

runners when executed in jinja templates should not print out to stdout #28354

runners when executed in jinja templates should not print out to stdout #28354

Comments

arthurzenika commented Oct 28, 2015

jfindlay commented Oct 29, 2015

gmolight commented Nov 19, 2015

jfindlay commented Nov 23, 2015

gmolight commented Nov 23, 2015

multani commented Dec 16, 2015

multani commented Dec 21, 2015

multani commented Feb 8, 2016

stale bot commented Mar 19, 2018