RFC: Returners that actually return to the master

[kaithar]

I've been toying with this idea for a while but I'm not sure how best to accomplish it.

At the moment, I can only see ways to get the returners run on the minions regardless of where the instruction was issued from. This has many problems, including:

• the need to expose connection details for reporting services to all minions
• having to deal with services that normally like to transmit in clear
• negotiating firewalls and NAT on both ends
• needing the minion to have the libraries needed to talk to the reporting service

If you're controlling a location diverse network and want to report data to a mysql or redis instance, you've basically got to either trust those minions with access and secure the connection ... or you've got to write a gateway service that mediates access and performs some kind of authentication.

Since a large part of the point of Salt's communications system is to mediate access, authenticate sources and provide secure transmission across potentially hostile networks, it seems horribly wrong that it doesn't support master side returners.

Have I missed some sort of official way for getting return behaviour on the master?
Is this an extension of Salt that others would want to see?
What might be a good way to implement it?

Lacking this is actually causing me issues in my roll out plan, I'd rather have a clean solution instead of hacking around it by quickly and repeatedly polling salt's job states.
I am, as usual, willing to put my coding where my mouth is on the issue and work on a PR... but in this instance I feel some discussion on mechanics before would be sensible.

Thoughts?

[techhat]

This can be a long discussion. Let's start by looking at the reasons why returners were added in the first place (ignoring for the moment the fact that things have changed a lot in the almost 3 years since).

Some of the first modules added were used for reporting: disk, status, etc. The idea was to send out a message to query vitals on the minions, then store the return data in a database. But why force the master to handle all that load? Why not just have the minion put the data there itself?

Things have changed a lot since then. States were added, and grains, pillars, and well, returners have been around for longer than almost anything else. Another feature that has been added since then is --out, which can be set to JSON, which is great for master-side data handling. Or you can just bypass the command line altogether and use the LocalClient interface to make the calls directly.

Some of the runners have some great examples of LocalClient usages. In fact, runners were designed to be command aggregators, which have the ability to handle targeting on their own. It shouldn't be hard to write a runner that accepts targets on the command line, but generally when people write a custom runner, they have a specific need that negates that need.

I know this doesn't lead you to having the salt command behave like a master-side returner, but hopefully it gives you an idea of how returners got to where they are, and gets you closer to implementing a temporary solution for the moment.

[kaithar]

States were added, and grains, pillars, and well, returners have been around for longer than almost anything else.

Legacy code... Woo!

At the moment I'm heavily wrapping LocalClient, but dumping output and parsing it in to the database feels like it leads to a large duplication of effort.
I started looking in to this specifically because I wanted something that could be put in to the repo rather than reinventing the wheel again.

The problem with runners is that you only get one ... you can fire multiple returners, so that's a bit of a step down. I suppose one could write a runner that loads multiple returner-like modules...
Having a runner that accepts modular returner-like objects isn't the worse idea in the world actually.

The other problem is that returners would work for async where as a runner would block, so you'd have to fire up multiple runners or accept them being sequential.

[whiteinge]

This was added in 2014.7 as the master_job_cache setting.

[anlutro]

I use event returners to run a returner on the master, which seems to work fine.

My main issue is that the naming of these configurations are not intuitive, and documentation is sparse on the subject.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.