Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ability to define nodegroups through master_tops system #5787

Closed
madduck opened this issue Jun 27, 2013 · 29 comments
Closed

Ability to define nodegroups through master_tops system #5787

madduck opened this issue Jun 27, 2013 · 29 comments
Labels
Core relates to code central or existential to Salt Feature new functionality including changes to functionality and code refactors, etc.
Milestone

Comments

@madduck
Copy link
Contributor

madduck commented Jun 27, 2013

Edit: master_tops is probably not the right place for this. I'll repurpose (and retitle) this issue as soon as the storm has left my brain (i.e. "brainstorming is done…").


At least with reclass, nodegroups are part of the inventory and thus could easily be made available to Salt, such that I could target commands or states at groups in my inventory.

Instead of defining a new interface (we already have master_tops and ext_pillar separately, do we want a third?), maybe the master_tops interface could be extended to allow the external source to return nodegroups?

I don't have a clear idea of how this would work yet, but if you give a general green light into this direction, I could start looking and evaluating various options. It is possible that the interface would have to be incompatibly changed, unfortunately, but I would document this appropriately and change the existing tops plugins.

@thatch45
Copy link
Contributor

I think this is a good idea, I am not putting it on the top of my list, but please let me know if you can see a clean way to make this a possibility.
I think that adding it into master_tops makes a lot of sense, since the pillar data maps to a very different context, but ideas will need to be cultivated here to determine exactly how to approach this feature.

@madduck
Copy link
Contributor Author

madduck commented Jun 27, 2013

Thanks for your time to reply, @thatch45!

@UtahDave and I are just discussing this on IRC and we think that master_tops is not the right place for this, because it just returns the same data as top.sls, and as long as top.sls is not used for node group definitions, master_tops shouldn't be either.

Right now, we are brainstorming about a third external information source, maybe something like an "inventory". It would not only provide node group membership information, but also be able to give a list of all minions and allow the master to report if a minion didn't respond.

We'll keep you posted!

@madduck
Copy link
Contributor Author

madduck commented Jun 27, 2013

If there were an inventory, then maybe minions that are not addressed by a published command could still respond with a NOOP, or a "yeah, I heard you, but I am not addressed". This would close the loop and allow the master to stay on top of things.

@thatch45
Copy link
Contributor

I am liking where this is going, it will be useful on many levels!

@arthurzenika
Copy link
Contributor

+1 on this feature

@mgwilliams
Copy link
Contributor

Inventory could be defined in yaml and/or by an ext_inventory module... I like this idea too. Could this somehow be combined with roster?

@baby-gnu
Copy link

+1

I can't imagine a configuration management without an inventory of “what I want” to compare to “what I have” and see the progress and configuration policy violations.

@bbinet
Copy link
Contributor

bbinet commented Jan 16, 2014

+1

2 similar comments
@nikicat
Copy link
Contributor

nikicat commented Jan 24, 2014

+1

@jnials
Copy link

jnials commented Feb 7, 2014

+1

@loz-hurst
Copy link
Contributor

I hope that the node groups defined through this new "inventory" mechanism will be available in the pillar top.sls file too - so that those of us who solve the "what groups does this node belong to" issue with ext_pillar interfaces to other inventory systems can (finally!) match on these groups in pillar/top.sls.

@mumblez
Copy link

mumblez commented Oct 3, 2014

+1

3 similar comments
@HontoNoRoger
Copy link

+1

@Galser
Copy link

Galser commented Mar 24, 2015

+1

@dharmab
Copy link

dharmab commented Jul 31, 2015

+1

@jfindlay jfindlay added P2 Priority 2 Core relates to code central or existential to Salt P1 Priority 1 and removed Low Severity P2 Priority 2 labels Aug 11, 2015
@stale
Copy link

stale bot commented Dec 18, 2017

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

@stale stale bot added the stale label Dec 18, 2017
@loz-hurst
Copy link
Contributor

We now do this with ext_pillar to import "group" pillar information based on a custom module that reads the data from a combination of external inventory systems. The addition of ext_pillar_first (available in stable after this bug was filed) means we can target using the "group" from ext_pillar in our pillar files.

In short, for our use case we have a workaround through other developments in salt in the meantime.

@stale
Copy link

stale bot commented Dec 19, 2017

Thank you for updating this issue. It is no longer marked as stale.

@stale stale bot removed the stale label Dec 19, 2017
@iavael
Copy link

iavael commented Dec 19, 2017

@loz-hurst

Unfortunately it doesn't help if there are only ext_pillar sources and one of them provides "groups" list and it's needed to use these groups to target pillars from other sources.

@loz-hurst
Copy link
Contributor

@iavael Indeed, which is why I said "for our use case". It's very much a workaround, not a fix, as we've had to create a single ext_pillar that interfaces with the other ext_pillars (and some sources that we've not bothered to write a proper ext_pillar interface around, since we can't use it direct because of this) so we can cross-reference and cross-use them before using (some of) the information in them to target the pillar_root files.

The functionality to do it properly in salt would be much better than mangling ext_pillar sources via our own hacky interface to get it to work.

@stale
Copy link

stale bot commented Apr 14, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

@stale stale bot added the stale label Apr 14, 2019
@iavael
Copy link

iavael commented Apr 18, 2019

As a workaround we currently target pillars to nodegroups in the external system. Basically, our internal service that provides nodegroups for state targeting via pillars, also began to provide some actual pillar data that was stored in git.
This solves some of the most painful moments (at least we got rid of bunch of cryptic regexps in pillar top.sls), but this is very kludgy, because targeting mechanism is now outside of saltstack.

@stale
Copy link

stale bot commented Apr 18, 2019

Thank you for updating this issue. It is no longer marked as stale.

@stale stale bot removed the stale label Apr 18, 2019
@sagetherage sagetherage removed the P1 Priority 1 label Apr 21, 2020
@bbinet
Copy link
Contributor

bbinet commented May 6, 2020

Any news on this issue?

Is it normal that I'm not able to target minions using pillar (salt -I or salt -J) when my pillar data sit exclusively in reclass ext_pillar ?
I don't understand why pillar data coming from ext_pillar could not be used to target minions like any other pillar data? shouldn't it be considered as a bug?

@iavael
Copy link

iavael commented May 6, 2020

@bbinet you can target jobs (via salt -I) and states (via top.sls) by pillar data (including ext_pillar). We successfully ise this in production for several years. You cannot target pillar via top.sls by pillar data.

@bbinet
Copy link
Contributor

bbinet commented May 6, 2020

@iavael it does not work for me for all my minions (version 2019.2.0), but I've just come across issue #52234 and I think I should upgrade my minions to latest 2019.2.4 which should have the fix.
I've just tried to target an old minion (2018.3.3) with salt -I and it does work!
So I think I just need to upgrade, thanks.

@iavael
Copy link

iavael commented May 6, 2020

@bbinet that's strange. We used such targeting since 2017.7. It could be available even earlier, but we didn't use saltstack then yet.
But if it works for you after upgrade, then it's fine, I suppose.

@whytewolf
Copy link
Collaborator

Is this still an issue? from the last comments on the ticket it sounds like it has been fixed

@whytewolf
Copy link
Collaborator

closeing this issue. if users are still having issues with this. please test in latest supported version to see if already addressed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Core relates to code central or existential to Salt Feature new functionality including changes to functionality and code refactors, etc.
Projects
None yet
Development

No branches or pull requests