New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ability to define nodegroups through `master_tops` system #5787

Open
madduck opened this Issue Jun 27, 2013 · 20 comments

Comments

Projects
None yet
@madduck
Contributor

madduck commented Jun 27, 2013

Edit: master_tops is probably not the right place for this. I'll repurpose (and retitle) this issue as soon as the storm has left my brain (i.e. "brainstorming is done…").


At least with reclass, nodegroups are part of the inventory and thus could easily be made available to Salt, such that I could target commands or states at groups in my inventory.

Instead of defining a new interface (we already have master_tops and ext_pillar separately, do we want a third?), maybe the master_tops interface could be extended to allow the external source to return nodegroups?

I don't have a clear idea of how this would work yet, but if you give a general green light into this direction, I could start looking and evaluating various options. It is possible that the interface would have to be incompatibly changed, unfortunately, but I would document this appropriately and change the existing tops plugins.

@thatch45

This comment has been minimized.

Show comment
Hide comment
@thatch45

thatch45 Jun 27, 2013

Member

I think this is a good idea, I am not putting it on the top of my list, but please let me know if you can see a clean way to make this a possibility.
I think that adding it into master_tops makes a lot of sense, since the pillar data maps to a very different context, but ideas will need to be cultivated here to determine exactly how to approach this feature.

Member

thatch45 commented Jun 27, 2013

I think this is a good idea, I am not putting it on the top of my list, but please let me know if you can see a clean way to make this a possibility.
I think that adding it into master_tops makes a lot of sense, since the pillar data maps to a very different context, but ideas will need to be cultivated here to determine exactly how to approach this feature.

@madduck

This comment has been minimized.

Show comment
Hide comment
@madduck

madduck Jun 27, 2013

Contributor

Thanks for your time to reply, @thatch45!

@UtahDave and I are just discussing this on IRC and we think that master_tops is not the right place for this, because it just returns the same data as top.sls, and as long as top.sls is not used for node group definitions, master_tops shouldn't be either.

Right now, we are brainstorming about a third external information source, maybe something like an "inventory". It would not only provide node group membership information, but also be able to give a list of all minions and allow the master to report if a minion didn't respond.

We'll keep you posted!

Contributor

madduck commented Jun 27, 2013

Thanks for your time to reply, @thatch45!

@UtahDave and I are just discussing this on IRC and we think that master_tops is not the right place for this, because it just returns the same data as top.sls, and as long as top.sls is not used for node group definitions, master_tops shouldn't be either.

Right now, we are brainstorming about a third external information source, maybe something like an "inventory". It would not only provide node group membership information, but also be able to give a list of all minions and allow the master to report if a minion didn't respond.

We'll keep you posted!

@madduck

This comment has been minimized.

Show comment
Hide comment
@madduck

madduck Jun 27, 2013

Contributor

If there were an inventory, then maybe minions that are not addressed by a published command could still respond with a NOOP, or a "yeah, I heard you, but I am not addressed". This would close the loop and allow the master to stay on top of things.

Contributor

madduck commented Jun 27, 2013

If there were an inventory, then maybe minions that are not addressed by a published command could still respond with a NOOP, or a "yeah, I heard you, but I am not addressed". This would close the loop and allow the master to stay on top of things.

@thatch45

This comment has been minimized.

Show comment
Hide comment
@thatch45

thatch45 Jun 27, 2013

Member

I am liking where this is going, it will be useful on many levels!

Member

thatch45 commented Jun 27, 2013

I am liking where this is going, it will be useful on many levels!

@arthurlogilab

This comment has been minimized.

Show comment
Hide comment
@arthurlogilab

arthurlogilab Sep 12, 2013

Contributor

+1 on this feature

Contributor

arthurlogilab commented Sep 12, 2013

+1 on this feature

@mgwilliams

This comment has been minimized.

Show comment
Hide comment
@mgwilliams

mgwilliams Oct 4, 2013

Contributor

Inventory could be defined in yaml and/or by an ext_inventory module... I like this idea too. Could this somehow be combined with roster?

Contributor

mgwilliams commented Oct 4, 2013

Inventory could be defined in yaml and/or by an ext_inventory module... I like this idea too. Could this somehow be combined with roster?

@baby-gnu

This comment has been minimized.

Show comment
Hide comment
@baby-gnu

baby-gnu Dec 31, 2013

+1

I can't imagine a configuration management without an inventory of “what I want” to compare to “what I have” and see the progress and configuration policy violations.

+1

I can't imagine a configuration management without an inventory of “what I want” to compare to “what I have” and see the progress and configuration policy violations.

@bbinet

This comment has been minimized.

Show comment
Hide comment
@bbinet

bbinet Jan 16, 2014

Contributor

+1

Contributor

bbinet commented Jan 16, 2014

+1

@nikicat

This comment has been minimized.

Show comment
Hide comment
@nikicat

nikicat Jan 24, 2014

Contributor

+1

Contributor

nikicat commented Jan 24, 2014

+1

@jnials

This comment has been minimized.

Show comment
Hide comment

jnials commented Feb 7, 2014

+1

@loz-hurst

This comment has been minimized.

Show comment
Hide comment
@loz-hurst

loz-hurst Sep 22, 2014

Contributor

I hope that the node groups defined through this new "inventory" mechanism will be available in the pillar top.sls file too - so that those of us who solve the "what groups does this node belong to" issue with ext_pillar interfaces to other inventory systems can (finally!) match on these groups in pillar/top.sls.

Contributor

loz-hurst commented Sep 22, 2014

I hope that the node groups defined through this new "inventory" mechanism will be available in the pillar top.sls file too - so that those of us who solve the "what groups does this node belong to" issue with ext_pillar interfaces to other inventory systems can (finally!) match on these groups in pillar/top.sls.

@mumblez

This comment has been minimized.

Show comment
Hide comment

mumblez commented Oct 3, 2014

+1

@HontoNoRoger

This comment has been minimized.

Show comment
Hide comment

+1

@Galser

This comment has been minimized.

Show comment
Hide comment

Galser commented Mar 24, 2015

+1

@dharmab

This comment has been minimized.

Show comment
Hide comment

dharmab commented Jul 31, 2015

+1

@stale

This comment has been minimized.

Show comment
Hide comment
@stale

stale bot Dec 18, 2017

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

stale bot commented Dec 18, 2017

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

@stale stale bot added the stale label Dec 18, 2017

@loz-hurst

This comment has been minimized.

Show comment
Hide comment
@loz-hurst

loz-hurst Dec 19, 2017

Contributor

We now do this with ext_pillar to import "group" pillar information based on a custom module that reads the data from a combination of external inventory systems. The addition of ext_pillar_first (available in stable after this bug was filed) means we can target using the "group" from ext_pillar in our pillar files.

In short, for our use case we have a workaround through other developments in salt in the meantime.

Contributor

loz-hurst commented Dec 19, 2017

We now do this with ext_pillar to import "group" pillar information based on a custom module that reads the data from a combination of external inventory systems. The addition of ext_pillar_first (available in stable after this bug was filed) means we can target using the "group" from ext_pillar in our pillar files.

In short, for our use case we have a workaround through other developments in salt in the meantime.

@stale

This comment has been minimized.

Show comment
Hide comment
@stale

stale bot Dec 19, 2017

Thank you for updating this issue. It is no longer marked as stale.

stale bot commented Dec 19, 2017

Thank you for updating this issue. It is no longer marked as stale.

@stale stale bot removed the stale label Dec 19, 2017

@iavael

This comment has been minimized.

Show comment
Hide comment
@iavael

iavael Dec 19, 2017

@loz-hurst

Unfortunately it doesn't help if there are only ext_pillar sources and one of them provides "groups" list and it's needed to use these groups to target pillars from other sources.

iavael commented Dec 19, 2017

@loz-hurst

Unfortunately it doesn't help if there are only ext_pillar sources and one of them provides "groups" list and it's needed to use these groups to target pillars from other sources.

@loz-hurst

This comment has been minimized.

Show comment
Hide comment
@loz-hurst

loz-hurst Dec 20, 2017

Contributor

@iavael Indeed, which is why I said "for our use case". It's very much a workaround, not a fix, as we've had to create a single ext_pillar that interfaces with the other ext_pillars (and some sources that we've not bothered to write a proper ext_pillar interface around, since we can't use it direct because of this) so we can cross-reference and cross-use them before using (some of) the information in them to target the pillar_root files.

The functionality to do it properly in salt would be much better than mangling ext_pillar sources via our own hacky interface to get it to work.

Contributor

loz-hurst commented Dec 20, 2017

@iavael Indeed, which is why I said "for our use case". It's very much a workaround, not a fix, as we've had to create a single ext_pillar that interfaces with the other ext_pillars (and some sources that we've not bothered to write a proper ext_pillar interface around, since we can't use it direct because of this) so we can cross-reference and cross-use them before using (some of) the information in them to target the pillar_root files.

The functionality to do it properly in salt would be much better than mangling ext_pillar sources via our own hacky interface to get it to work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment