Registration topic extra yamls in registration data #34

Merged
merged 2 commits into from Mar 12, 2012

Conversation

Projects
None yet
2 participants
Contributor

bobtfish commented Feb 7, 2012

Patch as per a discussion with Volcane on irc some weeks ago about how to get arbitrary data into registration.

Please let me know if this is suitable for inclusion, or if I'm missing the point here?

bobtfish added some commits Feb 7, 2012

@bobtfish bobtfish Don't die in an unclear way if you forget the registration is Meta co…
…nfig
6059fcf
@bobtfish bobtfish Allow inserting extra key into registration data.
From a directory of arbitrary YAML files which I want to get pushed up into Mongo with registration.

11:10 < t0m> Volcane: What's the reason behind squashing facts.yaml to be 1 level deep? I was considering doing something devious by making facts.yaml more complex (and using that data from mongodb), but it's not going to fly as facts/yaml.rb explicitly squashes the facts..
11:11 < oholiab> t0m: are you writing to facts.yaml from facter?
11:12 < t0m> I was writing to facts.yaml with vi to work out if it worked :)
11:18 < oholiab> so are you effectively trying to nest hashes and it won't work?
11:41 -!- e1mer [~erivera@unaffiliated/e1mer] has quit [Ping timeout: 255 seconds]
12:20 < Volcane> t0m: was all i needed initially for facter compat but been toying with ways to do nested data sanely - problem is mostly how do you use nested data on the cli
12:24 < t0m> Volcane: nod. I don't care about that so much though :) I have just smashed it, and I'm getting the deeper data into mongodb (bobtfish/marionette-collective@a30941f)
12:24 < Volcane> t0m: though been playing a lot with all that recently since the new rich filter language is effectively working on nested data and I am also supporting outputing JSON now and we wrote a grep tool for that that can grep in nested data
12:25 < t0m> cool, I haven't kept up with any of the new stuff, I'm still mostly playing catch up to mongodb stuff.. ;)
12:25 < Volcane> mco rpc puppetd last_run_summary -j|jgrep "data.resources.failed>0" -s sender
12:25 < Volcane> finds all machines that have failing resources, prints their mcollective id
12:26 < t0m> my actual problem which prompted this is I want to generate a load of graphs dynamically based on what's on the FS... I generate a yaml file with the data on the host, but I need to know a list of graphs to gen on the puppetmaster basically..
12:26 < Volcane> t0m: ah
12:26 < t0m> generate('/bin/cat', '/ohshit/this/is/on/wrong/system.yaml')
12:26 < Volcane> hehe
12:27 < t0m> etc etc :)
12:27 < Volcane> that jgrep did a grep over documents like http://www.pastie.org/2247985 one per host
12:27 < Volcane> so i am getting fairly happy with nested data on the cli, so might one day support it in facts
12:27  * t0m nods. That's exactly what I want really..
12:28 < t0m> Would be to make the registration plugin be able to pull in extra .yaml files, just for registration
12:28 < t0m> rather than crapping into facts, which is basically, a hack :)
12:28 < Volcane> yeah, i do that in mine
12:29 < t0m> ah, code visible anywhere?
12:29 < Volcane> also send registration data to a queue rather than to another mc agent cos i do entirely different things with reg
12:29 < t0m> cause that's what I really want, I'm just being a cheap-ass :)
12:29 < Volcane> nah but it should be trivial - look at the yaml.rb it already support merging yamls
12:29 < Volcane> so just nick that code
12:31 < t0m> I'll probably do that actually, if only for the fact I can then subclass your code, rather than maintaining a patch on it. Unless you can be persuaded to remove the flattening from the facts :)
12:32 < Volcane> well you wont see such a change soon is all i can say:)
12:32 < Volcane> we're on like a 6ish month production release cycle
12:33 < t0m> sure, I don't need to see it in a real release :)
12:33 < Volcane> https://github.com/puppetlabs/marionette-collective/blob/master/plugins/mcollective/facts/yaml_facts.rb#L23-33
12:33 < Volcane> thats how i merge the yamls in the fact source
12:33 < t0m> I'm happy for $work to work off master.. I'm just less happy about having to maintain own rebased branch.
12:34 < t0m> yy, I've spent the last couple of hours working out how all of this works ;)
12:34 < t0m> so I guess I'm looking for some advice for what patch would be acceptable upsteam (even if things in a release don't change for 6 months) to enable me to get custom things into the registration data :)
12:35 < t0m> *upstream
12:35 < Volcane> just write your own registration plugin - thats why its a plugin, the expectation is that most people will have their own
12:35 < t0m> this implies that I'm maintaining ruby :)
12:36 < Volcane> like they're this easy https://github.com/puppetlabs/marionette-collective/blob/master/plugins/mcollective/registration/agentlist.rb
12:36 < t0m> I'm entirely happy to write ruby. I just don't want to maintain it :)
12:36 < t0m> sure.
12:36 < t0m> I was hoping to be able to make the general one a bit more flexible, and then never have to touch it again, as maintaining it wouldn't be my problem ;)
12:37 < Volcane> ok, so https://github.com/puppetlabs/mcollective-plugins/blob/master/registration/meta.rb and around line 27 slap in the merge bit
12:37 < Volcane> and that I'd happily accepot
12:37 < Volcane> accept
12:39  * t0m nods - ok, cool, that's what I was after. So, just so that we're crystal, I'm going to add another hash to registration data which gets filled with yamls from config, or add the ability to merge extra .yamls onto the facts here?
12:39 < Volcane> hmm, probably best to do into a new key
12:39 < t0m> that was my thought also.
12:40 < Volcane> and make that key so it could be multiple yaml files merged up together
12:40 < t0m> I'm gonna go play with that then, I shall bug & pull request when I have something sane(r)
12:40 < t0m> thanks!
12:40 < Volcane> kewl kewl

This was a while ago - so if this is no longer appropriate or you have better ideas, please let me know
and I'll be happy to re-work.
350c312
Contributor

ripienaar commented Feb 7, 2012

Thanks, this looks great, can you please open tickets @ projects.puppeltabs.com so we can track these there as well as the contributor agreement status etc

Contributor

bobtfish commented Mar 9, 2012

ping! Any chances of getting this into a release at some point soon, given I've (I think) jumped through all the contribution hoops!

https://projects.puppetlabs.com/issues/12536

Contributor

ripienaar commented Mar 9, 2012

ah, wow, that ticket was in the puppet documentation project where I was unaware of it :) will ask someone to move it and then look at this

Contributor

bobtfish commented Mar 9, 2012

Oh good - I knew I'd have screwed it up somehow!

Thanks :)

@ripienaar ripienaar added a commit that referenced this pull request Mar 12, 2012

@ripienaar ripienaar Merge pull request #34 from bobtfish/registration_topic_extra_yamls_i…
…n_registration_data

Registration topic extra yamls in registration data
250a680

@ripienaar ripienaar merged commit 250a680 into puppetlabs:master Mar 12, 2012

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment