Browse files

Allow inserting extra key into registration data.

From a directory of arbitrary YAML files which I want to get pushed up into Mongo with registration.

11:10 < t0m> Volcane: What's the reason behind squashing facts.yaml to be 1 level deep? I was considering doing something devious by making facts.yaml more complex (and using that data from mongodb), but it's not going to fly as facts/yaml.rb explicitly squashes the facts..
11:11 < oholiab> t0m: are you writing to facts.yaml from facter?
11:12 < t0m> I was writing to facts.yaml with vi to work out if it worked :)
11:18 < oholiab> so are you effectively trying to nest hashes and it won't work?
11:41 -!- e1mer [~erivera@unaffiliated/e1mer] has quit [Ping timeout: 255 seconds]
12:20 < Volcane> t0m: was all i needed initially for facter compat but been toying with ways to do nested data sanely - problem is mostly how do you use nested data on the cli
12:24 < t0m> Volcane: nod. I don't care about that so much though :) I have just smashed it, and I'm getting the deeper data into mongodb (bobtfish/marionette-collective@a30941f)
12:24 < Volcane> t0m: though been playing a lot with all that recently since the new rich filter language is effectively working on nested data and I am also supporting outputing JSON now and we wrote a grep tool for that that can grep in nested data
12:25 < t0m> cool, I haven't kept up with any of the new stuff, I'm still mostly playing catch up to mongodb stuff.. ;)
12:25 < Volcane> mco rpc puppetd last_run_summary -j|jgrep "data.resources.failed>0" -s sender
12:25 < Volcane> finds all machines that have failing resources, prints their mcollective id
12:26 < t0m> my actual problem which prompted this is I want to generate a load of graphs dynamically based on what's on the FS... I generate a yaml file with the data on the host, but I need to know a list of graphs to gen on the puppetmaster basically..
12:26 < Volcane> t0m: ah
12:26 < t0m> generate('/bin/cat', '/ohshit/this/is/on/wrong/system.yaml')
12:26 < Volcane> hehe
12:27 < t0m> etc etc :)
12:27 < Volcane> that jgrep did a grep over documents like one per host
12:27 < Volcane> so i am getting fairly happy with nested data on the cli, so might one day support it in facts
12:27  * t0m nods. That's exactly what I want really..
12:28 < t0m> Would be to make the registration plugin be able to pull in extra .yaml files, just for registration
12:28 < t0m> rather than crapping into facts, which is basically, a hack :)
12:28 < Volcane> yeah, i do that in mine
12:29 < t0m> ah, code visible anywhere?
12:29 < Volcane> also send registration data to a queue rather than to another mc agent cos i do entirely different things with reg
12:29 < t0m> cause that's what I really want, I'm just being a cheap-ass :)
12:29 < Volcane> nah but it should be trivial - look at the yaml.rb it already support merging yamls
12:29 < Volcane> so just nick that code
12:31 < t0m> I'll probably do that actually, if only for the fact I can then subclass your code, rather than maintaining a patch on it. Unless you can be persuaded to remove the flattening from the facts :)
12:32 < Volcane> well you wont see such a change soon is all i can say:)
12:32 < Volcane> we're on like a 6ish month production release cycle
12:33 < t0m> sure, I don't need to see it in a real release :)
12:33 < Volcane>
12:33 < Volcane> thats how i merge the yamls in the fact source
12:33 < t0m> I'm happy for $work to work off master.. I'm just less happy about having to maintain own rebased branch.
12:34 < t0m> yy, I've spent the last couple of hours working out how all of this works ;)
12:34 < t0m> so I guess I'm looking for some advice for what patch would be acceptable upsteam (even if things in a release don't change for 6 months) to enable me to get custom things into the registration data :)
12:35 < t0m> *upstream
12:35 < Volcane> just write your own registration plugin - thats why its a plugin, the expectation is that most people will have their own
12:35 < t0m> this implies that I'm maintaining ruby :)
12:36 < Volcane> like they're this easy
12:36 < t0m> I'm entirely happy to write ruby. I just don't want to maintain it :)
12:36 < t0m> sure.
12:36 < t0m> I was hoping to be able to make the general one a bit more flexible, and then never have to touch it again, as maintaining it wouldn't be my problem ;)
12:37 < Volcane> ok, so and around line 27 slap in the merge bit
12:37 < Volcane> and that I'd happily accepot
12:37 < Volcane> accept
12:39  * t0m nods - ok, cool, that's what I was after. So, just so that we're crystal, I'm going to add another hash to registration data which gets filled with yamls from config, or add the ability to merge extra .yamls onto the facts here?
12:39 < Volcane> hmm, probably best to do into a new key
12:39 < t0m> that was my thought also.
12:40 < Volcane> and make that key so it could be multiple yaml files merged up together
12:40 < t0m> I'm gonna go play with that then, I shall bug & pull request when I have something sane(r)
12:40 < t0m> thanks!
12:40 < Volcane> kewl kewl

This was a while ago - so if this is no longer appropriate or you have better ideas, please let me know
and I'll be happy to re-work.
  • Loading branch information...
1 parent 6059fcf commit 350c312fad39b371acb7ba0f104eeeb9d1fefbe0 @bobtfish committed Feb 7, 2012
Showing with 13 additions and 1 deletion.
  1. +13 −1 agent/registration-mongodb/registration.rb
14 agent/registration-mongodb/registration.rb
@@ -13,7 +13,10 @@ module Agent
# - plugin.registration.mongohost where the mongodb is default: localhost
# - plugin.registration.mongodb the db name default: puppet
# - plugin.registration.collection the collection name default: nodes
- #
+ # - plugin.registration.extra_yaml_dir the presense of this key indicates a directory
+ # containing additional yaml files to push into an 'extra' key in the registration data. All files
+ # in the directory are pushed into a hash with keys being the file name (with the
+ # path and .yml extension stripped) and the value being the contents. default: false
# Each document will have the following data:
# - fqdn - the fqdn of the sender
# - lastseen - last time we got data from it
@@ -43,6 +46,7 @@ def initialize
@mongohost = @config.pluginconf["registration.mongohost"] || "localhost"
@mongodb = @config.pluginconf["registration.mongodb"] || "puppet"
@collection = @config.pluginconf["registration.collection"] || "nodes"
+ @yaml_dir = @config.pluginconf["registration.extra_yaml_dir"] || false
Log.instance.debug("Connecting to mongodb @ #{@mongohost} db #{@mongodb} collection #{@collection}")
@@ -63,6 +67,14 @@ def handlemsg(msg, connection)
req[:fqdn] = req[:facts]["fqdn"]
req[:lastseen] =
+ # Optionally send a list of extra yaml files
+ if (@yaml_dir != false)
+ req[:extra] = {}
+ Dir[@yaml_dir + "/*.yaml"].each do | f |
+ req[:extra][File.basename(f).split('.')[0]] = YAML.load_file(f)
+ end
+ end
# Sometimes facter doesnt send a fqdn?!
if req[:fqdn].nil?
Log.instance.debug("Got stats without a FQDN in facts")

0 comments on commit 350c312

Please sign in to comment.