Allowing to customize/override storage-schemas/aggregation #40

eladroz opened this Issue Mar 6, 2013 · 3 comments

2 participants


Currently storages-schemas.conf is static and storage-aggregation.conf is not handled at all. Personally I'd like to config both, but not sure of the best way.

FYI, changing these files does not require a service restart, so they could be changed externally easily. On the other hand, existing metrics are not affected by these changes unless you run a special script to refactor the existing files.

  1. Minimum change: don't do anything in graphite cookbook :-)
    On every chef run, both the carbon recipe & the user's recipe would write their own differing versions of storage-schemas on every chef run, last to write wins all... not very clean. Slim chance of very rare bug if a new metric comes in while our default file is in effect.

  2. A bit more: allow users to supply alternative template names from their wrapper cookbook, with no default for storage-aggregation (a bit like runit_service allows). Users then can be sure their template would be run at the right point, with the right ownership.

  3. Being nice to StatsD users: optionally creating a storage-aggregations file suitable for StatsD.

Whaddaya say?

Heavy Water Operations Cookbooks member
  1. The current preferred approach, with a template for storage-schemas.conf as default as possible.

  2. If users are writing a wrapper cookbook, they should already be able to do this. The template resource has a cookbook attribute and so you can pull templates from any other cookbook. I imagine many users are already overwriting the default storage-schemas.conf that is supplied since it's just a generic catchall.

  3. This might be worth having if it was a wholly separate recipe, say graphite::statsd. It does diverge from the scope of a graphite cookbook slightly, even though it's probably a common use case. I do feel like this sort of recipe belongs in a wrapper cookbook anyway since it's going to be custom to the users scenario.

Perhaps an LWRP, something like graphite_storage_schema would be general enough? User schemas could be in a data bag, loop through and render each in the template file. A generic LWRP would fit both 2 and 3, and is in keeping the idea of library cookbooks providing reusable components.

I'm happy to review any PRs that implement this stuff


I think data bags (and elaborate roles) are being slowly phased out as good practice since they are not versioned and cannot be bundled with the cookbooks you download. Instead there seems to be preference for node attributes.

I think we can use attributes here, as this ye olde PR suggested:

LWRP are overkill here IMHO, because they're usually used when you want to have multiple separate instances of something.

If users don't want our templates alltogether, they'd just set node.graphite.storage_schemas = nil. We can also switch to human-friendly retention and remove the priority entry, which seems deprecated. Optionally, for StatsD we can have a readymade aggregation rules hash that is not used by default, but can be easily used/customized.

Heavy Water Operations Cookbooks member

merged, closing

@webframp webframp closed this Apr 10, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment