Apply many files

[gutorc92]

Signed-off-by: Gustavo Coelho gutorc@hotmail.com

Solves #64

[malcolmholmes]

My suggestion here would be to just use jsonnet functionality to achieve this: you can create a single jsonnet file and have it import and merge the other ones.

[mavimo]

@malcolmholmes I agree with the @gutorc92 approach, having a "root doc" can make the generation complex (instead of building each single group of items need to process all items in a single one), another option should be to have a single arg that can be a glob (eg dashboards/*.jsonnetlib or mytools/**/grafana.jsonnetlib), WDYT?

[malcolmholmes]

@mavimo why does a "root doc" make things more complex? I'm curious to hear that.

I certainly agree that glob based targeting is a must. I don't recall having done it.

[mavimo]

@malcolmholmes a "complexity" can be the fact that need to include and reference every single new file in the "root document". In my use-case I have a folder (let's call it dashboards that can include dozens of dashboards) at the moment we are going to loop into every file and invoke grr to publish the dashboard, having a wildcard can make it easy.
If we want to include them in a single "root file" every time a new dashboard is created (or removed) we need to remember to edit also the root file. The root file is not "complex" di-per-se but add cognitive complexity to developers that work on it.

Another topic that (I'm not too much sure) is that the generation can be slower and consume much more memory if need to process a lot of dashboard every time (we expect to reach hundred of dashboards in short time), actually we are planning to do some "check" to publish only changed dashboard in CI (using some sort of git diff to detect changes) once we are going to merge all the dashboard in the "root file" we need to have a more complex algo to detect if a dashboard is changed (eg because is changes something in the root doc that impact the specific dashboard also if the dashboard is not changed..

Maybe my use-case is too specific to be a "real issues" 😄

[malcolmholmes]

This is a use-case I have been thinking about. Increasingly it is becoming clear to me that Grizzly has as much of a role outside of jsonnet, just dealing with raw JSON files. I haven't really thought through how we would cover the various scenarios. What you are suggesting is an interesting idea: a directory tree full of files, Grizzly can somehow identify the type of the file, then push it to its destination. Note JSON could be a datasource or a dashboard, YAML could be prometheus rules, etc.

Once we have this, for Grafana, there's an interesting possibility:

Grizzly will, via the file system, know the mapping between files and dashboard UIDs, be known. Thus, when a user uses grr listen, it will be possible to listen on all dashboards, and write them back to source files when a user saves. The thing I hadn't worked out was how dashboard files map to UIDs, and this solves that.

It is also a relatively significant diversion from the Tanka/Jsonnet pattern, so we need to make sure that we support both patterns, somehow.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[malcolmholmes]

This can now be done by pointing Grizzly at a directory instead of a file.