First iteration of the coffee script based CLI ES-stored dashboard impor...

[edasque]

First iteration of the coffee script based CLI ES-stored dashboard import/export utility

I sent you an email about this about a week ago but didn't ask. I think this could be useful to people from what I can tell. It's written in CoffeeScript so I'll need to provide some grunt tasks to compile. I need to clean it up a bit as well and provide unit tests. But before I start that work, I wanted to make sure there is interest.

Syntax and TODO follow:

Usage: coffee dashboard_import_export.coffee [options]

Options:

-h, --help                        output usage information
-V, --version                     output the version number
-h, --host [host]                 ElasticSearch host to export to/import from
-p, --port [port]
-e --export [DashboardName]       Export the named Grafana Dashboard from ElasticSearch
-o --overwrite                    Whether or not to overwrite should the export file already exist
-i --import [DashboardFile.json]  Import the given file in ElasticSearch

TODO:

• remove minimalist
• add unit testing
• add grunt task for coffee

[jaimegago]

I don't read coffeescript that well so forgive me if it's clear in the code, but does it export all dashboards if no [DashboardName] is specified? If so adding a "--zip zipfile" or "--tar tarball" option would be great so all dashboards are then found in a single file (ideally time stamped but that could easily be coded in the CLI call e.g. --zip $(date +"%Y%m%d%H%M")_grafana_dashboards.zip). Or maybe a --backup_to_tar option that exports all dashboards to a time stamped tarball?

[edasque]

This doesn't export all the dashboards, it wasn't useful in my use case but I could easily add it. Let me figure out if @torkelo is interested in the PR first, I'll probably need to add testing and a grunt task for coffeescript then I can easily add mass export. If I recall, there is a REST call in ES to get all the dashboard in a JSON file, then it's just a matter of grabbing each then zipping it up which will be easy on node.js.

[jaimegago]

There is http://my.elastiscsearch.fqdn/grafana-dash/dashboard/_search but that's not importable as is I think (the dashboards themselves are json in json), but it should be trivial to extract them and have one dashboard one json file within the tarball ( that's what I do for backing up our grafana dashboards https://gist.github.com/jaimegago/11229750 )

[edasque]

Oui, c'est assez simple. Compte sur moi 🇫🇷

Yup, looks easy enough. I'll do that once we make some progress on my PR.

[torkelo]

I might be more inclined to merge this if it was in js and not coffeescript. Then I am not sure if this should be in the main repo. Maybe in a contrib repository. I have created a grafana github organization, planing on moving grafana repo there when I have time. Maybe then we could have grafana/grafana (main repo), grafana/grafana-contrib (custom panels, utils, experimental stuff, etc)

[edasque]

@torkelo I intend to add a Grunt task to compile to JS so as to make it easy, I already have appended to the package.json for the correct packages. I understand the JS community is divided when it comes to CoffeeScript but as you know, it's just JS in the end. I hope you can get over that.

As for a contrib repo, I think that could be useful sure. Because of Grafana's nature as a browser based application (without a server component save for optional persistence through ES) user login, user management, roles, rights, etc will be a challenge (unless you require ES and manage all this through whole JSON files stored there). Without these things however, it's hard to do an enterprise deployment of Grafana since everyone can modify dashboards and there is no history, versioning. I am currently facing this challenge deploying Grafana for use by a large dev/devops group.

As I mentioned to @jaimegago, I am trying to enable the following workflow: people commit dashboards to a directory in our gitana repo. A process takes those files and imports there directly. That way we keep a log of imports and the commits linked to them. I'll eventually remove the import button from grafana for our users so people can't get around the dashboard update/create mechanism. Users (developers, devops) build their dashboards on their local grafana instance, export the json through the tool (easy grunt task) then commit their changes to the repo.

I'll happily retract the pull request but wanted to explain the rational behind this contribution.

[jaimegago]

+1 on moving to grafana/grafana and grafana/ repositories, I like the way Sensu is doing it (e.g. Sensu/Sensu-Puppet, Sensu/Sensu-Chef, Sensu/Sensu-community-plugins) but maybe there is no need for that many sub repos, sub dirs in contrib repo might be enough? There is also the question of documentation, if you go the grafana/grafana way, then the contrib repo code could be documented via .md files in each sub directories and then maybe a "contribution" section in the wiki

[torkelo]

@edasque I understand your case. And it is great that you explain it in detail, It give me some ideas on how to improve Grafana for larger teams/organizations in the future. And I think you wanting to contribute this is awsome! Just not sure where it fits in just now.

@jaimegago yes, having grafana/grafana-puppet, grafana/grafana-docker , and grafana/grafana-plugins is something I am planing. I looked into moving repos and it seems like Github has a feature to move a repository to an organization while keeping stars, watchers etc.

[edasque]

@torkelo great, let's create that -contrib repo and I'll happily contribute this to it.