Add an API to import/export dashboards

[jaimegago]

Unless I'm mistaken there is no programatic way to easily export/import user generated dashboards. Such API would be useful to pair with tooling (e.g. config management) so user generated dashboard are automatically managed.

[edasque]

I am guessing the only way to do this would be an import/export to/from elastic search since grafana has no server side components, it runs in the browser only.

[torkelo]

Yes, you are correct. Elasticsearch has a http API :)

[edasque]

Since I am interested as well in doing this (probably a node.js script making REST calls to ES), is there something we should know as to where the endpoints are? I imagine it's just a matter of issuing a GET for the right dashboard to export and a PUT to import a new dashboard?

I can't contribute (yet) to the AngularJS app but I might be able to help with some server side utilities

[torkelo]

elasticsearch has a pretty simple http api, look at the network tab in chrome dev tools to see the calls for GET and PUT for dashboards

[edasque]

So to write (import), it seems you'd do a PUT to http://edasquemac1:9200/grafana-dash/dashboard/Dashboard name with a JSON payload of { dashboard: " -- JSON source for the dashboard ", group: "guest",tags: [], title: "Name of the Dashboard", user: "guest" }

I'll try that.

[jaimegago]

In the meantime I've worked a python script to backup the user generated dashboards, not sure what to do with it other than write a short post about it.

https://gist.github.com/jaimegago/11229750

[edasque]

Funny, I just submitted a pull request to do this in JS/Coffeescript: https://github.com/torkelo/grafana/pull/340

[jaimegago]

I think there is definitively the need to produce something "official" that helps people deploying Grafana with automating the backup of "user generated dashboards", in particular since there is no built in authentication at the moment, it's too easy to accidentally destroy another user dashboard. I'm thinking a page in the wiki cleaning up our discussion here and referencing whatever gets in the grafana repo. I pinged @torkelo on twitter about this too. My gist is a "watered" version of what I run in prod as I've added a push of the tarball to an S3 bucket via config file for AWS credentials+bucket, I'll be happy to clean it up and add the "backup to S3" feature if there is interest for it.
I started this thread with Configuration Management tools in mind originally but it's trickier than what I thought, so since then I've lowered my ambitions to just backing up the dashboards. That said I could add an "import" option to my script but somehow it doesn't feel right in terms of idempotency, instead I'd rather have something that is native to the CM tool itself (e.g. an Ansible module).

[edasque]

It's similar to what I built though I do the Export & the Import and it's done in JS (well, coffeescript).

Our workflow is that people commit dashboards to a directory in our gitana repo. A process takes those files and imports there directly. That way we keep a log of imports and the commits linked to them. I'll eventually remove the import button from grafana for our users so people can't get around the dashboard update/create mechanism. Users (developers, devops) build their dashboards on their local grafana instance, export the json through the tool (easy grunt task) then commit their changes to the repo.

It can probably be refined by a competent devops but it works for us for now.

[jaimegago]

That flow makes a lot of sense at least to me and is very simple to implement, so I'm definitively going to propose something largely inspired by it, many thanks for sharing.
As for refining, I think it is more of a question of adapting to one's tooling environment and the targeted level of automation vs available human resources. In the end, I understood your workflow in the few lines it took to be explained and I will be able to propose something very similar that works in my environment so something is definitively right about it!

[edasque]

We finished implementing that flow here. We had to add clearing the dashboards and re-importing from fs on each commit to make things simpler.

[jaimegago]

We sticked to automated backups only because we have "non git" business users of grafana, sure we could have asked them to save then export their dashboards but that was adding an extra step.

[edasque]

The downside being that they can happily overwrite each other's changes....

[jaimegago]

Yes indeed one of the major ones, also backing up is not configuration management, I argued in favor of a git centric model, but then we would have had a need for a test instance to "develop" dashboards (no way I'd get business type of users to install grafana locally) and train people to git or have them open tickets with the dashboards JSON, either way extra steps vs automatically backed up to S3 every X min. It could be also improved just by being a push incremental backup instead of scheduled which would limit the overwriting issue.

[Dieterbe]

+1 for api to import graphs. it could be something like a python library that assists with creating the official json representation of a dashboard, so the input of the functions would be a graphite graph definition (render/?foo=bar&target=baz&...), a title, some tags, a user and group, and the provided api would know how to convert that into the correct json that grafana wants, so that we can then upload the json to grafana as new dashboard, or send it to ES/influxdb

[torkelo]

implemented in Grafana 2.0

[fengjuZhang]

There is no Http API import, I need to automatically generate the instrument panel by Http

[torkelo]

Here is a http api to create / update dashboarda

[fengjuZhang]

where

[fengjuZhang]

If I have a JSON I can directly generate the dashboard through the HTTP request.

[torkelo]

check the http api docs :)

[torkelo]

docs.grafana.org

[fengjuZhang]

thanks

[LuboVarga]

I have running grafana installed from rpm (latest version, 3.1.0), but it seems it is ignoring config file /etc/grafana/grafana.ini. I have modified it to enable json dashboards like this:

[dashboards.json]
;enabled = true
;path = /usr/share/grafana/public/dashboards

Given directory contains some json and js files. Accessing generated (*.js) dashboards is working and simple (also without moddification of settings file). I just open http://localhost:3000/dashboard/script/dashboardgenerator.js?environment=test8 and it is working. But when I want to provision dashoard to be visible in menu of grafana (in upper left menu with list of dashboards), I do not know, where should I put my dashboards. I need to make possible to search through available dashboards and not only to guess what parameter should be passed to my javascript.

Is it possible to somehow import dashboard and make it visible in dashboard list? (and by import I mean some programmatic way, not by playing in grafana web interface and importing things by hand)

[Djidiouf]

Hi LuboVarga, I just used

#################################### Dashboard JSON files ##########################
[dashboards.json]
enabled = true
path = /var/lib/grafana/dashboards

And as soon as I put a json dashboard in that folder, it appears on the dashboard list in Grafana. Perhaps try to have only .json in it and see what happens.

[LuboVarga]

Thanks @Djidiouf, it is working now. I am not sure, if I had wrong path (now it was wrong), or I have tried other than json suffixes.

Just for the record, I have solved provisioning of dashboards from saltstack through provisioning json/request files, curl and by using http api:

curl -X POST -d @/usr/share/grafana/public/dashboards/projection.json.dashboard.request 'http://localhost:3000/api/dashboards/db' --header 'Content-Type: application/json' --header 'Authorization: Bearer eyJrIjo_THIS_IS_API_KEY_joxfQ==' -sw 'STATUSCODE=%{http_code}' | grep 'STATUSCODE=200'

Dashboards get saved into /var/lib/grafana/grafana.db file (where are also other things, like api keys or data sources configured).

[mojili]

I solved the problem like this:
1- first create your datasource like this (In my case I used the combination of collectd, prometheus and grafana)

curl --user admin:admin 'http://IPADDR:3000/api/datasources' -X POST -H 'Content-Type: application/json;charset=UTF-8' --data-binary '{"name":"test","type":"prometheus","url":"http://localhost:9090","access":"proxy","basicAuth":false}'

2-For adding the customized json dashboard, edit grafana.ini file and enable Dashboard json file section like below:
;#################################### Dashboard JSON files ##########################
[dashboards.json]
enabled = true
path = /var/lib/grafana/dashboards

3- then copy dashboard json file to /var/lib/grafana/dashboards (you need to restart the service)

[leoauri]

In case anyone else struggles to get this happening a few years later, the options have moved to /etc/grafana/provisioning/dashboards/sample.yaml, where /etc/grafana/ is the install location on Ubuntu.