Skip to content

Commit

Permalink
Update tool to support folders
Browse files Browse the repository at this point in the history
Folders are available since Grafana v.5.0.*. Create scripts to migrate
folders and dashboards (with folders id)
  • Loading branch information
Matheus Figueredo committed Aug 16, 2018
1 parent b15a3d8 commit 74fa101
Show file tree
Hide file tree
Showing 8 changed files with 146 additions and 39 deletions.
37 changes: 25 additions & 12 deletions README.md
Expand Up @@ -4,27 +4,34 @@ Some python programs to call Grafana API to:

* Save every datasource to each datasource file.
* **saveDatasources.py**
* Save every folder to each datasource file.
* **saveFolders.py**
* Save every dashboard to each dashboard file.
* **saveDashboards.py**
* Create datasource from a backup file.
* **createDatasource.py**
* Create folder from a backup file.
* **createFolder.py**
* Create dashboard from a backup file.
* **createDashboard.py**

There a three convenient script files:
There a three convenient script files:

1. **backup_grafana.sh**
2. **restore_dashboards.sh**
3. **restore_datasources.sh**
3. **restore_datasources.sh**
3. **restore_folders.sh**

you can use them to
you can use them to

1. **backup all datasources and dashboards.**
1. **backup all datasources, dashboards and folders.**
2. ex: sh backup_grafana.sh
2. **restore dashboards from your dashboard backup folder.**
3. ex: sh restore_dashboards.sh /tmp/dashboards/2016-10-10_12:00:00
3. **restore datasources from your datasource backup folder.**
4. ex: sh restore_dashboards.sh /tmp/datasources/2016-10-10_12:00:00
3. **restore folders from your folder backup folder.**
4. ex: sh restore_folders.sh /tmp/folders/2016-10-10_12:00:00

[Grafana API document](http://docs.grafana.org/http_api/overview/)

Expand All @@ -37,8 +44,8 @@ you can use them to

1. Export the environment variables bellow
2. GRAFANA_URL (the default url is http://localhost:3000)
3. GRAFANA_TOKEN
3. GRAFANA_TOKEN

You can see how to get token from here: [Grafana Web page](http://docs.grafana.org/http_api/auth/)

## How to Use
Expand All @@ -49,16 +56,22 @@ You can see how to get token from here: [Grafana Web page](http://docs.grafana.o
* Use **saveDatasources.py** to save each datasources to each file under specific folder.
* ex: python saveDatasources.py **folder_path**

* Use **saveFolders.py** to save each folders to each file under specific folder.
* ex: python saveFolders.py **folder_path**

* Use **createDashboard.py** to read the dashboard file and create or update them on Grafana.
* ex: python createDashboard.py **file_path**

* Use **createDatasource.py** to read the datasource file and create or update them on Grafana.
* ex: python createDatasource.py **file_path**


* Use **backup_grafana.sh** to backup all your dashboards and datasources to **/tmp** folder.
* It will create
* two files: **/tmp/dashboards.tar.gz**, **/tmp/datasources.tar.gz**
* two folders contain dashboard files and datasource files: **/tmp/dashboards/$current_time**, **/tmp/datasources/$current_time**
* Use **createFolder.py** to read the folder file and create or update them on Grafana.
* ex: python createFolder.py **file_path**

* Use **backup_grafana.sh** to backup all your dashboards, datasources and folders to **/tmp** folder.
* It will create
* three files: **/tmp/dashboards.tar.gz**, **/tmp/datasources.tar.gz**
and **/tmp/folders.tar.gz**
* three folders contain dashboard files, datasource files and folders file: **/tmp/dashboards/$current_time**, **/tmp/datasources/$current_time** and **/tmp/folders/$current_time**
* ex:**sh backup_grafana.sh**
* result:**/tmp/dashboads.tar.gz**, **/tmp/datasourcess.tar.gz**, **/tmp/dashboards/2016-10-10_12:00:00**, **/tmp/datasources/2016-10-10_12:00:00**
* result:**/tmp/dashboads.tar.gz**, **/tmp/datasourcess.tar.gz**, **/tmp/folders.tar.gz**, **/tmp/dashboards/2016-10-10_12:00:00**, **/tmp/datasources/2016-10-10_12:00:00**, **/tmp/folders/2016-10-10_12:00:00**
28 changes: 18 additions & 10 deletions backup_grafana.sh
Expand Up @@ -4,22 +4,30 @@ current_path=`pwd`
current_time=`date +"%Y-%m-%d_%T"`
compressed_dashboards_name="dashboards.tar.gz"
compressed_datasources_name="datasources.tar.gz"
compressed_folders_name="folders.tar.gz"

echo $current_time

dashboard_backup_folder="/tmp/dashboards/$current_time"
datasource_backup_folder="/tmp/datasources/$current_time"
dashboard_backup_path="/tmp/dashboards/$current_time"
datasource_backup_path="/tmp/datasources/$current_time"
folders_backup_path="/tmp/folders/$current_time"

if [ ! -d "$dashboard_backup_folder" ]; then
mkdir -p "$dashboard_backup_folder"
if [ ! -d "$dashboard_backup_path" ]; then
mkdir -p "$dashboard_backup_path"
fi

if [ ! -d "$datasource_backup_folder" ]; then
mkdir -p "$datasource_backup_folder"
if [ ! -d "$datasource_backup_path" ]; then
mkdir -p "$datasource_backup_path"
fi

python "${current_path}/saveDashboards.py" $dashboard_backup_folder
python "${current_path}/saveDatasources.py" $datasource_backup_folder
if [ ! -d "$folders_backup_path" ]; then
mkdir -p "$folders_backup_path"
fi

python "${current_path}/saveDashboards.py" $dashboard_backup_path
python "${current_path}/saveDatasources.py" $datasource_backup_path
python "${current_path}/saveFolders.py" $folders_backup_path

tar -zcvf "/tmp/$compressed_dashboards_name" $dashboard_backup_folder
tar -zcvf "/tmp/$compressed_datasources_name" $datasource_backup_folder
tar -zcvf "/tmp/$compressed_dashboards_name" $dashboard_backup_path
tar -zcvf "/tmp/$compressed_datasources_name" $datasource_backup_path
tar -zcvf "/tmp/$compressed_folders_name" $folders_backup_path
14 changes: 9 additions & 5 deletions createDashboard.py
Expand Up @@ -10,9 +10,13 @@
with open(file_path, 'r') as f:
data = f.read()

dashboard = json.loads(data)
dashboard['dashboard']['id'] = None
content = json.loads(data)
content['dashboard']['id'] = None

db = {'dashboard': dashboard['dashboard']}
delete_dashboard(dashboard['meta']['slug'])
create_dashboard(json.dumps(db))
payload = {
'dashboard': content['dashboard'],
'folderId': get_folder_id_from_old_folder_url(content['meta']['folderUrl']),
'overwrite': True
}

create_dashboard(json.dumps(payload))
14 changes: 14 additions & 0 deletions createFolder.py
@@ -0,0 +1,14 @@
import json, sys, re, argparse
from dashboardApi import *

parser = argparse.ArgumentParser()
parser.add_argument('path', help='file path saved of folder\' setting')
args = parser.parse_args()

file_path = args.path

with open(file_path, 'r') as f:
data = f.read()

folder = json.loads(data)
create_folder(json.dumps(folder['dashboard']))
44 changes: 34 additions & 10 deletions dashboardApi.py
@@ -1,56 +1,80 @@
import requests, json
import requests, json, re
from grafanaSettings import *


def search_dashboard():
print "search dashboard in grafana:"
r = send_grafana_get(grafana_url + '/api/search/')
r = send_grafana_get(grafana_url + '/api/search/?type=dash-db')
return r.content


def get_dashboard(board_uri):
r = send_grafana_get(grafana_url + "/api/dashboards/{0}".format(board_uri))
status_code = r.status_code
content = r.content
print "query dashboard:{0}, status:{1}".format(board_uri, status_code)
return (status_code, content)


def delete_dashboard(board_uri):
r = requests.delete(grafana_url + "/api/dashboards/db/{0}".format(board_uri), headers=http_post_headers)
status_code = r.status_code
print "status: {0}".format(status_code)
print "msg: {0}".format(r.content)
return int(status_code)


def create_dashboard(payload):
r = send_grafana_post(grafana_url + '/api/dashboards/db', payload)
status_code = r.status_code
print "status: {0}".format(status_code)
print "msg: {0}".format(r.content)
return int(status_code)


def search_datasource():
r = send_grafana_get(grafana_url + '/api/datasources')
print "search datasources in grafana:"
return r.content


def create_datasource(payload):
r = send_grafana_post(grafana_url + '/api/datasources', payload)
status_code = r.status_code
print "status: {0}".format(status_code)
print "msg: {0}".format(r.content)
return int(status_code)

def search_folders():
print "search folder in grafana:"
r = send_grafana_get(grafana_url + '/api/search/?type=dash-folder')
return r.content

def get_folder(uid):
r = send_grafana_get(grafana_url + "/api/folders/{0}".format(uid))
status_code = r.status_code
content = r.content
print "query folder:{0}, status:{1}".format(uid, status_code)
return (status_code, content)

def get_folder_id_from_old_folder_url(folder_url):
if folder_url != "":
# Get folder uid
matches = re.search('dashboards\/[A-Za-z0-9]{1}\/(.*)\/.*', folder_url)
uid = matches.group(1)

response = get_folder(uid)
folder_data = json.loads(folder[1])

return folder_data['id']

return 0

def create_folder(payload):
r = send_grafana_post(grafana_url + '/api/folders', payload)
status_code = r.status_code
print "status: {0}".format(status_code)
print "msg: {0}".format(r.content)
return int(status_code)

def send_grafana_get(url):
r = requests.get(url, headers=http_get_headers)
return r


def send_grafana_post(url, json_payload):
r = requests.post(url, headers=http_post_headers, data=json_payload)
return r
return r
9 changes: 9 additions & 0 deletions restore_folders.sh
@@ -0,0 +1,9 @@
#!/bin/bash

current_path=`pwd`
folders_path="$1"

find "$folders_path" -mindepth 1 | while read f ; do
echo "$f"
python $current_path/createFolder.py "$f"
done
2 changes: 0 additions & 2 deletions saveDashboards.py
Expand Up @@ -8,7 +8,6 @@

folder_path = args.path


def get_all_dashboards_in_grafana():
content_of_all_dashboards = search_dashboard()
dashboards = json.loads(content_of_all_dashboards)
Expand All @@ -33,7 +32,6 @@ def get_indivisual_dashboard_setting_and_save(dashboards):
save_dashboard_setting(board['title'], status_code_and_content[1])
# save_dashboard_setting(board['title'], json.dumps(status_code_and_content[1]))


dashboards = get_all_dashboards_in_grafana()
print_horizontal_line()
dashboard_settings = get_indivisual_dashboard_setting_and_save(dashboards)
Expand Down
37 changes: 37 additions & 0 deletions saveFolders.py
@@ -0,0 +1,37 @@
import argparse
from dashboardApi import *
from commons import *

parser = argparse.ArgumentParser()
parser.add_argument('path', help='folder path to save folders')
args = parser.parse_args()

folder_path = args.path

def get_all_folders_in_grafana():
content_of_all_folders = search_folders()
folders = json.loads(content_of_all_folders)
print "There are {0} folders:".format(len(folders))
for board in folders:
print board['title']
return folders


def save_folder_setting(file_name, folder_settings):
file_path = folder_path + '/' + file_name + '.folder'
with open(file_path , 'w') as f:
f.write(folder_settings)
print "folder:{0} are saved to {1}".format(file_name, file_path)


def get_indivisual_folder_setting_and_save(folders):
for board in folders:
status_code_and_content = get_folder(board['uri'])
if status_code_and_content[0] == 200:
save_folder_setting(board['title'], status_code_and_content[1])

folders = get_all_folders_in_grafana()
print_horizontal_line()
folder_settings = get_indivisual_folder_setting_and_save(folders)
print_horizontal_line()

0 comments on commit 74fa101

Please sign in to comment.