Grafana 2.0: SQL Data source

[torkelo]

With the backend comes the possibility to have SQL data source.

My thinking is that when you add the data source you

• db type (initially only mysql and postgres and sqlite3)
• db connection details
• specify a metric query template (basically a SQL query with params)
• specify a annotation query template

Maybe also an option to allow RAW SQL queries from the panel metric query interface.

Any other ideas?

[syepes]

I would personally suggest some thing more in the line of Cassandra (CQL) or SparkSQL

[phagedorn]

Wow, this us very usefull.

• add a array of db servers for loadbalancing
• a field that concatenate a LIMIT X to the query
• a switch for ORDER BY ASC DESC
• chose db columns for TIME and for VALUE from mysql table ?

[torkelo]

@phagedorn some like that, not sure about the switch for order asc/desc, grafana always wants them in ascending order (by time). not sure about the limit x either.

more like a metric query template (written by user when they add the data source), this just an example, have not thought about this much yet.

SELECT MyValue as Value, Timestamp as Time 
  FROM MyMetrics
  WHERE SeriesName IN (?SeriesList) AND Time > (?TimeFrom) AND Time < (?TimeTo)
  ORDER BY Timestamp ASC 

@syepes I thinking as first step to implement something simple. Could you describe more what it would take to implement CQL or SparkSQL on top of a generic SQL database and what benefits? SparkSQL looks interesting from analaytics standpoint, any golang implementation?

[sysadminmike]

+1 for allowing raw SQL

[phagedorn]

@torkelo : Query templates sounds nice 👍
Would work for me for existing tables

[nucleusv]

Will this datasource has a suggestion editor? What DB backends for this datasource can be used ?

[rippy4]

also interested in allowing raw SQL

[torkelo]

Initially mysql and postgres, no auto complete isn't going to be included initially

[huhongbo]

interested +1

[sahilthapar]

👍 for postgres

[pskerrett]

👍 for postgres

[shirou]

👍 for postgres with JSON type

[PRI-mcallen]

👍 for postgres with JSON type

[juliusloman]

Any idea about the roadmap when SQL datasource will be available? In 2.0 or 2.1?

[torkelo]

@juliusloman not sure, maybe in 2.2 or 2.3, so much other stuff that was pused to 2.1 that needs to happen first

[torkelo]

But a PR is always welcomed

[hceylan97]

Having a SQL data source will definitely make grafana a top analytic tool. I would love to see it. Torkel, sorry not familar with github, what does PR entail?
+

[torkelo]

@hceylan97 it entails that someone tries implement it and submits it as pull request (i.e feature patch), I would love to implement this, but not sure when I will have to do it as I will probably have more high prio issues for the next couple of months

[hceylan97]

besides you, curious if anybody else is working on this?

[gsaray101]

👍

[Esysteme]

+1 mysql

[agilgur5]

@torkelo I might start a PR for this pending the expected amount of work. Would love to figure out how to make new datasources so we can integrate more into Grafana. Could you give a rundown of what needs to be done to add a new data source and the work remaining for generic SQL support?

[nucleusv]

+1 Oracle

[timgriffiths]

+1 MS SQL

[imZack]

👍 for postgres with JSON type

[pabloa]

+1 postgres / postgres with JSON type

[jleclanche]

Redshift SQL is just Postgres 8.x family, which should be compatible with the recently-landed Postgres support. Haven't tried it yet but also interested if there's any errors.

[svenklemm]

If you dont mind proxying data through postgres you can connect grafana to (almost) any database with postgres foreign data wrapper (https://wiki.postgresql.org/wiki/Foreign_data_wrappers).

[DenRozhko]

+1 Oracle DB

[tammytankian]

+1 for MS SQL

[646826]

+1 for SQLite

[linuxchips]

hi, any one interested in mssql, please check pr #10093

[gsaray101]

Has anyone worked on Oracle as data source? I would love to see it.

[jjeanis]

@gsaray101 and anyone who's interested - contact support@grafana.com if you'd like to test the Oracle beta datasource.

[daniellee]

We have merged the Microsoft SQL Server data source to Grafana and it will be released in Grafana 5.1 (#10093, #11298).

This means that Grafana now has support in core for MySQL, Postgres and MS SQL Server as data sources. We will not be adding any more sql databases as data sources to core Grafana so it is finally time to close this issue.

In the near future, we will have support for backend plugins so it will be possible to have other sql data sources as external plugins.

[blenard81]

Anyone looking at adding DB2 LUW support?

[mnlbox]

@daniellee What about Oracle and SQLite? 🤔 Any news about this?

[daniellee]

@mnlbox There is an Oracle plugin already: https://grafana.com/plugins/grafana-oracle-datasource (It is not open source however)

Sqlite as a datasource is not on our backlog at all and I haven't heard of anyone working on it.

[suntzu93]

+1 for Presto

[jd4900]

Any update on SQLite @daniellee ?

[quentinus95]

SQLite datasource would be very useful!

[OverWorldD]

Sqlite!!!!!

[646826]

Sqlite !!!!!

[adlerweb]

Sqlite!!!!!

Sqlite !!!!!

Please use the 👍 function to show your approval and comment only if you can supply additional information, helpful notes, patches and similar - well - comments helping to solve the issue. Spamming developers, contributors or participants probably won't convince anyone to implement your requests.

Daniellee stated above there are no further efforts taken to support additional datasources in core and plugins are the way to go. Also no one seems to have started working on SQLite so far. If you need a quick and dirty solution and don't want to write/commission/adapt a whole plugin for/to SQLite it should be somewhat easy to build a smapp proxy-script to serve your SQLite-data as JSON similar to doublemarkets RRD-Proxy. Not a great solution speedwise, but you probably wouldn't be using SQLite if thats a concern.

[daniellee]

As @adlerweb says, there are currently no plans for the core Grafana team for a core Sqlite datasource. I don't think we would accept a PR for it either. However, we would of course, publish an external datasource plugin on grafana.com if someone wrote it.

[oz123]

Any update on SQLite @daniellee ?

For those interested in SQLite support (or in fact those who wait for any data source), you don't need to wait long. It's pretty easy to write your own data source using Python. The documentation is a bit sparse (see https://github.com/grafana/simple-json-datasource), but it's possible. I have created a fairly extensive example in this repository and a blog post on how to visualize SQLite with Grafana . The repository also includes a working example of SQLite and a small database for this example.

[leandro518]

• SQLite vote !

[vevdokimov04041976]

vote for sqlite

[loveen]

With the backend comes the possibility to have SQL data source.

My thinking is that when you add the data source you

• db type (initially only mysql and postgres and sqlite3)
• db connection details
• specify a metric query template (basically a SQL query with params)
• specify a annotation query template

Maybe also an option to allow RAW SQL queries from the panel metric query interface.

Any other ideas?

For Sqlite

[connaisseur]

Vote for sqlite.

[Jul10l1r4]

Sqlite plsss

[oz123]

How come this is not mentioned here?
https://github.com/fr-ser/grafana-sqlite-datasource/blob/main/docs/installation.md