-
-
Notifications
You must be signed in to change notification settings - Fork 17
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support for multiple time series #16
Comments
+1 |
Hi @theodor-n . Thank you very much for your code contribution. I appreciate it very much! On the other hand I would still like to understand the issue a bit better before diving into code changes. If you would provide some more examples for me to grasp the problems with the dataframe format then just switching to a plain old timeseries might be the way to go here, but before jumping into code solutions I would like to clarify the problem a bit more. |
Hi, The problem with the charts is not the use of data frames, per se, but in the format of the data. As in the "old" way, there are (at least) two formats : Table and Time Series. (ref: https://grafana.com/docs/grafana/latest/developers/plugins/data-frames/ The plugin can currently provide output only in the table format, but not in the time series format. Note that the table format is not wrong, it also has its applications, as does the time series format. From this point on documentation is very non-specific, as you have already noticed, however, I did find this: I will shortly provide some examples, in a follow-up post. Edit: another useful source: https://github.com/grafana/grafana/blob/master/pkg/tsdb/sqleng/sql_engine.go |
Sample table data:
Response in table format, single data frame, as currently provided by the plugin: {
"Frames": [
{
"Name": "response",
"Fields": [
{
"Name": "time",
"Labels": null,
"Config": null,
"vector": [time01,time02,time03,time04,time01,time04]
},
{
"Name": "temperature",
"Labels": null,
"Config": null,
"vector": [10,11,20,21,25,26]
},
{
"Name": "city",
"Labels": null,
"Config": null,
"vector": ["London","London","New York","New York","Washington","Washington"]
}
],
"RefID": "",
"Meta": null
}
],
"Error": null
} Response in time series format, multiple data frames. One data frame for eash series, data is grouped by the "city" column {
"Frames": [
{
"Name": "London",
"Fields": [
{
"Name": "Time",
"Labels": null,
"Config": null,
"vector": [time01,time02]
},
{
"Name": "Value",
"Labels": null,
"Config": null,
"vector": [10,11]
}
],
"RefID": "",
"Meta": null
},
{
"Name": "New York",
"Fields": [
{
"Name": "Time",
"Labels": null,
"Config": null,
"vector": [time03,time04]
},
{
"Name": "Value",
"Labels": null,
"Config": null,
"vector": [20,21]
}
],
"RefID": "",
"Meta": null
},
{
"Name": "Washington",
"Fields": [
{
"Name": "Time",
"Labels": null,
"Config": null,
"vector": [time01,time04]
},
{
"Name": "Value",
"Labels": null,
"Config": null,
"vector": [25,26]
}
],
"RefID": "",
"Meta": null
}
],
"Error": null
} Note the fixed "magic" column names "Time" and "Value" (taken from here: https://github.com/grafana/grafana/blob/master/packages/grafana-data/src/dataframe/processDataFrame.ts, TIME_SERIES_TIME_FIELD_NAME and TIME_SERIES_VALUE_FIELD_NAME correspondingly) Again, the table format is not wrong and also has its applications. So ideally the plugin should be able to provide data in one or the other format, similar to the built-in database datasource plugins. |
In addition, here is a sample database and a dashboard: |
I read around in the documentation and backend code regarding data frames. https://grafana.com/docs/grafana/latest/developers/plugins/data-frames/#data-frames-as-time-series What I previously returned was the "long format" of time series'. There is, however, a method to convert the "long format" to the "wide format", which is supposedly better supported in the frontend right now. Please also check out the pre-release I created. It looks fine on the regular time series use cases, but the pie-chart plugin still seems to have trouble https://github.com/fr-ser/grafana-sqlite-datasource/releases/tag/v0.2.4-rc1 |
Is there another example except for the pie chart where the wide data frame is not supported? |
I don't know for other examples - I guess anything which expects multiple time series. About which columns and which conditions, there is this: Basically
I hope this helps, let me know if you have further doubts. |
I tried to go with the simplest possible approach and leveraging whatever the Grafana library already provides. Have a look and check if this makes sense. If I find no bugs in the coming weeks this is what I would release as the initial time series support (which still feels weird, but at least it works 🤷 ) |
Hi, |
No. I don't like mixing too many things together. I might have decided to skip the time series feature or do it in a year only. I'd rather have the related discussion there: #2 (comment) |
I created the new release and with this would consider this issue closed. |
Hi,
This is a somewhat mixed post, related to the PieChart issue and to visualisation panels requiring multiple time series.
It also accidentally covers the $__inteval_ms issue.
I took the liberty of experimenting with your plugin a bit, and I got it working.
What I impleneted:
example:
-- this is a SQL comment which includes the configuration options: format(timeseries)
SELECT "time"/1000.0 as "time", temperature, city
from temperature_data
where "time" >= $__from and "time" <= $__to
order by city, "time"
explanation:
time - time column, temperature - float column, city - string column
With the existing behaviour this would return a single data frame containing a table with 3 columns
with the extended behaviour:
This should somehow mimic the behaviour of the built-in data sources, still returning the output in data frames and not "in the old way"
In attachment there are 2 sources:
query.go - this is the original one with a few added code lines to activate the new behaviour
extension.go - this is the new code which implements the logic described above
extension.zip
Final words:
What I built is not very elegant (I'm an absolute novice in Go), its not integrated with your solution but basically sits on top of it.
I was a kind of hoping that having a working "prototype" would give you ideas on how to properly implement this (or analogous) functionality in the "official" plugin. Please feel free to use the attached code as you see fit.
Please let me know what you think and also share a preliminary opinion on whether you are willing to implement/integrate similar time series behaviour in the plugin.
Kind Regards,
Theodor
The text was updated successfully, but these errors were encountered: