Repository navigation
Support per-query timezone offsets #2074
Description
Activity
Doing a simple +/- is not sufficient for full timezone support because of daylight savings (PST vs PDT depending on the time of year).
For example, if you have a
GROUP BY time(1d), the size of the day/bucket can vary depending on it's time of year. This makes it extremely messy to implement with the current method. That said, it is exactly what I need for my application.Right now, I am planning on implementing an index on top of influx for this purpose, but would love to not have to do that.
- added this to the milestone
on Mar 25, 2015 IMO there should be both options.
Query-wide parameter covers most of the cases but function that just returns offset based on supplied timezone (which would be necessary for per-query operator anyway) is much more convenient than doing that client-side
Regarding "any other questions", can support be provided for falling edge v rising edge groupings?
time_zone as a "number" should be a four number string too (some XX30 tz)
Not sure if you're still debating format, but I'd recommend supporting the Olsen / tz database. See http://joda-time.sourceforge.net/timezones.html. It supports named timezones, as well as UTC offsets (which are confusingly backwards, ie. Etc/GMT+5 is actually GMT-5). This is the closest thing to a standard I've ever seen.
Reacted by Francis De Brabandere and Jiang Xu+1 for Olsen tz
... with timezone('america/phoenix')
why not write a timezone item in conf file?
+1
+1
another +1 for olsen db rather than literal offsets or any other named time zones
though I should also point out that timezones aren't the only problem. if I do a group by time(30m), I might want my groups to be between 15 and 45, so that the "middle" of the grouping is on the hour and half hour, when I expect my peak load due to user traffic patterns. and that has nothing to do with timezone - it's more "offset the grouping by a certain amount"
select mean(value) from cpu where time >= today() group by time(10m) time_zone(+8)It may be the best
I would use a configurable default timezone + query param to override it
I eagerly awaited this feature.
Without this feature,I can not use group by time(1d) .Make timezone can be configurable(in influxdb.conf) maybe is easier.
Can add this feature in the 0.9.2 release version first?Reacted by Airyland and ring0li+1
We are fetching sums and grouping by 1 day and can not get accurate stats because of this timezone issue.+1 for setting default timezone in config file.
Currently, I'm using a quick workaround which consists in using a cron script or program to retrieve the average of the day (executed at 00:01 AM for example) of the desired series from InfluxDB, using the correct start and end timestamps in the WHERE clause of the query.
After that, insert the data in a new series with the timestamp of the beginning of the day and you'll be able to retrieve the correct data in your queries.
56 remaining items
@mattbasta assuming 8h could be augmented with minute based offset, there's still the issue of DST which can't be dealt with like this :(
@mattbasta all good and well to hold the opinion, however there are ample use cases and it's a fairly standard requirement for a database dealing with datetimes to be capable of dealing with the real world and its complexities. Imho this is a long outstanding issue that needs resolving by the Influx core team and I know I am not alone. Until that happens should we implement Influx to meet our tsdb needs we will be limited to aggregating at a half hour level in the database and then completing any aggregation in the application. This puts it at a disadvantage when comparing with other options.
If one saves data in the database in UTC then daily aggregates can simply be done by converting to the right TZ before aggregration, this will handel DST too.
Aggregation should be something like this:select mean(value) from cpu group by time(1d,"CET")
And in the backend (i am not a go programmer):t := [time read from DB, should be in UTC] tz, _ := time.LoadLocation("CET") // get TZ identifier from SQL select clause t.In(tz) // convert time before group by ... // after this do the normal GROUP BY.Normal select queries will return data in UTC (as the data is saved in UTC) but Clients can convert to local TZ (which is good practice anyway since clients might be in diffrent TZ)
@mattbasta PST and PDT are not time zones. They are simply defined offsets (-8 and -7). Pacific Time is a time zone, and includes the information about when it changes offsets (DST), as well as nicknames for those offsets. And it's not hard for software to support it. There is a well maintained database of time zone information that software can use to know what the DST rules are for any given time zone in any given year (within practicality): https://en.wikipedia.org/wiki/Tz_database. Usually software will support the tzinfo data files, and rely on the operating system to keep them up to date.
I deleted my comment. I don't want to be involved in this discussion. Feel free to argue all you want.
Influx should store data with a UTC timestamp. Formatting timestamps should be a display detail.
The complete solution to timezone handling is to provide a timezone formatting mechanism. It should be possible to just use something like this with time: format(time,'Central Standard Time'). There can not be any solution based on using "server time" or "client time" etc. Queries simply need to be able to format time. Any query inferring time, such as group by and others, should be using the timezone of the locale that influx can detect at the server to shift time relative to UTC.@greggwon you're correct except for the rather large use case of aggregation methods by calendar days (weeks, months, years etc), then InfluxDB needs to be able to manage timezone offsets. In this case it is not merely a display detail but a functional requirement :)
+1
+1
Aggregation becomes a "display detail", because you are now visualizing. The functional definition as shown in the sample query is not my point. My point is that there should be a common time reference that is automatically known for all data. Data should not float around in varied Timezones and then have to be coerced to another in random fashion. Instead, there should be a single base timezone reference and then "visualization" can specify what timezone is important.
In the sample query, the 'time' comparison to "today()" also implies a visualization (extraction of the data for use) and so today() should also be offset by a timezone offset. today() needs the parameter, not a new function which will randomly interject problems to all other 'time' values in an expression. Each use of a time function, should by default use the server timezone offset but allow a parameter which specifies the timezone offset applicable.
Yes, that becomes more places that the offset must be specified, but it allows the developer/application the explicit control needed to make sure that we don't suddenly have offset math strewn about the query with various +21600000, and -21600000 etc.
I will be locking this issue. Litigating the result of a closed issue that was opened two years ago and resolved six months ago is unfair to the people subscribed to the issue and I think it is unfair to forcibly involve others in the discussion.
With that said, I am happy to share the rationale for why the issue was resolved in the way it was resolved. But, the proper location for that question is on our community forums and not the issue tracker. @greggwon if you want to continue discussion, please create an account on that website and post a new topic. I will keep an eye out for the next day to see if there is any discussion about time zones there.
If you have issues with the way it was resolved that pertain to a valid use case of time zone math, please open an issue. I do not want to fail to address valid use cases since I want the database to be as useful to as many people as possible. If there are problems with the existing implementation, I want to address them so that everyone benefits from a better database.
One clarification that might be needed for most people (since I have seen this confusion before), the TZ parameter does not affect any time values in the actual query. It affects time boundaries. The time
2000-01-01T00:00:00Zalready has a time listed. It's UTC and the time zone parameter cannot and will not affect that.Time zones are a disabled feature by default. If you don't include the
TZ()parameter, no time zone math will be done. And the previous group by offset feature introduced in 1.1 or 1.2 is still there for other use cases.I hope that clarification helps and look forward to your continued engagement with the InfluxData community.
- locked and limited conversation to collaborators
on Jul 21, 2017
Extending the discussion started in #2071 and re-starting the old discussion from #388, we should support time zones on a per-query basis.
A quick example might look like:
Or you could also do
time_zone(+8)ortime_zone(-2).Any other suggestions?