Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for daylight savings time #211

Closed
wgbartley opened this issue May 27, 2014 · 27 comments

Comments

@wgbartley
Copy link

commented May 27, 2014

Add support for daylight savings time to Time library, notably Time.zone().

https://community.spark.io/t/time-zone-and-daylight-savings-and-spark-synctime/4568

@kennethlimcp

This comment has been minimized.

Copy link
Contributor

commented Oct 25, 2014

This can be closed as Time.zone() is already implemented in the latest firmware and #280 has tested it to be working correctly except for minor improvement to be decided.

@m-mcgowan

This comment has been minimized.

Copy link
Contributor

commented Apr 5, 2015

Are you sure Time.zone() solves the problems requested in the thread above? It appears that at the very least a time-zone descriptor is needed, and that the cloud returns the current time zone offset for that descriptor and the time of the next change.

@andyw-lala

This comment has been minimized.

Copy link
Contributor

commented Apr 5, 2015

Seems to me that the timezone should be a persistent customer settable
attribute of a core. The cloud should always just deal with UTC. That
implies that the core needs to calculate local time, there are table-driven
shortcuts that can be used and/or refreshed from the cloud.
On Apr 5, 2015 7:48 AM, "Matthew McGowan" notifications@github.com wrote:

Are you sure Time.zone() solves the problems requested in the thread
above? It appears that at the very least a time-zone descriptor is needed,
and that the cloud returns the current time zone offset for that descriptor
and the time of the next change.


Reply to this email directly or view it on GitHub
#211 (comment).

@m-mcgowan

This comment has been minimized.

Copy link
Contributor

commented Apr 5, 2015

Moving the timezone calculations locally with updates from the cloud is a significant undertaking, considerably more than having the cloud perform these operations. What's the benefit of having this done locally when we already make use of a cloud connection to retrieve the current time?

I agree the cloud always works in UTC, and that doesn't preclude it also providing a timezone offset for a given time zone descriptor, which is then used by the core to calculate local time. The timezone descriptor may be stored locally, or the cloud may be able to provide the correct timezone via GeoIP services.

@andyw-lala

This comment has been minimized.

Copy link
Contributor

commented Apr 5, 2015

I guess if the core spits out the (locally configurable, and persistant) TZ
choice as part of the initial handshake, that would be perfectly fine,
except that doesn't handle transitions across clock forward/backward
events, so that would require the cloud to be able to push a new local time.

It is the corner cases that hurt with TZ implementations.

On Sun, Apr 5, 2015 at 10:09 AM, Matthew McGowan notifications@github.com
wrote:

Moving the timezone calculations locally with updates from the cloud is a
significant undertaking, considerably more than having the cloud perform
these operations. What's the benefit of having this done locally when we
already make use of a cloud connection to retrieve the current time?

I agree the cloud always works in UTC, and that doesn't preclude it also
providing a timezone offset for a given time zone descriptor, which is then
used by the core to calculate local time. The timezone descriptor may be
stored locally, or the cloud may be able to provide the correct timezone
via GeoIP services.


Reply to this email directly or view it on GitHub
#211 (comment).

Andy

@bkobkobko

This comment has been minimized.

Copy link
Contributor

commented Apr 5, 2015

So GeoIP is not the answer for daylight savings time and time zone. People need to be able to control it more finely and directly than that.

There are parts of Indiana in the US that are EST/EDT but other parts are CST/CDT depending mostly on whether or not the people living there work in Chicago on central time. Arizona is another exception as it has no DST but towns on the border are mixed.

The rules that are in a typical Unix timezone database still need to be tweaked for local operation.

I think the cloud should provide UTC and the core should have locally available controls to offset from that. If the cloud want to provide other services like translating "New York/New York" into TZ = -5, DST = 1, I think that should be a separate service so users can choose to use or ignore the return values as they see fit.

@andyw-lala

This comment has been minimized.

Copy link
Contributor

commented Apr 5, 2015

But the offset isn't set and forget unless you have local tables to handle
the transitions.

Code that supports the current known rules is obviously freely available, I
have not analysed the ram footprint (I figure we don't care that much
about code & const space.) This won't help when lawmakers decide to change
things, as happened in the US recently.

Alternate ideas are for the core to communicate a desired TZ, the cloud can
provide the next n years of offset data (n = 2 seems more than enough for a
device whose value prop is the cloud) - that would be 4 timestamps and
matching offsets - not an onerous burden. These would need to be refreshed
every time the TZ setting on a core changed, or when it was booted, or just
to ensure that there was at least n years of runway.

If the problem is punted completely to the cloud the core would be unable
of adapting local time at a transition, if it happened to be offline during
that window (imagine a data logger that accumulated data for a day, and
disconnected to save power, it would not be aware of the spring
forward/fall back activity at 2AM (or whenever) until it next connected.
Any attempt to provide local intelligence (according to a simple rule set)
quickly collapses onto the case described above.

This isn't rocket science, but it is subtle if you want cores to be
autonomous and accurate in the absence of an active connection to the cloud.

I think geo-ip is a very bad idea, I frequently VPN trunk connections all
over the place, and the "smart" software that wants to help me is
laughable. I think it will be much worse for electron, although you may
be able to get other geo hints from the carrier at that point. I strongly
believe letting the user set (and forget) it as an attribute is the right
way to go.

Not saying this needs to happen now, but I think having a solid design that
marks the point on the horizon you want to head towards has enormous value.

On Sun, Apr 5, 2015 at 12:18 PM, Brian Ogilvie notifications@github.com
wrote:

So GeoIP is not the answer for daylight savings time and time zone. People
need to be able to control it more finely and directly than that.

There are parts of Indiana in the US that are EST/EDT but other parts are
CST/CDT depending mostly on whether or not the people living there work in
Chicago on central time. Arizona is another exception as it has no DST but
towns on the border are mixed.

The rules that are in a typical Unix timezone database still need to be
tweaked for local operation.

I think the cloud should provide UTC and the core should have locally
available controls to offset from that. If the cloud want to provide other
services like translating "New York/New York" into TZ = -5, DST = 1, I
think that should be a separate service so users can choose to use or
ignore the return values as they see fit.


Reply to this email directly or view it on GitHub
#211 (comment).

Andy

@bkobkobko

This comment has been minimized.

Copy link
Contributor

commented Apr 5, 2015

My NTP library for Spark uses 46 bytes for US and 46 bytes for European DST rules, all in flash, to cover up to 2036. This was an arbitrary decision but more than 20 years felt good. And this could be simplified I am sure--I just looked up the transition points.

Using my library, you first set the time zone, and then say if you are using DST, and then further say if you want US or Euro DST. Three separate controls, basically. The exact API is not important but the controllability is, in my opinion.

@pomplesiegel

This comment has been minimized.

Copy link

commented May 20, 2015

As a Spark/Particle :) user, I find this feature quite essential. Our product contains a physical front panel w/ analog pots representing time settings. For this UI to be valid year-round, we'll have to send Time.zone() updates OTA to just our users observing DST - not the most elegant or scalable solution.
It would be fantastic if the Spark could obey local DST rules.

@bkobkobko

This comment has been minimized.

Copy link
Contributor

commented May 20, 2015

Hi @pomplesiegel

You will always have to give users control over DST vs no-DST in the US since there are some regions where DST rules are not followed.

That said I hope this feature gets some love in the future too.

@pomplesiegel

This comment has been minimized.

Copy link

commented May 20, 2015

Understood. Thanks for your response!

@m-mcgowan

This comment has been minimized.

Copy link
Contributor

commented Aug 23, 2015

Does anyone have any insight into resources we might use to build this. Is pushing the timezone setting out to the user a workable solution?

@andyw-lala

This comment has been minimized.

Copy link
Contributor

commented Aug 24, 2015

Does everyone agree that TZ location as a persistent attribute of a
particular device is a good idea ?

If we can agree on that, then we can work on the best cloud/local division
of labour and the appropriate mechanism.
On Aug 23, 2015 18:57, "Matthew McGowan" notifications@github.com wrote:

Does anyone have any insight into resources we might use to build this. Is
pushing the timezone setting out to the user a workable solution?


Reply to this email directly or view it on GitHub
#211 (comment).

@m-mcgowan

This comment has been minimized.

Copy link
Contributor

commented Aug 24, 2015

Sure, we can start there and see where it leads.

@andyw-lala

This comment has been minimized.

Copy link
Contributor

commented Aug 24, 2015

How does a (fleet) owner set the TZ for a particular device ? Should it be
a cloud-based attribute ?
On Aug 23, 2015 7:16 PM, "Matthew McGowan" notifications@github.com wrote:

Sure, we can start there and see where it leads.


Reply to this email directly or view it on GitHub
#211 (comment).

@bkobkobko

This comment has been minimized.

Copy link
Contributor

commented Aug 24, 2015

Does everyone agree that TZ location as a persistent attribute of a
particular device is a good idea ?

For Core and Photon this seems like an OK idea as service that (say) the cloud provides with a unique TZ value per device. Still there will be folks that want to run their sensor network in GMT anywhere in the world because they need to incorporate sensor data from lots of locations. I think you will want Spark.subscribe("spark/...") type of interface so users can opt in or out.

For Electron, I am not sure that TZ should be a persistent attribute of a particular device. It is just a much more mobile oriented platform.

@andyw-lala

This comment has been minimized.

Copy link
Contributor

commented Aug 24, 2015

My point is that the user/deployer gets to decide what TZ they want each
device in - given an API to manipulate it, then it can even be set byt the
device itself if the user writes startup/config code to trigger that
behaviour (think device with LCD screen & buttons, or a smartphone app for
user config.)

You could envisage an AUTO setting for the Electron (which returned an
error on Core/Photon) that used locality info from the carrier (if/when
available.)

On Sun, Aug 23, 2015 at 11:08 PM, Brian Ogilvie notifications@github.com
wrote:

Does everyone agree that TZ location as a persistent attribute of a
particular device is a good idea ?

For Core and Photon this seems like an OK idea as service that (say) the
cloud provides with a unique TZ value per device. Still there will be folks
that want to run their sensor network in GMT anywhere in the world because
they need to incorporate sensor data from lots of locations. I think you
will want Spark.subscribe("spark/...") type of interface so users can opt
in or out.

For Electron, I am not sure that TZ should be a persistent attribute of a
particular device. It is just a much more mobile oriented platform.


Reply to this email directly or view it on GitHub
#211 (comment).

Andy

@pomplesiegel

This comment has been minimized.

Copy link

commented Sep 28, 2015

Any love for this guy? :)
It seems a bit clunky that to manually implement DST now we'll have to set Time.zone() OTA on all devices in our fleet.

@m-mcgowan

This comment has been minimized.

Copy link
Contributor

commented Sep 30, 2015

I'm happy to implement something here, but I don't know the best way forward that will properly account for DST time in all cases. Advice gratefully received.

@pomplesiegel

This comment has been minimized.

Copy link

commented Sep 30, 2015

Totally understood!

From an interface perspective, providing FW support for

Time.enableDST(true)

and leaving the onus on the user application code designer to set that OTA using their own methods would still be a major step forward, and perhaps serve as a MVP for later features like settability through the Particle dashboard.

Currently we set Time.zone() OTA on our fleet depending on customer location (chosen by the user through the app), so an additional state to set like DST would be totally reasonable.

My thought is within the setup portion of our app, following successful WiFi setup the user would be prompted for

  1. Location/Time-zone (function call setting Time.zone() OTA )
  2. Enable/Disable DST (function call setting Time.enableDST() OTA)

Would these two pieces of information be enough for the Photon to locally determine the proper time based on a LUT or locally stored rules? Perhaps these could be compiled as Const into the system-fw code?

@m-mcgowan

This comment has been minimized.

Copy link
Contributor

commented Sep 30, 2015

I'm not understanding the connection of Time.zone()/Time.enableDST() and OTA (over the air?)

@pomplesiegel

This comment has been minimized.

Copy link

commented Sep 30, 2015

Sorry for the confusion.

Currently we call Time.zone() on a Photon in a customer's home OTA (over the air) using something similar to

particle call <photon name> multiFunc "timezone-<TZ offset>"
where multiFunc is a Particle.function() which is multiplexed out to many different possible function calls within the user FW, due to the restriction in the number of Particle.functions which can exist concurrently. In this case, by analyzing the first part of the string argument, it sees that you would like to call Time.zone(<TZ offset>), so it makes that call.

Enabling DST support on Photon, which could be enabled by Time.enableDST(true), would allow us to enable/disable it on each photon using a call like
particle call <photon name> multiFunc "enableDST-<1 || 0>"

  1. Does that make sense?
  2. As I asked before, would these two pieces of information (time zone + DST on/off) be enough for the Photon to locally determine the proper time based on a LUT or locally stored rules? Perhaps these could be compiled as Const into the system-fw code?
@m-mcgowan

This comment has been minimized.

Copy link
Contributor

commented Oct 1, 2015

Thanks for clarifying. Makes sense. From the discussion above, it seems that having locally stored rules doesn't provide a solution for all cases due to some anomalies of where DST is active and where it isn't. So I'm thinking that the system first provides the ability for some other source of truth to determine whether DST is active:

// Set the timezone without DST applied. (No change here, mentioning to reference how it works in conjunction with DST.)
Time.zone();

// add the offset from getDSTOffset() to the current time
Time.beginDST();
// do not add the offset from getDSTOffset() to the current time
Time.endDST();

// retrieve the current DST offset that is added to the current local time when Time.beginDST() has been called. The default is 1 hour
float Time.getDSTOffset(); 
Time.setDSTOffset(float offset); 

These are relatively low-level functions that allow the application developer/additional libraries to fully manage DST. The system makes no decision about whether DST is in effect and how much the DST offset is - that is all left to the application developer, or to additional libraries...

...Additional libraries!

This library here provides quite a high level interface to managing locales and timezones. https://github.com/JChristensen/Timezone. This would allow the user to define the most relevant TZ info for their app, plus perhaps having also a system default table that can be used.

TimeChangeRule usEDT = {"EDT", Second, Sun, Mar, 2, -240};  //UTC - 4 hours
TimeChangeRule usEST = {"EST", First, Sun, Nov, 2, -300};   //UTC - 5 hours
Timezone usEastern(usEDT, usEST);

// calls Time.zone(), Time.enableDST(), Time.setDSTOffset() using a DST setting determined from
// the current UTC time and the data in the Timezone instance.  
Time.timezone(usEstern);

How does that sound?

@pomplesiegel

This comment has been minimized.

Copy link

commented Oct 1, 2015

That looks great! Thanks for looking into this.

@m-mcgowan m-mcgowan added this to the 0.6.x milestone Mar 18, 2016

@wgbartley

This comment has been minimized.

Copy link
Author

commented Jun 3, 2016

This may be an old bump, but I noticed it was added to the 0.6.x milestone, so I thought I would think about it, even if just for inspiration for a real implementation.

Would it be possible to send a time zone offset as part of the cloud handshake when the time is sent? If so, the user could specify their desired time zone in string format (e.g. America/New_York) in the STARTUP(...) macro. That request could be sent as part of the initial cloud connection/handshake. When the cloud sends the time to the device, it could also send the timezone offset. The device would apply the offset to the time received. For bonus points, the cloud could also sent the next time DST changes so the device can handle it in the background at the appropriate time.

  1. User sets DST zone in STARTUP() macro (so it can be sent with initial cloud handshake).
  2. Cloud sends current UTC time, request time zone DST offset, and next DST change.
  3. Device automagically applies DST offset to UTC time received.
  4. Periodic cloud time syncs includes time, DST offset, and DST change.
  5. Device applies new DST offset at DST change time.
  6. Lather. Rinse. Repeat.

User STARTUP(...) might look like:

void my_startup_function() {
    Time.setDSTOffset('America/New_York');
}

STARTUP(my_startup_function);

IANA maintains a timezone database that can be periodically updated on the cloud servers -- http://www.iana.org/time-zones

@mebrunet

This comment has been minimized.

Copy link

commented Jun 7, 2016

@wgbartley - We've implemented this ourselves using a webhook.

  1. Device publishes its user set IANA timezone to a webhook event stream. e.g. "America/New_York"
  2. Device receives tzInfo: {currentOffset: -4.0, nextOffset: -5.0, nextOffsetAt: 1478412000.0}
  3. Device stores tzInfo in EEPROM.
  4. Run the below code on reboot and every 15 minutes:
if (Time.now() < nextOffsetAt) {
    Time.zone(currentOffset);
} else {
    Time.zone(nextOffset);
    reSyncTzInfo = true;
}  

Not familiar with the details of all foreign timezones, but from what I can tell this allows the device to run (possibly offline) for several months between syncs.

It would be nice if something like this was implemented natively... I'm sure lots of people need to deal with the same thing.

@m-mcgowan

This comment has been minimized.

Copy link
Contributor

commented Jun 7, 2016

That's a nice idea @wgbartley! It's more than we can implement this sprint for 0.6.x, but possibly something that can be made available in a future release. For 0.6.x, we'd like to provide the basic timezone hooks so that the system can save the timezone offset and provide hooks to enable/disable DST, as outlined here - #211 (comment)

avtolstoy added a commit that referenced this issue Jul 10, 2016
avtolstoy added a commit that referenced this issue Jul 10, 2016
@avtolstoy avtolstoy referenced this issue Jul 10, 2016
7 of 7 tasks complete

@technobly technobly closed this Aug 9, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
9 participants
You can’t perform that action at this time.