Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.
Sign upAmbiguities in UTC()'s spec and mis-specification of DaylightSavingTA() #725
Comments
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
ediosyncratic
Nov 1, 2016
Unfortunately, because the Date() constructor [0] embeds a call to UTC(), the problems above with UTC()'s spec do in fact imply problems for constructing Date() objects representing certain dates and times. Because the Date(y,mon,[d,h,min,s,ms]) constructor interprets its arguments in local time, yet yields a result that wants to believe it is a UTC time, there are times (in local time's fall-backs) that cannot be represented (the implicit disambiguation selects the other local time with the same seconds since epoch, but a different implicit nominal epoch).
If a zone makes its DST transition at midnight, this can break the "obvious" way to construct a Date() representing the day the transition happens; Date(year, month, day) selects 00:00 local-time at the day's start, but UTC()'s half-hearted attempt at correcting for local offset when determining DaylightSavingTA() leads to it getting a time actually an hour earlier, in the preceding day once represented as a local time: see [1].
[1] https://bugreports.qt.io/browse/QTBUG-54559
In short, the failure to make UTC(Localtime(t)) == t and Localtime(UTC(t)) == t, combined with embedding UTC() in Date(), makes some times (in any zone with DST) unrepresentable and (in some zones) forces unnatural handling of some dates as Date()s. This could be ameliorated by giving Date() more optional arguments, to avoid the ambiguity by letting the caller specify the given data is in fact in UTC already, or specify which side of the DST transition they are.
[Edit: well, "unrepresentable" claims too much; there are times that Date() can't directly give me, but other methods can solve the problem. It remains painful that Date() is broken.]
ediosyncratic
commented
Nov 1, 2016
•
|
Unfortunately, because the Date() constructor [0] embeds a call to UTC(), the problems above with UTC()'s spec do in fact imply problems for constructing Date() objects representing certain dates and times. Because the Date(y,mon,[d,h,min,s,ms]) constructor interprets its arguments in local time, yet yields a result that wants to believe it is a UTC time, there are times (in local time's fall-backs) that cannot be represented (the implicit disambiguation selects the other local time with the same seconds since epoch, but a different implicit nominal epoch). If a zone makes its DST transition at midnight, this can break the "obvious" way to construct a Date() representing the day the transition happens; Date(year, month, day) selects 00:00 local-time at the day's start, but UTC()'s half-hearted attempt at correcting for local offset when determining DaylightSavingTA() leads to it getting a time actually an hour earlier, in the preceding day once represented as a local time: see [1]. [1] https://bugreports.qt.io/browse/QTBUG-54559 In short, the failure to make UTC(Localtime(t)) == t and Localtime(UTC(t)) == t, combined with embedding UTC() in Date(), makes some times (in any zone with DST) unrepresentable and (in some zones) forces unnatural handling of some dates as Date()s. This could be ameliorated by giving Date() more optional arguments, to avoid the ambiguity by letting the caller specify the given data is in fact in UTC already, or specify which side of the DST transition they are. [Edit: well, "unrepresentable" claims too much; there are times that Date() can't directly give me, but other methods can solve the problem. It remains painful that Date() is broken.] |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
bterlson
Nov 14, 2016
Member
I can and will fix the lacking notes you mention (that UTC(Local(t)) !== t, Local(UTC(t)) !== t, and that t is presumed to be a UTC time), but most of this is a valid critique of the current date design. Changes here must be advanced via the standard staged proposal (see https://github.com/tc39/proposals for more details).
|
I can and will fix the lacking notes you mention (that UTC(Local(t)) !== t, Local(UTC(t)) !== t, and that t is presumed to be a UTC time), but most of this is a valid critique of the current date design. Changes here must be advanced via the standard staged proposal (see https://github.com/tc39/proposals for more details). |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
jungshik
Jan 11, 2017
Contributor
While working on https://bugs.chromium.org/p/v8/issues/detail?id=3547, I realized that Ecma262 spec has a few issues.
I second @ediosyncratic on Date() in general and the timezone offset handling ( UTC() , LocalTime() and Date(y,mon,[d,h,min,s,ms]) interpreting the input parameters in local timezone ) in particular.
Assuming that LocalTZA is constant over the history of a given timezone is very problematic. For instance, Europe/Moscow changed its timezone offset (and whether or not to use DST) multiple times, but LocalTZA is always calculated to be the LocalTZA at run-time.
One example: Europe/Moscow was UTC+0400 in Sep 2014, but it's now UTC+0300 (year-round). See the result below:
d8> Sep1_2014 = new Date("2014-09-01T20:15Z")
Mon Sep 01 2014 23:15:00 GMT+0300 (MSK) <=== Sep 02 2014 00:15:00 GMT+0400
d8> Sep1_2014.getHours()
23 <=== should be 0
d8> Sep1_2014.getDay()
1 <==== should be 2
|
While working on https://bugs.chromium.org/p/v8/issues/detail?id=3547, I realized that Ecma262 spec has a few issues. I second @ediosyncratic on Date() in general and the timezone offset handling ( UTC() , LocalTime() and Date(y,mon,[d,h,min,s,ms]) interpreting the input parameters in local timezone ) in particular. Assuming that LocalTZA is constant over the history of a given timezone is very problematic. For instance, Europe/Moscow changed its timezone offset (and whether or not to use DST) multiple times, but LocalTZA is always calculated to be the LocalTZA at run-time. One example: Europe/Moscow was UTC+0400 in Sep 2014, but it's now UTC+0300 (year-round). See the result below:
|
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
jungshik
Jan 17, 2017
Contributor
It turned out that Firefox does the right thing from a user's point of view not following the spec to its letters. (the result in my previous comment was taken from v8/Chromium).
WIth the timezone set to Europe/Moscow, I got this from Firefox.
Jul31_2014 = new Date("2014-07-31T20:15Z")
Date 2014-07-31T20:15:00.000Z
Jul31_2014.toString()
"Fri Aug 01 2014 00:15:00 GMT+0400 (MSK)"
Jul31_2014.toLocaleString("en")
"8/1/2014, 12:15:00 AM"
Jul31_2014.getHours()
0
Jul31_2014.getMonth()
7
Jul31_2014.getDate()
1
Jul31_2016 = new Date("2016-07-31T20:15Z")
Date 2016-07-31T20:15:00.000Z
Jul31_2016.toString()
"Sun Jul 31 2016 23:15:00 GMT+0300 (MSK)"
Jul31_2016.toLocaleString("en")
"7/31/2016, 11:15:00 PM"
As for skipped wall time (e.g. Spring forward) and repeated wall time (Fall backward), ICU Calendar API has SkippedWallTimeOption and RepeatedWallTimeOption
|
It turned out that Firefox does the right thing from a user's point of view not following the spec to its letters. (the result in my previous comment was taken from v8/Chromium). WIth the timezone set to Europe/Moscow, I got this from Firefox.
As for skipped wall time (e.g. Spring forward) and repeated wall time (Fall backward), ICU Calendar API has SkippedWallTimeOption and RepeatedWallTimeOption |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
jungshik
Jan 17, 2017
Contributor
Safari/JSC also behaves like Firefox.
From Firefox:
aug1_2014_0315_msklocal = new Date(2014,7,1,3,15)
Date 2014-07-31T23:15:00.000Z <= Europe/Moscow was UTC+04
aug1_2016_0315_msklocal = new Date(2016,7,1,3,15)
Date 2016-08-01T00:15:00.000Z <== Europe/Moscow was UTC+03
From Safari:
> aug1_2014_0315_msklocal = new Date(2014,7,1,3,15)
< Fri Aug 01 2014 03:15:00 GMT+0400 (MSK)
> aug1_2014_0315_msklocal.toLocaleString("en", {'timeZone': 'UTC'})
< "7/31/2014, 11:15:00 PM"
> aug1_2016_0315_msklocal = new Date(2016,7,1,3,15)
< Mon Aug 01 2016 03:15:00 GMT+0300 (MSK)
> aug1_2016_0315_msklocal.toLocaleString("en", {'timeZone': 'UTC'})
< "8/1/2016, 12:15:00 AM"
It appears that both Firefox and Safari treat LocalTZA as time-dependent (as it should) even though the spec does not say so (otoh, the spec makes it explicit that DST Adj is time-dependent).
I fully agree with @ediosyncratic that OffsetFromUTC(u) is a cleaner/better way to handle timezone offset than splitting the offset into two parts (LocalTZA and DST Adj). Moreover, a way to handle skipped or repeated wall times need to be introduced as well. That requires a rather extensive change in the spec. In the medium-to-long term, that must be done.
In the meantime, I wonder if it's possible to make a "minor edit" (maybe via a PR) of making LocalTZA time-dependent given that that's what at least two implementations do and I'm planning to align v8 with them. ( (will check Edge and IE 11) ).
@bterlson , @littledan , @ediosyncratic , what do you think of that?
20.3.1.7 Local Time Zone Adjustment#
An implementation of ECMAScript is expected to determine the local time zone adjustment. The
local time zone adjustment is a value LocalTZA measured in milliseconds which when added to UTC
represents the local standard time. Daylight saving time is not reflected by LocalTZA.
NOTE
It is recommended that implementations use the time zone information of the IANA Time Zone
Database http://www.iana.org/time-zones/.
20.3.1.8 Daylight Saving Time Adjustment#
An implementation dependent algorithm using best available information on time zones to
determine the local daylight saving time adjustment DaylightSavingTA(t), measured in milliseconds.
An implementation of ECMAScript is expected to make its best effort to determine the local daylight
saving time adjustment.
NOTE
It is recommended that implementations use the time zone information of the IANA Time Zone
Database http://www.iana.org/time-zones/.
|
Safari/JSC also behaves like Firefox. From Firefox:
From Safari:
It appears that both Firefox and Safari treat LocalTZA as time-dependent (as it should) even though the spec does not say so (otoh, the spec makes it explicit that DST Adj is time-dependent). I fully agree with @ediosyncratic that OffsetFromUTC(u) is a cleaner/better way to handle timezone offset than splitting the offset into two parts (LocalTZA and DST Adj). Moreover, a way to handle skipped or repeated wall times need to be introduced as well. That requires a rather extensive change in the spec. In the medium-to-long term, that must be done. In the meantime, I wonder if it's possible to make a "minor edit" (maybe via a PR) of making LocalTZA time-dependent given that that's what at least two implementations do and I'm planning to align v8 with them. ( (will check Edge and IE 11) ). @bterlson , @littledan , @ediosyncratic , what do you think of that?
|
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
bterlson
Jan 17, 2017
Member
@jungshik I would likely take such a PR, although I'm not an expert here (I wonder if @maggiepint has any thoughts). I'd suggest doing a PR this week so we can discuss next week in committee if necessary!
|
@jungshik I would likely take such a PR, although I'm not an expert here (I wonder if @maggiepint has any thoughts). I'd suggest doing a PR this week so we can discuss next week in committee if necessary! |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
jungshik
Jan 17, 2017
Contributor
@bterlson thank you for the reply. It seems more involved than a simple edit because UTC(t) is defined as following (OTOH, LocalTime(t) abstract operation would be well-defined with LocalTZA being t-dependent).
20.3.1.10 UTC ( t )
The abstract operation UTC with argument t converts t from local time to UTC is defined by performing the following steps:
Return t - LocalTZA - DaylightSavingTA(t - LocalTZA).
If we make LocalTZA depend on 't' where 't' is UTC, there'd be a 'cycle'.
t - LocalTZA (t - LocalTZA(t - LocalTZA(t - ... )))...) - DSTA(t - LocalTZA(t - LocalTZA(t - ...)))..
LocalTZA can be spec'd to take either UTC or local time (ugly...) and a warning can be added that it'd be ambiguous when there's a timezone offset change, but it's rather ugly.
Perhaps, better would be to introduce LocalTimeOffsetFromUTC(t_local) abstract operation with the skipped and repeated walltime behaviors matching existing implementations. Then, UTC(t_local) can be defined as t_local - LocalTimeOffsetFromUTC(t_local). In effect, I believe this is what JSC(Safari) and Spidermonkey(Firefox) do.
In the future, options for skipped and repeated walltimes can be added.
|
@bterlson thank you for the reply. It seems more involved than a simple edit because UTC(t) is defined as following (OTOH, LocalTime(t) abstract operation would be well-defined with LocalTZA being t-dependent).
If we make LocalTZA depend on 't' where 't' is UTC, there'd be a 'cycle'.
LocalTZA can be spec'd to take either UTC or local time (ugly...) and a warning can be added that it'd be ambiguous when there's a timezone offset change, but it's rather ugly. Perhaps, better would be to introduce In the future, options for skipped and repeated walltimes can be added. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
jungshik
Jan 17, 2017
Contributor
Europe/Moscow timezone changes in this decade are summarized at https://www.timeanddate.com/time/zone/russia/moscow
On 2011-02-27, 2:00 AM (localtime) => 3:00AM in Moscow
(new Date(2011,2,27,1,59)).toLocaleString("en", {'timeZone': 'UTC'})
"3/26/2011, 10:59:00 PM"
(new Date(2011,2,27,2,1)).toLocaleString("en", {'timeZone': 'UTC'}) <== skipped time
"3/26/2011, 10:01:00 PM"
(new Date(2011,2,27,3,1)).toLocaleString("en", {'timeZone': 'UTC'})
"3/26/2011, 11:01:00 PM"
On 2014-10-26, 2:00 AM (localtime) => 1:00 AM in Moscow
(new Date(2014,9,26,0,59)).toLocaleString("en", {"timeZone": 'UTC'})
"10/25/2014, 8:59:00 PM"
(new Date(2014,9,26,1,1)).toLocaleString("en", {"timeZone": 'UTC'}) <=== repeated time
"10/25/2014, 10:01:00 PM"
(new Date(2014,9,26,1,59)).toLocaleString("en", {"timeZone": 'UTC'}) <== repeated time
"10/25/2014, 10:59:00 PM"
(new Date(2014,9,26,2,1)).toLocaleString("en", {"timeZone": 'UTC'})
"10/25/2014, 11:01:00 PM"
Both Firefox and Safari use a 'new' offset (offset from UTC after a timezone change) to interpret skipped and repeated wall times.
|
Europe/Moscow timezone changes in this decade are summarized at https://www.timeanddate.com/time/zone/russia/moscow On 2011-02-27, 2:00 AM (localtime) => 3:00AM in Moscow
On 2014-10-26, 2:00 AM (localtime) => 1:00 AM in Moscow
Both Firefox and Safari use a 'new' offset (offset from UTC after a timezone change) to interpret skipped and repeated wall times. |
added a commit
to jungshik/ecma262
that referenced
this issue
Jan 20, 2017
jungshik
referenced this issue
Jan 20, 2017
Closed
Add TZOffset{UTC,Local} in place of LocalTZA/DSTA #771
added a commit
to jungshik/ecma262
that referenced
this issue
Jan 23, 2017
jungshik
referenced this issue
Jan 23, 2017
Merged
Make LocalTZA take 't' and 'isUTC' and drop DSTA(t). #778
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
jungshik
Jan 24, 2017
Contributor
Safari's handling of Pacific/Apia is interesting. At the end of 2011-12-29 local time, Pacific/Apia moved from UTC-1000 to UTC+1400. So, the entire day of 2011-12-30 is skipped in Pacific/Apia.
Up to (not including) 2011-12-30 23:00 (local time) is interpreted as in the time zone before the switch, but 2011-12-30 23:00 or later is treated in the time zone after the switch.
> (new Date(2011,11,30,11,0)).toUTCString()
< "Fri, 30 Dec 2011 21:00:00 GMT"
> (new Date(2011,11,30,12,0)).toUTCString()
< "Fri, 30 Dec 2011 22:00:00 GMT"
> (new Date(2011,11,30,13,0)).toUTCString()
< "Fri, 30 Dec 2011 23:00:00 GMT"
> (new Date(2011,11,30,14,0)).toUTCString()
< "Sat, 31 Dec 2011 00:00:00 GMT"
> (new Date(2011,11,30,15,0)).toUTCString()
< "Sat, 31 Dec 2011 01:00:00 GMT"
> (new Date(2011,11,30,16,0)).toUTCString()
< "Sat, 31 Dec 2011 02:00:00 GMT"
> (new Date(2011,11,30,17,0)).toUTCString()
< "Sat, 31 Dec 2011 03:00:00 GMT"
> (new Date(2011,11,30,18,0)).toUTCString()
< "Sat, 31 Dec 2011 04:00:00 GMT"
> (new Date(2011,11,30,19,0)).toUTCString()
< "Sat, 31 Dec 2011 05:00:00 GMT"
> (new Date(2011,11,30,20,0)).toUTCString()
< "Sat, 31 Dec 2011 06:00:00 GMT"
> (new Date(2011,11,30,21,0)).toUTCString()
< "Sat, 31 Dec 2011 07:00:00 GMT"
> (new Date(2011,11,30,22,0)).toUTCString()
< "Sat, 31 Dec 2011 08:00:00 GMT"
> (new Date(2011,11,30,23,0)).toUTCString()
< "Fri, 30 Dec 2011 09:00:00 GMT"
> (new Date(2011,11,31,0,0)).getTime() - (new Date(2011,11,30,23,0)).getTime()
< 3600000
> (new Date(2011,11,31,0,0)).getTime() - (new Date(2011,11,30,22,0)).getTime()
< -79200000
Anyway, JSC/Safari can't be blamed for this behavior because the spec was ambiguous. PR #778 clarified the behavior around a time zone offset change.
|
Safari's handling of Pacific/Apia is interesting. At the end of 2011-12-29 local time, Pacific/Apia moved from UTC-1000 to UTC+1400. So, the entire day of 2011-12-30 is skipped in Pacific/Apia. Up to (not including) 2011-12-30 23:00 (local time) is interpreted as in the time zone before the switch, but 2011-12-30 23:00 or later is treated in the time zone after the switch.
Anyway, JSC/Safari can't be blamed for this behavior because the spec was ambiguous. PR #778 clarified the behavior around a time zone offset change. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
jungshik
Jan 24, 2017
Contributor
Firefox cannot be tested because its handling of Pacific/Apia is broken at least on Mac OS.
((new Date(2011,11,31,0,0)).getTime() - (new Date(2011,11,29,23,59)).getTime()) / 1000
86460 <=== should be 60
|
Firefox cannot be tested because its handling of Pacific/Apia is broken at least on Mac OS.
|
ediosyncratic commentedNov 1, 2016
•
edited
As noted at the end of [1], UTC(Localtime(t)) is not always equal to t.
Indeed, Localtime(UTC(t)) is not always equal to t, either, although neither [0] nor [1] notes this.
Nor does [1] address the question of what to do when the parameter passed to UTC is either invalid (because it represents a time that local time skipped over when starting DST, during a "spring forward") or ambiguous (because it represents a time that was reused at the end of DST - first in DST then in standard time - in a "fall back").
The spec [2] of DaylightSavingTA() neglects to say that the input parameter, t, is presumed to be a UTC time; this is, however, made clear by the way it is used in the stipulated implementations of UTC() and Localtime(). Those uses make clear that, for any t, LocalTZA + DaylightSavingTA(t) should give local time's applicable offset from UTC at time t.
The spec [2] should make clear that t is presumed to be given in UTC.
LocalTZA is specified to be local time's present (at run-time, not at time t) standard offset from UTC, without any DST correction. If local time has ever had a change of standard offset, the tacit requirement that LocalTZA + DaylightSavingTA(t) was local time's applicable offset from UTC at time t then comes into conflict with the wording of the specification for DaylightSavingTA(); when t refers to a time the other side (from run-time) of a standard offset change in local time, DaylightSavingTA(t) needs to incorporate both the standard offset change and any DST that may have been in effect at time t, while the wording of the spec says it should only return the DST offset applicable at that time.
In the case of Pacific/Kiritimati looking at times before 1995, or Pacific/Apia before 2012, this implies a DaylightSavingTA() of 24 hours at times when DST was not in effect. For a zone that's now given up DST in favour of setting their standard offset to what used to be their summer offset, this shall mean reporting a negative DaylightSavingTA() before the change for non-DST times, while reporting a zero offset for when DST was in force.
The cause of this is the oversimplification of using LocalTZA as base offset and treating all deviations from it as DST. In practice, a good time offset API would have an OffsetFromUTC(t) method, taking a UTC time and returning the actual offset at that time; for legacy purposes, you can define LocalTZA as at present and DaylightSavingTA(t) as OffsetFromUTC(t) - LocalTZA. There may be some use to also providing StandardOffsetFromUTC(t) and DaylightSavingAdjustment(t) with which to decompose OffsetFromUTC(t), but it's OffsetFromUTC(t) that most code is going to actually care about.
You can then define Localtime(t) to be t + OffsetFromUTC(t). I encourage you to, at the same time, change the spec of UTC(); doing that right shall be contentious, but I do recommend looking at python's definitions [3] in this area.
Some mechanism for disambiguating fall-back hours is necessary; while client code shall often now know the needed hint, requiring sensible default behaviour (and matching present behaviour shall make sense for that), it is worth providing some mechanism for callers to
Whatever mechanism disambiguates fall-backs can also be used to decide what to do with invalid local time values in a spring forward. (My usual model for these is that someone stepped forward or backwards an hour, a day or a week from a valid time and landed in a gap. If the prior time was in DST, that step (backwards) should have landed in the hour before the spring forward; if the prior time was in standard time, the step (forward) should land in the hour after. So I select the time that actually contradicts the alleged DSTness, where handling of a fall-back seeks to match it.) Actual changes in a zone's standard offset can be handled the same way as a DST transition in the same direction.
Ideally it should be the case that UTC(t) returns a u for which u + OffsetFromUTC(u) is indeed t. Note, however, that OffsetFromUTC(u) need not be equal to OffsetFromUTC(t), so t - OffsetFromUTC(t) is not an adequately reliable answer, although it may be the best you can do, unless you arrange for your local-time values to come with an extra bit (or trit, to allow "unknown" as an option) of data, indicating their (presumed) DST-ness or some equivalent truth (such as python's "fold").
Time software would all be so much simpler if we all just stuck to UTC; I recommend doing so for all internal types and only meddling with the political mess of time zone variation at the very surface of software, where users are exposed to the data. Local time may seem like a convenience, but avoiding bugs at transitions annihilates what little convenience it actually confers.
I may be reached as edward.welbourne@qt.io (and shall now fix Qt's V4 engine to be less broken than it presently is - only as broken as the spec requires it to be).