POSIX-compatibility with java.util.Date

[MenoData]

The java.time.Instant - java.util.Date - conversion is defined in latter class via a simple algorithm to translate between Instant-seconds/nanos and millis on the other side. Basis is obviously a constant factor of 86400 secs per day. That is also the commonly expected behaviour of java.util.Date to just be a millisecond version of a unix timestamp.

What is disturbing here, is the javadoc/spec of java.time.Instant. There and on some other locations you introduce a so-called "java time scale", which shall be UTC-SLS, explicitly stated for class Instant. Leaving the fact aside that you try to revive an expired proposal, we surely agree that nobody in java world up to now has assumed or treated java.util.Date to be an implementation of UTC-SLS, but rather POSIX. The conversion between these time scales should either be reflected by conversion algorithm between Instant and j.u.Date (requiring a missing leap second table in the background) - or: The spec should be changed to replace this "java time scale" just by POSIX (which is probably the most honest, straight-forward and simpliest solution). Otherwise you will have behavioural incompatibility of j.u.Date with the spec (for ever?). We have enough confusion by pointless talk about non-supported leap seconds for example in the javadoc of j.u.Date. Just say: "We don't support leap seconds." Done.

Also important: By far the most used clock will be based on System.currentTimeMillis(). This clock while not very precise does never count any leap seconds and can just be seen as a kind of POSIX clock regarding the fact that all underlying UNIX operating systems use POSIX and that the JVM is heavily inspired by that (for example choosing the same epoch of 1970-01-01T00:00:00Z). So here again there is no space left for UTC-SLS.

[jodastephen]

It was the opinion of the expert group that leap-seconds could not be ignored or swept under the carpet. Saying we don't supoprt them was not an option available. Given that, I also wanted to ensure that leap-seconds never occur within user classes (Instant/LocalDateTime/ZonedDateTime etc). The Java time-scale is the best way to resovle the tension.

Note that java.util.Date does not ignore leap seconds. Instead, the getSeconds() method is allowed to return values of 60 or 61. This kind of behaviour is guaranteed to cause bugs, as virtually no developer will have thought about leap seconds or planned for the value 60 being returned.

POSIX does not solve any problems, in fact it creates them. It has no handling for leap seconds at all, with different ways that it could be implemented. Time is magically supposed to disappear around a leap second. Specifically, the java.util.Date class is not defined as a mapping of POSIX as it contains additional claims.

The Java time-scale simply defines a formal, implementable time-scale. This allows experts to provide implementations if they desire. There are no current plans for Java to do so and it is accepted that JDK 8 will not implement the time-scale.

The use of UTC-SLS is a practical choice. There is no point in redefining a smoothing mechanism when one has already been written up. Note that the definition of the Java time-scale allows for the possibility of a future formalised smoothing mechanism under a different name, by defining a new time-line segment.

Finally, I note that Google has its own smoothing mechanism, effectively defining a time-scale very similar to the Java time-scale. While Google's time-scale would not comply with the letter of the Java time-scale, it does comply with the principles.

Summary, the Java time-scale is a trade-off like all good design. It defines a correct way to handle leap-seconds while accepting that most implementations, such as the very poor POSIX, will not meet the definition.

[MenoData]

Stephen, no surprise, I disagree. Okay, having different opinions is no reason to be sad about. You have also not convinced most time experts on leapsecond mailing list when you presented your idea of adopting UTC-SLS in Java some time ago.

The term "Java time scale" presents one problem. It is too general. In fact we should remember what a time scale is from a mathematical point of view, namely a unique mapping from any kind of possibly complex chronological state onto a single incrementing number. So even a class like LocalDate has a time scale, namely defined by a variation of julian days, or LocalTime by elapsed time since midnight and and and... So also your "Java time scale" must be connected to a concrete type like java.time.Instant anyway. It does not exist in vacuum. But you even write and specify in Javadoc of class Instant:

"The Java time-scale is used for all date-time classes."

The same scale for LocalDate, Instant etc.??? I admit, I cannot follow completely. Instead I also make a difference between the time scales of POSIX and UTC and UTC-SLS. Just an example: Let us regard the physical time 2012-06-30T23:59:50.000Z (on true UTC time scale) which is exactly 11 SI-seconds before 2012-07-01T00:00:00.000 after the last leapsecond event. The same timestamp can mapped to the same numerical value on POSIX scale (although the leap second itself has no defined mapping in POSIX). How does it look in UTC-SLS? This idea says: Let's insert during the last 1000 seconds before a leap second an extra millisecond per every second, so we have instead of 2012-06-30T23:59:50.000Z the chronologically identical timestamp 2012-06-30T23:59:50.989Z on UTC-SLS-scale, apparently numerically not the same representation. So the mapping from POSIX to UTC-SLS is not just 1:1.

Of course I know that java.util.Date does not officially refer to POSIX standard. But it was shipped with Java 1.0, was heavily inspired by POSIX, C and UNIX era, has even adopted the theoreticallly impossible leap second value of 61 (as in an earlier POSIX paper as total misunderstanding what UTC is). Markus Kuhn has published his proposal of "UTC-SLS" in year 2005, ten!!! years later. The class java.util.Date has nothing to do with this proposal. And surely nobody in Java world has this association for just a moment. Using ducktyping arguments I would say, java.util.Date behaves like a POSIX implementation and therefore is one. Everyone uses it that way. Specifications have to be lived (not sure if this is correct English), to be filled with content. What the javadoc of java.util.Date says especially about leap seconds is rather not so important (in fact just evasive, delegating unsolved question to the underlying operating system and even theoretically wrong), but more the way how people use it. So it is justified to say that despite of the spec java.util.Date does not support leap seconds in reality.

Oh you complain about POSIX that "Time is magically supposed to disappear around a leap second". Indeed, but the same is true for UTC-SLS, just hidden in milli- and microsecond resolution. More astonishingly, you yourself admit that you have defined a Java time scale (UTC-SLS after 1972-11-03 at a practically hard to impossible to implement fuzzy time point where UTC == UT1, see also the data from IERS bulletin B for ca. 12th of October this year), and... no implementation of this spec. Furthermore, Oracle has actively removed already available implementations of UTC-SLS (represented by the threeten-class UTCInstant) and also removed a leap second table from TZDB data what is the technical precondition of any UTC-SLS implementation. So what is the point to specify the "Java time scale" based on UTC-SLS? And if you reimplement the mentioned removed stuff then the problem of behavioural incompatibilty between java.util.Date/java.time.Instant would automatically rise again if latter shall be like UTC-SLS, just see the timestamp example before.

Personally I have no problem to show to users timestamps like 2012-06-30T23:59:60Z and cannot follow what is so bad or frightening about this rare, but existing time anomaly. Anyway, I don't expect to convince you, but at least a (honest) compatibilty note about java.util.Date and UTC-SLS (or this "Java time-scale") would be nice for all Java users. That is also my wish addressed towards Oracle.

[jodastephen]

System.currentTimeMillis() effectively has the fiction that there are 86400 seconds per day. So does JSR-310. The difference is that JSR-310 defines the fiction in a clear manner, not that any "normal" user of Java SE will ever care.

Bear in mind that leap seconds may cease to exist in a few years through US pressure, and as such adding classes or data to support leap seconds is of potentially marginal value. Personally I would have kept the data, but I would not at this point support a JDK class for leaps (as they are only useful if you have an accurate clock, and Java does not).

[MenoData]

My main criticism is not that you want to let the user ignore leap seconds. It is your right to design (and limit) your api that way so developers with more scientific interest are not served. But what is not good is that your "Java time-scale" is not really more clear than posix in detail. I agree that posix is not a good standard, but it is the de-facto-standard in wide areas of IT-world and we don't need another new fuzzy time-scale instead of it. Your "Java time-scale" is in my honest opinion not even implementable (when it comes to precision).

[RogerRiggs]

The Java time scale describes the requirements on the default clocks. The java clock must be consistent with the system clock or a whole variety of application errors could occur because of skew between clocks that are assumed to be in sync. The same is true across systems and typically they rely on a broader clock distribution like NNTP for their time source; the result is that none of the clocks meet scientific standards and just as the os can't make guarantees about time, linearity, etc. neither can the Java runtime.
An application is free to implement a clock with the characteristics it requires if suitable hardware or infrastructure is available.

[MenoData]

Okay, so for you the Java time scale is only related to the (default?) clocks. But in the javadoc of java.time.Instant we can read: "The Java time-scale is used for all date-time classes." (even including LocalDate!). I think we need to make a difference between clock implementations and the inner state of java.time.Instant. I am mainly speaking about how to interprete the inner state of the class Instant.

Considering your remark about "scientific standards" it appears to me that we agree on the statement your java time scale does not meet any scientific requirments. But I also fear that we have no common minimal understanding about what a time scale is. For me, a time scale is statically associated to types like java.time.Instant (via the definition of second and the conversion algorithm to other types like j.u.Date), so it affects the interpretation of the inner state of every Instant object.

[RogerRiggs]

Clocks are the source of the value of 'now'. The rest of the API operates on the definition in Instant, with a fixed number of seconds per day and defined to be aligned at noon. All of the computations in the date-time types use the same basis and it simplified the API significantly. Mapping the defined time scale to any other is difficult, mostly because the Java Runtime operates on systems which do not have rigorously defined clocks. The current dates and times used within the Java runtime are dependent on the operating systems and use its time sources. The design and definitions support the more useful to maintain the assumptions about the relationship of the time as the OS views it with the Date and Time classes in the Java Runtime than it is to be synchronized with an external time source. The reference UTC-SLS provides a specific definition for the dilatation of time around leap seconds.

[MenoData]

Okay, now my last try to get a reasonable answer. Roger, I am actually not concerned about your requirements for clocks in the scope of this issue. A clock is not even necessary to create an instant. So let's go back to the original issue and let us consider following code:

GregorianCalendar gcal = new GregorianCalendar();
gcal.setTimeZone(TimeZone.getTimeZone("GMT"));
gcal.set(Calendar.MILLISECOND, 0);
gcal.set(2012, Calendar.JUNE, 30, 23, 59, 50); // see also discussion above

Date d = gcal.getTime(); // let us assume the underlying OS is a Linux server based on POSIX
Instant instant = d.toInstant(); // note it has the same numerical value as variable d (mapping 1:1)

Now my question, what is the time scale of this instant? A simple question, isn't it? Do you still seriously insist on UTC-SLS?

[RogerRiggs]

By definition the time scale of Instant is the Java time scale.
The value for seconds and nanoseconds is taken from Date and will be equal.
(I'm assuming you picked that date because a leap second is about to occur).
Since Linux is the OS, maybe using NNTP, maybe with different patches or versions installed, not much can be said about its time scale. It may or may not already being adjusted via NNTP or some other local mechanism.
If you have a better citation for a timescale that is adjusted to be monotonic and without leap seconds, that might be just as useful as UTC-SLS at communicating the properties needed.

[MenoData]

Come on, you don't know the time scale on a linux server (a unix derivate)? Okay, then just close this issue without any outcome. At least I hope it was an interesting discussion for all observers.

[jodastephen]

The only time that the time-scale matters is around a leap second. At all other times, the two forms will be the same without issue. I assert that Linux/Unix/Posix is unreliable around leap seconds such that you do not know what answer you will get. With the Java time-scale, your answer is likely to be similarly unreliable, but you have the option to ensure your system is reliable by creating an accurate clock. I consider that to be a step forward.

Note that if leap seconds are abolished, then the whole problem/difference goes away.

[MenoData]

Thanks for getting the point in my code example, Stephen. So we have following situation: If we value (scientific) accuracy then every conversion between LocalDateTime + ZoneOffset to Instant has a tiny error near leap second events. This mainly because despite of pinning the label UTC-SLS on Instant you simply apply trivial posix-like conversion logic internally.

But the target audience of JSR-310, standard business developers (see JEP 150) don't really care much about even second accuracy. So when you also tolerate such tiny errors then I ask you why claiming UTC-SLS at all. POSIX is sufficient from that perspective.It also declares 86400 secs per day. The rest outside of threeten scope does it, too. For example all reference timestamps in IANA-timezone-DB are on that basis. What is the surplus value of UTC-SLS? You write about the option of creating an accurate clock. Well, people can do this. The question is only, shall we replace POSIX by UTC-SLS or shall we add specialized methods in such a clock in addition to standard methods like getEpochTime(), for example getUTCTime()? Can we afford to ignore POSIX? I don't think, if we like POSIX or not.

And by the way, if your speculation about the end of leap seconds becomes true then UTC-SLS will die, too. But POSIX will continue.

So I give the advise to consult again all experts at Oracle, IBM, Apache, Red Hat, Google and in Unix world how useful or harmful your UTC-SLS attitude is in a POSIX dominated environment. A change of the spec/javadoc in class Instant should still be possible since it does not affect any API signatures.

[jodastephen]

"every conversion between LocalDateTime + ZoneOffset to Instant has a tiny error near leap second events"

This is not so. A LocalDateTime is declared to have 86400 "seconds" per day, as does Instant. Both have the same length "seconds". There is no inconsistency within JSR-310. The only point of question is when converting/comparing between a JSR-310 class and j.u.Date and j.u.Calendar, and only then around leap seconds.