Timezone databases

[dkhalanskyjb]

Timezone Database API

Currently, kotlinx-datetime only allows using the system timezone database. There's an established practice in Android development to bundle timezone databases with the application so that older devices that no longer receive updates can also access up-to-date timezone information.

(In this document, users are software developers that don't need to write their own timezone database implementations and don't publish their code as libraries.)

Figuring out the best API for this is challenging due to a tradeoff between convenience for users, flexibility in supporting niche use cases, and risks of using the API incorrectly. We have discussed several approaches that optimize for different aspects of usability, and we would like to receive input from the community to see which aspects you value the most.

There are four comments under this discussion, each with the name of one approach. Please upvote the comment with the approach of you like the most and downvote the comment with the approach you dislike the most. Also please leave any comments with concerns, suggestions, or use cases that come to mind.

Approach 1: ServiceLoader-style

Using the system timezone database: works as before.

Bundling the IANA timezone database: enabled by adding a runtime dependency on kotlinx-datetime-zoneinfo.

Using a custom timezone database: enabled by adding a runtime dependency on the library providing it.

The API for obtaining timezones stays the same as it is today:

val tz = TimeZone.of("Europe/Berlin")
println(Clock.System.now().toLocalDateTime(tz))

Library code freely accesses timezones:

package third.party.library

import kotlinx.datetime.*

public fun Clock.timeInBerlin(): LocalTime {
    val tz = TimeZone.of("Europe/Berlin")
    return now().toLocalDateTime(tz).time
}

of checks which timezone databases the runtime classpath contains and chooses among them based on the priority specified by the authors of timezone database implementations.
By default, a platform-specific timezone database is used: on the JVM, it's the timezone database provided by the JVM, on iOS, it reads the OS-level timezone database from the filesystem.

kotlinx-datetime will start publishing a separate artifact that provides a timezone database implementation corresponding to the latest IANA version (https://www.iana.org/time-zones).
The IANA timezone database is the timezone database that you are most likely already using; it's the one distributed with Android, iOS, MacOS, Linux, and used for .NET applications on Windows and the JVM.

Benefits of this approach:

• The code stays simple and straightforward, as timezone databases can not be configured there.
• Library authors don't have to expose using TimeZoneDatabase in their API and can just write normal code.

Downsides of this approach:

• Only one timezone database may be used per program invocation. It's impossible to combine the behaviors of different timezone database implementations (for example, by providing fallback implementations in case one of them is inaccessible) or to use different timezone databases at different moments.
• Implementing timezone databases becomes quite difficult. ServiceLoader is not available for multiplatform Kotlin currently, it is a JVM-only mechanism, so different platforms require different non-obvious workarounds to emulate the same behavior.
• Passing parameters to timezone database implementations isn't clean. For example, a hypothetical timezone database that queries the server may be parameterized with the server's URL, but there is nowhere to put that URL by default. It needs to be exposed as a global variable or a function.
• Libraries must take care not to accidentally expose a TimeZoneDatabase implementation needlessly, for example, by adding a runtime dependency on a timezone database implementation.
• It becomes impossible to say which timezone database was used just by looking at the code.

Approach 2: explicit interface TimeZoneDatabase, with no default common-code implementation

Using the system timezone database: the correct TimeZoneDatabase instance must be manually used for each platform separately. TimeZone.of and other singleton functions are not available.

Bundling the IANA timezone database: a dependency on kotlinx-datetime-zoneinfo provides a new TimeZoneDatabase implementation, which must be manually used.

Using a custom timezone database: the corresponding TimeZoneDatabase implementation must be manually used.

val tz = tzdb.get("Europe/Berlin")
println(Clock.System.now().toLocalDateTime(tz))

package third.party.library

import kotlinx.datetime.*

public fun Clock.timeInBerlin(tzdb: TimeZoneDatabase): LocalTime {
    val tz = tzdb.get("Europe/Berlin")
    return now().toLocalDateTime(tz).time
}

There is no TimeZone.of, and the only way to obtain a timezone is to use a timezone database, which is passed explicitly.

There is no singleton like TimeZoneDatabase.System. Instead, on the JVM, there would be TimeZoneDatabase.Jvm, on iOS, TimeZoneDatabase.Filesystem and TimeZoneDatabase.Foundation, and on Windows, TimeZoneDatabase.Registry (names are not final and are just examples).

There is also a common-code TimeZoneDatabase.Iana published in a separate artifact that needs to be used manually if one wants a timezone database implementation that's guaranteed to be up-to-date.

Benefits:

• Multiplatform library authors are automatically guided into exposing an interface suitable for passing custom TimeZoneDatabase instances, which means that usages of TimeZoneDatabase are made transparent.
• Configuration is fully exposed to the user, who is free to configure timezone database behavior arbitrarily, mix together several timezone databases, combine them, or have multiple parts of the system work with different timezone databases.

Downsides:

• In a typical scenario, every user now will need to spend some effort deciding which timezone database to use on which platform, instead of being able to rely on a reasonable default.
• It is possible to accidentally mix several timezone databases together and obtain inconsistent results.

Approach 3: explicit interface TimeZoneDatabase with a non-overridable TimeZoneDatabase.System

Using the system timezone database: the TimeZoneDatabase.System instance must be manually used. TimeZone.of and other singleton functions are not available.

Bundling the IANA timezone database: a dependency on kotlinx-datetime-zoneinfo provides a new TimeZoneDatabase implementation, which must be manually used.

Using a custom timezone database: the corresponding TimeZoneDatabase implementation must be manually used.

// either
val tz = tzdb.get("Europe/Berlin")
println(clock.now().toLocalDateTime(tz))

// or
val tz = TimeZoneDatabase.System.get("Europe/Berlin")
println(Clock.System.now().toLocalDateTime(tz))

package third.party.library

import kotlinx.datetime.*

public fun Clock.timeInBerlin(tzdb: TimeZoneDatabase): LocalTime {
    // it's incorrect to use `TimeZoneDatabase.System` here!
    val tz = tzdb.get("Europe/Berlin")
    return now().toLocalDateTime(tz).time
}

This is similar to the previous approach, but there is TimeZoneDatabase.System in common code.
It always refers to the system timezone database, even if TimeZoneDatabase.Iana is available.

Benefits:

• Configuration is fully exposed to the users who want it, but in a typical scenario, it's also possible to use TimeZoneDatabase.System in user code.

Downsides:

• Library authors may forget to expose TimeZoneDatabase in their APIs, which defeats the purpose of TimeZoneDatabase.Iana and other custom timezone databases.
• Also, the risk of obtaining inconsistent results due to different parts of the system accidentally relying on different timezone databases is higher because of that.

Approach 4: interface TimeZoneDatabase with TimeZoneDatabase.System overridable only by Iana

Using the system timezone database: the TimeZoneDatabase.System instance must be manually used. TimeZone.of and other singleton functions are not available.

Bundling the IANA timezone database: using the TimeZoneDatabase.System automatically delegates to the IANA timezone database if there's a runtime dependency on kotlinx-datetime-zoneinfo.

Using a custom timezone database: the corresponding TimeZoneDatabase implementation must be manually used.

val tz = TimeZoneDatabase.System.get("Europe/Berlin")
println(Clock.System.now().toLocalDateTime(tz))

package third.party.library

import kotlinx.datetime.*

public fun Clock.timeInBerlin(tzdb: TimeZoneDatabase): LocalTime {
    // using `TimeZoneDatabase.System` instead is discouraged,
    // but only users with the most exotic needs will be affected
    // if `System` is used
    val tz = tzdb.get("Europe/Berlin")
    return now().toLocalDateTime(tz).time
}

A combination of approaches 3 and 1.
If TimeZoneDatabase.Iana is present, System will default to using it via a ServiceLoader-like mechanism.

Benefits:

• The most common use case of replacing System with a more up-to-date Iana timezone database is covered without any extra boilerplate.
• Those that want full flexibility can access it.
• Even if a library author doesn't expose their usage of TimeZoneDatabase.System, the right thing will still happen if Iana is available.

Downsides:

• First-party implementations of timezone databases are decidedly more convenient to use than third-party ones, whereas in the ServiceLoader-based approach, they are treated as equals.

[dkhalanskyjb]

Approach 1: ServiceLoader-style. Upvote this if this is your favorite approach. Downvote this if this is your least favorite approach.

[eygraber]

Not sure if this is still applicable, but I think ServiceLoader had performance implications on Android in the past.

[JakeWharton]

It does, but R8 mostly obviates them if you interact with it correctly. They have experience in this from its usage in coroutines for dispatcher loading, though, so it wouldn't be an additional problem to worry about.

[dkhalanskyjb]

Approach 2: explicit interface TimeZoneDatabase, with no default common-code implementation. Upvote this if this is your favorite approach. Downvote this if this is your least favorite approach.

[dkhalanskyjb]

Approach 3: explicit interface TimeZoneDatabase with a non-overridable TimeZoneDatabase.System. Upvote this if this is your favorite approach. Downvote this if this is your least favorite approach.

[JakeWharton]

I'm a bit wary about the delicate suggestion because the system tzdb is not only reasonable but often desired in a lot of configurations.

[JakeWharton]

@hrach Unhelpful strawman, as the laws of physics are not changed multiple times per year like the timezone database.

[russhwolf]

@JakeWharton That's fair. What I like about opt-ins is they make it much harder to miss the documentation about when you should or shouldn't use something. But I can see the argument that it might be a step too far here.

[dkhalanskyjb]

Also, it's not clear from the description, but I would hope this version also removes TimeZone.of()

Fixed the description, thanks! Indeed, of gets removed in these scenarios.

marking TimeZoneDatabase.System with a @DelicateApi opt-in

System as a concept is fine.

Pros over Iana:

• Your application size doesn't have to increase by 50-500 kilobytes (depending on the accuracy of the timezone database you go for).
• The behavior of your application is consistent with the system applications and basically everything else on your device.

Cons:

• May return inaccurate results on old, unsupported devices.
• Unpredictable, harder to test against.

I'd expect most applications to knowingly go with System instead of bundling their own IANA tzdb, it doesn't feel right to me to make System inconvenient to use.

[swankjesse]

I wonder if the correct name for the time zone database that ships with my app is BundledTimeZoneDatabase and not Iana.

That way developers are deciding between the host operating system and their bundled app, which is a trade-off that fits our responsibilities.

Choosing between the host operating system and Iana feels awkward because the host OS is probably also based on Iana.

Also also if you wanna be Very Fancy, you could offer this method:

  fun latest(option: TimeZoneDatabase, vararg more: TimeZoneDatabase): TimeZoneDatabase

You'd use that to dynamically take the system or bundled DB, whichever was newer. In my opinion that's the nicest policy. The caveat is that these things would need a timestamp field for their data.

[dkhalanskyjb]

Approach 4: interface TimeZoneDatabase with TimeZoneDatabase.System overridable only by Iana. Upvote this if this is your favorite approach. Downvote this if this is your least favorite approach.

[kevincianfarini]

Originally, I had downvoted the traditional service loader approach, but I actually think this approach might be the worst of all worlds. The lackluster parts of the service loader approach (e.g. any library can change what tzdb implementation I'm using) still exist while also still having the overhead of needing to pass around a TimeZoneDatabase instance.

The way I've read this is that even if this is restricted to just first-party implementations of the tzdb, like the Iana one, the system database could still be overridden with old versions of that dependency. Am I understanding this properly?

[JakeWharton]

Yeah, it undermines the inversion of control to create a somewhat orthogonal overriding system based on a known-problematic mechanism which only works on one platform.

[dkhalanskyjb]

the system database could still be overridden with old versions of that dependency. Am I understanding this properly?

If I understood the question correctly, there's a lot of nuance to this. In our design, Iana would know its version, and everywhere but Kotlin/Native for mingw64, the SYSTEM timezone database is also IANA with a known version. This approach and the ServiceLoader-based approach can inspect the versions of all available IANA timezone databases and merge the information the databases have together (some versions may contain more historical data, some may support newer recurring rules).

With non-ServiceLoader-based approaches (and with custom timezone databases in this approach), we could consider providing an API for developers to do this manually, in essence, something like (TimeZoneDatabase.System as? IanaTimeZoneDatabase).combineIanaTzdb(TimeZoneDatabase.Iana) ?: TimeZoneDatabase.Iana, but this wouldn't happen automatically, since these would be separate singletons. If you were just to use TimeZoneDatabase.Iana (without merging it with System or using System), you could potentially be using an older version than System, no one would stop you.

To sum it up, is it possible to accidentally use an older version of a timezone database when kotlinx-datetime sees a new one available?

• Approach 1 (ServiceLoader): only if an incorrect implementation of a timezone database causes this, but typically, no.
• Approaches 2 and 3: yes, by just using Iana without attempting to merge it with System.
• Approach 4 (ServiceLoader for System): yes when using custom timezone databases that don't benefit from service-loading, yes when using TimeZoneDatabase.Iana explicitly, no when using TimeZoneDatabase.System enhanced by Iana (though this last bit of behavior is debatable).

which only works on one platform

This approach would work on all platforms, it's not a JVM-only solution.

[hrach]

Have you considered providing ~ TimeZoneDatabase.Default? This would be an addition to Approach 3.

There are multiple ways to approach this, but basically it would be configurable; it may fall back to System, xor it may throw when not configured.

The main point is that the libs would, without a strong use case to do otherwise, use this tzdb source without any obstacles.