TZInfo::AmbiguousTime: 1895-12-31 23:59:59 UTC is an ambiguous local time. (Asia/Taipei)

[yorkxin]

I have a Rails app that defaults time zone to "Asia/Taipei" (UTC+8). Sometimes an exception like this would be raised:

TZInfo::AmbiguousTime: 1895-12-31 23:59:59 UTC is an ambiguous local time.

With tzinfo 0.3.37 (which Rails depends on), this exception can be reproduced in irb:

> time = Time.new(1895,12,31,23,59,59)
 => 1895-12-31 23:59:59 +0800

> TZInfo::Timezone.get("Asia/Taipei").local_to_utc(time)
 => TZInfo::AmbiguousTime: Time: 1895-12-31 23:59:59 UTC is an ambiguous local time.

> TZInfo::Timezone.get("Asia/Shanghai").local_to_utc(time)
 => 1895-12-31 15:54:02 UTC

Both Asia/Taipei and Asia/Shanghai are in UTC+8 today, and have no official time zone in 1895.

This issue does not happen if I load tzinfo 1.0.1, which uses TZ Database directly from OS. If I install both tzinfo 1.0.1 and tzinfo-data gems, this issue would happen again (in this case tzinfo takes tzinfo-data instead of OS-provided TZ DB). However Rails binds version of tzinfo to >= 0.3.37, so I cannot work around this by simply upgrading tzinfo to 1.0.x.

I confirmed that such exception would happen if the local time to convert is between 1895-12-31 23:54:00 and 1895-12-31 23:59:59, and noticed that, in the original TZ Database, the following definition exists:

# Zone  NAME        GMTOFF  RULES   FORMAT  [UNTIL]
Zone    Asia/Taipei 8:06:00 -       LMT     1896 # or Taibei or T'ai-pei
                    8:00    Taiwan  C%sT

I want to make sure:

• Which is the expected behavior? Ambiguous time? Or correctly converted like Asia/Shanghai?
• Is there a rare case that is not handled in the converter of tzinfo-data?

Thanks.

[philr]

Hi,

For Asia/Taipei, in the range you describe, there are two possible UTC times for each local time. The AmbiguousTime exception is therefore the correct result. According to the TZ data base rules, the local clock in Taipei went back 6 minutes at 15:54 UTC on 1895-12-31.

To resolve the ambiguity, you'll need to pass a block to local_to_utc to select one of the periods. For example:

> TZInfo::Timezone.get("Asia/Taipei").local_to_utc(time) {|periods| periods.first }
=> 1895-12-31 15:53:59 UTC

You'll see a similar region of ambiguity with the Asia/Shanghai zone between 23:54:03 and 23:59:59 local time on 1927-12-31.

The reason you're not seeing the AmbiguousTime exception when using the system zoneinfo files is probably a 32-bit support issue (the earliest 32-bit UNIX timestamp being 1901-12-13 20:45:52).

It looks like you're running a version of Ruby that supports 64-bit timestamps (since you're able to instantiate a Time prior to 1901), so the issue is probably with your zoneinfo files.

Older zoneinfo files only support 32-bit timestamps. Newer files support 32-bit and 64-bit timestamps. To check which format you have available, you can run the following code (using TZInfo v1.0.1):

TZInfo::DataSource.set(:zoneinfo)
dir = TZInfo::DataSource.get.zoneinfo_dir
File.open(File.join(dir, 'Asia', 'Taipei'), 'r') {|f| f.read(5) }

If the last line returns "TZif\x00", then your zoneinfo files only support 32-bit timestamps. It is possible that your platform only includes 32-bit zoneinfo files on 32-bit systems. It would also be worth checking that dir is the official location for zoneinfo files on your system.

If the last line returns "TZif2", then you have zoneinfo files that support 32-bit and 64-bit timestamps, but it would appear that TZInfo is failing to read them correctly. Please could you send me the Asia/Taipei file and let me know what platform you are running on (operating system, Ruby versions, etc)?

Kind regards,

Phil

[yorkxin]

Hi Phil,

I reproduced that issue on my development env, Mac OS X 10.8.6 with Ruby 1.9.3. Here is the result, as you expected, it seems to be 32-bit-timestamps only:

$ uname -a
Darwin 40TEM1024.local 12.4.0 Darwin Kernel Version 12.4.0: Wed May  1 17:57:12 PDT 2013; root:xnu-2050.24.15~1/RELEASE_X86_64 x86_64

$ irb
> TZInfo::DataSource.set(:zoneinfo)
> dir = TZInfo::DataSource.get.zoneinfo_dir
> File.open(File.join(dir, 'Asia', 'Taipei'), 'r') { |f| f.read(5) }
 => "TZif\x00"

>  time = Time.new(1895,12,31,23,59,59)
 => 1895-12-31 23:59:59 +0800
> TZInfo::Timezone.get("Asia/Taipei").local_to_utc(time)
 => 1895-12-31 15:59:59 UTC

If I do the same experiment on the production server (Debian Squeeze), it seems supporting 64-bit timestamp, and the AmbiguousTime behavior is the same as using tzdata-info:

$ uname -a
Linux [redacted] 2.6.32-5-amd64 #1 SMP Sun Sep 23 10:07:46 UTC 2012 x86_64 GNU/Linux

$ irb
> TZInfo::DataSource.set(:zoneinfo)
> dir = TZInfo::DataSource.get.zoneinfo_dir
> File.open(File.join(dir, 'Asia', 'Taipei'), 'r') {|f| f.read(5) }
 => "TZif2"

> time = Time.new(1895,12,31,23,59,59)
 => 1895-12-31 23:59:59 +0800
>  TZInfo::Timezone.get("Asia/Taipei").local_to_utc(time)
TZInfo::AmbiguousTime: Time: 1895-12-31 23:59:59 UTC is an ambiguous local time.

OS X 10.8.4 ships TZ database version 2013c, both source code and /usr/local/zoneinfo/+VERSION indicates this. However the bundled tz-related programs might not be in the same version.

I surveyed for some time, and learned the following things (please correct me if I am wrong):

1. tz is a single project, but separately releases two packages, tzdata for plaintext data, and tzcode for programs. An unofficial development repository on GitHub can be found at eggert/tz
2. zic is a program that compiles tzdata into binary zoneinfo files.
3. I can generate zoneinfo files manually, which can be read by TZInfo gem.
4. The version of a compiled zoneinfo file is embedded when generating zoneinfo files, and is hard-coded in ZIC_VERSION macro, in zic.c.

If I take zic shipped with OS X (/usr/sbin/zic), it generates old version of zoneinfo files; the first 5 bytes are TZif\0. According to your response, this indicates that the zoneinfo files does not support 64-bit timestamps. The version number of this zic is zic: @(#)zic.c 7.116. It seems that such version number is generated by a version control system called SCCS, and tz migrated to Git on July 19, 2012 at eggert/tz@dccd5a1 .

The version number of zic on my Linux server shows zic (Debian EGLIBC 2.11.3-4) @(#)zic.c 8.19.

If I compile tz 2013d or 2013c manually, either form IANA's website, or 2013c from Apple's Open Source page, I'll get a database that seems supports 64-bit timestamp, and get the correct result from TZInfo gem:

$ mkdir work
$ make TOPDIR=work/ CFLAGS=-DSTD_INSPIRED  install
$ ./zic --version
zic (tzcode) 2013c # or 2013d
$ head -c 5 work/etc/zoneinfo/Asia/Taipei
TZif2
$ irb -rtzinfo
> TZInfo::DataSource.set(:zoneinfo, "work/etc/zoneinfo")
> time = Time.new(1895,12,31,23,59,59)
> TZInfo::Timezone.get("Asia/Taipei").local_to_utc(time)
TZInfo::AmbiguousTime: Time: 1895-12-31 23:59:59 UTC is an ambiguous local time.

I've traced zic.c and tzinfo.h and found that the magic string TZif2 is embedded into the compiled database, during the execution of zic, as early as Feb 20, 2006, and was released at version 2006b for "64-bit code".

From the evidences above, I guess that: OS X 10.8.4 ships an older zic version that does not generate newer zoneinfo files.

Finally, I'll take the workaround you provided: choose any of the ambiguous UTC times, to mute that error in the future. Also, due to the fact that the OS-provided TZ database may not support ancient timestamp, I think I'll always take tzinfo-data in my Ruby application if ancient timestamp is vital.

By the way, I found that rails/rails#10826 is going to upgrade TZInfo to 1.0.1. I think such incorrect local-to-UTC conversion could happen, although it is not common in modern web applications.

Thank you very much!

Sincerely,
Yu-Cheng Chuang

[philr]

Hi,

I've just checked on an old Mac OS X 10.6 machine. It also has 32-bit zoneinfo files with TZif\0 headers and a copy of zic with the same @(#)zic.c 7.116 version string.

Everything you've written about the tz project looks correct to me.

I'll add something to the TZInfo documentation regarding 32-bit zoneinfo files and the issues they can cause.

One of the motivations for making TZInfo work with the system zoneinfo files was to allow it to be included in Linux distributions without duplicating all the time zone data (see Debian Bug #503591 and Red Hat Bug 729763). I think that having Rails not include a dependency on tzinfo-data is the best option - people can always install tzinfo-data or compile their own zoneinfo files if necessary.

You may like to consider using the second dst parameter of local_to_utc as well as using the block. This helps with resolving ambiguities during the annual turning back of the clock in time zones that observe daylight savings time. The dst parameter takes a Boolean and allows you to select whether you would prefer the UTC time corresponding to daylight savings time (true) or standard time (false). If the ambiguity cannot be resolved using the dst parameter (as would be the case at 23:59:59 on 1895-12-31 in Asia/Taipei), then the block will be executed.

Kind regards,

Phil