Test failures in the MRI 1.9 suite (Time class) when TZ is set #948

Closed
Sinjo opened this Issue Aug 7, 2013 · 7 comments

Projects

None yet

2 participants

@Sinjo

On a fresh checkout and build of JRuby, running rake test:mri19 results in a number of test failures related to timezones. When I have TZ=Europe/London I get the following output at the end of the test run:

  1) Skipped:
test_civil__gregorian(TestDateBase) [/home/sinjo/projects/jruby/lib/ruby/gems/shared/gems/minitest-4.7.5/lib/minitest/unit.rb:1258]:
Skipped, no message given

  2) Skipped:
test_civil__julian(TestDateBase) [/home/sinjo/projects/jruby/lib/ruby/gems/shared/gems/minitest-4.7.5/lib/minitest/unit.rb:1258]:
Skipped, no message given

  3) Skipped:
test_commercial__gregorian(TestDateBase) [/home/sinjo/projects/jruby/lib/ruby/gems/shared/gems/minitest-4.7.5/lib/minitest/unit.rb:1258]:
Skipped, no message given

  4) Skipped:
test_nth_kday(TestDateBase) [/home/sinjo/projects/jruby/lib/ruby/gems/shared/gems/minitest-4.7.5/lib/minitest/unit.rb:1258]:
Skipped, no message given

  5) Skipped:
test_ordinal__gregorian(TestDateBase) [/home/sinjo/projects/jruby/lib/ruby/gems/shared/gems/minitest-4.7.5/lib/minitest/unit.rb:1258]:
Skipped, no message given

  6) Skipped:
test_ordinal__julian(TestDateBase) [/home/sinjo/projects/jruby/lib/ruby/gems/shared/gems/minitest-4.7.5/lib/minitest/unit.rb:1258]:
Skipped, no message given

  7) Skipped:
test_weeknum(TestDateBase) [/home/sinjo/projects/jruby/lib/ruby/gems/shared/gems/minitest-4.7.5/lib/minitest/unit.rb:1258]:
Skipped, no message given

  8) Skipped:
test_weeknum__2(TestDateBase) [/home/sinjo/projects/jruby/lib/ruby/gems/shared/gems/minitest-4.7.5/lib/minitest/unit.rb:1258]:
Skipped, no message given

  9) Skipped:
test_base(TestSH) [/home/sinjo/projects/jruby/lib/ruby/gems/shared/gems/minitest-4.7.5/lib/minitest/unit.rb:1258]:
Skipped, no message given

 10) Failure:
test_gen_Europe_London_92(TestTimeTZ) [/home/sinjo/projects/jruby/test/externals/ruby1.9/ruby/test_time_tz.rb:248]:
TZ=Europe/London Time.local(59, 59, 1, 16, 3, 1947, nil, nil, true, nil).
<"1947-03-16 01:59:59 +0000"> expected but was
<"1947-03-16 00:59:59 +0000">.

 11) Failure:
test_gen_Europe_London_93(TestTimeTZ) [/home/sinjo/projects/jruby/test/externals/ruby1.9/ruby/test_time_tz.rb:247]:
TZ=Europe/London Time.local(0, 0, 3, 16, 3, 1947, nil, nil, false, nil).
<"1947-03-16 03:00:00 +0100"> expected but was
<"1947-03-16 04:00:00 +0100">.

 12) Failure:
test_gen_Europe_London_94(TestTimeTZ) [/home/sinjo/projects/jruby/test/externals/ruby1.9/ruby/test_time_tz.rb:247]:
TZ=Europe/London Time.local(59, 59, 1, 13, 4, 1947, nil, nil, false, nil).
<"1947-04-13 01:59:59 +0100"> expected but was
<"1947-04-13 03:59:59 +0200">.

 13) Failure:
test_gen_Europe_London_95(TestTimeTZ) [/home/sinjo/projects/jruby/test/externals/ruby1.9/ruby/test_time_tz.rb:247]:
TZ=Europe/London Time.local(0, 0, 3, 13, 4, 1947, nil, nil, false, nil).
<"1947-04-13 03:00:00 +0200"> expected but was
<"1947-04-13 05:00:00 +0200">.

 14) Failure:
test_gen_Europe_London_96(TestTimeTZ) [/home/sinjo/projects/jruby/test/externals/ruby1.9/ruby/test_time_tz.rb:257]:
TZ=Europe/London Time.local(59, 59, 2, 10, 8, 1947, nil, nil, true, nil).
<"1947-08-10 02:59:59 +0200"> expected but was
<"1947-08-10 02:59:59 +0100">.

 15) Failure:
test_gen_Europe_London_97(TestTimeTZ) [/home/sinjo/projects/jruby/test/externals/ruby1.9/ruby/test_time_tz.rb:264]:
TZ=Europe/London Time.local(0, 0, 2, 10, 8, 1947, nil, nil, false, nil).
<"1947-08-10 02:00:00 +0100"> expected but was
<"1947-08-10 03:00:00 +0100">.

 16) Failure:
test_gen_Europe_London_98(TestTimeTZ) [/home/sinjo/projects/jruby/test/externals/ruby1.9/ruby/test_time_tz.rb:258]:
TZ=Europe/London Time.new(1947, 11, 2, 2, 59, 59, :dst).
<"1947-11-02 02:59:59 +0100"> expected but was
<"1947-11-02 02:59:59 +0000">.

4923 tests, 753218 assertions, 7 failures, 0 errors, 9 skips

When I export TZ=UTC I'm left with just the 9 skipped tests, and no failures.

I'm running Ubuntu 12.04, and the version string of my JRuby build looks like:

jruby 1.7.5.dev (1.9.3p392) 2013-08-07 8b9d926 on OpenJDK 64-Bit Server VM 1.7.0_25-b30 [linux-amd64]

@BanzaiMan
JRuby Team member

Interesting. If you set TZ, the tests involving that time zone will fail.

$ env TZ=Europe/Moscow jruby test/externals/ruby1.9/ruby/test_time_tz.rb
Run options:

# Running tests:

................................................................................................F..FF......................

Finished tests in 0.382000s, 321.9895 tests/s, 185.8639 assertions/s.

  1) Failure:
test_gen_Europe_Moscow_100(TestTimeTZ) [test/externals/ruby1.9/ruby/test_time_tz.rb:248]:
TZ=Europe/Moscow Time.local(59, 59, 1, 19, 1, 1992, nil, nil, true, nil).
<"1992-01-19 01:59:59 +0200"> expected but was
<"1992-01-19 00:59:59 +0200">.

  2) Failure:
test_gen_Europe_Moscow_103(TestTimeTZ) [test/externals/ruby1.9/ruby/test_time_tz.rb:247]:
TZ=Europe/Moscow Time.local(0, 0, 0, 29, 3, 1992, nil, nil, false, nil).
<"1992-03-29 00:00:00 +0400"> expected but was
<"1992-03-29 01:00:00 +0400">.

  3) Failure:
test_gen_Europe_Moscow_104(TestTimeTZ) [test/externals/ruby1.9/ruby/test_time_tz.rb:258]:
TZ=Europe/Moscow Time.new(1992, 9, 26, 22, 59, 59, :dst).
<"1992-09-26 22:59:59 +0400"> expected but was
<"1992-09-26 22:59:59 +0300">.

123 tests, 71 assertions, 3 failures, 0 errors, 0 skips
@BanzaiMan
JRuby Team member

Hmm. These tests are sketchy at best.

If you do not set TZ, no tests are run:

$ ruby -v test/externals/ruby1.9/ruby/test_time_tz.rb                 
ruby 1.9.3p448 (2013-06-27 revision 41675) [x86_64-darwin12.4.0]
test/externals/ruby1.9/ruby/test_time_tz.rb:139: warning: assigned but unused variable - tz
test/externals/ruby1.9/ruby/test_time_tz.rb:239: warning: assigned but unused variable - u
Run options: 

# Running tests:

...........................................................................................................................

Finished tests in 0.005433s, 22639.4257 tests/s, 0.0000 assertions/s.

123 tests, 0 assertions, 0 failures, 0 errors, 0 skips

If you set RUBY_FORCE_TIME_TZ_TEST to yes, the tests run, but quite a few of them fail:

$ env RUBY_FORCE_TIME_TZ_TEST=yes ruby -v test/externals/ruby1.9/ruby/test_time_tz.rb                                                                                                                           
ruby 1.9.3p448 (2013-06-27 revision 41675) [x86_64-darwin12.4.0]
test/externals/ruby1.9/ruby/test_time_tz.rb:139: warning: assigned but unused variable - tz
test/externals/ruby1.9/ruby/test_time_tz.rb:239: warning: assigned but unused variable - u
Run options: 

# Running tests:

......F.........................................................................................................FFFFFFFF.FF

Finished tests in 0.032408s, 3795.3592 tests/s, 21075.0432 assertions/s.
⋮
⋮
123 tests, 683 assertions, 11 failures, 0 errors, 0 skips
@BanzaiMan
JRuby Team member

The design of these tests is such that only when TZ is set, run the tests that pertain to that TZ value. Unless it's on linux (why?) or RUBY_FORCE_TIME_TZ_TEST is set to yes.

This is not to say that there are no JRuby bugs.

$ env RUBY_FORCE_TIME_TZ_TEST=yes jruby -v test/externals/ruby1.9/ruby/test_time_tz.rb
jruby 1.7.5.dev (1.9.3p392) 2013-08-07 531c0e6 on Java HotSpot(TM) 64-Bit Server VM 1.7.0_25-b15 [darwin-x86_64]
Run options:

# Running tests:

E...F.E.......FF.F.F.....FF.FFF..F.........F...F..........FFF.....FF.F.....FF...........FFFFFFF.F..FF...........FFFFFFFF.FF

Finished tests in 0.960000s, 128.1250 tests/s, 635.4167 assertions/s.

⋮
⋮

123 tests, 610 assertions, 41 failures, 2 errors, 0 skips
@Sinjo

Unsurprisingly, these tests pass in MRI. Also looks like they run a whole bunch more assertions by default:

sinjo@widdershins:~$ rvm use 1.9.3
Using /home/sinjo/.rvm/gems/ruby-1.9.3-p448
sinjo@widdershins:~$ env TZ=Europe/London ruby ~/projects/jruby/test/externals/ruby1.9/ruby/test_time_tz.rb
Run options: 

# Running tests:

...........................................................................................................................

Finished tests in 0.053304s, 2307.5320 tests/s, 13713.8690 assertions/s.

123 tests, 731 assertions, 0 failures, 0 errors, 0 skips
@Sinjo

So it looks like these tests are dealing with weird edge cases (at least looking at the ones that run for Europe/London). It turns out that the UK went a bit mad with British Summer Time that year. Details at:

http://www.timeanddate.com/worldclock/clockchange.html?n=136&year=1947
http://en.wikipedia.org/wiki/British_Summer_Time#Periods_of_deviation

@Sinjo

Actually, while that may be why that date was picked in those tests, it could be a red herring for this bug. With a more recent date, in JRuby:

irb(main):002:0> Time.local(59, 59, 1, 16, 3, 2012, nil, nil, true, nil)
=> 2012-03-16 00:59:59 +0000

and in MRI:

1.9.3-p448 :002 > Time.local(59, 59, 1, 16, 3, 2012, nil, nil, true, nil)                                                                                                 
 => 2012-03-16 01:59:59 +0000 
@Sinjo

So I wrote a small script, just looking at 2012 for now to avoid the extra complications with DST in 1947:

require 'date'

puts 'With DST false'
(1..30).each do |day|
  puts Time.local(30, 30, 2, day, 3, 2012, nil, nil, false, nil)
end

puts '-------------'
puts 'With DST true'
(1..30).each do |day|
  puts Time.local(30, 30, 2, day, 3, 2012, nil, nil, true, nil)
end

The interesting parts of the output are, in JRuby:

With DST false
2012-03-23 02:30:30 +0000
2012-03-24 02:30:30 +0000
2012-03-25 03:30:30 +0100
2012-03-26 03:30:30 +0100
-------------
With DST true
2012-03-23 01:30:30 +0000
2012-03-24 01:30:30 +0000
2012-03-25 02:30:30 +0100
2012-03-26 02:30:30 +0100

and in MRI 1.9.3-p448:

With DST false
2012-03-23 02:30:30 +0000
2012-03-24 02:30:30 +0000
2012-03-25 02:30:30 +0100
2012-03-26 02:30:30 +0100
-------------
With DST true
2012-03-23 02:30:30 +0000
2012-03-24 02:30:30 +0000
2012-03-25 02:30:30 +0100
2012-03-26 02:30:30 +0100

Commenting out the entire contents of this if statement makes the behaviour in JRuby line up with that of MRI.

That leaves tests 96 and 98 failing:

  1) Failure:
test_gen_Europe_London_96(TestTimeTZ) [test/externals/ruby1.9/ruby/test_time_tz.rb:257]:
TZ=Europe/London Time.local(59, 59, 2, 10, 8, 1947, nil, nil, true, nil).
<"1947-08-10 02:59:59 +0200"> expected but was
<"1947-08-10 02:59:59 +0100">.

  2) Failure:
test_gen_Europe_London_98(TestTimeTZ) [test/externals/ruby1.9/ruby/test_time_tz.rb:257]:
TZ=Europe/London Time.local(59, 59, 2, 2, 11, 1947, nil, nil, true, nil).
<"1947-11-02 02:59:59 +0100"> expected but was
<"1947-11-02 02:59:59 +0000">.

I'm a bit hesitant to say that just commenting out that entire statement is a valid thing to do, and surprised it doesn't make any extra tests fail. Would definitely appreciate any comments on this, and I'll keep investigating in the meantime.

@Sinjo Sinjo added a commit that referenced this issue Sep 4, 2013
@Sinjo Sinjo Fix handling of :dst Symbol in Time.new
Completely fixes #948
4d388b8
@jrubyci jrubyci pushed a commit that closed this issue Sep 6, 2013
@Sinjo Sinjo Fix handling of Time offsets at DST boundaries.
Partly fixes #948
480beb7
@jrubyci jrubyci closed this in 480beb7 Sep 6, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment