Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test failures in the MRI 1.9 suite (Time class) when TZ is set #948

Closed
Sinjo opened this Issue Aug 7, 2013 · 7 comments

Comments

Projects
None yet
2 participants
@Sinjo
Copy link
Contributor

commented Aug 7, 2013

On a fresh checkout and build of JRuby, running rake test:mri19 results in a number of test failures related to timezones. When I have TZ=Europe/London I get the following output at the end of the test run:

  1) Skipped:
test_civil__gregorian(TestDateBase) [/home/sinjo/projects/jruby/lib/ruby/gems/shared/gems/minitest-4.7.5/lib/minitest/unit.rb:1258]:
Skipped, no message given

  2) Skipped:
test_civil__julian(TestDateBase) [/home/sinjo/projects/jruby/lib/ruby/gems/shared/gems/minitest-4.7.5/lib/minitest/unit.rb:1258]:
Skipped, no message given

  3) Skipped:
test_commercial__gregorian(TestDateBase) [/home/sinjo/projects/jruby/lib/ruby/gems/shared/gems/minitest-4.7.5/lib/minitest/unit.rb:1258]:
Skipped, no message given

  4) Skipped:
test_nth_kday(TestDateBase) [/home/sinjo/projects/jruby/lib/ruby/gems/shared/gems/minitest-4.7.5/lib/minitest/unit.rb:1258]:
Skipped, no message given

  5) Skipped:
test_ordinal__gregorian(TestDateBase) [/home/sinjo/projects/jruby/lib/ruby/gems/shared/gems/minitest-4.7.5/lib/minitest/unit.rb:1258]:
Skipped, no message given

  6) Skipped:
test_ordinal__julian(TestDateBase) [/home/sinjo/projects/jruby/lib/ruby/gems/shared/gems/minitest-4.7.5/lib/minitest/unit.rb:1258]:
Skipped, no message given

  7) Skipped:
test_weeknum(TestDateBase) [/home/sinjo/projects/jruby/lib/ruby/gems/shared/gems/minitest-4.7.5/lib/minitest/unit.rb:1258]:
Skipped, no message given

  8) Skipped:
test_weeknum__2(TestDateBase) [/home/sinjo/projects/jruby/lib/ruby/gems/shared/gems/minitest-4.7.5/lib/minitest/unit.rb:1258]:
Skipped, no message given

  9) Skipped:
test_base(TestSH) [/home/sinjo/projects/jruby/lib/ruby/gems/shared/gems/minitest-4.7.5/lib/minitest/unit.rb:1258]:
Skipped, no message given

 10) Failure:
test_gen_Europe_London_92(TestTimeTZ) [/home/sinjo/projects/jruby/test/externals/ruby1.9/ruby/test_time_tz.rb:248]:
TZ=Europe/London Time.local(59, 59, 1, 16, 3, 1947, nil, nil, true, nil).
<"1947-03-16 01:59:59 +0000"> expected but was
<"1947-03-16 00:59:59 +0000">.

 11) Failure:
test_gen_Europe_London_93(TestTimeTZ) [/home/sinjo/projects/jruby/test/externals/ruby1.9/ruby/test_time_tz.rb:247]:
TZ=Europe/London Time.local(0, 0, 3, 16, 3, 1947, nil, nil, false, nil).
<"1947-03-16 03:00:00 +0100"> expected but was
<"1947-03-16 04:00:00 +0100">.

 12) Failure:
test_gen_Europe_London_94(TestTimeTZ) [/home/sinjo/projects/jruby/test/externals/ruby1.9/ruby/test_time_tz.rb:247]:
TZ=Europe/London Time.local(59, 59, 1, 13, 4, 1947, nil, nil, false, nil).
<"1947-04-13 01:59:59 +0100"> expected but was
<"1947-04-13 03:59:59 +0200">.

 13) Failure:
test_gen_Europe_London_95(TestTimeTZ) [/home/sinjo/projects/jruby/test/externals/ruby1.9/ruby/test_time_tz.rb:247]:
TZ=Europe/London Time.local(0, 0, 3, 13, 4, 1947, nil, nil, false, nil).
<"1947-04-13 03:00:00 +0200"> expected but was
<"1947-04-13 05:00:00 +0200">.

 14) Failure:
test_gen_Europe_London_96(TestTimeTZ) [/home/sinjo/projects/jruby/test/externals/ruby1.9/ruby/test_time_tz.rb:257]:
TZ=Europe/London Time.local(59, 59, 2, 10, 8, 1947, nil, nil, true, nil).
<"1947-08-10 02:59:59 +0200"> expected but was
<"1947-08-10 02:59:59 +0100">.

 15) Failure:
test_gen_Europe_London_97(TestTimeTZ) [/home/sinjo/projects/jruby/test/externals/ruby1.9/ruby/test_time_tz.rb:264]:
TZ=Europe/London Time.local(0, 0, 2, 10, 8, 1947, nil, nil, false, nil).
<"1947-08-10 02:00:00 +0100"> expected but was
<"1947-08-10 03:00:00 +0100">.

 16) Failure:
test_gen_Europe_London_98(TestTimeTZ) [/home/sinjo/projects/jruby/test/externals/ruby1.9/ruby/test_time_tz.rb:258]:
TZ=Europe/London Time.new(1947, 11, 2, 2, 59, 59, :dst).
<"1947-11-02 02:59:59 +0100"> expected but was
<"1947-11-02 02:59:59 +0000">.

4923 tests, 753218 assertions, 7 failures, 0 errors, 9 skips

When I export TZ=UTC I'm left with just the 9 skipped tests, and no failures.

I'm running Ubuntu 12.04, and the version string of my JRuby build looks like:

jruby 1.7.5.dev (1.9.3p392) 2013-08-07 8b9d926 on OpenJDK 64-Bit Server VM 1.7.0_25-b30 [linux-amd64]

@BanzaiMan

This comment has been minimized.

Copy link
Member

commented Aug 9, 2013

Interesting. If you set TZ, the tests involving that time zone will fail.

$ env TZ=Europe/Moscow jruby test/externals/ruby1.9/ruby/test_time_tz.rb
Run options:

# Running tests:

................................................................................................F..FF......................

Finished tests in 0.382000s, 321.9895 tests/s, 185.8639 assertions/s.

  1) Failure:
test_gen_Europe_Moscow_100(TestTimeTZ) [test/externals/ruby1.9/ruby/test_time_tz.rb:248]:
TZ=Europe/Moscow Time.local(59, 59, 1, 19, 1, 1992, nil, nil, true, nil).
<"1992-01-19 01:59:59 +0200"> expected but was
<"1992-01-19 00:59:59 +0200">.

  2) Failure:
test_gen_Europe_Moscow_103(TestTimeTZ) [test/externals/ruby1.9/ruby/test_time_tz.rb:247]:
TZ=Europe/Moscow Time.local(0, 0, 0, 29, 3, 1992, nil, nil, false, nil).
<"1992-03-29 00:00:00 +0400"> expected but was
<"1992-03-29 01:00:00 +0400">.

  3) Failure:
test_gen_Europe_Moscow_104(TestTimeTZ) [test/externals/ruby1.9/ruby/test_time_tz.rb:258]:
TZ=Europe/Moscow Time.new(1992, 9, 26, 22, 59, 59, :dst).
<"1992-09-26 22:59:59 +0400"> expected but was
<"1992-09-26 22:59:59 +0300">.

123 tests, 71 assertions, 3 failures, 0 errors, 0 skips
@BanzaiMan

This comment has been minimized.

Copy link
Member

commented Aug 9, 2013

Hmm. These tests are sketchy at best.

If you do not set TZ, no tests are run:

$ ruby -v test/externals/ruby1.9/ruby/test_time_tz.rb                 
ruby 1.9.3p448 (2013-06-27 revision 41675) [x86_64-darwin12.4.0]
test/externals/ruby1.9/ruby/test_time_tz.rb:139: warning: assigned but unused variable - tz
test/externals/ruby1.9/ruby/test_time_tz.rb:239: warning: assigned but unused variable - u
Run options: 

# Running tests:

...........................................................................................................................

Finished tests in 0.005433s, 22639.4257 tests/s, 0.0000 assertions/s.

123 tests, 0 assertions, 0 failures, 0 errors, 0 skips

If you set RUBY_FORCE_TIME_TZ_TEST to yes, the tests run, but quite a few of them fail:

$ env RUBY_FORCE_TIME_TZ_TEST=yes ruby -v test/externals/ruby1.9/ruby/test_time_tz.rb                                                                                                                           
ruby 1.9.3p448 (2013-06-27 revision 41675) [x86_64-darwin12.4.0]
test/externals/ruby1.9/ruby/test_time_tz.rb:139: warning: assigned but unused variable - tz
test/externals/ruby1.9/ruby/test_time_tz.rb:239: warning: assigned but unused variable - u
Run options: 

# Running tests:

......F.........................................................................................................FFFFFFFF.FF

Finished tests in 0.032408s, 3795.3592 tests/s, 21075.0432 assertions/s.
⋮
⋮
123 tests, 683 assertions, 11 failures, 0 errors, 0 skips
@BanzaiMan

This comment has been minimized.

Copy link
Member

commented Aug 9, 2013

The design of these tests is such that only when TZ is set, run the tests that pertain to that TZ value. Unless it's on linux (why?) or RUBY_FORCE_TIME_TZ_TEST is set to yes.

This is not to say that there are no JRuby bugs.

$ env RUBY_FORCE_TIME_TZ_TEST=yes jruby -v test/externals/ruby1.9/ruby/test_time_tz.rb
jruby 1.7.5.dev (1.9.3p392) 2013-08-07 531c0e6 on Java HotSpot(TM) 64-Bit Server VM 1.7.0_25-b15 [darwin-x86_64]
Run options:

# Running tests:

E...F.E.......FF.F.F.....FF.FFF..F.........F...F..........FFF.....FF.F.....FF...........FFFFFFF.F..FF...........FFFFFFFF.FF

Finished tests in 0.960000s, 128.1250 tests/s, 635.4167 assertions/s.

⋮
⋮

123 tests, 610 assertions, 41 failures, 2 errors, 0 skips
@Sinjo

This comment has been minimized.

Copy link
Contributor Author

commented Aug 10, 2013

Unsurprisingly, these tests pass in MRI. Also looks like they run a whole bunch more assertions by default:

sinjo@widdershins:~$ rvm use 1.9.3
Using /home/sinjo/.rvm/gems/ruby-1.9.3-p448
sinjo@widdershins:~$ env TZ=Europe/London ruby ~/projects/jruby/test/externals/ruby1.9/ruby/test_time_tz.rb
Run options: 

# Running tests:

...........................................................................................................................

Finished tests in 0.053304s, 2307.5320 tests/s, 13713.8690 assertions/s.

123 tests, 731 assertions, 0 failures, 0 errors, 0 skips
@Sinjo

This comment has been minimized.

Copy link
Contributor Author

commented Aug 10, 2013

So it looks like these tests are dealing with weird edge cases (at least looking at the ones that run for Europe/London). It turns out that the UK went a bit mad with British Summer Time that year. Details at:

http://www.timeanddate.com/worldclock/clockchange.html?n=136&year=1947
http://en.wikipedia.org/wiki/British_Summer_Time#Periods_of_deviation

@Sinjo

This comment has been minimized.

Copy link
Contributor Author

commented Aug 10, 2013

Actually, while that may be why that date was picked in those tests, it could be a red herring for this bug. With a more recent date, in JRuby:

irb(main):002:0> Time.local(59, 59, 1, 16, 3, 2012, nil, nil, true, nil)
=> 2012-03-16 00:59:59 +0000

and in MRI:

1.9.3-p448 :002 > Time.local(59, 59, 1, 16, 3, 2012, nil, nil, true, nil)                                                                                                 
 => 2012-03-16 01:59:59 +0000 
@Sinjo

This comment has been minimized.

Copy link
Contributor Author

commented Aug 26, 2013

So I wrote a small script, just looking at 2012 for now to avoid the extra complications with DST in 1947:

require 'date'

puts 'With DST false'
(1..30).each do |day|
  puts Time.local(30, 30, 2, day, 3, 2012, nil, nil, false, nil)
end

puts '-------------'
puts 'With DST true'
(1..30).each do |day|
  puts Time.local(30, 30, 2, day, 3, 2012, nil, nil, true, nil)
end

The interesting parts of the output are, in JRuby:

With DST false
2012-03-23 02:30:30 +0000
2012-03-24 02:30:30 +0000
2012-03-25 03:30:30 +0100
2012-03-26 03:30:30 +0100
-------------
With DST true
2012-03-23 01:30:30 +0000
2012-03-24 01:30:30 +0000
2012-03-25 02:30:30 +0100
2012-03-26 02:30:30 +0100

and in MRI 1.9.3-p448:

With DST false
2012-03-23 02:30:30 +0000
2012-03-24 02:30:30 +0000
2012-03-25 02:30:30 +0100
2012-03-26 02:30:30 +0100
-------------
With DST true
2012-03-23 02:30:30 +0000
2012-03-24 02:30:30 +0000
2012-03-25 02:30:30 +0100
2012-03-26 02:30:30 +0100

Commenting out the entire contents of this if statement makes the behaviour in JRuby line up with that of MRI.

That leaves tests 96 and 98 failing:

  1) Failure:
test_gen_Europe_London_96(TestTimeTZ) [test/externals/ruby1.9/ruby/test_time_tz.rb:257]:
TZ=Europe/London Time.local(59, 59, 2, 10, 8, 1947, nil, nil, true, nil).
<"1947-08-10 02:59:59 +0200"> expected but was
<"1947-08-10 02:59:59 +0100">.

  2) Failure:
test_gen_Europe_London_98(TestTimeTZ) [test/externals/ruby1.9/ruby/test_time_tz.rb:257]:
TZ=Europe/London Time.local(59, 59, 2, 2, 11, 1947, nil, nil, true, nil).
<"1947-11-02 02:59:59 +0100"> expected but was
<"1947-11-02 02:59:59 +0000">.

I'm a bit hesitant to say that just commenting out that entire statement is a valid thing to do, and surprised it doesn't make any extra tests fail. Would definitely appreciate any comments on this, and I'll keep investigating in the meantime.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.