Run specs also on TruffleRuby on Travis CI

[deepj]

TruffleRuby works on Travis finally. travis-ci/travis-build#1604 It always installs the latest version of TruffleRuby.

Sequel specs don't pass now. One of the reasons why not passing Object#clone(freeze) is not supported yet but the fix is coming soon in the next release oracle/truffleruby#1454

I believe running specs on TruffleRuby would help its developers

[jeremyevans]

Thanks for the patch! This seems fine for me if it helps out TruffleRuby and doesn't put too much load on Travis. I'll merge this and see how long the TruffleRuby job takes.

[deepj]

@jeremyevans the biggest issue is recently with Bundler when bundle install is very slow oracle/truffleruby#1398 maybe @eregon will know if some improvement of the issue is coming

[jeremyevans]

The TruffleRuby job takes 18 minutes (compared to 2-3 minutes for MRI and about 5 minutes for JRuby), and doesn't even run any specs as it fails to install nokogiri. To save time on Travis, I'm going to revert this change. Please resubmit after the nokogiri issue has been addressed.

[deepj]

@jeremyevans interesting. In my case, I can install Nokogiri and use under TruffleRuby. But I use macOS personally.

irb(main):001:0> RUBY_ENGINE
=> "truffleruby"
irb(main):002:0> require 'open-uri'
=> true
irb(main):003:0> require 'nokogiri'
=> true
irb(main):004:0> Nokogiri::HTML(open('http://www.nokogiri.org/tutorials/installing_nokogiri.html'))
=> #<Nokogiri::HTML::Document:0x28ba name="document" children=[#<Nokogiri::XML::DTD:0x876 name="html">, #<Nokogiri::XML::Comment:0x878 "[if lt IE 7]>...

I can't see Sequel's Travis would be publicly accessible. Is it possible to share the Nokogiri installation issue? I could report it.

[jeremyevans]

Sequel's Travis output is publicly accessible: https://travis-ci.org/jeremyevans/sequel/jobs/460952287

[chrisseaton]

Yes sorry it's slow to install gems at the moment - we're working on it.

[MaxLap]

For the nokogiri issue mentionned, you need to add bundle config build.nokogiri --use-system-libraries before installing the gems on truffleruby.

[deepj]

@MaxLap I'm counting to add this (I hope @jeremyevans is not against it) in the next PR when bundle install won't be so slow and installation of sequel_pg won't fail as well. That is already fixed oracle/truffleruby#1440 anyway.

@jeremyevans Will you be OK when running specs would be around 4 - 8 minutes?

[jeremyevans]

@deepj Yes, 4-8 minutes is fine.

[eregon]

Just as a note, Bundler tries to install nokogiri 3 times in that log, because the default TravisCI install command is bundle install --jobs=3 --retry=3. That might impact a bit the time, but installing gems/bundle install is still too slow.

[deepj]

@jeremyevans I've tried newer RC10 and Bundler is still quite slow but everything is okay except one thing. One spec "should allow mathematical or string operations on numerics when argument is a generic or numeric expressions" fails and in this expression

BigDecimal('1.0') * Sequel[:a][:y]

It throws NoMethodError: undefined method 'to_f' for #<Sequel::SQL::QualifiedIdentifier @column=>:y, @table=>"a">:Sequel::SQL::QualifiedIdentifier

Any idea why it happens? I thing the problem is in TruffleRuby's BigDecimal implementation but I'm not sure how to debug it under MRI or TruffleRuby. See what everything is called.

[deepj]

See oracle/truffleruby#1507

[jeremyevans]

I agree it looks like an issue in TruffleRuby. Sequel uses Ruby's coercion protocol for such things: https://github.com/jeremyevans/sequel/blob/master/lib/sequel/sql.rb#L787. Is TruffleRuby's BigDecimal implementation not calling coerce on the Sequel::SQL::QualifiedIdentifier instance?

[deepj]

@jeremyevans yes, it's caused by that. @janko-m was faster to make a reproducible example oracle/truffleruby#1507 (comment)

After fixing this, Sequel's specs would pass on TruffleRuby (including sequel-pg)

[deepj]

@jeremyevans Hello Jeremy, it is possible to try it again in Travis CI? From the last, several things have been improved (cc: @eregon) including some things around TruffleRuby and Travis CI. But still, Bundler and RubyGems are not so quick. There is a chance, the time has been reduced.

Recently, only one test failing due to this oracle/truffleruby#1598

➜  sequel git:(master) rake spec
~/.rubies/truffleruby-1.0.0-rc13/bin/truffleruby spec/core_spec.rb
Run options: --seed 14136

# Running:

............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................E..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

Finished in 51.111929s, 36.9190 runs/s, 100.0158 assertions/s.

  1) Error:
Blockless Ruby Filters#test_0024_should allow mathematical or string operations on numerics when argument is a generic or numeric expressions:
TypeError: Truffle doesn't have a case for the org.truffleruby.stdlib.bigdecimal.BigDecimalNodesFactory$MultOpNodeFactory$MultOpNodeGen node with values of type  BigDecimal(com.oracle.truffle.object.basic.DynamicObjectBasic) Sequel::SQL::QualifiedIdentifier(com.oracle.truffle.object.basic.DynamicObjectBasic)
    ~/dev/bugs/truffleruby/sequel/spec/core/expression_filters_spec.rb:211:in `*'
    ~/dev/bugs/truffleruby/sequel/spec/core/expression_filters_spec.rb:211:in `test_0024_should allow mathematical or string operations on numerics when argument is a generic or numeric expressions'

1887 runs, 5112 assertions, 0 failures, 1 errors, 0 skips
rake aborted!
Command failed with status (1): [~/.rubies/truffleruby-1.0.0-rc1...]
~/dev/bugs/truffleruby/sequel/rakefile:73:in `block in <top (required)>'
~/dev/bugs/truffleruby/sequel/rakefile:80:in `call'
~/dev/bugs/truffleruby/sequel/rakefile:80:in `block (2 levels) in <top (required)>'
~/.gem/truffleruby/2.4.4/gems/rake-12.3.2/exe/rake:27:in `<top (required)>'
Tasks: TOP => spec => spec_core
(See full trace by running task with --trace)
➜  sequel git:(master)

[jeremyevans]

Sure, I can enable it again and see how it goes. It still appears to be 4x longer than JRuby and over 15x longer than CRuby for the core spec.

[deepj]

Thanks. Is the duration of running specs in TruffleRuby acceptable for you? I believe it will be improving over time and even in RC14 all specs will pass as well.

I tried hard to report everything around sequel/sequel_pg to be able to run in TruffleRuby :)

[jeremyevans]

@deepj At a little over 8 minutes, it isn't too bad. The core spec is one of the faster ones, though.

I am certainly excited to see TruffleRuby getting mature enough to run all of Sequel. I look forward to benchmarks.

[deepj]

@jeremyevans Good! Anyway, the last problem was solved by oracle/truffleruby#1598 so Sequel specs would pass in RC14!