Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sequel fails with -Xcompile.invokedynamic=true #4455

Closed
mwpastore opened this issue Jan 25, 2017 · 5 comments
Closed

Sequel fails with -Xcompile.invokedynamic=true #4455

mwpastore opened this issue Jan 25, 2017 · 5 comments
Labels
invokedynamic jit JRuby 9000 needs tests
Milestone
JRuby 9.1.8.0

Comments

@mwpastore
Copy link

Environment

macOS Sierra

$ jruby -v
jruby 9.1.7.0 (2.3.1) 2017-01-11 68056ae Java HotSpot(TM) 64-Bit Server VM 25.51-b03 on 1.8.0_51-b16 +indy +jit [darwin-x86_64]
$ uname -a
Darwin poseidon.local 16.3.0 Darwin Kernel Version 16.3.0: Thu Nov 17 20:23:58 PST 2016; root:xnu-3789.31.2~1/RELEASE_X86_64 x86_64

Ubuntu Server 14.04 LTS (Trusty)

$ jruby -v
jruby 9.1.7.0 (2.3.1) 2017-01-11 68056ae Java HotSpot(TM) 64-Bit Server VM 25.121-b13 on 1.8.0_121-b13 +jit [linux-x86_64]
$ uname -a
Linux tfb-all 4.4.0-59-generic #80~14.04.1-Ubuntu SMP Fri Jan 6 18:02:02 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

Gemfile:

source 'https://rubygems.org'

gem 'sequel'
gem 'jdbc-postgres'

test.rb:

# frozen_string_literal: true
require 'sequel'

DB = Sequel.connect('jdbc:postgresql://localhost/foo')

class Bar < Sequel::Model
end

Steps to reproduce:

  1. Install and start PostgreSQL server 9.3-9.6
  2. Create a foo database accessible by your current user (createdb foo) without any password (or edit the connection string in test.rb to suit)
    • You do not need to create any tables
  3. bundle install
  4. JRUBY_OPTS=<see below> bundle exec ruby test.rb

Expected Behavior

$ JRUBY_OPTS=-Xcompile.invokedynamic=true bundle exec ruby test.rb
$

Actual Behavior

$ JRUBY_OPTS=-Xcompile.invokedynamic=true bundle exec ruby test.rb
NoMethodError: undefined method `identifier_input_method' for nil:NilClass
              identifier_input_method at /Users/mwp/.rbenv/versions/jruby-9.1.7.0/lib/ruby/gems/shared/gems/sequel-4.42.1/lib/sequel/extensions/identifier_mangling.rb:141
                     input_identifier at /Users/mwp/.rbenv/versions/jruby-9.1.7.0/lib/ruby/gems/shared/gems/sequel-4.42.1/lib/sequel/extensions/identifier_mangling.rb:183
              quote_identifier_append at /Users/mwp/.rbenv/versions/jruby-9.1.7.0/lib/ruby/gems/shared/gems/sequel-4.42.1/lib/sequel/dataset/sql.rb:717
              quote_identifier_append at /Users/mwp/.rbenv/versions/jruby-9.1.7.0/lib/ruby/gems/shared/gems/sequel-4.42.1/lib/sequel/dataset/sql.rb:-1
                    identifier_append at /Users/mwp/.rbenv/versions/jruby-9.1.7.0/lib/ruby/gems/shared/gems/sequel-4.42.1/lib/sequel/dataset/sql.rb:1103
      qualified_identifier_sql_append at /Users/mwp/.rbenv/versions/jruby-9.1.7.0/lib/ruby/gems/shared/gems/sequel-4.42.1/lib/sequel/dataset/sql.rb:703
                          to_s_append at /Users/mwp/.rbenv/versions/jruby-9.1.7.0/lib/ruby/gems/shared/gems/sequel-4.42.1/lib/sequel/sql.rb:120
            literal_expression_append at /Users/mwp/.rbenv/versions/jruby-9.1.7.0/lib/ruby/gems/shared/gems/sequel-4.42.1/lib/sequel/dataset/sql.rb:1238
                       literal_append at /Users/mwp/.rbenv/versions/jruby-9.1.7.0/lib/ruby/gems/shared/gems/sequel-4.42.1/lib/sequel/dataset/sql.rb:86
        aliased_expression_sql_append at /Users/mwp/.rbenv/versions/jruby-9.1.7.0/lib/ruby/gems/shared/gems/sequel-4.42.1/lib/sequel/dataset/sql.rb:356
                          to_s_append at /Users/mwp/.rbenv/versions/jruby-9.1.7.0/lib/ruby/gems/shared/gems/sequel-4.42.1/lib/sequel/sql.rb:120
            literal_expression_append at /Users/mwp/.rbenv/versions/jruby-9.1.7.0/lib/ruby/gems/shared/gems/sequel-4.42.1/lib/sequel/dataset/sql.rb:1238
            literal_expression_append at /Users/mwp/.rbenv/versions/jruby-9.1.7.0/lib/ruby/gems/shared/gems/sequel-4.42.1/lib/sequel/dataset/sql.rb:-1
                       literal_append at /Users/mwp/.rbenv/versions/jruby-9.1.7.0/lib/ruby/gems/shared/gems/sequel-4.42.1/lib/sequel/dataset/sql.rb:86
      block in expression_list_append at /Users/mwp/.rbenv/versions/jruby-9.1.7.0/lib/ruby/gems/shared/gems/sequel-4.42.1/lib/sequel/dataset/sql.rb:1033
                                 each at org/jruby/RubyArray.java:1733
               expression_list_append at /Users/mwp/.rbenv/versions/jruby-9.1.7.0/lib/ruby/gems/shared/gems/sequel-4.42.1/lib/sequel/dataset/sql.rb:1031
                   column_list_append at /Users/mwp/.rbenv/versions/jruby-9.1.7.0/lib/ruby/gems/shared/gems/sequel-4.42.1/lib/sequel/dataset/sql.rb:937
                   select_columns_sql at /Users/mwp/.rbenv/versions/jruby-9.1.7.0/lib/ruby/gems/shared/gems/sequel-4.42.1/lib/sequel/dataset/sql.rb:1366
                           select_sql at /Users/mwp/.rbenv/versions/jruby-9.1.7.0/lib/ruby/gems/shared/gems/sequel-4.42.1/lib/sequel/dataset/sql.rb:237
                                 each at /Users/mwp/.rbenv/versions/jruby-9.1.7.0/lib/ruby/gems/shared/gems/sequel-4.42.1/lib/sequel/dataset/actions.rb:149
                                  map at org/jruby/RubyEnumerable.java:830
                                  map at /Users/mwp/.rbenv/versions/jruby-9.1.7.0/lib/ruby/gems/shared/gems/sequel-4.42.1/lib/sequel/dataset/actions.rb:455
                   schema_parse_table at /Users/mwp/.rbenv/versions/jruby-9.1.7.0/lib/ruby/gems/shared/gems/sequel-4.42.1/lib/sequel/adapters/shared/postgres.rb:1133
                               schema at /Users/mwp/.rbenv/versions/jruby-9.1.7.0/lib/ruby/gems/shared/gems/sequel-4.42.1/lib/sequel/database/query.rb:161
               block in get_db_schema at /Users/mwp/.rbenv/versions/jruby-9.1.7.0/lib/ruby/gems/shared/gems/sequel-4.42.1/lib/sequel/model/base.rb:1044
  block in check_non_connection_error at /Users/mwp/.rbenv/versions/jruby-9.1.7.0/lib/ruby/gems/shared/gems/sequel-4.42.1/lib/sequel/model/base.rb:937
                 block in transaction at /Users/mwp/.rbenv/versions/jruby-9.1.7.0/lib/ruby/gems/shared/gems/sequel-4.42.1/lib/sequel/database/transactions.rb:162
                                 hold at /Users/mwp/.rbenv/versions/jruby-9.1.7.0/lib/ruby/gems/shared/gems/sequel-4.42.1/lib/sequel/connection_pool/threaded.rb:107
                          synchronize at /Users/mwp/.rbenv/versions/jruby-9.1.7.0/lib/ruby/gems/shared/gems/sequel-4.42.1/lib/sequel/database/connecting.rb:285
                          transaction at /Users/mwp/.rbenv/versions/jruby-9.1.7.0/lib/ruby/gems/shared/gems/sequel-4.42.1/lib/sequel/database/transactions.rb:156
           check_non_connection_error at /Users/mwp/.rbenv/versions/jruby-9.1.7.0/lib/ruby/gems/shared/gems/sequel-4.42.1/lib/sequel/model/base.rb:937
                        get_db_schema at /Users/mwp/.rbenv/versions/jruby-9.1.7.0/lib/ruby/gems/shared/gems/sequel-4.42.1/lib/sequel/model/base.rb:1044
                          set_dataset at /Users/mwp/.rbenv/versions/jruby-9.1.7.0/lib/ruby/gems/shared/gems/sequel-4.42.1/lib/sequel/model/base.rb:824
                                Model at /Users/mwp/.rbenv/versions/jruby-9.1.7.0/lib/ruby/gems/shared/gems/sequel-4.42.1/lib/sequel/model/base.rb:200
                       block in Model at /Users/mwp/.rbenv/versions/jruby-9.1.7.0/lib/ruby/gems/shared/gems/sequel-4.42.1/lib/sequel/model/base.rb:145
                               <main> at test.rb:6

Notes

  • It doesn't always fail, so run it a few times (or add additional Sequel::Model subclasses to increase the odds of it happening on any given run)
  • It never fails with JRUBY_OPTS= (i.e. blanked out)
  • It never fails gem 'jdbc-sqlite3' and a connection string of jdbc:sqlite::memory:
@olleolleolle
Copy link
Member

olleolleolle commented Feb 8, 2017 •
edited

The exact code line in Sequel 4.42.1 See also line 183.

@headius
Copy link
Member

headius commented Feb 9, 2017

You can force JIT to happen sooner with -Xjit.threshold=0. That will help confirm it's JIT+indy causing this, but I have no doubt you're right.

@headius headius added this to the JRuby 9.1.8.0 milestone Feb 9, 2017
@headius
Copy link
Member

headius commented Feb 22, 2017

Thank you for the reproduction. I've managed to reproduce the error locally with various different errors. Setting -Xjit,threshold=0 produces the following error consistently:

~/projects/jruby/pgsql_error $ JRUBY_OPTS='-Xcompile.invokedynamic -Xjit.threshold=0' bundle exec ruby test.rb
ArgumentError: odd number of arguments for Hash
                                   [] at org/jruby/RubyHash.java:186
                      initialize_copy at /home/headius/projects/jruby/lib/ruby/gems/shared/gems/sequel-4.43.0/lib/sequel/dataset/misc.rb:336
                                clone at org/jruby/RubyKernel.java:1881
                                clone at /home/headius/projects/jruby/lib/ruby/gems/shared/gems/sequel-4.43.0/lib/sequel/dataset/query.rb:100
                                 from at /home/headius/projects/jruby/lib/ruby/gems/shared/gems/sequel-4.43.0/lib/sequel/dataset/query.rb:263
                   schema_parse_table at /home/headius/projects/jruby/lib/ruby/gems/shared/gems/sequel-4.43.0/lib/sequel/adapters/shared/postgres.rb:1123
                               schema at /home/headius/projects/jruby/lib/ruby/gems/shared/gems/sequel-4.43.0/lib/sequel/database/query.rb:161
               block in get_db_schema at /home/headius/projects/jruby/lib/ruby/gems/shared/gems/sequel-4.43.0/lib/sequel/model/base.rb:1048
  block in check_non_connection_error at /home/headius/projects/jruby/lib/ruby/gems/shared/gems/sequel-4.43.0/lib/sequel/model/base.rb:941
                 block in transaction at /home/headius/projects/jruby/lib/ruby/gems/shared/gems/sequel-4.43.0/lib/sequel/database/transactions.rb:162
                                 hold at /home/headius/projects/jruby/lib/ruby/gems/shared/gems/sequel-4.43.0/lib/sequel/connection_pool/threaded.rb:107
                          synchronize at /home/headius/projects/jruby/lib/ruby/gems/shared/gems/sequel-4.43.0/lib/sequel/database/connecting.rb:285
                          transaction at /home/headius/projects/jruby/lib/ruby/gems/shared/gems/sequel-4.43.0/lib/sequel/database/transactions.rb:156
           check_non_connection_error at /home/headius/projects/jruby/lib/ruby/gems/shared/gems/sequel-4.43.0/lib/sequel/model/base.rb:941
                        get_db_schema at /home/headius/projects/jruby/lib/ruby/gems/shared/gems/sequel-4.43.0/lib/sequel/model/base.rb:1048
                          set_dataset at /home/headius/projects/jruby/lib/ruby/gems/shared/gems/sequel-4.43.0/lib/sequel/model/base.rb:828
                            inherited at /home/headius/projects/jruby/lib/ruby/gems/shared/gems/sequel-4.43.0/lib/sequel/model/base.rb:651
                               <main> at test.rb:6

I'll start here.

@headius
Copy link
Member

headius commented Feb 23, 2017

The problem was that invokedynamic was incorrectly binding instance variables in the cases where we allocate regular Java fields for them. The field variables of the object in my error above were at "logical" instance variable indices of 0, 1, and 2. The @opts variable accessed above was in the additional table of instance variables at index 1. When indy bound that variable, it used the table offset when calling methods that wanted the logical offset, so the object field at index 1 was accessed instead. It was empty in this case, resulting in a nil being passed to Hash[] and triggering the error above.

I also ran your example without the JIT threshold set, and with it set to a low non-zero value. All runs repeatedly passed, so I think we're good.

@Marahin
Copy link

Marahin commented Mar 31, 2017

@headius I can confirm that on jruby-9.1.8.0 the problem is no longer seen, where as mentioned error was seen on 9.1.7.0.
sequel (4.44.0)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
invokedynamic jit JRuby 9000 needs tests
Projects
None yet
Milestone
JRuby 9.1.8.0
Development

No branches or pull requests
4 participants
@olleolleolle @headius @Marahin @mwpastore