Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Auto-indexes in SQLite have no SQL definition in master table #377

Closed
Holek opened this issue Apr 18, 2013 · 4 comments
Closed

Auto-indexes in SQLite have no SQL definition in master table #377

Holek opened this issue Apr 18, 2013 · 4 comments
Labels
Milestone

Comments

@Holek
Copy link
Contributor

Holek commented Apr 18, 2013

I have a problem in indexes method of arjdbc/sqlite3/adapter.rb file.

Auto-indexes in my SQLite database have not been populated with their responsive SQL statements in SQLite master table.

Therefore, I assume that index summary is not reliable, and shouldn't be used by jdbc adapter.

What I'm trying to do is to run these migrations and then rollback on this database: fusioneer-ltd/shiva/blob/specs/spec/fixtures/databases/ponies.sqlite. This error occurs whether it's a database created by hand, or with migrations.

I get an exception in jdbc-adapter, trying to look up indexes in it.

Stacktrace

undefined method `match' for nil:NilClass
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-jdbc-adapter-1.2.9/lib/arjdbc/sqlite3/adapter.rb:220:in `indexes'
org/jruby/RubyArray.java:2361:in `map'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-jdbc-adapter-1.2.9/lib/arjdbc/sqlite3/adapter.rb:217:in `indexes'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-jdbc-adapter-1.2.9/lib/arjdbc/jdbc/missing_functionality_helper.rb:50:in `copy_table_indexes'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-jdbc-adapter-1.2.9/lib/arjdbc/jdbc/missing_functionality_helper.rb:43:in `copy_table'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-jdbc-adapter-1.2.9/lib/arjdbc/jdbc/missing_functionality_helper.rb:20:in `move_table'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-jdbc-adapter-1.2.9/lib/arjdbc/jdbc/missing_functionality_helper.rb:13:in `alter_table'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-3.2.13/lib/active_record/connection_adapters/abstract/database_statements.rb:192:in `transaction'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-jdbc-adapter-1.2.9/lib/arjdbc/jdbc/missing_functionality_helper.rb:10:in `alter_table'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-jdbc-adapter-1.2.9/lib/arjdbc/sqlite3/adapter.rb:311:in `remove_column'
org/jruby/RubyArray.java:1613:in `each'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-jdbc-adapter-1.2.9/lib/arjdbc/sqlite3/adapter.rb:310:in `remove_column'
org/jruby/RubyBasicObject.java:1671:in `__send__'
org/jruby/RubyKernel.java:2094:in `send'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-3.2.13/lib/active_record/migration.rb:466:in `method_missing'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-3.2.13/lib/active_record/migration.rb:438:in `say_with_time'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/1.9/benchmark.rb:280:in `measure'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-3.2.13/lib/active_record/migration.rb:438:in `say_with_time'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-3.2.13/lib/active_record/migration.rb:458:in `method_missing'
org/jruby/RubyBasicObject.java:1671:in `__send__'
org/jruby/RubyKernel.java:2094:in `send'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-3.2.13/lib/active_record/migration.rb:334:in `method_missing'
./spec/migrations/ponies/2_add_race.rb:7:in `down'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-3.2.13/lib/active_record/migration.rb:376:in `down'
org/jruby/RubyBasicObject.java:1659:in `__send__'
org/jruby/RubyKernel.java:2086:in `send'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-3.2.13/lib/active_record/migration.rb:410:in `migrate'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/1.9/benchmark.rb:280:in `measure'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-3.2.13/lib/active_record/migration.rb:410:in `migrate'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-3.2.13/lib/active_record/connection_adapters/abstract/connection_pool.rb:129:in `with_connection'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-3.2.13/lib/active_record/migration.rb:389:in `migrate'
org/jruby/RubyBasicObject.java:1665:in `__send__'
c:0:in `migrate'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-3.2.13/lib/active_record/migration.rb:720:in `migrate'
org/jruby/RubyProc.java:249:in `call'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-3.2.13/lib/active_record/migration.rb:775:in `ddl_transaction'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-3.2.13/lib/active_record/connection_adapters/abstract/database_statements.rb:192:in `transaction'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-3.2.13/lib/active_record/transactions.rb:208:in `transaction'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-3.2.13/lib/active_record/migration.rb:775:in `ddl_transaction'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-3.2.13/lib/active_record/migration.rb:719:in `migrate'
org/jruby/RubyArray.java:1613:in `each'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-3.2.13/lib/active_record/migration.rb:700:in `migrate'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-3.2.13/lib/active_record/migration.rb:574:in `down'
org/jruby/RubyBasicObject.java:1671:in `__send__'
org/jruby/RubyKernel.java:2094:in `send'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-3.2.13/lib/active_record/migration.rb:656:in `move'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-3.2.13/lib/active_record/migration.rb:562:in `rollback'
./lib/shiva/migrator.rb:16:in `rollback'
./lib/shiva/migrator.rb:34:in `using_connection'
./lib/shiva/migrator.rb:15:in `rollback'
./spec/shiva/migrator_spec.rb:61:in `(root)'
org/jruby/RubyBasicObject.java:1691:in `instance_eval'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/rspec-core-2.13.1/lib/rspec/core/example.rb:237:in `instance_eval'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/rspec-core-2.13.1/lib/rspec/core/hooks.rb:21:in `run'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/rspec-core-2.13.1/lib/rspec/core/hooks.rb:66:in `run'
org/jruby/RubyArray.java:1613:in `each'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/rspec-core-2.13.1/lib/rspec/core/hooks.rb:66:in `run'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/rspec-core-2.13.1/lib/rspec/core/hooks.rb:418:in `run_hook'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/rspec-core-2.13.1/lib/rspec/core/example_group.rb:334:in `run_before_each_hooks'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/rspec-core-2.13.1/lib/rspec/core/example.rb:300:in `run_before_each'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/rspec-core-2.13.1/lib/rspec/core/example.rb:113:in `run'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/rspec-core-2.13.1/lib/rspec/core/example.rb:254:in `with_around_each_hooks'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/rspec-core-2.13.1/lib/rspec/core/example.rb:111:in `run'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/rspec-core-2.13.1/lib/rspec/core/example_group.rb:390:in `run_examples'
org/jruby/RubyArray.java:2361:in `map'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/rspec-core-2.13.1/lib/rspec/core/example_group.rb:386:in `run_examples'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/rspec-core-2.13.1/lib/rspec/core/example_group.rb:371:in `run'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/rspec-core-2.13.1/lib/rspec/core/example_group.rb:372:in `run'
org/jruby/RubyArray.java:2361:in `map'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/rspec-core-2.13.1/lib/rspec/core/example_group.rb:372:in `run'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/rspec-core-2.13.1/lib/rspec/core/example_group.rb:372:in `run'
org/jruby/RubyArray.java:2361:in `map'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/rspec-core-2.13.1/lib/rspec/core/example_group.rb:372:in `run'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/rspec-core-2.13.1/lib/rspec/core/example_group.rb:372:in `run'
org/jruby/RubyArray.java:2361:in `map'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/rspec-core-2.13.1/lib/rspec/core/example_group.rb:372:in `run'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/rspec-core-2.13.1/lib/rspec/core/command_line.rb:28:in `run'
org/jruby/RubyArray.java:2361:in `map'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/rspec-core-2.13.1/lib/rspec/core/command_line.rb:28:in `run'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/rspec-core-2.13.1/lib/rspec/core/reporter.rb:34:in `report'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/rspec-core-2.13.1/lib/rspec/core/command_line.rb:25:in `run'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/rspec-core-2.13.1/lib/rspec/core/runner.rb:80:in `run'
c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/rspec-core-2.13.1/lib/rspec/core/runner.rb:17:in `autorun'
@kares
Copy link
Member

kares commented Apr 18, 2013

right, it's not aligned with AR master (which already fixed this), try patching it as :

ArJdbc::SQLite3.module_eval do
    def indexes(table_name, name = nil) # :nodoc:
      exec_query("PRAGMA index_list(#{quote_table_name(table_name)})", 'SCHEMA').map do |row|
        name = row['name']; unique = row['unique'] != 0
        columns = exec_query("PRAGMA index_info('#{name}')", 'SCHEMA').map { |col| col['name'] }
        IndexDefinition.new(table_name, name, unique, columns)
      end
   end
end

@Holek
Copy link
Contributor Author

Holek commented Apr 19, 2013

There also might be a related issue with this:

Right now you're creating a temporary table in SQLite to perform all table alterations.

alter_table goes to copy_table, that to copy_table_indexes, in the end ending up in index_name_exists? which tries to check whether an index exists on a table with no indexes yet.

Therefore it returns just 0:Integer, and not ActiveRecord::Result

       ActiveRecord::JDBCError: query does not return ResultSet: PRAGMA index_list("aponies")
     # arjdbc/jdbc/RubyJdbcConnection.java:504:in `execute_query'
     # c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-jdbc-adapter-1.3.0.DEV/lib/arjdbc/jdbc/adapter.rb:235:in `exec_query'
     # c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-3.2.13/lib/active_record/connection_adapters/abstract_adapter.rb:280:in `log'
     # c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activesupport-3.2.13/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
     # c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activesupport-3.2.13/lib/active_support/notifications/instrumenter.rb:19:in `instrument'
     # c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-3.2.13/lib/active_record/connection_adapters/abstract_adapter.rb:275:in `log'
     # c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-jdbc-adapter-1.3.0.DEV/lib/arjdbc/jdbc/adapter.rb:235:in `exec_query'
     # ./spec/support/ar_jdbc_sqlite3_monkey_patch.rb:9:in `indexes'
     # c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-3.2.13/lib/active_record/connection_adapters/abstract/schema_statements.rb:406:in `index_name_exists?'
     # c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-3.2.13/lib/active_record/connection_adapters/abstract/schema_statements.rb:575:in `add_index_options'
     # c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-3.2.13/lib/active_record/connection_adapters/abstract/schema_statements.rb:351:in `add_index'
     # c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-jdbc-adapter-1.3.0.DEV/lib/arjdbc/jdbc/missing_functionality_helper.rb:68:in `copy_table_indexes'
     # org/jruby/RubyArray.java:1613:in `each'
     # c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-jdbc-adapter-1.3.0.DEV/lib/arjdbc/jdbc/missing_functionality_helper.rb:52:in `copy_table_indexes'
     # c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-jdbc-adapter-1.3.0.DEV/lib/arjdbc/jdbc/missing_functionality_helper.rb:45:in `copy_table'
     # c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-jdbc-adapter-1.3.0.DEV/lib/arjdbc/jdbc/missing_functionality_helper.rb:20:in `move_table'
     # c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-jdbc-adapter-1.3.0.DEV/lib/arjdbc/jdbc/missing_functionality_helper.rb:14:in `alter_table'
     # c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-3.2.13/lib/active_record/connection_adapters/abstract/database_statements.rb:192:in `transaction'
     # c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-jdbc-adapter-1.3.0.DEV/lib/arjdbc/jdbc/missing_functionality_helper.rb:11:in `alter_table'
     # c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-jdbc-adapter-1.3.0.DEV/lib/arjdbc/sqlite3/adapter.rb:356:in `remove_column'
     # org/jruby/RubyArray.java:1613:in `each'
     # c:/dev/torquebox/torquebox-2.3.0/jruby/lib/ruby/gems/shared/gems/activerecord-jdbc-adapter-1.3.0.DEV/lib/arjdbc/sqlite3/adapter.rb:355:in `remove_column'
     # org/jruby/RubyBasicObject.java:1671:in `__send__'
     # org/jruby/RubyKernel.java:2094:in `send'

You can check it out yourself by cloning fusioneer-ltd/shiva@specs, installing your gems from source running rspec spec

@kares kares closed this as completed in 0fcafa1 Apr 19, 2013
@kares
Copy link
Member

kares commented Apr 19, 2013

right, my bad I did not test it actually just basically copied it from Rails master ... should work fine if we let the JDBC API (and thus SQLite3's driver) to handle the index resolution ... please try out master if you can or undef :indexes

@Holek
Copy link
Contributor Author

Holek commented Apr 19, 2013

Seems to be working, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants