Error on schema:load in rails 6

[DylanAndrews]

During the upgrade to rails 6.0.1 from 5.2 we started getting this error when doing db:schema:load --trace

** Invoke db:schema:load (first_time)
** Invoke db:load_config (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute db:load_config
** Invoke db:check_protected_environments (first_time)
** Invoke db:load_config
** Execute db:check_protected_environments
** Invoke environment
** Execute db:schema:load
rake aborted!
NoMethodError: undefined method `spec' for #<ActiveRecord::ConnectionAdapters::NullPool:0x00007fa648b78a40>
Did you mean?  inspect
/Users/Documents/vendor/ruby/2.6.0/gems/activerecord-6.0.1/lib/active_record/connection_adapters/abstract_adapter.rb:168:in `schema_migration'
/Users/Documents/vendor/ruby/2.6.0/gems/activerecord-6.0.1/lib/active_record/schema.rb:54:in `define'
/Users/Documents/vendor/ruby/2.6.0/gems/activerecord-6.0.1/lib/active_record/schema.rb:47:in `define'
/Users/Documents/db/schema.rb:13:in `<main>'
/Users/Documents/vendor/ruby/2.6.0/gems/bootsnap-1.4.5/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:54:in `load'
/Users/Documents/vendor/ruby/2.6.0/gems/bootsnap-1.4.5/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:54:in `load'
/Users/Documents/vendor/ruby/2.6.0/gems/activerecord-6.0.1/lib/active_record/tasks/database_tasks.rb:328:in `load_schema'
/Users/Documents/vendor/ruby/2.6.0/gems/activerecord-6.0.1/lib/active_record/tasks/database_tasks.rb:425:in `block in load_schema_current'
/Users/Documents/vendor/ruby/2.6.0/gems/activerecord-6.0.1/lib/active_record/tasks/database_tasks.rb:479:in `block (2 levels) in each_current_configuration'
/Users/Documents/vendor/ruby/2.6.0/gems/activerecord-6.0.1/lib/active_record/tasks/database_tasks.rb:476:in `each'
/Users/Documents/vendor/ruby/2.6.0/gems/activerecord-6.0.1/lib/active_record/tasks/database_tasks.rb:476:in `block in each_current_configuration'
/Users/Documents/vendor/ruby/2.6.0/gems/activerecord-6.0.1/lib/active_record/tasks/database_tasks.rb:475:in `each'
/Users/Documents/vendor/ruby/2.6.0/gems/activerecord-6.0.1/lib/active_record/tasks/database_tasks.rb:475:in `each_current_configuration'
/Users/Documents/vendor/ruby/2.6.0/gems/activerecord-6.0.1/lib/active_record/tasks/database_tasks.rb:424:in `load_schema_current'
/Users/Documents/vendor/ruby/2.6.0/gems/activerecord-6.0.1/lib/active_record/railties/databases.rake:397:in `block (3 levels) in <main>'
/Users/Documents/vendor/ruby/2.6.0/gems/rake-13.0.1/lib/rake/task.rb:281:in `block in execute'
/Users/Documents/vendor/ruby/2.6.0/gems/rake-13.0.1/lib/rake/task.rb:281:in `each'
/Users/Documents/vendor/ruby/2.6.0/gems/rake-13.0.1/lib/rake/task.rb:281:in `execute'
/Users/Documents/vendor/ruby/2.6.0/gems/bugsnag-6.12.1/lib/bugsnag/integrations/rake.rb:19:in `execute'
/Users/Documents/vendor/ruby/2.6.0/gems/rake-13.0.1/lib/rake/task.rb:219:in `block in invoke_with_call_chain'
/Users/.asdf/installs/ruby/2.6.3/lib/ruby/2.6.0/monitor.rb:230:in `mon_synchronize'
/Users/Documents/vendor/ruby/2.6.0/gems/rake-13.0.1/lib/rake/task.rb:199:in `invoke_with_call_chain'
/Users/Documents/vendor/ruby/2.6.0/gems/rake-13.0.1/lib/rake/task.rb:188:in `invoke'
/Users/Documents/vendor/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:160:in `invoke_task'
/Users/Documents/vendor/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:116:in `block (2 levels) in top_level'
/Users/Documents/vendor/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:116:in `each'
/Users/Documents/vendor/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:116:in `block in top_level'
/Users/Documents/vendor/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:125:in `run_with_threads'
/Users/Documents/vendor/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:110:in `top_level'
/Users/Documents/vendor/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:83:in `block in run'
/Users/Documents/vendor/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:186:in `standard_exception_handling'
/Users/Documents/vendor/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:80:in `run'
/Users/Documents/vendor/ruby/2.6.0/gems/rake-13.0.1/exe/rake:27:in `<top (required)>'
/Users/Documents/vendor/ruby/2.6.0/bin/rake:23:in `load'
/Users/Documents/vendor/ruby/2.6.0/bin/rake:23:in `<top (required)>'
/Users/.asdf/installs/ruby/2.6.3/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/cli/exec.rb:74:in `load'
/Users/.asdf/installs/ruby/2.6.3/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/cli/exec.rb:74:in `kernel_load'
/Users/.asdf/installs/ruby/2.6.3/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/cli/exec.rb:28:in `run'
/Users/.asdf/installs/ruby/2.6.3/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/cli.rb:463:in `exec'
/Users/.asdf/installs/ruby/2.6.3/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/Users/.asdf/installs/ruby/2.6.3/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
/Users/.asdf/installs/ruby/2.6.3/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/vendor/thor/lib/thor.rb:387:in `dispatch'
/Users/.asdf/installs/ruby/2.6.3/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/cli.rb:27:in `dispatch'
/Users/.asdf/installs/ruby/2.6.3/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/vendor/thor/lib/thor/base.rb:466:in `start'
/Users/.asdf/installs/ruby/2.6.3/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/cli.rb:18:in `start'
/Users/.asdf/installs/ruby/2.6.3/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/exe/bundle:30:in `block in <top (required)>'
/Users/.asdf/installs/ruby/2.6.3/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/friendly_errors.rb:124:in `with_friendly_errors'
/Users/.asdf/installs/ruby/2.6.3/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/exe/bundle:22:in `<top (required)>'
/Users/.asdf/installs/ruby/2.6.3/bin/bundle:23:in `load'
/Users/.asdf/installs/ruby/2.6.3/bin/bundle:23:in `<main>'

Gemfile

source 'https://rubygems.org'

ruby '~> 2.6.3'

gem 'rails', '6.0.1'

gem 'active_model_serializers', '~> 0.10.10'
gem 'activeadmin', '~> 2.5'
gem 'active_admin_datetimepicker', '~> 0.7'
gem 'active_elastic_job', '~> 2.0'
gem 'analytics-ruby', '~> 2.2', require: 'segment'
gem 'aws-sdk', '~> 2.1'
gem 'bootsnap', '>= 1.4.5', require: false
gem 'bugsnag', '~> 6.12'
gem 'cancancan', '~> 3.0'
gem 'combine_pdf', '~> 1.0'
gem 'crawler_detect', '~> 0.1'
gem 'devise', '~> 4.7'
gem 'ddtrace', '~> 0.22'
gem 'embedly', '~> 1.9'
gem 'emoji_data', '~> 0.2'
gem 'ey_config', '~> 0.0.7'
gem 'fast_jsonapi', '~> 1.4'
gem 'geocoder', '~> 1.5'
gem 'httparty', '~> 0.17'
gem 'jwplayer-api-client', '~> 0.1.0'
gem 'jwt', '~> 2.1'
gem 'lograge', '~> 0.11'
gem 'mini_magick', '~> 4.9'
gem 'mysql2', '~> 0.5'
gem 'mysql_online_migrations', '~> 1.0'
gem 'paperclip', '~> 5.3'
gem 'paranoia', '~> 2.4'
gem 'prawn', '~> 2.2'
gem 'puma', '3.12'
gem 'rack-attack', '~> 6.0'
gem 'rack-cors', '~> 1.0', require: 'rack/cors'
gem 'rack-rewrite', '~> 1.5'
gem 'rails-observers', '~> 0.1.5'
# This version needs to stay the same
gem 'redis', '3.3.5'
# This version needs to stay the same
gem 'redis-objects', '1.3'
gem 'redis-rails', '~> 5.0'
gem 'responders', '~> 3.0'
gem 'roboto', '~> 1.0.0'
gem 'ruby-hmac', '~> 0.4'
gem 'ruby-progressbar', '~> 1.10'
gem 'sendgrid-ruby', '~> 1.1'
gem 'stringex', '~> 2.8'
gem 'stripe', '~> 2.12'
gem 'uservoice-ruby', '~> 0.0.11'
gem 'versionist', '~> 1.7'
gem 'will_paginate', '~> 3.1'

# Assets
gem 'autoprefixer-rails', '~> 6.7'
gem 'bourbon', '~> 3.2.4'
gem 'coffee-rails', '~> 5.0'
gem 'neat', '~> 1.9.0'
gem 'sass-rails', '~> 6.0'
gem 'uglifier', '>= 1.3.0'

group :development, :test do
  gem 'awesome_print'
  gem 'brakeman'
  gem 'bullet'
  gem 'factory_bot_rails'
  gem 'fasterer'
  gem 'figaro'
  gem 'fuubar'
  gem 'guard-rspec', require: false
  gem 'guard-rubocop'
  gem 'pry-rails'
  gem 'rails_best_practices'
  gem 'rb-fsevent'
  gem 'reek'
  gem 'rspec-rails', '4.0.0.beta3'
  gem 'spring'
  gem 'spring-commands-rspec'
end

group :test do
  gem 'rails-controller-testing'
  gem 'semaphore_test_boosters'
  gem 'shoulda-matchers', '~> 3.1'
  gem 'simplecov', require: false
  gem 'timecop'
  gem 'webmock'
end

database.yml

defaults: &defaults
  adapter: mysql2
  encoding: utf8
  host: <%= ENV['RDS_HOSTNAME'] %>
  username: <%= ENV['RDS_USERNAME'] %>
  password: <%= ENV['RDS_PASSWORD'] %>
  port: <%= ENV.fetch('RDS_PORT', 3306) %>
  pool: <%= ENV.fetch('RDS_POOL', 5) %>

development:
  primary:
    <<: *defaults
    database: <%= ENV['RDS_DATABASE'] %>
  archive:
    <<: *defaults
    database: <%= ENV['RDS_ARCHIVE_DATABASE'] %>
    migrations_paths: db/archive_migrate
  archive_read_only:
    <<: *defaults
    host: <%= ENV['RDS_RO_HOSTNAME'] %>
    database: <%= ENV['RDS_ARCHIVE_DATABASE'] %>
    replica: true

production:
  primary:
    <<: *defaults
    database: <%= ENV['RDS_DATABASE'] %>
  archive:
    <<: *defaults
    database: <%= ENV['RDS_ARCHIVE_DATABASE'] %>
    migrations_paths: db/archive_migrate
  archive_read_only:
    <<: *defaults
    host: <%= ENV['RDS_RO_HOSTNAME'] %>
    database: <%= ENV['RDS_ARCHIVE_DATABASE'] %>
    replica: true
  read_only:
    <<: *defaults
    host: <%= ENV['RDS_PROD_RO_HOSTNAME'] %>
    database: <%= ENV['RDS_PROD_DATABASE'] %>
    username: <%= ENV['RDS_PROD_USERNAME'] %>
    password: <%= ENV['RDS_PROD_PASSWORD'] %>

test:
  primary:
    <<: *defaults
    database: <%= ENV['RDS_DATABASE'] %>
  archive:
    <<: *defaults
    database: <%= ENV['RDS_ARCHIVE_DATABASE'] %>
    migrations_paths: db/archive_migrate
  archive_read_only:
    <<: *defaults
    database: <%= ENV['RDS_ARCHIVE_DATABASE'] %>
    replica: true

We have no idea what is causing this and have done the following to try and get the bottom of it, and nothing has helped fix the issue.

• Open the active_record gem and put binding.pry in multiple to places to see what is going on.
• Have a coworker pull down my branch to ensure it is not an issue with the setup on my machine (it's not)
• Remove the usage of the figaro gem in the database.yml file
• Open a question on stack overflow.

We have gotten around it by setting config.active_record.schema_format = :sql and by using db:structure:load to create .sql files instead of schema.rb files, but we would rather not have to do this.

Any help would be deeply appreciated.

Thanks.

System configuration

Rails version: 6.0.1

Ruby version: 2.6.3

[rafaelfranca]

For some reason the connection is using the NullPool as connection pool and I don't know how this state can be achieved. If a connection is checkout, the pull is assigned. And that method checkout a connection https://github.com/rails/rails/blob/6-0-stable/activerecord/lib/active_record/schema.rb#L54. Do you have any monkey patch or gem that could change the connection pool code?

[Edouard-chin]

I can reproduce the problem, your issue is in this gem https://github.com/anthonyalberto/mysql_online_migrations

[DylanAndrews]

@Edouard-chin You nailed it!!! Thanks man!

[DylanAndrews]

@rafaelfranca Thanks for the tip. You were right, a gem was the issue.

[JuanitoFatas]

Sharing in case future people coming from search to here like me. I encountered the same error and found out the change in #37253 breaks database_rewinder, a patch has been sent if you need it ☺️