Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is rake db:create accidentally treating the default config as an environment on par with development/test/production? #46577

Closed
monarchwadia opened this issue Nov 25, 2022 · 10 comments · Fixed by #47028

Comments

@monarchwadia
Copy link

monarchwadia commented Nov 25, 2022

Steps to reproduce

Not 100% sure about this, and I apologize if this turns out to be a red herring or non-reproducible; but at least one other person seems to be having the same issue as I am, so I'm creating this ticket.

Issue seems to be happening specifically in apps created with the --api --database=postgresql flags.

Previous discussion in this stackoverflow is related.

Test case

rails new testapp --api --database=postgresql
cd testapp
rake db:create

This comes back with

ActiveRecord::NoDatabaseError: We could not find your database: postgres. Which can be found in the database configuration file located at config/database.yml.

Expected behavior

It should not try to connect to postgres database.

Actual behavior

It tries to connect to postgres database. It seems to be picking up the default config as an actual environment... the pg gem's default database name is postgres... put 2 + 2 together and you might have yourself a bug.

System configuration

Rails version: 7.0.4

Ruby version: ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [x86_64-linux]

@monarchwadia monarchwadia changed the title Is default config being picked as an environment by rake db:create? Is default config being picked as an environment by rake db:create? Nov 25, 2022
@monarchwadia monarchwadia changed the title Is default config being picked as an environment by rake db:create? Is rake db:create accidentally treating the default config as an environment on par with development/test/production? Nov 25, 2022
@maxcal
Copy link

maxcal commented Nov 25, 2022

Cannot be reproduced. You're most likely mistakenly attributing an error with connecting your Postgres server to Rails - say for example that the server isn't actually running or can't be connected to with the default connection settings.

maxcal@MSI ~/projects> rails _7.0.4_ new testapp --api --database=postgresql
      create
      create  README.md
      create  Rakefile
      create  .ruby-version
      create  config.ru
      create  .gitignore
      create  .gitattributes
      create  Gemfile
         run  git init from "."
Initialized empty Git repository in /home/maxcal/projects/testapp/.git/
      create  app
      create  app/assets/config/manifest.js
      create  app/assets/stylesheets/application.css
      create  app/channels/application_cable/channel.rb
      create  app/channels/application_cable/connection.rb
      create  app/controllers/application_controller.rb
      create  app/helpers/application_helper.rb
      create  app/jobs/application_job.rb
      create  app/mailers/application_mailer.rb
      create  app/models/application_record.rb
      create  app/views/layouts/application.html.erb
      create  app/views/layouts/mailer.html.erb
      create  app/views/layouts/mailer.text.erb
      create  app/assets/images
      create  app/assets/images/.keep
      create  app/controllers/concerns/.keep
      create  app/models/concerns/.keep
      create  bin
      create  bin/rails
      create  bin/rake
      create  bin/setup
      create  config
      create  config/routes.rb
      create  config/application.rb
      create  config/environment.rb
      create  config/cable.yml
      create  config/puma.rb
      create  config/storage.yml
      create  config/environments
      create  config/environments/development.rb
      create  config/environments/production.rb
      create  config/environments/test.rb
      create  config/initializers
      create  config/initializers/assets.rb
      create  config/initializers/content_security_policy.rb
      create  config/initializers/cors.rb
      create  config/initializers/filter_parameter_logging.rb
      create  config/initializers/inflections.rb
      create  config/initializers/new_framework_defaults_7_0.rb
      create  config/initializers/permissions_policy.rb
      create  config/locales
      create  config/locales/en.yml
      create  config/master.key
      append  .gitignore
      create  config/boot.rb
      create  config/database.yml
      create  db
      create  db/seeds.rb
      create  lib
      create  lib/tasks
      create  lib/tasks/.keep
      create  lib/assets
      create  lib/assets/.keep
      create  log
      create  log/.keep
      create  public
      create  public/404.html
      create  public/422.html
      create  public/500.html
      create  public/apple-touch-icon-precomposed.png
      create  public/apple-touch-icon.png
      create  public/favicon.ico
      create  public/robots.txt
      create  tmp
      create  tmp/.keep
      create  tmp/pids
      create  tmp/pids/.keep
      create  tmp/cache
      create  tmp/cache/assets
      create  vendor
      create  vendor/.keep
      create  test/fixtures/files
      create  test/fixtures/files/.keep
      create  test/controllers
      create  test/controllers/.keep
      create  test/mailers
      create  test/mailers/.keep
      create  test/models
      create  test/models/.keep
      create  test/helpers
      create  test/helpers/.keep
      create  test/integration
      create  test/integration/.keep
      create  test/channels/application_cable/connection_test.rb
      create  test/test_helper.rb
      create  storage
      create  storage/.keep
      create  tmp/storage
      create  tmp/storage/.keep
      remove  app/assets
      remove  lib/assets
      remove  tmp/cache/assets
      remove  app/helpers
      remove  test/helpers
      remove  app/views/layouts/application.html.erb
      remove  public/404.html
      remove  public/422.html
      remove  public/500.html
      remove  public/apple-touch-icon-precomposed.png
      remove  public/apple-touch-icon.png
      remove  public/favicon.ico
      remove  config/initializers/assets.rb
      remove  app/assets/config/manifest.js
      remove  app/assets/config
      remove  app/assets/stylesheets/application.css
      remove  config/initializers/content_security_policy.rb
      remove  config/initializers/permissions_policy.rb
      remove  config/initializers/new_framework_defaults_7_0.rb
         run  bundle install
Fetching gem metadata from https://rubygems.org/...........
Resolving dependencies...
Using rake 13.0.6
Using concurrent-ruby 1.1.10
Using websocket-extensions 0.1.5
Using marcel 1.0.2
Using mini_mime 1.1.2
Using timeout 0.3.0
Using msgpack 1.6.0
Using bundler 2.2.25
Using rack 2.2.4
Using method_source 1.0.0
Using pg 1.4.5
Using thor 1.2.1
Using zeitwerk 2.6.6
Using i18n 1.12.0
Using tzinfo 2.0.5
Using websocket-driver 0.7.5
Using mail 2.7.1
Using net-protocol 0.1.3
Using bootsnap 1.14.0
Using rack-test 2.0.2
Using net-imap 0.3.1
Using net-pop 0.1.2
Using net-smtp 0.3.3
Using minitest 5.16.3
Using builder 3.2.4
Using activesupport 7.0.4
Using racc 1.6.0
Using globalid 1.0.0
Using io-console 0.5.11
Using nio4r 2.5.8
Using reline 0.3.1
Using puma 5.6.5
Using irb 1.5.0
Using nokogiri 1.13.9 (x86_64-linux)
Using debug 1.6.3
Using rails-dom-testing 2.0.3
Using activemodel 7.0.4
Using crass 1.0.6
Using activerecord 7.0.4
Using activejob 7.0.4
Using loofah 2.19.0
Using erubi 1.11.0
Using rails-html-sanitizer 1.4.3
Using actionview 7.0.4
Using actionpack 7.0.4
Using actioncable 7.0.4
Using activestorage 7.0.4
Using actionmailer 7.0.4
Using actionmailbox 7.0.4
Using actiontext 7.0.4
Using railties 7.0.4
Using rails 7.0.4
Bundle complete! 6 Gemfile dependencies, 52 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
         run  bundle binstubs bundler
maxcal@MSI ~/projects> cd testapp/
maxcal@MSI ~/p/testapp (main)> rails db:create; rails db:migrate
Created database 'testapp_development'
Created database 'testapp_test'
maxcal@MSI ~/p/testapp (main)> rails db
psql (12.8 (Ubuntu 12.8-0ubuntu0.20.04.1))
Type "help" for help.

testapp_development=>

@monarchwadia
Copy link
Author

Thanks for checking & going through this with me. This can probably be closed.

@bensheldon
Copy link
Contributor

I wanted to offer that this could be possible due to a misconfiguration and/or state leaking in from the environment. It's unlikely, but possible!

If you open up rails c you can inspect all of your database configurations with ActiveRecord::Base.configurations.configs_for. Here's an example from an application I work on:

[5] pry(main)> ActiveRecord::Base.configurations.configs_for
=> [#<ActiveRecord::DatabaseConfigurations::HashConfig:0x0000000147b99298
  @configuration_hash=
   {:adapter=>"postgresql", :encoding=>"unicode", :host=>"localhost", :pool=>10},
  @env_name="default",
  @name="primary">,
 #<ActiveRecord::DatabaseConfigurations::HashConfig:0x0000000147b98f78
  @configuration_hash=
   {:adapter=>"postgresql",
    :encoding=>"unicode",
    :host=>"localhost",
    :pool=>10,
    :database=>"good_job_development"},
  @env_name="development",
  @name="primary">,
 #<ActiveRecord::DatabaseConfigurations::HashConfig:0x0000000147b98d70
  @configuration_hash=
   {:adapter=>"postgresql",
    :encoding=>"unicode",
    :host=>"localhost",
    :pool=>10,
    :database=>"good_job_test",
    :reaping_frequency=>0},
  @env_name="test",
  @name="primary">,
 #<ActiveRecord::DatabaseConfigurations::HashConfig:0x0000000147b98b18
  @configuration_hash=
   {:adapter=>"postgresql",
    :encoding=>"unicode",
    :host=>"localhost",
    :pool=>10,
    :database=>"good_job_development"},
  @env_name="production",
  @name="primary">,
 #<ActiveRecord::DatabaseConfigurations::UrlConfig:0x0000000147b988c0
  @configuration_hash=
   {:adapter=>"postgresql",
    :encoding=>"unicode",
    :host=>"localhost",
    :pool=>10,
    :database=>"good_job_demo",
    :url=>nil},
  @env_name="demo",
  @name="primary",
  @url=nil>]

You'll notice that the default key is present, which is how YAML anchors work (they aren't excluded from the deserealized output). Digging into the Rails source, Rails will filter these items to exclude configurations environments that do not define a database value:

def each_local_configuration
configs_for.each do |db_config|
next unless db_config.database

While my default environment configuration block doesn't define a database value, it's possible that somehow something (or several things) are leaking in from the environment. Could you share the output when you run:

bin/rails runner "pp ActiveRecord::Base.configurations.configs_for"

...if you don't see a database value defined in the configuration block for default then I agree with @maxcal that this is likely a misattribution. But if you do, then I think it warrants further debugging.

@monarchwadia
Copy link
Author

Thanks @bensheldon , I just followed my repro steps from the first post and created a new Rails app using the commands...

rails new testapp --api --database=postgresql
cd testapp
rake db:create

Which resulted in the same issue:

# after rake db:create
We could not find your database: postgres. Which can be found in the database configuration file located at config/database.yml.

My output for bin/rails runner "pp ActiveRecord::Base.configurations.configs_for" is as follows

# output for bin/rails runner "pp ActiveRecord::Base.configurations.configs_for"
[#<ActiveRecord::DatabaseConfigurations::HashConfig:0x00007efe5e11cd68
  @configuration_hash={:adapter=>"postgresql", :encoding=>"unicode", :pool=>5},
  @env_name="default",          
  @name="primary">,             
 #<ActiveRecord::DatabaseConfigurations::HashConfig:0x00007efe5e11cc50
  @configuration_hash={:adapter=>"postgresql", :encoding=>"unicode", :pool=>5, :database=>"testapp_development"},
  @env_name="development",      
  @name="primary">,             
 #<ActiveRecord::DatabaseConfigurations::HashConfig:0x00007efe5e11cb88
  @configuration_hash={:adapter=>"postgresql", :encoding=>"unicode", :pool=>5, :database=>"testapp_test"},
  @env_name="test",             
  @name="primary">,             
 #<ActiveRecord::DatabaseConfigurations::HashConfig:0x00007efe5e11cac0
  @configuration_hash=
   {:adapter=>"postgresql",
    :encoding=>"unicode",
    :pool=>5,
    :database=>"testapp_production",
    :username=>"testapp",
    :password=>nil},
  @env_name="production",
  @name="primary">]

@bensheldon
Copy link
Contributor

I can reproduce this issue locally if I drop the default postgres database: DROP DATABASE postgres;.

Rails hardcodes/expects the presence of the postgres database, which is created by default by Postgres. If you're in a situation (Docker?) in which the database is in an unexpected configuration then the command will fail.

def public_schema_config
configuration_hash.merge(database: "postgres", schema_search_path: "public")
end

Background: In order to create a database on Postgres, the connection must already be connected to an existing database. Here's a good explanation: https://dba.stackexchange.com/a/144289

If you're using the Docker image, it's possible that the postgres database isn't created: the image creates a database named POSTGRES_USER if POSTGRES_DB is empty.

I ultimately think the problem is on the application for misconfiguring Postgres and/or Rail's database configuration (all of my Rails-on-Docker experience is to have the Postgres docker image create the database rather than rails db:create).

And I think the error message from Rails could be more accurate/helpful because the database is not defined in config/database.yml

@spryffee
Copy link

I'm also having this issue with Rails 7 on Ubuntu 22.04 with WSL2. Spent a lot of time to debug with no luck. The postgres database is in place by default. I was assuming that WSL connect to postgresql on Windows via Unix socket. However, I don't have any postgres installed on the Windows. I have more environments with same or similar setup but this is only magicly reproducible with Rails 7, nothing helps

@Shedrack-Sunday
Copy link

Same issues with @spryffee. Also running on WSL, and can't seem to figure it out. Did you find a solution @spryffee ?

@bensheldon
Copy link
Contributor

Here is my complete error output. Do you see the part at the very bottom that says "Caused by:" where it outputs the initial exception (PG::ConnectionBad: connection to server at "::1", port 5432 failed: FATAL: database "postgres" does not exist). Can you share what the exception message is that you're seeing?

 bin/rails db:create

We could not find your database: postgres. Which can be found in the database configuration file located at config/database.yml.

To resolve this issue:

- Did you create the database for this app, or delete it? You may need to create your database.
- Has the database name changed? Check your database.yml config has the correct database name.

To create your database, run:

        bin/rails db:create
Couldn't create 'good_job_development' database. Please check your configuration.
rails aborted!
ActiveRecord::NoDatabaseError: We could not find your database: postgres. Which can be found in the database configuration file located at config/database.yml.

To resolve this issue:

- Did you create the database for this app, or delete it? You may need to create your database.
- Has the database name changed? Check your database.yml config has the correct database name.

To create your database, run:

        bin/rails db:create


Caused by:
PG::ConnectionBad: connection to server at "::1", port 5432 failed: FATAL:  database "postgres" does not exist

Tasks: TOP => db:create
(See full trace by running task with --trace)

@Shedrack-Sunday
Copy link

Here is my complete error output. Do you see the part at the very bottom that says "Caused by:" where it outputs the initial exception (PG::ConnectionBad: connection to server at "::1", port 5432 failed: FATAL: database "postgres" does not exist). Can you share what the exception message is that you're seeing?

 bin/rails db:create

We could not find your database: postgres. Which can be found in the database configuration file located at config/database.yml.

To resolve this issue:

- Did you create the database for this app, or delete it? You may need to create your database.
- Has the database name changed? Check your database.yml config has the correct database name.

To create your database, run:

        bin/rails db:create
Couldn't create 'good_job_development' database. Please check your configuration.
rails aborted!
ActiveRecord::NoDatabaseError: We could not find your database: postgres. Which can be found in the database configuration file located at config/database.yml.

To resolve this issue:

- Did you create the database for this app, or delete it? You may need to create your database.
- Has the database name changed? Check your database.yml config has the correct database name.

To create your database, run:

        bin/rails db:create


Caused by:
PG::ConnectionBad: connection to server at "::1", port 5432 failed: FATAL:  database "postgres" does not exist

Tasks: TOP => db:create
(See full trace by running task with --trace)

here's mine:
image

@bensheldon
Copy link
Contributor

@Shedrack-Sunday Thank you! That's really helpful, and also a different error than we identified earlier.

Just to summarize, I think there is a Rails bug here, which is specifically that Rails is providing an inaccurate error message and masking (or distracting from) the underlying error message.

...and then there are the two (so far!) system configuration problems, that are outside the scope of the Rails issue and the workaround is to ignore everything above the "Caused By" part of the output and instead try to address:

  • Not having a postgres database
  • Not being able to connect to Postgresql from WSL

Just to focus on the inaccurate error message, I think this is because the logic is keying off of the presence of the conn_params[:dbname] which is postgres when trying to db:create and thus always outputting the first branch of this logic because the message will very frequently contain the fragment postgres regardless of the specifics of the message:

rescue ::PG::Error => error
if conn_params && conn_params[:dbname] && error.message.include?(conn_params[:dbname])
raise ActiveRecord::NoDatabaseError.db_error(conn_params[:dbname])
elsif conn_params && conn_params[:user] && error.message.include?(conn_params[:user])
raise ActiveRecord::DatabaseConnectionError.username_error(conn_params[:user])
elsif conn_params && conn_params[:hostname] && error.message.include?(conn_params[:hostname])
raise ActiveRecord::DatabaseConnectionError.hostname_error(conn_params[:hostname])
else
raise ActiveRecord::ConnectionNotEstablished, error.message
end
end

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants