Add rails db:system:change command #34832

gmcgibbon · 2018-12-31T07:03:05Z

Part two of #34710.

After studying up commands and generators, I decided to make a command that delegates to a generator (similar to rails new / app generator). This seems to give us the most flexibility with how the command interacts with files, and if the user wants to accept the config overwrites or not.

TODO:

Fix three level nested generator lookup
Add tests for generator with file assertions
Add chagelog entry

eileencodes

I like this. I left 2 minor comments.

eileencodes · 2019-01-02T14:40:53Z

railties/lib/rails/generators/database.rb

+        when "jdbcsqlite3"    then ["activerecord-jdbcsqlite3-adapter", nil]
+        when "jdbcpostgresql" then ["activerecord-jdbcpostgresql-adapter", nil]
+        when "jdbc"           then ["activerecord-jdbc-adapter", nil]
+        else [database, nil]


Should this raise an error if we get to the else? Otherwise I think this will silently fail.

I don't think so, sqlite3 and ibm_db have a direct adapter name to gem name mapping with no version locking. The value itself should be validated by the #initialize check in ChangeGenerator.

railties/lib/rails/generators/rails/db/system/change/change_generator.rb

kaspth · 2019-01-02T20:36:21Z

railties/lib/rails/generators.rb

+
+        if context
+          lookups << "#{name}:#{context}"
+        end


What do these new lines do? I've always found the base and context naming here confusing?

I'm essentially only adding lookups << "rails:#{base}:#{name}" if a base exists. This is necessary due to the second part of the lookup code:

unless base || context unless name.to_s.include?(?:) lookups << "#{name}:#{name}" lookups << "rails:#{name}" end lookups << "#{name}" end

Specifically, lookups << "rails:#{name}" which limits us to looking up single segment generator names in the toplevel rails namespace.

When looking up rails g db:system:change, name is the last segment ("change"), base is the namespace path ("db:system"), and context is nil. I think context is used if name isn't the command name, but I'm not sure why. Maybe we should change base to path?

So it's possible for users to call rails g db:system:change? I thought this generator was internal and sufficiently called via it's class start method from the command?

That is currently possible, yes. If we don't want it to be possible, and only called via the command, I can remove this patch to #find_by_namespace. WDYT?

I'll go ahead and revert the changes to the generator lookup and hide this generator so it can only be called through the command. I'm assuming we would prefer only one way of changing databases.

kaspth

You're getting pretty adept at finding your way around the codebase! Lovely to see 💪

kaspth · 2019-01-02T20:36:45Z

railties/lib/rails/commands/db/system/change/change_command.rb

+    module Db
+      module System
+        class ChangeCommand < Base # :nodoc:
+          class_option :to, desc: "The DBMS to switch to."


Do we generally say DBMS elsewhere?

Only once in a comment of activerecord/lib/active_record/connection_adapters/abstract_adapter.rb. I see RDBMS in guides/source/active_record_basics.md and activerecord/lib/active_record/connection_adapters/abstract/schema_statements.rb, but I don't think that's much better. I'll change it to database for now.

Try "database type"? It's not brilliant, but maybe an improvement? Or "database platform"?

Using "database system" would match the command name and reinforce the concept. But it's perhaps not expounding on what a "database system" means, that meaning only comes from seeing MySQL, Postgres and the gang.

Agree that database system makes the most sense given the command name. Will use that

railties/lib/rails/generators/app_name.rb

kaspth · 2019-01-02T20:39:02Z

railties/lib/rails/generators/app_name.rb

+
+        def valid_const?
+          if /^\d/.match?(app_const)
+            raise Error, "Invalid application name #{original_app_name}. Please give a name which does not start with numbers."


Curiosity for another time: since 2.6 allows constants to start with non ascii characters, does that mean we could name a Rails app, say, 666?

@kaspth not 666, because it needs to begin with something the unicode database as included in the particular ruby version thinks is an uppercase letter. Just not limited to ascii.

railties/lib/rails/generators/database.rb

railties/lib/rails/generators/rails/db/system/change/change_generator.rb

kaspth · 2019-01-02T20:45:54Z

railties/test/generators/generators_test_helper.rb

+        gem 'turbolinks', '~> 5'
+        # Build JSON APIs with ease. Read more: https://github.com/rails/jbuilder
+        gem 'jbuilder', '~> 2.5'
+      GEMFILE


How will we have to maintain these Gemfiles going forward?

Good question. I initially tried to generate all test apps without --skip-gemfile, but then rails will run bundle install and crash because rails 6 isn't released yet. I just noticed we have skip_bundle as an app generator option, so I'll try using that, and using the gem template in generator tests.

kaspth · 2019-01-02T20:47:16Z

railties/test/generators/db_system_change_generator_test.rb

+              assert_match "adapter: postgresql", content
+              assert_match "database: test_app", content
+            end
+            assert_file("Gemfile") do |content|


Adding a newline after output and between the assert_file seems like it would increase readability in these tests.

kaspth · 2019-01-02T20:48:20Z

railties/lib/rails/commands/db/system/change/change_command.rb

+  module Command
+    module Db
+      module System
+        class ChangeCommand < Base # :nodoc:


For another time: seeing this deep module nesting makes me think we should consider adding something akin to Action Mailboxes' routing for commands.

kaspth · 2019-01-02T20:50:05Z

Forgot to ask, how are we handling cases where people who may have changed their config/database.yml when calling this command?

gmcgibbon · 2019-01-03T17:20:15Z

Forgot to ask, how are we handling cases where people who may have changed their config/database.yml when calling this command?

The generator will detect a configuration file already exists and gives options to overwrite, show a diff, reject the change, or merge the two files together.

Eg.

rails g db:system:change --to mysql
    conflict  config/database.yml
Overwrite /Users/gannon/dummy/config/database.yml? (enter "h" for help) [Ynaqdhm] h
        Y - yes, overwrite
        n - no, do not overwrite
        a - all, overwrite this and all others
        q - quit, abort
        d - diff, show the differences between the old and the new
        h - help, show this help
        m - merge, run merge tool
Overwrite /Users/gannon/dummy/config/database.yml? (enter "h" for help) [Ynaqdhm]

Add `rails db:system:change` command for changing databases. ``` bin/rails db:system:change --to=postgresql force config/database.yml gsub Gemfile ``` The change command copies a template `config/database.yml` with the target database adapter into your app, and replaces your database gem with the target database gem.

gmcgibbon · 2019-01-11T20:52:57Z

I've hidden the change generator, reverted the patch to call it with rails g, and removed the static gemfile strings in favour of template evaluation. Let me know if there's anything else this PR needs to get shipped!

bogdanvlviv · 2019-01-21T13:16:51Z

Hey, Thanks for adding this!
~~Since this PR is merged we should close #34710, and #34732?~~
Also, I think we should exclude db:change from commands (railties/lib/rails/command.rb file) in order to not show it on rails --help, (we should show db:system:change instead).
Sorry for the too late review.

Edited: Seems like Rails::Command::Db::System::ChangeCommand.printing_commands returns ["db:change"] instead of [db:system:change].

rafaelfranca · 2019-01-23T20:04:07Z

cc @gmcgibbon

gmcgibbon · 2019-01-23T20:19:59Z

Thanks for pointing that out @bogdanvlviv, I'll see what I can do!

We sometimes say "✂️ newline after `private`" in a code review (e.g. rails#18546 (comment), rails#34832 (comment)). Now `Layout/EmptyLinesAroundAccessModifier` cop have new enforced style `EnforcedStyle: only_before` (rubocop/rubocop#7059). That cop and enforced style will reduce the our code review cost.

Add rails db:system:change command rails/rails#34832

simi · 2019-12-28T02:42:19Z

Implementing this as a command makes it inconsistent at few places:

bin/rails db:system:change --help shows wrong usage (there is just change instead of db:system:change)

[retro@retro  test-app (master #%)]❤ bin/rails db:system:change --help
Usage:
  rails change [options]

Options:
  [--to=TO]  # The database system to switch to.

this command is not present for bin/rake -T, but all other db:* commands are
this command is not present for bin/rails --tasks, but all other db:* commands are

But it is present for rails command.

[retro@retro  test-app (master #%)]❤ bin/rails
The most common rails commands are:
 generate     Generate new code (short-cut alias: "g")
 console      Start the Rails console (short-cut alias: "c")
 server       Start the Rails server (short-cut alias: "s")
 test         Run tests except system tests (short-cut alias: "t")
 test:system  Run system tests
 dbconsole    Start a console for the database specified in config/database.yml
              (short-cut alias: "db")

 new          Create a new Rails application. "rails new my_app" creates a
              new application called MyApp in "./my_app"


All commands can be run with -h (or --help) for more information.
In addition to those commands, there are:

  about
  action_mailbox:ingress:exim
  action_mailbox:ingress:postfix
  action_mailbox:ingress:qmail
  action_mailbox:install
  action_text:install
  action_text:install:migrations
  active_storage:install
  app:template
  app:update
  assets:clean[keep]
  assets:clobber
  assets:environment
  assets:precompile
  cache_digests:dependencies
  cache_digests:nested_dependencies
  credentials:diff
  credentials:edit
  credentials:show
  db:create
  db:drop
  db:environment:set
  db:fixtures:load
  db:migrate
  db:migrate:down
  db:migrate:redo
  db:migrate:status
  db:migrate:up
  db:prepare
  db:reset
  db:rollback
  db:schema:cache:clear
  db:schema:cache:dump
  db:schema:dump
  db:schema:load
  db:seed
  db:seed:replant
  db:setup
  db:structure:dump
  db:structure:load
  db:system:change
  db:version
  destroy
  dev:cache
  encrypted:edit
  encrypted:show
  initializers
  log:clear
  middleware
  notes
  restart
  routes
  runner
  secret
  secrets:edit
  secrets:setup
  secrets:show
  stats
  test:db
  time:zones[country_or_offset]
  tmp:clear
  tmp:create
  version
  webpacker
  webpacker:binstubs
  webpacker:check_binstubs
  webpacker:check_node
  webpacker:check_yarn
  webpacker:clean[keep]
  webpacker:clobber
  webpacker:compile
  webpacker:info
  webpacker:install
  webpacker:install:angular
  webpacker:install:coffee
  webpacker:install:elm
  webpacker:install:erb
  webpacker:install:react
  webpacker:install:stimulus
  webpacker:install:svelte
  webpacker:install:typescript
  webpacker:install:vue
  webpacker:verify_install
  webpacker:yarn_install
  yarn:install
  zeitwerk:check

We sometimes say "✂️ newline after `private`" in a code review (e.g. rails/rails#18546 (comment), rails/rails#34832 (comment)). Now `Layout/EmptyLinesAroundAccessModifier` cop have new enforced style `EnforcedStyle: only_before` (rubocop/rubocop#7059). That cop and enforced style will reduce the our code review cost.

rails-bot bot added the railties label Dec 31, 2018

gmcgibbon force-pushed the db_system_change_command branch 8 times, most recently from 9342aa9 to 6e19f90 Compare January 2, 2019 06:50

eileencodes requested changes Jan 2, 2019

View reviewed changes

kaspth reviewed Jan 2, 2019

View reviewed changes

gmcgibbon changed the title ~~[WIP] Add rails db:system:change command~~ Add rails db:system:change command Jan 2, 2019

gmcgibbon force-pushed the db_system_change_command branch from b0a95b7 to 9c2ef0f Compare January 3, 2019 17:15

gmcgibbon force-pushed the db_system_change_command branch from 9c2ef0f to 55a27f4 Compare January 3, 2019 18:38

Move application generator naming and database code to concerns

e3204b9

gmcgibbon force-pushed the db_system_change_command branch 2 times, most recently from 2131e71 to 88e1d38 Compare January 9, 2019 19:19

gmcgibbon added 2 commits January 9, 2019 14:23

Revise wording on invalid database error messages

d49899c

gmcgibbon force-pushed the db_system_change_command branch from 88e1d38 to d49899c Compare January 9, 2019 19:23

Merge branch 'master' into db_system_change_command

cb3f78a

kaspth merged commit 90536eb into rails:master Jan 16, 2019

gmcgibbon deleted the db_system_change_command branch January 23, 2019 20:20

gmcgibbon mentioned this pull request Jan 23, 2019

Fix deeply nested namespace command printing #35033

Merged

kamipo mentioned this pull request Jun 13, 2019

Enable Layout/EmptyLinesAroundAccessModifier cop #36472

Merged

suketa added a commit to suketa/rails_sandbox that referenced this pull request Jul 20, 2019

try063

3019638

Add rails db:system:change command rails/rails#34832

simi mentioned this pull request Dec 29, 2019

Show possible options for db:system:change command. #38109

Closed

ghiculescu mentioned this pull request Apr 2, 2022

Add rails db:change SYSTEM=postgresql|mysql|whatever #34710

Closed

nshun583 mentioned this pull request Nov 4, 2022

change RDB from SQLite to MySQL nshun583/my-first-rails#14

Closed

11 tasks

Add rails db:system:change command #34832

Add rails db:system:change command #34832

gmcgibbon commented Dec 31, 2018 •

edited

eileencodes left a comment

eileencodes Jan 2, 2019

gmcgibbon Jan 2, 2019

kaspth Jan 2, 2019

gmcgibbon Jan 3, 2019

kaspth Jan 3, 2019 •

edited

gmcgibbon Jan 3, 2019 •

edited

gmcgibbon Jan 9, 2019

kaspth left a comment

kaspth Jan 2, 2019

gmcgibbon Jan 3, 2019

matthewd Jan 3, 2019

kaspth Jan 3, 2019

gmcgibbon Jan 3, 2019 •

edited

kaspth Jan 2, 2019

jrochkind Jan 23, 2019

kaspth Jan 2, 2019

gmcgibbon Jan 3, 2019 •

edited

kaspth Jan 2, 2019

kaspth Jan 2, 2019

kaspth commented Jan 2, 2019

gmcgibbon commented Jan 3, 2019 •

edited

gmcgibbon commented Jan 11, 2019

bogdanvlviv commented Jan 21, 2019 •

edited

rafaelfranca commented Jan 23, 2019

gmcgibbon commented Jan 23, 2019

simi commented Dec 28, 2019

Add rails db:system:change command #34832

Add rails db:system:change command #34832

Conversation

gmcgibbon commented Dec 31, 2018 • edited

eileencodes left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kaspth Jan 3, 2019 • edited

Choose a reason for hiding this comment

gmcgibbon Jan 3, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kaspth left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gmcgibbon Jan 3, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gmcgibbon Jan 3, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kaspth commented Jan 2, 2019

gmcgibbon commented Jan 3, 2019 • edited

gmcgibbon commented Jan 11, 2019

bogdanvlviv commented Jan 21, 2019 • edited

rafaelfranca commented Jan 23, 2019

gmcgibbon commented Jan 23, 2019

simi commented Dec 28, 2019

gmcgibbon commented Dec 31, 2018 •

edited

kaspth Jan 3, 2019 •

edited

gmcgibbon Jan 3, 2019 •

edited

gmcgibbon Jan 3, 2019 •

edited

gmcgibbon Jan 3, 2019 •

edited

gmcgibbon commented Jan 3, 2019 •

edited

bogdanvlviv commented Jan 21, 2019 •

edited