Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement rake proxy for rails cli #22288

Closed

Conversation

@dharamgollapudi
Copy link
Contributor

@dharamgollapudi dharamgollapudi commented Nov 14, 2015

  • This implements a thin proxy layer for rails cli to invoke the underlying rake tasks.
  • With this change all the rake tasks can be called/invoked directly with rails. (ex: rails stats, rails db:version, etc)
  • Also included a rake subcommand with shortcut set to k which can be used to query
    version and help info of rake, which supports all three formats as -h, --help and help.
  • This addresses #18878, and #21012
@rails-bot
Copy link

@rails-bot rails-bot commented Nov 14, 2015

Thanks for the pull request, and welcome! The Rails team is excited to review your changes, and you should hear from @pixeltrix (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

@timbreitkreutz
Copy link

@timbreitkreutz timbreitkreutz commented Nov 14, 2015

@dharamgollapudi Did you happen to notice all the discussion in #21254 ? I was looking at doing something for #18878 also but got overwhelmed by the discussion there.

@dharamgollapudi
Copy link
Contributor Author

@dharamgollapudi dharamgollapudi commented Nov 14, 2015

@timbreitkreutz yes I did. My understanding per #18878 as well as follow up comments by @dhh and others in #21012, is that we needed a thin layer that proxies rake tasks, as we are not going to deprecate or move over all the existing tasks as they have dependencies. That is exactly what this pull request addresses, which handles all the rake tasks through rails cli.

Also the implementation in #21254 probably may have some performance penalties as it seems to define methods that call the underlying rake tasks. As rails cli is not a service, this will happen on every run.

@kaspth kaspth assigned kaspth and unassigned pixeltrix Nov 22, 2015
@kaspth
Copy link
Member

@kaspth kaspth commented Nov 22, 2015

This isn't exactly what we're going for. In addition to funneling tasks through Rails, we want to deprecate running tasks with Rake (regardless of whether we use Rake to run tasks internally).

What we've been trying to do in #21254 is create a lightweight command structure to give us easier parameter parsing in the future. It's been taking longer than I've wanted, so I'm open to also move things along here. Thanks 👍

@kaspth
Copy link
Member

@kaspth kaspth commented Dec 9, 2015

@dharamgollapudi hey, since writing the above message I've changed my opinion and think your change is good. Let me know if you still want to work on this and I'll review it more in depth. Thanks! ❤️

@dharamgollapudi
Copy link
Contributor Author

@dharamgollapudi dharamgollapudi commented Dec 10, 2015

@kaspth I am a bit confused now.
Upon reviewing/following #22457, I thought that we are going in that direction.
I am more that happy to work on this, given a proper direction and input.

@kaspth
Copy link
Member

@kaspth kaspth commented Dec 10, 2015

Sure, I can understand. We are generally going in that direction, but I don't think we will be there in time for Rails 5*. As such I think this is a great cover that gives leverage to work on the underpinnings without a deadline. I'll go through the PR now, 👍.

  • Partly because the command API doesn't hold its own yet, and I don't think it's worth it to just force all the tasks and existing commands over.
@kaspth
kaspth reviewed Dec 10, 2015
View changes
railties/lib/rails/commands.rb Outdated
@@ -8,6 +8,7 @@
"db" => "dbconsole",
"r" => "runner",
"t" => "test",
"k" => "rake"

This comment has been minimized.

@kaspth

kaspth Dec 10, 2015
Member

Don't think querying rake version and help info is a common enough command to warrant an alias. Let's take it out.

@kaspth
kaspth reviewed Dec 10, 2015
View changes
railties/lib/rails/commands/commands_tasks.rb Outdated
when 'help' then ARGV.unshift('--help')
when 'version' then ARGV.unshift('--version')
else ARGV.unshift(command)
end

This comment has been minimized.

@kaspth

kaspth Dec 10, 2015
Member

parse_command turns --help into help and this undoes it. We should find a way to not run the command through parse_command.

This comment has been minimized.

@dharamgollapudi

dharamgollapudi Dec 11, 2015
Author Contributor

I totally removed querying for help/version from the rake, as users will be able to find that info directly using rake.

@kaspth
kaspth reviewed Dec 10, 2015
View changes
railties/lib/rails/commands/commands_tasks.rb Outdated

def initialize(argv)
def initialize(argv = ARGV)

This comment has been minimized.

@kaspth

kaspth Dec 10, 2015
Member

No default; we should keep it a required argument.

@kaspth
kaspth reviewed Dec 10, 2015
View changes
railties/lib/rails/commands/commands_tasks.rb Outdated
if COMMAND_WHITELIST.include?(command)
send(command)
else
write_error_message(command)
ARGV.unshift(command)
send(:rake)

This comment has been minimized.

@kaspth

kaspth Dec 10, 2015
Member

If we rename the parse_command result variable something like rails_command, we don't have to undo parse_command in rake.

This comment has been minimized.

@dharamgollapudi

dharamgollapudi Dec 11, 2015
Author Contributor

Renaming result variable of parse_command won't help here, as the ARGV.shift is done at https://github.com/rails/rails/blob/master/railties/lib/rails/commands.rb#L13 and we need to put it back/unshift it for rake to take over, otherwise rake won't be able to find the right task to run and defaults to the rake test.

@kaspth
kaspth reviewed Dec 10, 2015
View changes
railties/lib/rails/commands/commands_tasks.rb Outdated
EOT

COMMAND_WHITELIST = %w(plugin generate destroy console server dbconsole runner new version help test)
COMMAND_WHITELIST = %w(plugin generate destroy console server dbconsole runner new version help test rake)

This comment has been minimized.

@kaspth

kaspth Dec 10, 2015
Member

We still want to phase rake out. You see you have added this command so people can query Rake version and help info.

Can't they get the Rake version from their Gemfile.lock?
What does the Rake help info help people with?

I'm leaning towards removing this extra command.

@kaspth
kaspth reviewed Dec 10, 2015
View changes
railties/lib/rails/commands/commands_tasks.rb Outdated
'version'
when '--help', '-h'
when '--help', '-h', 'help'

This comment has been minimized.

@kaspth

kaspth Dec 10, 2015
Member

No need to add 'help' or 'version', they would just hit the else branch here, which would let them through just the same.

@kaspth
kaspth reviewed Dec 10, 2015
View changes
railties/lib/rails/commands/rake_proxy.rb Outdated
module RakeProxy

RAKE_TASKS_HELP_MESSAGE = <<-EOT
In addition to those, you can now run the rake tasks as rails commands:

This comment has been minimized.

@kaspth

kaspth Dec 10, 2015
Member

now doesn't make sense in the context of released software. When Rails 5 is 3 months old, is it still now? 😄

We also want to go for as smooth a transition as possible, so the rake commands should just show up in the standard help output, I think.

@kaspth
kaspth reviewed Dec 10, 2015
View changes
railties/lib/rails/commands/rake_proxy.rb Outdated
require 'rake'

module Rails
module RakeProxy

This comment has been minimized.

@kaspth

kaspth Dec 10, 2015
Member

Add #:nodoc: — we don't want this showing up in documentation 😁

@kaspth
kaspth reviewed Dec 10, 2015
View changes
railties/lib/rails/commands/rake_proxy.rb Outdated
Rake::TaskManager.record_task_metadata = true
Rake.application.instance_variable_set(:@name, 'rails')
Rails.application.load_tasks
@rake_tasks ||= Rake.application.tasks.select { |t| t.comment }

This comment has been minimized.

@kaspth

kaspth Dec 10, 2015
Member

Seems like the ivar should wrap the whole method, otherwise we're loading tasks twice.

Also do select(&:comment)

@kaspth
kaspth reviewed Dec 10, 2015
View changes
railties/lib/rails/commands/rake_proxy.rb Outdated
Rake.application.load_rakefile
Rake.application.top_level
end
end

This comment has been minimized.

@kaspth

kaspth Dec 10, 2015
Member

All the methods here should be under a private mark. We prefer it like so:

private
  def invoke_rake
  end
@kaspth
kaspth reviewed Dec 10, 2015
View changes
railties/lib/rails/commands/rake_proxy.rb Outdated
end
end
end
end

This comment has been minimized.

@kaspth

kaspth Dec 10, 2015
Member

Add a new line at the end of the file.

@kaspth
kaspth reviewed Dec 10, 2015
View changes
railties/lib/rails/commands/rake_proxy.rb Outdated
width = rake_tasks.map { |t| t.name_with_args.length }.max || 10
rake_tasks.each do |t|
printf("#{Rake.application.name} %-#{width}s # %s\n", t.name_with_args, t.comment)
end

This comment has been minimized.

@kaspth

kaspth Dec 10, 2015
Member

Can you give an example of the output?

This comment has been minimized.

@dharamgollapudi

dharamgollapudi Dec 10, 2015
Author Contributor

Following is the output by running rails (with no arguments) within the newly generated app:

Usage: rails COMMAND [ARGS]

The most common rails commands are:
 generate    Generate new code (short-cut alias: "g")
 console     Start the Rails console (short-cut alias: "c")
 server      Start the Rails server (short-cut alias: "s")
 test        Run tests (short-cut alias: "t")
 dbconsole   Start a console for the database specified in config/database.yml
             (short-cut alias: "db")
 new         Create a new Rails application. "rails new my_app" creates a
             new application called MyApp in "./my_app"

In addition to those, there are:
 destroy      Undo code generated with "generate" (short-cut alias: "d")
 plugin new   Generates skeleton for developing a Rails plugin
 runner       Run a piece of code in the application environment (short-cut alias: "r")

All commands can be run with -h (or --help) for more information.

In addition to those, you can run the rake tasks as rails commands:

rails about                              # List versions of all Rails frameworks and the environment
rails assets:clean[keep]                 # Remove old compiled assets
rails assets:clobber                     # Remove compiled assets
rails assets:environment                 # Load asset compile environment
rails assets:precompile                  # Compile all the assets named in config.assets.precompile
rails cache_digests:dependencies         # Lookup first-level dependencies for TEMPLATE (like messages/show or comments/_comment.html)
rails cache_digests:nested_dependencies  # Lookup nested dependencies for TEMPLATE (like messages/show or comments/_comment.html)
rails db:create                          # Creates the database from DATABASE_URL or config/database.yml for the current RAILS_ENV (use db:create:all to create all databases in the config)
rails db:drop                            # Drops the database from DATABASE_URL or config/database.yml for the current RAILS_ENV (use db:drop:all to drop all databases in the config)
rails db:fixtures:load                   # Loads fixtures into the current environment's database
rails db:migrate                         # Migrate the database (options: VERSION=x, VERBOSE=false, SCOPE=blog)
rails db:migrate:status                  # Display status of migrations
rails db:rollback                        # Rolls the schema back to the previous version (specify steps w/ STEP=n)
rails db:schema:cache:clear              # Clears a db/schema_cache.dump file
rails db:schema:cache:dump               # Creates a db/schema_cache.dump file
rails db:schema:dump                     # Creates a db/schema.rb file that is portable against any DB supported by Active Record
rails db:schema:load                     # Loads a schema.rb file into the database
rails db:seed                            # Loads the seed data from db/seeds.rb
rails db:setup                           # Creates the database, loads the schema, and initializes with the seed data (use db:reset to also drop the database first)
rails db:structure:dump                  # Dumps the database structure to db/structure.sql
rails db:structure:load                  # Recreates the databases from the structure.sql file
rails db:version                         # Retrieves the current schema version number
rails initializers                       # Print out all defined initializers in the order they are invoked by Rails
rails log:clear                          # Truncates all *.log files in log/ to zero bytes (specify which logs with LOGS=test,development)
rails middleware                         # Prints out your Rack middleware stack
rails notes                              # Enumerate all annotations (use notes:optimize, :fixme, :todo for focus)
rails notes:custom                       # Enumerate a custom annotation, specify with ANNOTATION=CUSTOM
rails rails:template                     # Applies the template supplied by LOCATION=(/path/to/template) or URL
rails rails:update                       # Update configs and some other initially generated files (or use just update:configs or update:bin)
rails restart                            # Restart app by touching tmp/restart.txt
rails riak:restart                       # Restart riak locally
rails riak:start                         # Start riak locally
rails riak:stop                          # Stop riak locally
rails routes                             # Print out all defined routes in match order, with names
rails secret                             # Generate a cryptographically secure secret key (this is typically used to generate a secret for cookie sessions)
rails stats                              # Report code statistics (KLOCs, etc) from the application or engine
rails test                               # Runs all tests in test folder
rails test:db                            # Run tests quickly, but also reset db
rails time:zones:all                     # Displays all time zones, also available: time:zones:us, time:zones:local -- filter with OFFSET parameter, e.g., OFFSET=-6
rails tmp:clear                          # Clear cache and socket files from tmp/ (narrow w/ tmp:cache:clear, tmp:sockets:clear)
rails tmp:create                         # Creates tmp directories for cache, sockets, and pids
@kaspth
kaspth reviewed Dec 10, 2015
View changes
railties/lib/rails/commands/rake_proxy.rb Outdated
def rake_tasks
require_application_and_environment!
Rake::TaskManager.record_task_metadata = true
Rake.application.instance_variable_set(:@name, 'rails')

This comment has been minimized.

@kaspth

kaspth Dec 10, 2015
Member

There's no other way to set the name?

This comment has been minimized.

@dharamgollapudi

dharamgollapudi Dec 10, 2015
Author Contributor

Unfortunately there is no other way to set the name, as rake doesn't expose the setter.
I can submit a pull request to rake project maintainer, if you think it makes sense to expose the setter.

@dharamgollapudi dharamgollapudi force-pushed the dharamgollapudi:rake_proxy_for_rails_cli branch to 435e7f2 Dec 11, 2015
@dharamgollapudi
Copy link
Contributor Author

@dharamgollapudi dharamgollapudi commented Dec 11, 2015

@kaspth Implemented the changes per the feedback.
Let me know of any further changes.

@dharamgollapudi
Copy link
Contributor Author

@dharamgollapudi dharamgollapudi commented Dec 11, 2015

By running any of the following commands within the newly generated rails app:
rails
rails -h
rails --help
rails help

following is the output:

Usage: rails COMMAND [ARGS]

The most common rails commands are:
 generate    Generate new code (short-cut alias: "g")
 console     Start the Rails console (short-cut alias: "c")
 server      Start the Rails server (short-cut alias: "s")
 test        Run tests (short-cut alias: "t")
 dbconsole   Start a console for the database specified in config/database.yml
             (short-cut alias: "db")
 new         Create a new Rails application. "rails new my_app" creates a
             new application called MyApp in "./my_app"

In addition to those, there are:
 destroy      Undo code generated with "generate" (short-cut alias: "d")
 plugin new   Generates skeleton for developing a Rails plugin
 runner       Run a piece of code in the application environment (short-cut alias: "r")

All commands can be run with -h (or --help) for more information.

In addition to those, you can run the rake tasks as rails commands:

rails about                              # List versions of all Rails frameworks and the environment
rails assets:clean[keep]                 # Remove old compiled assets
rails assets:clobber                     # Remove compiled assets
rails assets:environment                 # Load asset compile environment
rails assets:precompile                  # Compile all the assets named in config.assets.precompile
rails cache_digests:dependencies         # Lookup first-level dependencies for TEMPLATE (like messages/show or comments/_comment.html)
rails cache_digests:nested_dependencies  # Lookup nested dependencies for TEMPLATE (like messages/show or comments/_comment.html)
rails db:create                          # Creates the database from DATABASE_URL or config/database.yml for the current RAILS_ENV (use db:create:all to create all databases in the config)
rails db:drop                            # Drops the database from DATABASE_URL or config/database.yml for the current RAILS_ENV (use db:drop:all to drop all databases in the config)
rails db:fixtures:load                   # Loads fixtures into the current environment's database
rails db:migrate                         # Migrate the database (options: VERSION=x, VERBOSE=false, SCOPE=blog)
rails db:migrate:status                  # Display status of migrations
rails db:rollback                        # Rolls the schema back to the previous version (specify steps w/ STEP=n)
rails db:schema:cache:clear              # Clears a db/schema_cache.dump file
rails db:schema:cache:dump               # Creates a db/schema_cache.dump file
rails db:schema:dump                     # Creates a db/schema.rb file that is portable against any DB supported by Active Record
rails db:schema:load                     # Loads a schema.rb file into the database
rails db:seed                            # Loads the seed data from db/seeds.rb
rails db:setup                           # Creates the database, loads the schema, and initializes with the seed data (use db:reset to also drop the database first)
rails db:structure:dump                  # Dumps the database structure to db/structure.sql
rails db:structure:load                  # Recreates the databases from the structure.sql file
rails db:version                         # Retrieves the current schema version number
rails initializers                       # Print out all defined initializers in the order they are invoked by Rails
rails log:clear                          # Truncates all *.log files in log/ to zero bytes (specify which logs with LOGS=test,development)
rails middleware                         # Prints out your Rack middleware stack
rails notes                              # Enumerate all annotations (use notes:optimize, :fixme, :todo for focus)
rails notes:custom                       # Enumerate a custom annotation, specify with ANNOTATION=CUSTOM
rails rails:template                     # Applies the template supplied by LOCATION=(/path/to/template) or URL
rails rails:update                       # Update configs and some other initially generated files (or use just update:configs or update:bin)
rails restart                            # Restart app by touching tmp/restart.txt
rails riak:restart                       # Restart riak locally
rails riak:start                         # Start riak locally
rails riak:stop                          # Stop riak locally
rails routes                             # Print out all defined routes in match order, with names
rails secret                             # Generate a cryptographically secure secret key (this is typically used to generate a secret for cookie sessions)
rails stats                              # Report code statistics (KLOCs, etc) from the application or engine
rails test                               # Runs all tests in test folder
rails test:db                            # Run tests quickly, but also reset db
rails time:zones:all                     # Displays all time zones, also available: time:zones:us, time:zones:local -- filter with OFFSET parameter, e.g., OFFSET=-6
rails tmp:clear                          # Clear cache and socket files from tmp/ (narrow w/ tmp:cache:clear, tmp:sockets:clear)
rails tmp:create                         # Creates tmp directories for cache, sockets, and pids
throwing an error message. If the command is valid, a method of the same name
is called.
If the command is part of the COMMAND_WHITELIST, a method of the same name is called,
if not we proxy it to rake.

This comment has been minimized.

@kaspth

kaspth Dec 11, 2015
Member

I'd rather keep the old comment, we already copy the implementation, so we don't need to add more implementation details.

@@ -102,6 +110,10 @@ def new
end
end

def rake

This comment has been minimized.

@kaspth

kaspth Dec 11, 2015
Member

Remove this method.

@@ -39,6 +44,9 @@ def run_command!(command)

if COMMAND_WHITELIST.include?(command)
send(command)
else
ARGV.unshift(command)
send(:rake)

This comment has been minimized.

@kaspth

kaspth Dec 11, 2015
Member

Let's go with:

run_rake_task(command)
EOT

private

This comment has been minimized.

@kaspth

kaspth Dec 11, 2015
Member

Remove this newline.

@kaspth
Copy link
Member

@kaspth kaspth commented Dec 11, 2015

It seems we should just make the rake tasks flow straight with:

In addition to those, there are:
 destroy      Undo code generated with "generate" (short-cut alias: "d")
 plugin new   Generates skeleton for developing a Rails plugin
 runner       Run a piece of code in the application environment (short-cut alias: "r")
 about        List versions of all Rails frameworks and the environment
 # Rest of the tasks...

All commands can be run with -h (or --help) for more information.

So a one space delimiter and then 3 spaces minimum. Note: this means we have to make the HELP_MESSAGE more dynamic.

Rake.application.load_rakefile
Rake.application.top_level
end
end

This comment has been minimized.

@kaspth

kaspth Dec 11, 2015
Member

Should be:

def run_rake_task(task)
  ARGV.unshift(task) # Prepend the command, so Rake knows how to run it.

  Rake.application...
end
private

def write_rake_tasks_help_message
puts RAKE_TASKS_HELP_MESSAGE

This comment has been minimized.

@kaspth

kaspth Dec 11, 2015
Member

Don't think we need this constant anymore.

def write_rake_tasks
width = rake_tasks.map { |t| t.name_with_args.length }.max || 10
rake_tasks.each do |t|
printf("#{Rake.application.name} %-#{width}s # %s\n", t.name_with_args, t.comment)

This comment has been minimized.

@kaspth

kaspth Dec 11, 2015
Member

Since I don't think we should print Rake.application.name, can we avoid setting it?

@kaspth
Copy link
Member

@kaspth kaspth commented Dec 13, 2015

@dharamgollapudi thanks so much for the work! We needed this in sooner, so I addressed my comment in the merge commit here: 3e65c3d.

Thanks again ❤️

@kaspth kaspth closed this Dec 13, 2015
@dharamgollapudi dharamgollapudi deleted the dharamgollapudi:rake_proxy_for_rails_cli branch Jun 13, 2016
pardel added a commit to Nexmo/nexmo-developer that referenced this pull request Jan 4, 2019
Bringing them in line with the [current standards](https://guides.rubyonrails.org/getting_started.html)
* [rake proxy for rails cli](rails/rails#22288)
mheap added a commit to Nexmo/nexmo-developer that referenced this pull request Jan 9, 2019
… update (#1346)

This PR includes 2 minor changes and 1 show-stopper (I've installed everything from scratch on a new machine).

**Minor changes:**

- updated required min version for Ruby as defined in `.ruby-version`.
- updated the Rails project setup instructions to use the new [rake proxy for rails cli](rails/rails#22288). I've also compressed all 3 operations into one.

**Show-stopper** 

An incompatibility exists between yarn and node versions as specified in `package.json`:

```
{
  ...
  "engines": {
    "node": "> 8.0.0 < 10.0.0",
    "yarn": ">=0.25.2"
  }
}

```

Error:

```
$ ./bin/yarn install
yarn install v1.12.3
[1/5] 🔍  Validating package.json...
error @1.1.0: The engine "node" is incompatible with this module. Expected version "> 8.0.0 < 10.0.0". Got "11.6.0"
error Found incompatible module
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
```

The fix was to use NVM to use node 9.x locally
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

5 participants
You can’t perform that action at this time.