New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Yarn support in new apps using --yarn option #26836

Merged
merged 4 commits into from Nov 29, 2016

Conversation

@Liceth
Contributor

Liceth commented Oct 20, 2016

This add a package.json and the settings needed to get npm nodules integrated in new apps when the --yarn option is used (e.g rails new blog --yarn).

The next changes should be done to finish this:

  • Add package.json in new apps when --yarn option is passed
  • Run yarn install after bundle install in new apps
  • Add rails-ujs dependency to package.json
  • Add node_modules folder to asset paths
  • Fix existing broken tests
  • Add new tests
  • Changelog
@rails-bot

This comment has been minimized.

Show comment
Hide comment
@rails-bot

rails-bot Oct 20, 2016

Thanks for the pull request, and welcome! The Rails team is excited to review your changes, and you should hear from @eileencodes (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

This repository is being automatically checked for code quality issues using Code Climate. You can see results for this analysis in the PR status below. Newly introduced issues should be fixed before a Pull Request is considered ready to review.

Please see the contribution instructions for more information.

rails-bot commented Oct 20, 2016

Thanks for the pull request, and welcome! The Rails team is excited to review your changes, and you should hear from @eileencodes (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

This repository is being automatically checked for code quality issues using Code Climate. You can see results for this analysis in the PR status below. Newly introduced issues should be fixed before a Pull Request is considered ready to review.

Please see the contribution instructions for more information.

@guilleiguaran guilleiguaran self-assigned this Oct 20, 2016

@guilleiguaran guilleiguaran changed the title from Add Npm support in new apps using option to [WIP] Add Npm support in new apps using option Oct 20, 2016

@guilleiguaran guilleiguaran changed the title from [WIP] Add Npm support in new apps using option to [WIP] Add npm support in new apps using --npm option Oct 20, 2016

Show outdated Hide outdated railties/lib/rails/generators/app_base.rb
@@ -404,6 +412,49 @@ def run_bundle
bundle_command("install") if bundle_install?
end
def npm_path
commands = ["npm"]

This comment has been minimized.

@guilleiguaran

guilleiguaran Oct 20, 2016

Member

I think we can add also to "yarn" command here (https://code.facebook.com/posts/1840075619545360)

@guilleiguaran

guilleiguaran Oct 20, 2016

Member

I think we can add also to "yarn" command here (https://code.facebook.com/posts/1840075619545360)

Show outdated Hide outdated railties/lib/rails/generators/app_base.rb
gems << GemfileEntry.version("#{options[:javascript]}-rails", nil,
"Use #{options[:javascript]} as the JavaScript library")
packagejson_exist = File.exist?("package.json")

This comment has been minimized.

@guilleiguaran

guilleiguaran Oct 20, 2016

Member

We are using this line code in multiple parts of code so I think we can extract it to an package_json_exist? function

@guilleiguaran

guilleiguaran Oct 20, 2016

Member

We are using this line code in multiple parts of code so I think we can extract it to an package_json_exist? function

Show outdated Hide outdated railties/lib/rails/generators/app_base.rb
@@ -33,6 +33,9 @@ def self.add_shared_options_for(name)
class_option :javascript, type: :string, aliases: "-j", default: "jquery",
desc: "Preconfigure for selected JavaScript library"
class_option :npm, type: :boolean, default: false,
desc: "Preconfigure for NPM for assets management"

This comment has been minimized.

@guilleiguaran

guilleiguaran Oct 20, 2016

Member

"Preconfigure NPM for assets management"

@guilleiguaran

guilleiguaran Oct 20, 2016

Member

"Preconfigure NPM for assets management"

Show outdated Hide outdated railties/lib/rails/generators/rails/app/templates/package.json
"dependencies": {
"jquery": "~3.1.0",
"jquery-ujs": "~1.2.2"
}

This comment has been minimized.

@guilleiguaran

guilleiguaran Oct 20, 2016

Member

I think we will be able to remove jquery from here and change jquery-ujs to rails-ujs after of having #25208 finished

@guilleiguaran

guilleiguaran Oct 20, 2016

Member

I think we will be able to remove jquery from here and change jquery-ujs to rails-ujs after of having #25208 finished

Show outdated Hide outdated railties/lib/rails/generators/app_base.rb
unless npm_path
puts %{Warning: npm option passed but npm executable was
not detected in the system. Download Node.js in https://nodejs.org/en/download/}

This comment has been minimized.

@guilleiguaran

guilleiguaran Oct 20, 2016

Member

Let's move this warning message to https://github.com/rails/rails/pull/26836/files#diff-248322211d56bb3b2ab59ac68a2839f5R212, right now it's being displayed everytime the npm_path function is called if command is not found

@guilleiguaran

guilleiguaran Oct 20, 2016

Member

Let's move this warning message to https://github.com/rails/rails/pull/26836/files#diff-248322211d56bb3b2ab59ac68a2839f5R212, right now it's being displayed everytime the npm_path function is called if command is not found

@rafaelfranca

This comment has been minimized.

Show comment
Hide comment
@rafaelfranca

rafaelfranca Oct 20, 2016

Member

Can we change npm with yarn? I think only the npm install command that
needs to change
On Wed, 19 Oct 2016 at 21:58 Ruby on Rails Bot notifications@github.com
wrote:

Thanks for the pull request, and welcome! The Rails team is excited to
review your changes, and you should hear from @eileencodes
https://github.com/eileencodes (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra
commits. This ensures that the reviewer can see what has changed since they
last reviewed the code. Due to the way GitHub handles out-of-date commits,
this should also make it reasonably obvious what issues have or haven't
been addressed. Large or tricky changes may require several passes of
review and changes.

This repository is being automatically checked for code quality issues
using Code Climate https://codeclimate.com. You can see results for
this analysis in the PR status below. Newly introduced issues should be
fixed before a Pull Request is considered ready to review.

Please see the contribution instructions
http://edgeguides.rubyonrails.org/contributing_to_ruby_on_rails.html
for more information.


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#26836 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAC66CqJBNFf1G3_6OUb7rSrr1YlfQVeks5q1rzGgaJpZM4KbnZu
.

Member

rafaelfranca commented Oct 20, 2016

Can we change npm with yarn? I think only the npm install command that
needs to change
On Wed, 19 Oct 2016 at 21:58 Ruby on Rails Bot notifications@github.com
wrote:

Thanks for the pull request, and welcome! The Rails team is excited to
review your changes, and you should hear from @eileencodes
https://github.com/eileencodes (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra
commits. This ensures that the reviewer can see what has changed since they
last reviewed the code. Due to the way GitHub handles out-of-date commits,
this should also make it reasonably obvious what issues have or haven't
been addressed. Large or tricky changes may require several passes of
review and changes.

This repository is being automatically checked for code quality issues
using Code Climate https://codeclimate.com. You can see results for
this analysis in the PR status below. Newly introduced issues should be
fixed before a Pull Request is considered ready to review.

Please see the contribution instructions
http://edgeguides.rubyonrails.org/contributing_to_ruby_on_rails.html
for more information.


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#26836 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAC66CqJBNFf1G3_6OUb7rSrr1YlfQVeks5q1rzGgaJpZM4KbnZu
.

@Liceth

This comment has been minimized.

Show comment
Hide comment
@Liceth

Liceth Oct 20, 2016

Contributor

yes, we can add "yarn" as first option to supported commands list and it will be used instead of npm if installed

Contributor

Liceth commented Oct 20, 2016

yes, we can add "yarn" as first option to supported commands list and it will be used instead of npm if installed

@rafaelfranca

This comment has been minimized.

Show comment
Hide comment
@rafaelfranca

rafaelfranca Oct 20, 2016

Member

I'd not even bother to support npm, just jump straight to yarn.

On Wed, 19 Oct 2016 at 22:57 Liceth Ovalles Rodriguez <
notifications@github.com> wrote:

yes, we can add "yarn" as first option to supported commands list and it
will be used instead of npm if installed


You are receiving this because you commented.

Reply to this email directly, view it on GitHub
#26836 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAC66LVJhBn3WNYgCrQQT1xk7LRggIDcks5q1sqJgaJpZM4KbnZu
.

Member

rafaelfranca commented Oct 20, 2016

I'd not even bother to support npm, just jump straight to yarn.

On Wed, 19 Oct 2016 at 22:57 Liceth Ovalles Rodriguez <
notifications@github.com> wrote:

yes, we can add "yarn" as first option to supported commands list and it
will be used instead of npm if installed


You are receiving this because you commented.

Reply to this email directly, view it on GitHub
#26836 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAC66LVJhBn3WNYgCrQQT1xk7LRggIDcks5q1sqJgaJpZM4KbnZu
.

@guilleiguaran guilleiguaran added this to the 5.1.0 milestone Oct 20, 2016

@Liceth Liceth changed the title from [WIP] Add npm support in new apps using --npm option to Add npm support in new apps using --npm option Oct 22, 2016

@Liceth

This comment has been minimized.

Show comment
Hide comment
@Liceth

Liceth Oct 22, 2016

Contributor

@eileencodes @rafaelfranca @guilleiguaran This is ready for review and failures in TravisCI aren't related to my changes (mysql2 tests are failing since yesterday).

Contributor

Liceth commented Oct 22, 2016

@eileencodes @rafaelfranca @guilleiguaran This is ready for review and failures in TravisCI aren't related to my changes (mysql2 tests are failing since yesterday).

@guilleiguaran

This comment has been minimized.

Show comment
Hide comment
@guilleiguaran

guilleiguaran Oct 23, 2016

Member

@Liceth I've fixed the mysql2 tests in f13ec72 and updated your branch

Member

guilleiguaran commented Oct 23, 2016

@Liceth I've fixed the mysql2 tests in f13ec72 and updated your branch

Show outdated Hide outdated railties/lib/rails/generators/rails/app/app_generator.rb
@@ -205,6 +209,9 @@ def create_root_files
build(:readme)
build(:rakefile)
build(:configru)
if options[:npm]
build(:packagejson)
end

This comment has been minimized.

@guilleiguaran

guilleiguaran Oct 24, 2016

Member

This if can be done in a single line:

build(:packagejson) if options[:npm]
@guilleiguaran

guilleiguaran Oct 24, 2016

Member

This if can be done in a single line:

build(:packagejson) if options[:npm]

This comment has been minimized.

@Liceth

Liceth Oct 24, 2016

Contributor

Thanks for the observation @guilleiguaran I already made the changes

@Liceth

Liceth Oct 24, 2016

Contributor

Thanks for the observation @guilleiguaran I already made the changes

@bouk

This comment has been minimized.

Show comment
Hide comment
@bouk

bouk Oct 25, 2016

Member

Please note that yarn is not a total drop-in replacement for npm, we can't try one or the other. A project that uses yarn gets a yarn.lock file (analogous to Gemfile.lock). NPM doesn't create a file like that and doesn't interpret it.

I fully support not even bothering with NPM and just using yarn, it's a lot more robust in my experience. Requiring people to install it separately might take away from the plug-and-play nature of Rails.

Member

bouk commented Oct 25, 2016

Please note that yarn is not a total drop-in replacement for npm, we can't try one or the other. A project that uses yarn gets a yarn.lock file (analogous to Gemfile.lock). NPM doesn't create a file like that and doesn't interpret it.

I fully support not even bothering with NPM and just using yarn, it's a lot more robust in my experience. Requiring people to install it separately might take away from the plug-and-play nature of Rails.

@eileencodes eileencodes removed their assignment Oct 25, 2016

@Liceth

This comment has been minimized.

Show comment
Hide comment
@Liceth

Liceth Oct 25, 2016

Contributor

I'm fine wherever you decide about yarn/npm, only let me know the final decision on it.

Contributor

Liceth commented Oct 25, 2016

I'm fine wherever you decide about yarn/npm, only let me know the final decision on it.

@connorshea

This comment has been minimized.

Show comment
Hide comment
@connorshea

connorshea Oct 26, 2016

Contributor

Don't you install Yarn using npm? I 100% support using Yarn as the package manager, but it won't avoid npm entirely.

Contributor

connorshea commented Oct 26, 2016

Don't you install Yarn using npm? I 100% support using Yarn as the package manager, but it won't avoid npm entirely.

Show outdated Hide outdated railties/lib/rails/generators/app_base.rb
@@ -33,6 +33,9 @@ def self.add_shared_options_for(name)
class_option :javascript, type: :string, aliases: "-j", default: "jquery",
desc: "Preconfigure for selected JavaScript library"
class_option :npm, type: :boolean, default: false,
desc: "Preconfigure for NPM assets management"

This comment has been minimized.

@connorshea

connorshea Oct 26, 2016

Contributor

npm should be lowercase in all situations as far as I know.

@connorshea

connorshea Oct 26, 2016

Contributor

npm should be lowercase in all situations as far as I know.

Show outdated Hide outdated railties/lib/rails/generators/app_base.rb
npm_command("install")
else
say_status :warning, "npm option passed but npm executable was not detected in the system.", :yellow
say_status :warning, "Download Node.js in https://nodejs.org/en/download/.", :yellow

This comment has been minimized.

@connorshea

connorshea Oct 27, 2016

Contributor

Should be "Download Node.js at https://nodejs.org/en/download/"

@connorshea

connorshea Oct 27, 2016

Contributor

Should be "Download Node.js at https://nodejs.org/en/download/"

This comment has been minimized.

@Liceth

Liceth Oct 27, 2016

Contributor

Hi @connorshea Thanks for you review

@Liceth

Liceth Oct 27, 2016

Contributor

Hi @connorshea Thanks for you review

Show outdated Hide outdated ...ib/rails/generators/rails/app/templates/config/initializers/assets.rb.tt
@@ -5,6 +5,9 @@ Rails.application.config.assets.version = '1.0'
# Add additional assets to the asset load path
# Rails.application.config.assets.paths << Emoji.images_path
<%- if options[:npm] -%>
Rails.application.config.assets.paths << Rails.root.join('node_modules')

This comment has been minimized.

@connorshea

connorshea Oct 27, 2016

Contributor

Should this have a comment explaining what it does?

@connorshea

connorshea Oct 27, 2016

Contributor

Should this have a comment explaining what it does?

@bouk

This comment has been minimized.

Show comment
Hide comment
@bouk

bouk Oct 28, 2016

Member

For installing yarn there's options available for most common package managers (brew, apt, Chocolatey) https://yarnpkg.com/en/docs/install

Member

bouk commented Oct 28, 2016

For installing yarn there's options available for most common package managers (brew, apt, Chocolatey) https://yarnpkg.com/en/docs/install

@bouk

This comment has been minimized.

Show comment
Hide comment
@bouk

bouk Oct 28, 2016

Member

It might be handy to add updating of the JS packages to the bin/update and bin/setup templates

Member

bouk commented Oct 28, 2016

It might be handy to add updating of the JS packages to the bin/update and bin/setup templates

@guilleiguaran

This comment has been minimized.

Show comment
Hide comment
@guilleiguaran

guilleiguaran Oct 31, 2016

Member

@Liceth we have decided to skip NPM and go directly with Yarn because of the improvements of this last. Can you update your pull request?

Also update the bin/update and bin/setup files as @bouk suggested.

Thank you very much for your work on this!!!

Member

guilleiguaran commented Oct 31, 2016

@Liceth we have decided to skip NPM and go directly with Yarn because of the improvements of this last. Can you update your pull request?

Also update the bin/update and bin/setup files as @bouk suggested.

Thank you very much for your work on this!!!

@Liceth

This comment has been minimized.

Show comment
Hide comment
@Liceth

Liceth Oct 31, 2016

Contributor

Sure @guilleiguaran I wil do the changes in the pull request.
Thank you 👍

Contributor

Liceth commented Oct 31, 2016

Sure @guilleiguaran I wil do the changes in the pull request.
Thank you 👍

@Liceth Liceth changed the title from Add npm support in new apps using --npm option to Add Yarn support in new apps using --yarn option Oct 31, 2016

@Liceth

This comment has been minimized.

Show comment
Hide comment
@Liceth

Liceth Nov 18, 2016

Contributor

@rafaelfranca @guilleiguaran I'm finishing this but I have a quick question related to jQuery removal (#25208)

If the users pass the -j option when they're creating a new app do we need to add the library to the Gemfile (e.g jquery-rails) or to the package.json?

Also note that there are some libraries that haven't been published to npm since they haven't received updates in many years like prototype.js, prototype_ujs and mootools_ujs

Contributor

Liceth commented Nov 18, 2016

@rafaelfranca @guilleiguaran I'm finishing this but I have a quick question related to jQuery removal (#25208)

If the users pass the -j option when they're creating a new app do we need to add the library to the Gemfile (e.g jquery-rails) or to the package.json?

Also note that there are some libraries that haven't been published to npm since they haven't received updates in many years like prototype.js, prototype_ujs and mootools_ujs

@guilleiguaran

This comment has been minimized.

Show comment
Hide comment
@guilleiguaran

guilleiguaran Nov 19, 2016

Member

Let's keep the actual behavior of -j option, if the users pass it let's add the gem instead of the npm dependency.

Member

guilleiguaran commented Nov 19, 2016

Let's keep the actual behavior of -j option, if the users pass it let's add the gem instead of the npm dependency.

@guilleiguaran

This comment has been minimized.

Show comment
Hide comment
@guilleiguaran

guilleiguaran Nov 19, 2016

Member

Also please rebase this and squash commits

Member

guilleiguaran commented Nov 19, 2016

Also please rebase this and squash commits

@dhh

This comment has been minimized.

Show comment
Hide comment
@dhh

dhh Dec 1, 2016

Member

I'd be happy in time to move Gemfile/Gemfile.lock there as well.

Member

dhh commented Dec 1, 2016

I'd be happy in time to move Gemfile/Gemfile.lock there as well.

@benlangfeld

This comment has been minimized.

Show comment
Hide comment
@benlangfeld

benlangfeld Dec 1, 2016

@dhh In cases where Rails engines are also npm modules for the purpose of delivering assets, the package.json they have is meta-data about the package being delivered in the same vain as the gemspec. In the event this change propagated to engines, would you support these files staying in the engine root alongside the gemspec?

I ask mostly because this is something I'm working through on our engine-based app, but also because I recall a traumatic incident as a child when I "tidied up" some messy DLLs from C:/ to C:/Windows and "bricked" the family computer. 😄

benlangfeld commented Dec 1, 2016

@dhh In cases where Rails engines are also npm modules for the purpose of delivering assets, the package.json they have is meta-data about the package being delivered in the same vain as the gemspec. In the event this change propagated to engines, would you support these files staying in the engine root alongside the gemspec?

I ask mostly because this is something I'm working through on our engine-based app, but also because I recall a traumatic incident as a child when I "tidied up" some messy DLLs from C:/ to C:/Windows and "bricked" the family computer. 😄

@dhh

This comment has been minimized.

Show comment
Hide comment
@dhh

dhh Dec 1, 2016

Member

Can you walk me through the workflow you have for engines where there is a package.json? Are you using that file to publish the JS parts of the engine to NPM? Or are you thinking that this is a way to integrate matters once we get to Webpack?

Because I get that package.json is basically both Gemfile and .gemspec in one. In Rails, your own app doesn't have a .gemspec, but your engine might. Here, we're using package.json as Gemfile, just to manage the vendored JS dependencies, not as .gemspec with the intention of publishing the app somewhere.

Member

dhh commented Dec 1, 2016

Can you walk me through the workflow you have for engines where there is a package.json? Are you using that file to publish the JS parts of the engine to NPM? Or are you thinking that this is a way to integrate matters once we get to Webpack?

Because I get that package.json is basically both Gemfile and .gemspec in one. In Rails, your own app doesn't have a .gemspec, but your engine might. Here, we're using package.json as Gemfile, just to manage the vendored JS dependencies, not as .gemspec with the intention of publishing the app somewhere.

@benlangfeld

This comment has been minimized.

Show comment
Hide comment
@benlangfeld

benlangfeld Dec 1, 2016

@dhh We're following an internal workflow based on the ideas at http://cbra.info, the overall goal being rescuing a 200kloc, 600 model application from chaos.

This app also uses react_on_rails, which uses Webpack rather than the asset pipeline, and my current effort is on linking engine's assets into the app. This is what I'm actively working on as of today, so my thoughts are not yet well formed.

once we get to Webpack?

Is some core usage of Webpack already on the roadmap? Are there details available of what this will look like?

benlangfeld commented Dec 1, 2016

@dhh We're following an internal workflow based on the ideas at http://cbra.info, the overall goal being rescuing a 200kloc, 600 model application from chaos.

This app also uses react_on_rails, which uses Webpack rather than the asset pipeline, and my current effort is on linking engine's assets into the app. This is what I'm actively working on as of today, so my thoughts are not yet well formed.

once we get to Webpack?

Is some core usage of Webpack already on the roadmap? Are there details available of what this will look like?

@dhh

This comment has been minimized.

Show comment
Hide comment
@dhh

dhh Dec 1, 2016

Member

Gotcha. Yeah, let's try to sort it all out.

I'm starting work on --webpack now. Should have a preliminary PR up soon. There's definitely a ton more work for us to get there compared to --yarn, but I'd like to target 5.1 with that option as well.

Member

dhh commented Dec 1, 2016

Gotcha. Yeah, let's try to sort it all out.

I'm starting work on --webpack now. Should have a preliminary PR up soon. There's definitely a ton more work for us to get there compared to --yarn, but I'd like to target 5.1 with that option as well.

@dhh

This comment has been minimized.

Show comment
Hide comment
@dhh

dhh Dec 1, 2016

Member

Not a fan of using components to decompose the Majestic Monolith. But whatever floats your boat! We have a 300-model, 35kloc app in Basecamp 3. Shopify has, I believe, a 700-model 200kloc app. Both Majestic Monoliths, and happily so.

If you, for organizational reasons, want to split the Majestic Monolith, then I think going with completely separate apps is a better way to go.

Member

dhh commented Dec 1, 2016

Not a fan of using components to decompose the Majestic Monolith. But whatever floats your boat! We have a 300-model, 35kloc app in Basecamp 3. Shopify has, I believe, a 700-model 200kloc app. Both Majestic Monoliths, and happily so.

If you, for organizational reasons, want to split the Majestic Monolith, then I think going with completely separate apps is a better way to go.

@TheLarkInn

This comment has been minimized.

Show comment
Hide comment
@TheLarkInn

TheLarkInn Dec 1, 2016

Feel free and reach out for webpack assistance :) our org would love to track the efforts to adding webpack as a replacement to sprockets etc and assist where valuable.

TheLarkInn commented Dec 1, 2016

Feel free and reach out for webpack assistance :) our org would love to track the efforts to adding webpack as a replacement to sprockets etc and assist where valuable.

@dhh

This comment has been minimized.

Show comment
Hide comment
@dhh

dhh Dec 2, 2016

Member

Thanks @TheLarkInn. I'm sure we can use the help! I'll ping you on the PR once it's ready.

The current thinking is to let the asset pipeline focus on JS sprinkles, css, images, etc. I know that web pack technically CAN manage those things, but I don't see a big benefit to doing so.

Where Webpack really shines is for application-level JS. So that's also a way for having the two systems coexist, especially for existing apps. For Basecamp, we're considering writing another calendaring feature, which is right in the sweet spot for client-side MVC. We can use Webpack for that feature, while we continue to use the asset pipeline for the JS sprinkles that add sparkle to pages with less demanding UI.

Anyway, I've been diving in over the last several days. Playing with everything within Basecamp. So thinking subject to change!

What's clear is that we want the core Rails principles of convention over configuration to carry over. Nobody should HAVE to configure their own webpack.config.js. It should surely be possible, but not required.

Member

dhh commented Dec 2, 2016

Thanks @TheLarkInn. I'm sure we can use the help! I'll ping you on the PR once it's ready.

The current thinking is to let the asset pipeline focus on JS sprinkles, css, images, etc. I know that web pack technically CAN manage those things, but I don't see a big benefit to doing so.

Where Webpack really shines is for application-level JS. So that's also a way for having the two systems coexist, especially for existing apps. For Basecamp, we're considering writing another calendaring feature, which is right in the sweet spot for client-side MVC. We can use Webpack for that feature, while we continue to use the asset pipeline for the JS sprinkles that add sparkle to pages with less demanding UI.

Anyway, I've been diving in over the last several days. Playing with everything within Basecamp. So thinking subject to change!

What's clear is that we want the core Rails principles of convention over configuration to carry over. Nobody should HAVE to configure their own webpack.config.js. It should surely be possible, but not required.

@TheLarkInn

This comment has been minimized.

Show comment
Hide comment
@TheLarkInn

TheLarkInn Dec 2, 2016

Totalll, I think that's a great start/approach. I'm making a plugin that can potentially work with multipage MVC's multipage-webpack-plugin. It's an oss plugin inspired at work for the need for an easier script injection process for server routes MVC's (we use laravel), so if you find benefit to match this kind of rails arch, let.me know and we can work on it or i can implement additional feature. I'll keep an eye out for an email when you get a PR ready. 👏👏👏

TheLarkInn commented Dec 2, 2016

Totalll, I think that's a great start/approach. I'm making a plugin that can potentially work with multipage MVC's multipage-webpack-plugin. It's an oss plugin inspired at work for the need for an easier script injection process for server routes MVC's (we use laravel), so if you find benefit to match this kind of rails arch, let.me know and we can work on it or i can implement additional feature. I'll keep an eye out for an email when you get a PR ready. 👏👏👏

@moonglum

This comment has been minimized.

Show comment
Hide comment
@moonglum

moonglum Dec 2, 2016

@dhh A different approach would be to offer integration to external asset pipelines, without choosing a specific one like Webpack. I've built a gem for that purpose and am using that in production with a (Webpack-based) Node.js asset pipeline. The idea is simple:

The external asset pipeline compiles the files. In addition, it creates so called manifest files that map artifact names (like application.js) to a path where the file can be found (like /javascripts/application-gjgkj3499292.js). The gem simply reads the manifest files so you can still keep using helpers like stylesheet_link_tag or image_tag without loosing the capability to use cache busting for those files. The manifest file is a simple JSON file.

If you're interested in that approach, I'm more than happy to contribute that to Rails.

moonglum commented Dec 2, 2016

@dhh A different approach would be to offer integration to external asset pipelines, without choosing a specific one like Webpack. I've built a gem for that purpose and am using that in production with a (Webpack-based) Node.js asset pipeline. The idea is simple:

The external asset pipeline compiles the files. In addition, it creates so called manifest files that map artifact names (like application.js) to a path where the file can be found (like /javascripts/application-gjgkj3499292.js). The gem simply reads the manifest files so you can still keep using helpers like stylesheet_link_tag or image_tag without loosing the capability to use cache busting for those files. The manifest file is a simple JSON file.

If you're interested in that approach, I'm more than happy to contribute that to Rails.

@dhh

This comment has been minimized.

Show comment
Hide comment
@dhh

dhh Dec 2, 2016

Member

Lucas, that's an interesting idea. I'd like to start with Webpack first and see how things go. For starters, we're only going to use Webpack for app-style JS. Not images, css, fonts, etc. I think it's more important out the gate that we nail the experience for the pipelines we provide in the box (Rails' own and Webpack) than going full extendable. But those goals are not in opposition.

I'd be happy to see what an exploration looks like. I'd just temper expectations about what's on a likely path for Rails 5.1.

It'll probably be easier to see how this is going to fit together once we land the current plans for --webpack in master.

Member

dhh commented Dec 2, 2016

Lucas, that's an interesting idea. I'd like to start with Webpack first and see how things go. For starters, we're only going to use Webpack for app-style JS. Not images, css, fonts, etc. I think it's more important out the gate that we nail the experience for the pipelines we provide in the box (Rails' own and Webpack) than going full extendable. But those goals are not in opposition.

I'd be happy to see what an exploration looks like. I'd just temper expectations about what's on a likely path for Rails 5.1.

It'll probably be easier to see how this is going to fit together once we land the current plans for --webpack in master.

@FND

This comment has been minimized.

Show comment
Hide comment
@FND

FND Dec 2, 2016

Contributor

@dhh's approach seems reasonable - however, it might still be advisable to choose a compiler-agnostic name for the CLI option, to prevent API changes when the underlying infrastructure changes? (For straightforward webpack configurations, Rollup can be used as a drop-in replacement.)

Contributor

FND commented Dec 2, 2016

@dhh's approach seems reasonable - however, it might still be advisable to choose a compiler-agnostic name for the CLI option, to prevent API changes when the underlying infrastructure changes? (For straightforward webpack configurations, Rollup can be used as a drop-in replacement.)

@dhh

This comment has been minimized.

Show comment
Hide comment
@dhh

dhh Dec 2, 2016

Member

Don't think the CLI options matter that much, since even if we did go with a generic solution, we can still support that. So --webpack would trigger the Webpack variant of that. Then --other-pipe could trigger that. Or we could extend with --pipeline webpack. This CLI name is only relevant for the start of a new project anyway. So it's not like there'd be backwards incompatibility issues going forward.

But yeah, I think all this is up in the air. Let's get a real PR going, then it'll be easier to talk about specifics.

Member

dhh commented Dec 2, 2016

Don't think the CLI options matter that much, since even if we did go with a generic solution, we can still support that. So --webpack would trigger the Webpack variant of that. Then --other-pipe could trigger that. Or we could extend with --pipeline webpack. This CLI name is only relevant for the start of a new project anyway. So it's not like there'd be backwards incompatibility issues going forward.

But yeah, I think all this is up in the air. Let's get a real PR going, then it'll be easier to talk about specifics.

@mrreynolds

This comment has been minimized.

Show comment
Hide comment
@mrreynolds

mrreynolds Dec 2, 2016

Contributor

+1 for --pipeline webpack.

Contributor

mrreynolds commented Dec 2, 2016

+1 for --pipeline webpack.

@kayakyakr

This comment has been minimized.

Show comment
Hide comment
@kayakyakr

kayakyakr Dec 2, 2016

Contributor

This triggered a long conversation in our Slack about the same and even with just 2 of us mostly talking, we came up with 4 different opinions on how it should be done... In general, I think we'd favor a combination of @moonglum and @dhh's approaches.

Dunno if anyone's had experience with https://github.com/thoughtbot/ember-cli-rails, but they've mostly hit the sweet spot as far as I'm concerned.

Contributor

kayakyakr commented Dec 2, 2016

This triggered a long conversation in our Slack about the same and even with just 2 of us mostly talking, we came up with 4 different opinions on how it should be done... In general, I think we'd favor a combination of @moonglum and @dhh's approaches.

Dunno if anyone's had experience with https://github.com/thoughtbot/ember-cli-rails, but they've mostly hit the sweet spot as far as I'm concerned.

@dhh

This comment has been minimized.

Show comment
Hide comment
@dhh

dhh Dec 2, 2016

Member

@kayakyakr That's always been a big part of the value with Rails. There's an infinite number of ways things COULD be done, but there's an immense benefit to simply picking a path of running with it. That's the core promise behind Convention over Configuration.

Member

dhh commented Dec 2, 2016

@kayakyakr That's always been a big part of the value with Rails. There's an infinite number of ways things COULD be done, but there's an immense benefit to simply picking a path of running with it. That's the core promise behind Convention over Configuration.

@FND

This comment has been minimized.

Show comment
Hide comment
@FND

FND Dec 2, 2016

Contributor

FWIW, without wanting to belabor this point, just a quick clarification: The benefits of Convention over Configuration are of course undisputed - the reason this issue seemed worth raising is because the JavaScript precompiler shouldn't surface to application developers, i.e. the respective source code should be unaffected by whatever is decided here.

Contributor

FND commented Dec 2, 2016

FWIW, without wanting to belabor this point, just a quick clarification: The benefits of Convention over Configuration are of course undisputed - the reason this issue seemed worth raising is because the JavaScript precompiler shouldn't surface to application developers, i.e. the respective source code should be unaffected by whatever is decided here.

@kayakyakr

This comment has been minimized.

Show comment
Hide comment
@kayakyakr

kayakyakr Dec 2, 2016

Contributor

@FND that's not going to be the case if the implementation is to run sprockets and webpack side-by-side, though. Each pipeline wants its own assets path. Only taking the path of full pipeline replacement gets you the ability to be precompiler-agnostic.

Contributor

kayakyakr commented Dec 2, 2016

@FND that's not going to be the case if the implementation is to run sprockets and webpack side-by-side, though. Each pipeline wants its own assets path. Only taking the path of full pipeline replacement gets you the ability to be precompiler-agnostic.

@moonglum

This comment has been minimized.

Show comment
Hide comment
@moonglum

moonglum Dec 2, 2016

In general, I think we'd favor a combination of @moonglum and @dhh's approaches.

I think that makes sense. If the ability to read manifest files as described above would be built-into Rails, then all the --webpack flag would need to do is create a webpack config that creates the according manifest files. Adding a --whatever-other-solution-you-need flag would then be simple.

This would offer the easy out of the box experience that @dhh described, but would still be flexible to use with other external asset pipelines.

moonglum commented Dec 2, 2016

In general, I think we'd favor a combination of @moonglum and @dhh's approaches.

I think that makes sense. If the ability to read manifest files as described above would be built-into Rails, then all the --webpack flag would need to do is create a webpack config that creates the according manifest files. Adding a --whatever-other-solution-you-need flag would then be simple.

This would offer the easy out of the box experience that @dhh described, but would still be flexible to use with other external asset pipelines.

@NekR

This comment has been minimized.

Show comment
Hide comment
@NekR

NekR Dec 4, 2016

I know that web pack technically CAN manage those things, but I don't see a big benefit to doing so.

It might be useful if someone would, say, use a ServiceWorker webpack plugin to generate it for static assets. I know there is ServiceWorker plugin for rails' itself, but still, that might be possible.

NekR commented Dec 4, 2016

I know that web pack technically CAN manage those things, but I don't see a big benefit to doing so.

It might be useful if someone would, say, use a ServiceWorker webpack plugin to generate it for static assets. I know there is ServiceWorker plugin for rails' itself, but still, that might be possible.

@steakknife

This comment has been minimized.

Show comment
Hide comment
@steakknife

steakknife Dec 4, 2016

Looks good. Previously there was webpack-rails (more resources: Migrating from Rails’ asset pipeline to Node’s webpack, react-webpack-rails-tutorial), which might serve as an example to build upon configuration / conventions and integration behaviors.

steakknife commented Dec 4, 2016

Looks good. Previously there was webpack-rails (more resources: Migrating from Rails’ asset pipeline to Node’s webpack, react-webpack-rails-tutorial), which might serve as an example to build upon configuration / conventions and integration behaviors.

@akabekobeko

This comment has been minimized.

Show comment
Hide comment
@akabekobeko

akabekobeko Dec 5, 2016

webpack is popular and multi-functional. However, it is mere module and it is not standard as Node and npm. As with yarn, these are optional as npm and npm-scripts as standard.

If jQuery dependence is to be stopped, it is better to be cautious about non-standard product dependence.

If future yarn and webpack are obsolete (...It may be impossible), npm and npm-scripts will remain.

akabekobeko commented Dec 5, 2016

webpack is popular and multi-functional. However, it is mere module and it is not standard as Node and npm. As with yarn, these are optional as npm and npm-scripts as standard.

If jQuery dependence is to be stopped, it is better to be cautious about non-standard product dependence.

If future yarn and webpack are obsolete (...It may be impossible), npm and npm-scripts will remain.

@justin808

This comment has been minimized.

Show comment
Hide comment
@justin808

justin808 Dec 5, 2016

@steakknife Thanks for the shout-out for the react-webpack-rails-tutorial. Its sibling package is react_on_rails.

justin808 commented Dec 5, 2016

@steakknife Thanks for the shout-out for the react-webpack-rails-tutorial. Its sibling package is react_on_rails.

@justin808

This comment has been minimized.

Show comment
Hide comment
@justin808

justin808 Dec 5, 2016

@dhh wrote:

Lucas, that's an interesting idea. I'd like to start with Webpack first and see how things go. For starters, we're only going to use Webpack for app-style JS. Not images, css, fonts, etc.

A huge advantage of Webpack is CSS-Modules. If you like tidy, you'll love CSS-Modules. See Sass, CSS Modules, and Twitter Bootstrap Integration.

Here's an example of it:

justin808 commented Dec 5, 2016

@dhh wrote:

Lucas, that's an interesting idea. I'd like to start with Webpack first and see how things go. For starters, we're only going to use Webpack for app-style JS. Not images, css, fonts, etc.

A huge advantage of Webpack is CSS-Modules. If you like tidy, you'll love CSS-Modules. See Sass, CSS Modules, and Twitter Bootstrap Integration.

Here's an example of it:

@josepjaume

This comment has been minimized.

Show comment
Hide comment
@josepjaume

josepjaume Dec 5, 2016

Contributor

@dhh in my experience, the major pain points I've found integrating any sort of pre-compiling using Webpack and feeding it to Sprockets are asset fingerprinting. The way Sprockets works right now, it's nearly impossible to create, let's say, a react component that references a particular asset (an image, for example), without getting React to read Sprocket's manifest file, or Sprockets read Webpack's manifest file.

There's two approaches I believe might work:

  • The first one, just let Webpack handle asset fingerprinting, but you already said that's not desirable.

  • Another other option would be to let Sprockets fingerprint all the assets and rewrite the actual assets just at the end of the build process, instead of relying on the original code to write the right asset locations - something like what the Phoenix framework does with brunch. That'd would also decouple the contract between webpack (or any other build tool) and sprockets, allowing us to integrate with virtually any other build tool.

Maybe the guys on https://github.com/shakacode/react_on_rails/ (specially @justin808) could chime in on the matter, they seem to have given it a lot of thought.

Contributor

josepjaume commented Dec 5, 2016

@dhh in my experience, the major pain points I've found integrating any sort of pre-compiling using Webpack and feeding it to Sprockets are asset fingerprinting. The way Sprockets works right now, it's nearly impossible to create, let's say, a react component that references a particular asset (an image, for example), without getting React to read Sprocket's manifest file, or Sprockets read Webpack's manifest file.

There's two approaches I believe might work:

  • The first one, just let Webpack handle asset fingerprinting, but you already said that's not desirable.

  • Another other option would be to let Sprockets fingerprint all the assets and rewrite the actual assets just at the end of the build process, instead of relying on the original code to write the right asset locations - something like what the Phoenix framework does with brunch. That'd would also decouple the contract between webpack (or any other build tool) and sprockets, allowing us to integrate with virtually any other build tool.

Maybe the guys on https://github.com/shakacode/react_on_rails/ (specially @justin808) could chime in on the matter, they seem to have given it a lot of thought.

@justin808

This comment has been minimized.

Show comment
Hide comment
@justin808

justin808 Dec 6, 2016

@josepjaume We have the problem solved in React on Rails, such that "it just works". The solution involves:

  1. Allow webpack to fingerprint assets.
  2. Publish the assets using the Rails asset pipeline, which creates double fingerprints.
  3. Create symlinks at deployment, using the manifest file, so that we have single-fingerprinted symlink names that point to the double fingerprinted files.

This way, the correct file names are available on deployment.

Any questions?

justin808 commented Dec 6, 2016

@josepjaume We have the problem solved in React on Rails, such that "it just works". The solution involves:

  1. Allow webpack to fingerprint assets.
  2. Publish the assets using the Rails asset pipeline, which creates double fingerprints.
  3. Create symlinks at deployment, using the manifest file, so that we have single-fingerprinted symlink names that point to the double fingerprinted files.

This way, the correct file names are available on deployment.

Any questions?

@dhh

This comment has been minimized.

Show comment
Hide comment
@dhh

dhh Dec 7, 2016

Member

For those interested, I've published the first work on --webpack here: #27288.

Re: cross-reading of manifests, @josepjaume, I think that's actually how we should make this work. Make it possible for each side to read the others manifest and construct what they need from there. The current strategy is to use Webpack exclusively for app-like JS, and nothing else. Asset pipeline for everything else (JS sprinkles, images, fonts, css, etc).

Member

dhh commented Dec 7, 2016

For those interested, I've published the first work on --webpack here: #27288.

Re: cross-reading of manifests, @josepjaume, I think that's actually how we should make this work. Make it possible for each side to read the others manifest and construct what they need from there. The current strategy is to use Webpack exclusively for app-like JS, and nothing else. Asset pipeline for everything else (JS sprinkles, images, fonts, css, etc).

@iantheparker

This comment has been minimized.

Show comment
Hide comment
@iantheparker

iantheparker Feb 28, 2017

@josepjaume this is also how the bower-rails gem resolved relative asset paths.

Another other option would be to let Sprockets fingerprint all the assets and rewrite the actual assets just at the end of the build process, instead of relying on the original code to write the right asset locations

Has this option gotten any love recently?

iantheparker commented Feb 28, 2017

@josepjaume this is also how the bower-rails gem resolved relative asset paths.

Another other option would be to let Sprockets fingerprint all the assets and rewrite the actual assets just at the end of the build process, instead of relying on the original code to write the right asset locations

Has this option gotten any love recently?

@krushi123

This comment has been minimized.

Show comment
Hide comment
@krushi123

krushi123 Jul 4, 2017

My Rails Version is 5.1.2 and Ruby 2.4 - I am using Yarn

yarn add font-awesome

Now, in application.css.scss I imported the font-awesome

@import font-awesome/scss/font-awesome;

Now, when I refer to fa-icon in the view, I am getting 404 Not found error for the font file in development. Any idea what I am doing wrong?

krushi123 commented Jul 4, 2017

My Rails Version is 5.1.2 and Ruby 2.4 - I am using Yarn

yarn add font-awesome

Now, in application.css.scss I imported the font-awesome

@import font-awesome/scss/font-awesome;

Now, when I refer to fa-icon in the view, I am getting 404 Not found error for the font file in development. Any idea what I am doing wrong?

@Schwad

This comment has been minimized.

Show comment
Hide comment
@Schwad

Schwad Jul 25, 2017

Hey all, really interested in what's going on with these changes and updating from Rails 5.0 to Rails 5.1. Quick (hopefully not too stupid) question; does this only apply to new Rails Apps with rails new blog --yarn, or are there any out of the box options included for updating existing Rails 5.1 apps with this?

Thanks so much for your work on this feature!!! 🎉

Schwad commented Jul 25, 2017

Hey all, really interested in what's going on with these changes and updating from Rails 5.0 to Rails 5.1. Quick (hopefully not too stupid) question; does this only apply to new Rails Apps with rails new blog --yarn, or are there any out of the box options included for updating existing Rails 5.1 apps with this?

Thanks so much for your work on this feature!!! 🎉

@aried3r

This comment has been minimized.

Show comment
Hide comment
@aried3r

aried3r Aug 28, 2017

Contributor

@Schwad, there is a rails yarn:install task, but it doesn't really do everything that happens when a new Rails app is created, notably editing the .gitignore file, adding node_modules to the assets path etc.

You can find a few things if you search for skip_yarn in the codebase.

I'm not sure if this could be changed or is an intentional omission, but I couldn't find an official guide for adding yarn to existing Rails projects upgrading to 5.1.

Contributor

aried3r commented Aug 28, 2017

@Schwad, there is a rails yarn:install task, but it doesn't really do everything that happens when a new Rails app is created, notably editing the .gitignore file, adding node_modules to the assets path etc.

You can find a few things if you search for skip_yarn in the codebase.

I'm not sure if this could be changed or is an intentional omission, but I couldn't find an official guide for adding yarn to existing Rails projects upgrading to 5.1.

@aried3r

This comment has been minimized.

Show comment
Hide comment
@aried3r

aried3r Aug 29, 2017

Contributor

@Schwad, also, if you run the rails app:update task, Rails will offer you to add all necessary changes except for the .gitignore file I think.

Contributor

aried3r commented Aug 29, 2017

@Schwad, also, if you run the rails app:update task, Rails will offer you to add all necessary changes except for the .gitignore file I think.

@steakknife

This comment has been minimized.

Show comment
Hide comment
@steakknife

steakknife Aug 30, 2017

Just came by to mass unsubscribe.
(start bitching)
I don't use Rails for new development anymore, but yarn and broccoli awesome. Yarn is face-meltingly fast and a nearly drop-in replacement for slow, unlocked npm. Webpack proved to be brittle, inflexible and slow, despite having a loyal, zealous herd. Hobby projects are free to use anything, but development for maintainable production has many different needs.
(end bitching)

steakknife commented Aug 30, 2017

Just came by to mass unsubscribe.
(start bitching)
I don't use Rails for new development anymore, but yarn and broccoli awesome. Yarn is face-meltingly fast and a nearly drop-in replacement for slow, unlocked npm. Webpack proved to be brittle, inflexible and slow, despite having a loyal, zealous herd. Hobby projects are free to use anything, but development for maintainable production has many different needs.
(end bitching)

@amrutjadhav

This comment has been minimized.

Show comment
Hide comment
@amrutjadhav

amrutjadhav Feb 19, 2018

@krushi123 You need to overwrite the $fa-font-path variable which is used by font-awesome to point to font directory. Just add these line to your application.css.scss:
$fa-font-path: "font-awesome/fonts/";
before
@import font-awesome/scss/font-awesome;

amrutjadhav commented Feb 19, 2018

@krushi123 You need to overwrite the $fa-font-path variable which is used by font-awesome to point to font directory. Just add these line to your application.css.scss:
$fa-font-path: "font-awesome/fonts/";
before
@import font-awesome/scss/font-awesome;

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment