Switch away from Webpacker to Shakapacker

[alecslupu]

🎩 What? Why?

Decidim has migrated from sprockets to Webpacker in #7464, almost 2 years ago. Since then, the rails team has decided to retire the webpacker

Based on the discussions we had in the thread: #8783 we could not find a consensus on what is the best way of approaching the webpacker retirement, yet, since there are no new releases to webpacker, and since there are bugfixes and security improvements happening javascript ecosystem, we need to make upgrades so we can keep a high standard to the application.

I have chosen to migrate to Shakapacker, because we need to perform upgrades to the system, and Shakapacker is the only tool that offers backwards compatibility with 0.26 and 0.27. I did not liked the ideea in the first place, based on the arguments that have been provided in the discussion, yet, now, seems more stable and more promising that the current system.

Upgrade and Testing

In order to perform the update, you will need to make sure that you do not have webpacker in your Gemfile.
If you have it, please remove it, or change it to

gem "shakapacker", "~> 6.5"

Please backup the following files, to make sure that you save any customizations you may have done to webpacker:

config/webpacker.yml
config/webpack/*
package.json
postcss.config.js

After you have performed the backup, make sure you run:

bundle update decidim
bin/rails decidim:upgrade
bin/rails db:migrate

Then run the below command, and replace all the configuration with the one that Decidim is providing by default:

bundle exec rake decidim:webpacker:install

Compiling the JS and CSS at the runtime is a very slow process. Additionally, compiling JS and CSS in the test suite will cause your pipeline to timeout.
To avoid that, you could export the following env variable:

export WEBPACKER_RUNTIME_COMPILE="false"

Then use the command to watch and compile the javascript and css upon changes:

bundle exec bin/webpacker-dev-server

♥️ Thank you!

[ahukkanen]

The issue with the test assets not compiling automatically can be solved by adding this to the default RSpec configuration:

    config.before(:all, type: :system) do
      Dir.chdir(Rails.root) { Webpacker.compile }
    end

(note: I understand this will only recompile if the assets have changed)

[ahukkanen]

OK, I think I've given it enough time, so here are the remaining issues from the previous comments I'd still like to get resolved before merging this:

• Lock both JS and Ruby shakapacker versions at 6.6.x to avoid breaking our pipelines (see comment)
• Run the decidim:procfile:install rake task during application generation (see comment)
• Chmod the bin/dev script correctly (see comment)
• Change the default for WEBPACKER_RUNTIME_COMPILE to false (see comment)
• Remove unnecessary lines from the map specs after the refactoring (see comment)
• Compile the assets automatically during tests if they haven't been compiled before (see comment)

And remaining issue after this which can be handled in a separate PR, find out a better way to handle the Tailwind configuration generation to avoid overriding the runner (see thread).

[alecslupu]

@ahukkanen , all the review recommendations have been applied.

[ahukkanen]

Tested both development app generation and test app generation and both work just fine now. The ./bin/dev script works on develop and during tests the assets were correctly compiled when they weren't available.

I have noticed few more issues after the latest changes, could you please have a look?

Also, could you revert the branch name changes that you have marked yourself. It should be good to go after these.

[ahukkanen]

Great work @alecslupu ! Works really nicely now.

This gives us good time to think where to go from here to improve the assets generation.