Webpacker can't find pack in test environment

[sdellis]

I'm getting this error on my CircleCI test server (but not when I run tests locally):

     ActionView::Template::Error:
       Webpacker can't find filemanager.js in /home/ubuntu/figgy/public/packs-test/manifest.json. Possible causes:
       1. You want to set webpacker.yml value of compile to true for your environment
          unless you are using the `webpack -w` or the webpack-dev-server.
       2. Webpack has not yet re-run to reflect updates.
       3. You have misconfigured Webpacker's config/webpacker.yml file.
       4. Your Webpack configuration is not creating a manifest.
       Your manifest contains:
       {
       }

I have the following set in my webpacker.yml:

  source_path: valhalla/app/javascript
  source_entry_path: packs
  public_output_path: packs
  cache_path: tmp/cache/webpacker

and

test:
  <<: *default
  compile: true

  # Compile test packs to a separate directory
  public_output_path: packs-test

The directory structure is a little unconventional as the Rails devs I'm working with are separating out potentially sharable pieces of the app via the valhalla directory. The structure looks like this:

figgy
 |--app
 |--public
 |... other stuff
 |--valhalla
      |--app
           |--javascript
                |--packs
                     |--filemanager.js

I am confused as to why I don't get this error in my local test environment and only on CircleCI.

[gauravtiwari]

@sdellis You would need to precompile assets for test environment before running circle ci (in one of the steps).

[gauravtiwari]

steps: 
  run: NODE_ENV=test bundle exec rails webpacker:compile

[sdellis]

Thanks for the help. When I do that I get this compilation error using webpack-dev-server 2.9.4 :

Using /home/ubuntu/figgy/config/webpacker.yml file for setting up webpack paths
Compiling…
Compilation failed:

Invalid configuration object. Webpack has been initialised using a configuration object that does not match the API schema.
 - configuration has an unknown property 'build'. These properties are valid:
   object { amd?, bail?, cache?, context?, dependencies?, devServer?, devtool?, entry, externals?, loader?, module?, name?, node?, output?, parallelism?, performance?, plugins?, profile?, recordsInputPath?, recordsOutputPath?, recordsPath?, resolve?, resolveLoader?, stats?, target?, watch?, watchOptions? }
   For typos: please correct them.
   For loader options: webpack 2 no longer allows custom properties in configuration.
     Loaders should be updated to allow passing options via loader options in module.rules.
     Until loaders are updated one can use the LoaderOptionsPlugin to pass these options to the loader:
     plugins: [
       new webpack.LoaderOptionsPlugin({
         // test: /\.xxx$/, // may apply this only for some modules
         options: {
           build: ...
         }
       })
     ]

I can try playing with different versions of webpack-dev-server to try to get this to work.

[gauravtiwari]

How does your webpack/test.js file looks like? It seems there is an unknown build option in the config.

configuration has an unknown property 'build'. These properties are valid:

[sdellis]

@gauravtiwari I was having trouble setting up my test environment for Jest, but I can't remember exactly what the issues were since it was a while ago. I remember that I had to customize the test environment following instructions here to get it working. However, I've recently upgraded Webpacker and now find that the defaults work. Closing out this non-issue. Thanks again for your help!

[codebycliff]

@gauravtiwari Sorry for blowing you up today. I have another question that I would like some clarification on. After making the following upgrades

Webpacker: 4.0.0.pre.pre.2
@rails/webpacker: 4.0.0.pre.pre.2
webpack-dev-server: 3.1.3

I started receiving the above error on CircleCI. I was able to fix it by adding a manual step to compile webpack like above (e.g. run: bundle exec rails webpacker:compile). However, it seemed strange that it worked before but now requires an explicit step. So I continued to look into it. I noticed that I was exporting NODE_ENV=test in my circleci config.yml. This would end up overriding the new code at the top of config/test.js (i.e. process.env.NODE_ENV = process.env.NODE_ENV || 'development'). After removing this environment variable from the config, the error went away. Is there something in the webpacker compile process that depends on the NODE_ENV equaling development? Is this the intended behavior?

[gauravtiwari]

@codebycliff In node world, test environment doesn't really mean much since most libraries out there are tailored to two environments - development or production.

For ex: React enables some optimizations if you set NODE_ENV to production.

In this PR #1359, I have made RAILS_ENV and NODE_ENV independent of each other meaning you can set RAILS_ENV anything you want and it won't affectNODE_ENV, which should either be production or development in most cases. You can use RAILS_ENV to load configuration from webpacker.yml.

RAILS_ENV=test bundle exec rails webpacker:compile

The above will load test env configuration from webpacker.yml but will compile packs in production mode. So you can remove any explicit NODE_ENV set on circleCI and just use RAILS_ENV to load configuration from webpacker.yml.

For example:

circle:
  <<: *default
  # Setting this compiles packs on the first request, I guess this is why you didn't have to compile before?
  compile: true

  # Compile test packs to a separate directory
  public_output_path: packs-circle

Does this make sense?

[codebycliff]

Yes it does. Thank you for your detailed response. Removing the NODE_ENV environment variable was enough for me. I can't even recall why it was added in the first place. Thanks again for your time.

[jsugarman]

Just to add that bundle exec rails webpacker:compile did not work for me but bundle exec rails assets:precompile did. This is with a vanilla rails 6.0.2.1 using webpack/webpacker/yarn.

[AxelTheGerman]

Thanks @jsugarman for the pointer. After running asset pre-compilation it still did not work for me, but pointed me (finally) to my root cause:

webpack binstubs not found.
Have you run rails webpacker:install ?
Make sure the bin directory or binstubs are not included in .gitignore

I have a global ~/.gitignore_global that ignores /bin and therefore ignored my webpack binstubs - I added them explicitly via git add -f but I assume you could also re-run rails webpacker:binstubs every time in CI (that's why my tests passed on local but not CI)

[dmolesUC]

What is the point of the compile: true directive for test and development in webpacker.yml, if you have to precompile anyway?

[AxelTheGerman]

@dmolesUC I did not need the pre-compilation but it showed me the error that the binstubs were not there. Adding the binstubs to git fixed it

[hebergentilin]

What is the point of the compile: true directive for test and development in webpacker.yml, if you have to precompile anyway?

It works like a charm, but this is painful when run the test, it waste lot of time.
I do prefer precompile all before