Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Stop bundling dependencies with slashes #1634

Merged
merged 1 commit into from
Jun 25, 2019

Conversation

Rob--W
Copy link
Member

@Rob--W Rob--W commented Jun 21, 2019

See commit message.

Fixes #1629

@Rob--W Rob--W requested a review from rpl June 21, 2019 00:39
External dependencies should not be bundled, because they are already
referenced via "dependencies" in package.json.

Before this patch, external modules with slashes in their identifier
were bundled anyway because there is no directory with that name in the
`node_modules` directory (which in turn prevents them from being added
to the `externals` object in the webpack config).

This is fixed by looking up dependencies in `package.json` instead.
@coveralls
Copy link

Coverage Status

Coverage remained the same at 100.0% when pulling 4854788 on Rob--W:fix-webpack-externals into f7abb75 on mozilla:master.

1 similar comment
@coveralls
Copy link

Coverage Status

Coverage remained the same at 100.0% when pulling 4854788 on Rob--W:fix-webpack-externals into f7abb75 on mozilla:master.

Copy link
Member

@rpl rpl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Rob--W

This looks good to me.

It would be great to also tweak how our "production mode" function tests run, so that we make sure that we trigger this kind of issue from the CI jobs.

I briefly took a look and the following changes seems to be enough to make TEST_PRODUCTION_MODE=1 npm run test-function to trigger the Cannot find module 'es6-promise' error without this fix (and to pass as expected with this fix applied):

diff --git a/scripts/lib/config.js b/scripts/lib/config.js
index 331262a..5c11b85 100644
--- a/scripts/lib/config.js
+++ b/scripts/lib/config.js
@@ -1,14 +1,13 @@
 module.exports = {
   clean: ['dist/*'],
   copy: {
-    productionModeArtifacts: {
+    productionModeAssets: {
       src: [
         'package.json',
         'package-lock.json',
         'dist/**',
         'bin/**',
       ],
-      dest: './artifacts/production',
     },
   },
   watch: {
diff --git a/scripts/test-functional b/scripts/test-functional
index 0fac1a2..f636f93 100755
--- a/scripts/test-functional
+++ b/scripts/test-functional
@@ -1,6 +1,9 @@
 #!/usr/bin/env node
 
+const path = require('path');
+
 const shell = require('shelljs');
+const tmp = require('tmp');
 
 const config = require('./lib/config');
 const {mochaFunctional} = require('./lib/mocha');
@@ -9,36 +12,32 @@ shell.set('-e');
 
 const testProductionMode = process.env.TEST_PRODUCTION_MODE === '1';
 
-let execBuildOptions = {};
 let execMochaOptions = {};
 
+shell.exec('npm run build', testProductionMode ? {
+  env: {
+    ...process.env,
+    NODE_ENV: 'production',
+  },
+} : {});
+
 if (testProductionMode) {
-  execBuildOptions = {
-    env: {
-      ...process.env,
-      NODE_ENV: 'production',
-    },
-  };
+  const destDir = tmp.tmpNameSync();
+  const packageDir = tmp.tmpNameSync();
 
   execMochaOptions = {
     env: {
       ...process.env,
-      TEST_WEB_EXT_BIN: './artifacts/production/bin/web-ext',
+      TEST_WEB_EXT_BIN: path.join(destDir, 'node_modules', '.bin', 'web-ext'),
     },
   };
-}
-
-shell.exec('npm run build', execBuildOptions);
 
-if (testProductionMode) {
   shell.echo('\nPreparing web-ext production mode environment...\n');
-  shell.rm('-rf', config.copy.productionModeArtifacts.dest);
-  shell.mkdir('-p', config.copy.productionModeArtifacts.dest);
-  shell.cp('-rf', config.copy.productionModeArtifacts.src,
-           config.copy.productionModeArtifacts.dest);
-
-  shell.pushd(config.copy.productionModeArtifacts.dest);
-  shell.exec('npm install --production');
+  shell.rm('-rf', destDir, packageDir);
+  shell.mkdir('-p', destDir, packageDir);
+  shell.cp('-rf', config.copy.productionModeAssets.src, packageDir);
+  shell.pushd(destDir);
+  shell.exec(`npm install --production --legacy-bundling ${packageDir}`);
   shell.popd();
   shell.echo('\nProduction mode environment successfully created.\n');
 }

@Rob--W
Copy link
Member Author

Rob--W commented Jun 25, 2019

--legacy-bundling is legacy behavior. To match production, we should try to run with the default / most common configuration as much as possible.

@rpl
Copy link
Member

rpl commented Jun 25, 2019

--legacy-bundling is legacy behavior. To match production, we should try to run with the default / most common configuration as much as possible.

I agree, but it does still concern me that we are not able to catch this kind of dependency loading issues.

I'm fine with proceeding with merging the fix as is, but it would be great (at least in a follow up) to look into trigger this kind of issue (that's the reason why we have a "test-functional running in production mode" as part of our CI jobs, "having to test this manually" kills its purpose) at least while there is still a way (e.g. using an additional npm option) to trigger it.

@Rob--W
Copy link
Member Author

Rob--W commented Jun 25, 2019

--legacy-bundling is legacy behavior. To match production, we should try to run with the default / most common configuration as much as possible.

I agree, but it does still concern me that we are not able to catch this kind of dependency loading issues.

My concern with using --legacy-bundling only is that we won't catch problems if we ever have an issue related to the non-legacy way of bundling dependencies.

I'm fine with proceeding with merging the fix as is, but it would be great (at least in a follow up) to look into trigger this kind of issue (that's the reason why we have a "test-functional running in production mode" as part of our CI jobs, "having to test this manually" kills its purpose) at least while there is still a way (e.g. using an additional npm option) to trigger it.

Could you open a new PR with your proposed patch (and try to address my review feedback if you agree that it's a potential issue)?

@rpl
Copy link
Member

rpl commented Jun 25, 2019

--legacy-bundling is legacy behavior. To match production, we should try to run with the default / most common configuration as much as possible.

I agree, but it does still concern me that we are not able to catch this kind of dependency loading issues.

My concern with using --legacy-bundling only is that we won't catch problems if we ever have an issue related to the non-legacy way of bundling dependencies.

I'm fine with proceeding with merging the fix as is, but it would be great (at least in a follow up) to look into trigger this kind of issue (that's the reason why we have a "test-functional running in production mode" as part of our CI jobs, "having to test this manually" kills its purpose) at least while there is still a way (e.g. using an additional npm option) to trigger it.

Could you open a new PR with your proposed patch (and try to address my review feedback if you agree that it's a potential issue)?

What I was proposing is to start from the set of changes linked in my previous comments to prepare an additional 'legacy-bundling production mode" test step in the CI job, but if you prefer I can take a look into it as part of a follow up.

@rpl rpl merged commit fd336bb into mozilla:master Jun 25, 2019
@rpl rpl added this to In progress in 3.1.0 via automation Jul 1, 2019
@rpl rpl moved this from In progress to Done in 3.1.0 Jul 1, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
3.1.0
  
Done
Development

Successfully merging this pull request may close these issues.

'run' command fails due to absent 'es6-promise'
3 participants