Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doesn't play nice with Webpack #93

Open
njlr opened this issue Dec 6, 2019 · 60 comments
Open

Doesn't play nice with Webpack #93

njlr opened this issue Dec 6, 2019 · 60 comments

Comments

@njlr
Copy link

njlr commented Dec 6, 2019

Edit by @berstend:

see here for the workaround: webpack/webpack#4175 (comment) and another one specific to the stealth plugin: #93 (comment)

Original issue:

When bundling I get this error:

WARNING in ./node_modules/puppeteer-extra/dist/index.esm.js 294:22-35
Critical dependency: the request of a dependency is an expression
@ ./src/processor.js
@ ./src/index.js
@ multi @babel/polyfill ./src/index.js

And then at run-time:

A plugin listed 'puppeteer-extra-plugin-stealth/evasions/chrome.runtime' as dependency,
which is currently missing. Please install it:

      yarn add puppeteer-extra-plugin-stealth

      Note: You don't need to require the plugin yourself,
      unless you want to modify it's default settings.

Error: Cannot find module 'puppeteer-extra-plugin-stealth/evasions/chrome.runtime'

Of course, puppeteer-extra-plugin-stealth is already in the package.json.

@njlr
Copy link
Author

njlr commented Dec 9, 2019

Work-around is to import and apply the plugins manually:

import puppeteerVanilla from 'puppeteer';
import { addExtra } from 'puppeteer-extra';
import StealthPlugin from 'puppeteer-extra-plugin-stealth';
import AcceptLanguagePlugin from 'puppeteer-extra-plugin-stealth/evasions/accept-language';
import ChromeRuntimePlugin from 'puppeteer-extra-plugin-stealth/evasions/chrome.runtime';
import ConsoleDebugPlugin from 'puppeteer-extra-plugin-stealth/evasions/console.debug';
import IFrameContentWindowPlugin from 'puppeteer-extra-plugin-stealth/evasions/iframe.contentWindow';
import MediaCodecsPlugin from 'puppeteer-extra-plugin-stealth/evasions/media.codecs';
import NavigatorLanguagesPlugin from 'puppeteer-extra-plugin-stealth/evasions/navigator.languages';
import NavigatorPermissionsPlugin from 'puppeteer-extra-plugin-stealth/evasions/navigator.permissions';
import NavigatorPlugins from 'puppeteer-extra-plugin-stealth/evasions/navigator.plugins';
import WebdriverPlugin from 'puppeteer-extra-plugin-stealth/evasions/navigator.webdriver';
import UserAgentPlugin from 'puppeteer-extra-plugin-stealth/evasions/user-agent';
import WebglVendorPlugin from 'puppeteer-extra-plugin-stealth/evasions/webgl.vendor';
import WindowOuterDimensionsPlugin from 'puppeteer-extra-plugin-stealth/evasions/window.outerdimensions';

async () => {
  const puppeteer = addExtra(puppeteerVanilla);

  const plugins = [
    StealthPlugin(),
    AcceptLanguagePlugin(),
    ChromeRuntimePlugin(),
    ConsoleDebugPlugin(),
    IFrameContentWindowPlugin(),
    MediaCodecsPlugin(),
    NavigatorLanguagesPlugin(),
    NavigatorPermissionsPlugin(),
    NavigatorPlugins(),
    WebdriverPlugin(),
    UserAgentPlugin(),
    WebglVendorPlugin(),
    WindowOuterDimensionsPlugin(),
  ];

  const browser = await puppeteer.launch();

  for (const plugin of plugins) {
    await plugin.onBrowser(browser);
  }

  const [ page ] = await browser.pages();

  for (const plugin of plugins) {
    await plugin.onPageCreated(page);
  }

  // ...
};

@berstend
Copy link
Owner

Hmm, interesting. May I ask why you opted to use Webpack for a NodeJS project?

If you could provide a minimal webpack based project with that issue that'd be great, as I then can take a quick look and see how to best fix this. :)

@njlr
Copy link
Author

njlr commented Dec 10, 2019

I use Webpack with Node because it's a simpler way to use Babel, bundle node_modules and minify code.

webpack.config.js:

const path = require('path');
const TerserPlugin = require('terser-webpack-plugin');

const { env } = process;

const isProduction = env['NODE_ENV'] === 'production';

const mode = isProduction ?
  'production' :
  'development';

console.log({ mode });

module.exports = {
  entry: [
    '@babel/polyfill',
    './src/index.js',
  ],
  target: 'node',
  devtool: isProduction ? false : 'source-map',
  mode,
  output: {
    path: path.join(__dirname, 'build'),
    filename: 'index.js'
  },
  module: {
    rules: [
      {
        test: /\.m?js$/,
        exclude: /(node_modules|bower_components)/,
        use: {
          loader: 'babel-loader',
          options: {
            babelrc: true,
          },
        },
      },
      {
        test: /\.js$/,
        loader: 'unlazy-loader'
      }
    ],
  },
  optimization: {
    minimize: isProduction,
    minimizer: [
       new TerserPlugin(),
    ],
  },
  resolve: {
    alias: {
      'pg-native': path.resolve(__dirname, 'aliases/pg-native'),
    },
  },
};

@berstend
Copy link
Owner

Thanks for providing the config, I'll look into it when I find time.

I use Webpack with Node because it's a simpler way to use Babel, bundle node_modules and minify code.

So it's to use new ES language features? Bundling and minifying shouldn't matter on backend code (hence my question). I personally used Babel with NodeJS back in the day to be able to use ES6 imports but eventually stopped doing that as I noticed issues with that and figured it's not worth it and causes more problems than it does good :)

@njlr
Copy link
Author

njlr commented Dec 10, 2019

Yes I use the latest ES6 features.

I also use loaders for other languages (such as Fable) and data (such as JSON, CSS).

Bundling everything can also be useful for targets like AWS lambda.

@njlr
Copy link
Author

njlr commented Dec 10, 2019

I think as rule-of-thumb, only using ES6 import / export will always give a library that works with Webpack, Rollup, etc.

@berstend
Copy link
Owner

berstend commented Dec 10, 2019

Got it :) Anyway, puppeteer-extra should still be able to work with Webpack. It might be that the internal dependency system is just not aware of the bundler already taking care of the dependencies and a flag to disable the internal dependency resolution is sufficient.

@berstend berstend self-assigned this Dec 10, 2019
@berstend berstend added the work-in-progress This is currently being worked on label Dec 10, 2019
@berstend
Copy link
Owner

berstend commented Dec 10, 2019

Couple of avenues I'll explore to fix this:

  • Add options to puppeteer-extra with something like disableInternalDependencyResolution: true
  • Add support for an ENV variable to detect the presence of bundlers
  • Modify the esm rollup build to disable the internal dependency management (not sure about that, as regular NodeJS will soon(?) use esm exports as well?)
  • Make the dependency thing a warning rather than an error (would still spam log output during webpack build but that's ok I guess)

@njlr
Copy link
Author

njlr commented Dec 10, 2019

What is the purpose of the internal dependency management?

Could a simpler design work like this?

import puppeteerExtra from 'puppeteer-extra';
import MyPlugin from 'my-plugin';

const puppeteer = pupeteerExtra({
  plugins: [
    MyPlugin(),
  ],
});

async () => {
  const browser = await puppeteer.launch();

  // etc...
};

@berstend
Copy link
Owner

berstend commented Dec 10, 2019

The above already exists (with puppeteer.use()) :)

The idea behind the dependency plugin system was to make it easy to re-use plugins within plugins. E.g. the stealth plugin needs to anonymize the UA and instead of copy pasting that code we just load the anonymize-ua plugin internally as a dependency.

The code for that is here: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra/src/index.ts#L326-L334

Thinking of it now (I wrote this in the very first version) I think we don't need to handle that within puppeteer-extra, but can just use the native package.json methods to declare them.

What I currently don't understand: This should still work with Webpack regardless, as they should transform the require() statements to point at bundled resources instead. I need to take a closer look to see what's going on there.

@berstend
Copy link
Owner

So yeah, it's just webpack being bad at dynamic imports.

This fix should work:
webpack/webpack#4175 (comment)

@berstend
Copy link
Owner

So basically adding this to the webpack module.rules (untested):

{
  // regex for the files that are problematic
  test: \.\/node_modules\/puppeteer-extra\/dist\/index\.esm\.js,
  loader: 'string-replace-loader',
  options: {
    // match a require function call where the argument isn't a string
    // also capture the first character of the args so we can ignore it later
    search: 'require[(]([^\'"])',
    // replace the 'require(' with a '__non_webpack_require__(', meaning it will require the files at runtime
    // $1 grabs the first capture group from the regex, the one character we matched and don't want to lose
    replace: '__non_webpack_require__($1',
    flags: 'g'
  }
}

also you will need to install string-replace-loader: https://github.com/Va1/string-replace-loader

@njlr
Copy link
Author

njlr commented Dec 11, 2019

The above already exists (with puppeteer.use()) :)

I meant this usage but with internal dependency management removed entirely.

The idea behind the dependency plugin system was to make it easy to re-use plugins within plugins. E.g. the stealth plugin needs to anonymize the UA and instead of copy pasting that code we just load the anonymize-ua plugin internally as a dependency.

The code for that is here: https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra/src/index.ts#L326-L334

Thinking of it now (I wrote this in the very first version) I think we don't need to handle that within puppeteer-extra, but can just use the native package.json methods to declare them.

So the plugins would declare the other plugins that they depend on using package.json dependencies? Sounds like the best solution to me.

What I currently don't understand: This should still work with Webpack regardless, as they should transform the require() statements to point at bundled resources instead. I need to take a closer look to see what's going on there.

string-replace-loader looks like a good work-around. Thanks!

@berstend
Copy link
Owner

string-replace-loader looks like a good work-around. Thanks!

Would be great to hear if it fixes the webpack issue, then I can add it to the documentation :)

So the plugins would declare the other plugins that they depend on using package.json dependencies? Sounds like the best solution to me.

Unfortunately that won't fix the issue with dynamic imports + webpack, we still need to do dynamic require() under the hood (which works fine, just not with webpack). :)

@njlr
Copy link
Author

njlr commented Dec 11, 2019

we still need to do dynamic require() under the hood

Why is this?

@berstend
Copy link
Owner

we still need to do dynamic require() under the hood

Why is this?

It's always good to question assumptions, I'm a big believer in that :)

Let's take the stealth plugin as an example: it comes with a set of "evasions" (which are just regular plugins) and acts as an "umbrella" plugin, so the user doesn't need to add the specific evasions one-by-one.

One feature is that the user can add or remove evasions on that list, before puppeteer-extra will require these files (and thereby code mods).

I'm not aware of a way to accomplish that without dynamic require(). :)

@berstend
Copy link
Owner

Assuming the fix mentioned in #93 (comment) works I'm gonna demote this issue to "nice to have in a future version". :)

@berstend berstend added idea workaround-available and removed work-in-progress This is currently being worked on labels Dec 14, 2019
@njlr
Copy link
Author

njlr commented Dec 15, 2019

we still need to do dynamic require() under the hood

Why is this?

It's always good to question assumptions, I'm a big believer in that :)

Let's take the stealth plugin as an example: it comes with a set of "evasions" (which are just regular plugins) and acts as an "umbrella" plugin, so the user doesn't need to add the specific evasions one-by-one.

One feature is that the user can add or remove evasions on that list, before puppeteer-extra will require these files (and thereby code mods).

I'm not aware of a way to accomplish that without dynamic require(). :)

The stealth plugin could require all evasions statically. This would cover most use-cases.

Users who need something more custom could manually import the ones that they need. This wouldn't be much code and the full import list could be copied from the stealth plugin then edited.

@ioannist
Copy link

I got this working locally but when I used webpack to bundle it and send it over to aws lambda, this line

StealthPlugin();

results to the error below. Adding "kind-of": "^6.0.2" to the project's package.json does not resolve the problem.

Error: Cannot find module 'kind-of'
at a (/var/task/index.js:145:1835)
at Function.o [as typeOf] (/var/task/index.js:145:1440)
at i (/var/task/index.js:145:854)
at e.exports (/var/task/index.js:145:371)
at new n (/var/task/index.js:139:191)
at new i (/var/task/index.js:133:12241)
at e.exports (/var/task/index.js:133:12910)
at Object.startBrowser (/var/task/index.js:127:99655)
at Runtime.t.handler (/var/task/index.js:127:86330)
at processTicksAndRejections (internal/process/task_queues.js:93:5) {
code: 'MODULE_NOT_FOUND'
}

@levz0r
Copy link

levz0r commented Dec 21, 2019

I got this working locally but when I used webpack to bundle it and send it over to aws lambda, this line

StealthPlugin();

results to the error below. Adding "kind-of": "^6.0.2" to the project's package.json does not resolve the problem.

Error: Cannot find module 'kind-of'
at a (/var/task/index.js:145:1835)
at Function.o [as typeOf] (/var/task/index.js:145:1440)
at i (/var/task/index.js:145:854)
at e.exports (/var/task/index.js:145:371)
at new n (/var/task/index.js:139:191)
at new i (/var/task/index.js:133:12241)
at e.exports (/var/task/index.js:133:12910)
at Object.startBrowser (/var/task/index.js:127:99655)
at Runtime.t.handler (/var/task/index.js:127:86330)
at processTicksAndRejections (internal/process/task_queues.js:93:5) {
code: 'MODULE_NOT_FOUND'
}

I had the same issue. unlazy-loader solved this for me...

@berstend
Copy link
Owner

berstend commented Jan 6, 2020

@levz0r Interesting. Would you mind providing a full example using unlazy-loader for others? :)

@levz0r
Copy link

levz0r commented Jan 17, 2020

@levz0r Interesting. Would you mind providing a full example using unlazy-loader for others? :)

Hey, sorry for the long delay...

Nothing special actually... Just add

rules: [
      {
        test: /\.js$/,
        use: "unlazy-loader"
      }
    ]

To webpack.config.js. That's it.

Hope it helps.

@SaajidJoosab
Copy link

SaajidJoosab commented May 1, 2020

I'm getting a similar problem to this with serverless-bundle. Any ideas how to work around this with serverless-bundle?

@Seblor
Copy link

Seblor commented Dec 3, 2020

@berstend None of the solutions above worked for me. Do you have any update on the rework ?

@njlr
Copy link
Author

njlr commented Dec 3, 2020

@berstend None of the solutions above worked for me. Do you have any update on the rework ?

Did you try #93 (comment) ? It is highly likely to work because it is pure ES6 with static imports.

@berstend
Copy link
Owner

berstend commented Dec 3, 2020

thanks @njlr pretty sure your workaround would still work. @Seblor just verify that you cover all evasion imports (check the index.js file in the stealth repo for all currently loaded evasions).

@Seblor
Copy link

Seblor commented Dec 3, 2020

After many hours, I managed to make it work, albeit only halfway.

I had to use @njlr's linked solution in addition to unlazy-loader.

However even though it works nicely on bot.sannysoft.com (it even has one more passing criteria that my browser), it still gets detected by reCaptcha. Possibly something to do with it being in an iframe ?

Interestingly enough, removing puppeteer-extra (and all plugins) and simply adding the --disable-blink-features=AutomationControlled flag (and no other "stealth" mechanisms) is enough to fool reCaptcha. Which is weird since I saw in an other issue that this flag was supposed to be included in the puppeteer-extra stealth plugin.

Thank you for the quick replies, though.

@berstend
Copy link
Owner

berstend commented Dec 3, 2020

@Seblor this is out of the scope of this webpack issue but the iframe.contentWindow evasion is known to cause issues, more context and workaround here: #137 (comment)

Which is weird since I saw in an other issue that this flag was supposed to be included in the puppeteer-extra stealth plugin.

It is, as you can see in the code of the navigator.webdriver evasion. Make sure you use the latest versions of everything.

@danniel-isiah-libor
Copy link

@berstend I am getting the error:

Error: Cannot find module 'kind-of'
    at webpackEmptyContext (D:\node_projects\vueapp\dist_electron\index.js:34338:10)
    at Function.getter [as typeOf] (D:\node_projects\vueapp\dist_electron\index.js:91020:29)
    at cloneDeep (D:\node_projects\vueapp\dist_electron\index.js:34370:17)
    at mergeDeep (D:\node_projects\vueapp\dist_electron\index.js:100090:16)
    at new PuppeteerExtraPlugin (D:\node_projects\vueapp\dist_electron\index.js:159981:22)
    at new StealthPlugin (D:\node_projects\vueapp\dist_electron\index.js:159809:5)
    at defaultExport (D:\node_projects\vueapp\dist_electron\index.js:159905:31)
    at Module../src/main/gen.js (D:\node_projects\vueapp\dist_electron\index.js:221678:15)
    at __webpack_require__ (D:\node_projects\vueapp\dist_electron\index.js:20:30)
    at Module../src/main/main.js (D:\node_projects\vueapp\dist_electron\index.js:221764:62)

This is my vue.config.js

module.exports = {
    configureWebpack: {
        // Configuration applied to all builds
        target: "electron-renderer",
        devtool: 'source-map',
        module:{
            rules:[
                {
                    test: /\.js$/,
                    use: "unlazy-loader"
                },
                {
                    // regex for the files that are problematic
                    test: /node_modules\/puppeteer-extra\/dist\/index\.esm\.js/,
                    loader: 'string-replace-loader',
                    options: {
                        // match a require function call where the argument isn't a string
                        // also capture the first character of the args so we can ignore it later
                        search: 'require[(]([^\'"])',
                        // replace the 'require(' with a '__non_webpack_require__(', meaning it will require the files at runtime
                        // $1 grabs the first capture group from the regex, the one character we matched and don't want to lose
                        replace: '__non_webpack_require__($1',
                        flags: 'g'
                    }
                }
            ]
        }
    },
    pluginOptions: {
        electronBuilder: {
            builderOptions: {
                // options placed here will be merged with default configuration and passed to electron-builder
            }
        }
    }
}

babel.config.js:

module.exports = {
  presets: [
    '@vue/cli-plugin-babel/preset'
  ]
}

This only happens if i use StealthPlugin() code

I'm also getting this error. I'm also using vue webpack with this the same configs. StealthPlugin() is not working for me. I need help TIA

@danniel-isiah-libor
Copy link

Error: Cannot find module 'kind-of'
    at webpackEmptyContext (webpack:///./node_modules/merge-deep/node_modules/clone-deep_sync?:2:10)
    at Function.getter [as typeOf] (webpack:///./node_modules/merge-deep/node_modules/clone-deep/node_modules/lazy-cache/index.js?:39:29)
    at cloneDeep (webpack:///./node_modules/merge-deep/node_modules/clone-deep/index.js?:14:17)
    at mergeDeep (webpack:///./node_modules/merge-deep/index.js?:19:16)
    at new PuppeteerExtraPlugin (webpack:///./node_modules/puppeteer-extra-plugin/dist/index.esm.js?:59:22)     
    at new StealthPlugin (webpack:///./node_modules/puppeteer-extra-plugin-stealth/index.js?:74:5)
    at defaultExport (webpack:///./node_modules/puppeteer-extra-plugin-stealth/index.js?:171:31)
    at eval (webpack:///./src/background.js?:452:103)
    at electron/js2c/browser_init.js:209:558 {
  code: 'MODULE_NOT_FOUND'
}

vue.config

module.exports = {
  transpileDependencies: [
    'vuetify'
  ],
  pluginOptions: {
    electronBuilder: {
      externals: [
        'puppeteer',
        'puppeteer-extra',
        'puppeteer-extra-plugin-stealth'
      ],
      nodeModulesPath: ['../../node_modules', './node_modules'],
      nodeIntegration: true
    }
  },
  configureWebpack: {
    externals: {
      puppeteer: "require('puppeteer')",
      'puppeteer-extra': "require('puppeteer-extra')",
      'puppeteer-extra-plugin-stealth': "require('puppeteer-extra-plugin-stealth')"
    }
  }
}

Im using VueJS and ElectronJS for my project, this config of mine is working in dev but once I build the project to prod I receive that error above. Any Idea? thanks in advance!

By the way Im using this vue-cli-plugin-electron-builder

@berstend
Copy link
Owner

berstend commented Mar 6, 2021

@dannielibor well, depending on your level of dedication you could manually import the stealth evasions as mentioned higher up :-)

I need help TIA

A transient ischemic attack (TIA) is a temporary period of symptoms similar to those of a stroke.
🤔

@danniel-isiah-libor
Copy link

Hi @berstend thanks for your response, but I already tried all that even applying those webpack configs mentioned above.

import puppeteerVanilla from 'puppeteer'
import { addExtra } from 'puppeteer-extra'
import StealthPlugin from 'puppeteer-extra-plugin-stealth'

    const puppeteer = addExtra(puppeteerVanilla)

    const plugins = [
      StealthPlugin()
    ]

    const browser = await puppeteer.launch()

    for (const plugin of plugins) {
      await plugin.onBrowser(browser)
    }

    const [page] = await browser.pages()

    for (const plugin of plugins) {
      await plugin.onPageCreated(page)
    }

@danniel-isiah-libor
Copy link

By the way, TIA = thanks in advance :D

@danniel-isiah-libor
Copy link

I can confirm that it is working fine in a simple node project but after integrating it to my vue-electron project its not working

@gtsigner
Copy link

"peerDependencies": {
"puppeteer-extra-plugin": "*",
"merge-deep": "github:godtoy/merge-deep#master"
},
image

it works now

@gtsigner
Copy link

just upgrade clone-deep version

@frontendphil
Copy link

💯 what @godtoy said. I've fiddled with this just to also get to the point that the 0.2.4 version of clone-deep uses a require syntax that webpack doesn't like. I've added a resolution to 4.0.1 and everything works just fine.

@kurohoan
Copy link

As @godtoy says, I updated clone-deep to 4.0.1, but I'm getting the same error.

I also put the following in webpack.json, but it didn't change. What should I do?

"peerDependencies":{
      "puppeteer-extra-plugin": "*",
      "merge-deep": "github:godtoy / merge-deep #master"
  },
```

@kurohoan
Copy link

@godtoy
I forgot to mention that I'm building Electron with Vue CLI.
I can't get it to compile like you are saying, any solutions?

@danniel-isiah-libor
Copy link

danniel-isiah-libor commented Jun 29, 2021

Hi guys, I made it working. Want to share what I got. Hope it works to y'all too.

Inside vue.config.js:

var webpack = require('webpack')

module.exports = {
  transpileDependencies: [
    'vuetify'
  ],
  pluginOptions: {
    electronBuilder: {
      externals: [
        'puppeteer',
        'puppeteer-extra',
        'puppeteer-extra-plugin-stealth'
      ],
      nodeModulesPath: ['../../node_modules', './node_modules'],
      nodeIntegration: true,
      builderOptions: {
        asarUnpack: 'node_modules/puppeteer/.local-chromium/**/*',
        copyright: 'Copyright © 2021',
        nsis: {
          oneClick: false,
          allowToChangeInstallationDirectory: true
        },
        publish: [
          {
            provider: '<your provider>',
            repo: '<your repo here>',
            owner: '<owner>',
            token: '<token here>',
            releaseType: 'draft',
            publishAutoUpdate: true,
            private: true
          }
        ]
      }
    }
  },
  configureWebpack: {
    externals: {
      puppeteer: "require('puppeteer')",
      'puppeteer-extra': "require('puppeteer-extra')",
      'puppeteer-extra-plugin-stealth': "require('puppeteer-extra-plugin-stealth')"
    }
  }
}

Basically you have to expose these packages and include those inside devDependencies and dependencies to make it working.

@kurohoan
Copy link

@dannielibor
It works now thanks to you!

@loopiezlol
Copy link

anyone got the stealth plugin to work with serverless-bundle?

tried most suggestions in this thread, but none seem to work. I've yet to look into unlazy-loader as serverless-bundle uses an internal, default webpack config which I'd rather not override/mess with

@JasonHoku
Copy link

Hello!

If you're using webpack and electron I found a fix for this by making sure to include any merge-deep dependencies as an external and to declare the node_modules path relative to your package.json in your package.json as an 'extraResource'.

@kaptinlin
Copy link

@JasonHoku can you give a example package.json?

@JasonHoku
Copy link

@JasonHoku can you give a example package.json?

I've trimmed it to relevancy for you here:

However I now use a script to automate this and have made it much easier to pack and update multiple modules.

update and pack pup

import fs from 'fs';
import chalk from 'chalk';
import path from 'path';

const fse = require('fs-extra');

// get chromium rev info https://commondatastorage.googleapis.com/chromium-browser-snapshots/

console.log(
  `${chalk.whiteBright.bgYellow.bold('Updating Puppeteer Script WIP Check Notes In update-puppeteer-deps .')}`
);

try {
  // fse.copySync('./resources/puppeteer-core/', './node_modules/puppeteer-core/', { overwrite: true });
  console.log(
    `${chalk.whiteBright.bgGreen.bold(
      'Manually run npm i in resources/pup-updater and run the post-install pup core patch!'
    )}`
  );
  console.log(`${chalk.whiteBright.bgGreen.bold('Manually update resources/puppeteer-core next')}`);
  console.log(
    `${chalk.whiteBright.bgGreen.bold(
      'Manually replace "fs/promises") with fs").promises (4 occurances as of Feb 28 2023 2 double and 2 single quoted)to convert to Electron8 '
    )}`
  );
  console.log(
    `${chalk.whiteBright.bgGreen.bold('Manually run yarn build Then build-dll And then electron-rebuild Lastly')}`
  );
} catch (err) {
  console.error(err);
}

//  Electron 8 package.json extraResources param to pack modules to custom directory:
// {
//   "from": "resources/node_modules",
//   "to": "node_modules"
// }

Package.json ex

{
  "win": {
    "extraResources": [
      "content.zip",
      "app/node_modules/",
      "node_modules/crypto-js",
      
      // new method
        {
          "from": "resources/node_modules",
          "to": "node_modules"
        }
    ]
  }
}

ivan added a commit to ludios/scrape-things that referenced this issue Jun 30, 2023
…really bundler-compatible

Some people had success with webpack in berstend/puppeteer-extra#93 but webpack is pretty complicated
@smn729
Copy link

smn729 commented Mar 7, 2024

I use (electron-forge+webpack+typescript+puppeteer-extra) got this error:

App threw an error during load
Error: Cannot find module 'is-plain-object'
    at webpackEmptyContext (/Path/To/Your/Project/electron-webpack/.webpack/main/index.js:17478:10)

I finally got it work, here is what I did: add "externals" in webpack.main.config.ts

export const mainConfig: Configuration = {
   .....
   .....
  externals: {
    "puppeteer": "puppeteer",
    "puppeteer-extra": "puppeteer-extra",
    "puppeteer-extra-plugin-stealth": "puppeteer-extra-plugin-stealth",
  }
};

@sivertheisholt
Copy link

I use (electron-forge+webpack+typescript+puppeteer-extra) got this error:

App threw an error during load
Error: Cannot find module 'is-plain-object'
    at webpackEmptyContext (/Path/To/Your/Project/electron-webpack/.webpack/main/index.js:17478:10)

I finally got it work, here is what I did: add "externals" in webpack.main.config.ts

export const mainConfig: Configuration = {
   .....
   .....
  externals: {
    "puppeteer": "puppeteer",
    "puppeteer-extra": "puppeteer-extra",
    "puppeteer-extra-plugin-stealth": "puppeteer-extra-plugin-stealth",
  }
};

I also use electron-forge+webpack+typescript+puppeteer-extra. This worked, thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests