[Bug] TypeError: utils.forOwn is not a function #690

LeiG · 2022-07-30T07:07:12Z

Hi, I'm running into this strange bug when using puppeteer-extra, puppeteer-extra-plugin-stealth with Typescript and the Serverless Framework. I will describe my setup and bug encountered below. I've tried a few things such as changing from puppeteer-core to puppeteer, but didn't have any luck. I'm not sure if I missed anything obvious. Thanks for the help in advance!

The full error message is below. The error throws when calling puppeteerExtra.use(StealthPlugin());. I tried other plugins such as puppeteer-extra-plugin-adblocker and the error persists so I don't think it's the issue with a specific plugin.

✖ utils.forOwn is not a function
✖ TypeError: utils.forOwn is not a function
      at cloneObjectDeep (/Users/leigong/git/friday/lambda/node_modules/merge-deep/node_modules/clone-deep/index.js:27:11)
      at cloneDeep (/Users/leigong/git/friday/lambda/node_modules/merge-deep/node_modules/clone-deep/index.js:16:14)
      at mergeDeep (/Users/leigong/git/friday/lambda/node_modules/merge-deep/index.js:19:16)
      at new PuppeteerExtraPlugin (/Users/leigong/git/friday/lambda/node_modules/puppeteer-extra-plugin/src/index.ts:77:18)
      at new StealthPlugin2 (/Users/leigong/git/friday/lambda/node_modules/puppeteer-extra-plugin-stealth/index.js:74:5)
      at defaultExport (/Users/leigong/git/friday/lambda/node_modules/puppeteer-extra-plugin-stealth/index.js:172:31)
      at Object.<anonymous> (/Users/leigong/git/friday/lambda/src/functions/hermes/handler.ts:13:20)
      at Module._compile (node:internal/modules/cjs/loader:1105:14)
      at Module._extensions..js (node:internal/modules/cjs/loader:1159:10)
      at Object.require.extensions.<computed> [as .js] (/Users/leigong/git/friday/lambda/node_modules/ts-node/src/index.ts:1587:43)
      at Module.load (node:internal/modules/cjs/loader:981:32)
      at Function.Module._load (node:internal/modules/cjs/loader:822:12)
      at Module.require (node:internal/modules/cjs/loader:1005:19)
      at require (node:internal/modules/cjs/helpers:102:18)
      at /Users/leigong/git/friday/lambda/node_modules/serverless-offline/dist/lambda/handler-runner/in-process-runner/InProcessRunner.js:166:133
      at processTicksAndRejections (node:internal/process/task_queues:96:5)
      at async InProcessRunner.run (/Users/leigong/git/friday/lambda/node_modules/serverless-offline/dist/lambda/handler-runner/in-process-runner/InProcessRunner.js:166:9)

Here is a simplified version of the code snippet in .ts

import { Browser } from 'puppeteer';
import { addExtra } from 'puppeteer-extra';
import StealthPlugin from 'puppeteer-extra-plugin-stealth';
import chromium from 'chrome-aws-lambda';

const puppeteerExtra = addExtra(chromium.puppeteer as any);
puppeteerExtra.use(StealthPlugin());

browser = await puppeteerExtra.launch({
            args: chromium.args,
            defaultViewport: chromium.defaultViewport,
            executablePath: await chromium.executablePath,
            headless: chromium.headless,
            ignoreHTTPSErrors: true,
        })

        const page = await browser.newPage();
        // other scraping logics

The relevant version of the libraries are below

"puppeteer": "^10.1.0",
"puppeteer-extra": "^3.3.4",
"puppeteer-extra-plugin-adblocker": "^2.13.4",
"puppeteer-extra-plugin-stealth": "^2.11.0",

The text was updated successfully, but these errors were encountered:

zkschmitz · 2022-07-31T15:53:08Z

+1 (same problem)

LeiG · 2022-08-01T08:30:18Z

Not sure if this is the root cause - the "clone-deep" package in one of the dependencies is at a pretty old version 0.2.4 and the error seems to be pointing to this line here https://github.com/jonschlinkert/clone-deep/blob/8ab50fb93db4eb9288a6a9b6042dbe3e9a1ac7fb/utils.js#L14, which looks like should be require('for-own', 'forOwn');?

As a js n00b, I'm not sure how to fix or if it's something Typescript related.

cdotte · 2022-09-20T00:19:20Z

(same problem) any update?

Randdalf · 2022-10-25T16:37:15Z

I worked around this by overriding the version of "clone-deep" in my package.json

  "overrides": {
    "clone-deep": "^4.0.1"
  }

ledesmablt · 2022-10-29T11:32:18Z

^^ The above fix works for me, though had to delete node_modules & package-lock.json

gwak2837 · 2022-12-17T13:54:03Z

I worked around this by overriding the version of "clone-deep" in my package.json when using yarn berry

{
  "resolutions": {
    "clone-deep": "^4.0.1"
  }
}

dbjpanda · 2023-07-21T15:27:39Z

+1 Need a fix here

AdrKacz · 2023-09-25T13:51:01Z

Adding "clone-deep": "^4.0.1" to my resolution didn't fix the issue on my side, anyone to help?

I am using yarn with workspaces, I added the resolutions, deleted the node_modules and yarn.lock, but I still have the error TypeError: utils.forOwn is not a function

jeanhdev · 2023-09-25T13:56:05Z

So I fixed this issue - what you need to do is force the resolutions of the clone-deep and merge-deep latest versions cc @AdrKacz like below in your package.json

"resolutions": { "merge-deep": "^3.0.3", "clone-deep": "^4.0.1" }

However, I'm still getting the error below;
A plugin listed 'puppeteer-extra-plugin-stealth/evasions/chrome.app' as dependency, which is currently missing.

If you have any idea on how to solve my issue, I'd gladly take your recommendation. Cheers

AdrKacz · 2023-09-25T14:07:11Z

Thank you for your help @jeanhdev, unfortunately using "resolutions": { "merge-deep": "^3.0.3", "clone-deep": "^4.0.1" } I still have the exact same error.

Probably has an impact, I have to use puppeteer-core version 19.4.0 (so not the latest).

Here is what in my yarn.lock

clone-deep@^0.2.4, clone-deep@^4.0.1:
  version "4.0.1"
  resolved "https://registry.yarnpkg.com/clone-deep/-/clone-deep-4.0.1.tgz#c19fd9bdbbf85942b4fd979c84dcf7d5f07c2387"
  integrity sha512-neHB9xuzh/wk0dIHweyAXv2aPGZIVk3pLMe+/RNzINf17fe0OG96QroktYAUm7SM1PBnzTabaLboqqxDyMU+SQ==
  dependencies:
    is-plain-object "^2.0.4"
    kind-of "^6.0.2"
    shallow-clone "^3.0.0"
    
...

merge-deep@^3.0.1, merge-deep@^3.0.3:
  version "3.0.3"
  resolved "https://registry.yarnpkg.com/merge-deep/-/merge-deep-3.0.3.tgz#1a2b2ae926da8b2ae93a0ac15d90cd1922766003"
  integrity sha512-qtmzAS6t6grwEkNrunqTBdn0qKwFgNWvlxUbAV8es9M7Ot1EbyApytCnvE0jALPa46ZpKDUo527kKiaWplmlFA==
  dependencies:
    arr-union "^3.1.0"
    clone-deep "^0.2.4"
    kind-of "^3.0.2"

sh1kxrv · 2023-10-19T20:22:42Z

just add to your package.json in workspace repository

  "dependencies": {,
      "puppeteer": "^10.1.0",
      "puppeteer-extra": "^3.3.4",
      "puppeteer-extra-plugin-adblocker": "^2.13.4",
      "puppeteer-extra-plugin-stealth": "^2.11.0",
  },

shakkky · 2023-11-24T23:44:13Z

hey @AdrKacz, did it manage to figure it out? I'm running into the same issue. Also using puppeteer-core

AdrKacz · 2023-11-25T09:04:04Z

No, I had to build my solution without puppeteer-extra unfortunately 😞

Juaoie · 2024-05-22T10:11:38Z

如果你使用的是yarn，你在yarn.lock中就可以找到，是哪个包依赖了clone-deep，我这边显示的是merge-deep，你也可以通过查看源码看到依赖关系，这里依赖的clone-deep是0.2.4版本，你要修复这个bug，就只用把clone-deep升级就可以了，而不是安装两个clone-deep版本

能理解我意思吗？

最后分析一下产生这个问题的原因，我自己尝试使用官方的demo去执行，发现不会这个bug，但是只要上ts就会有问题，很明显是使用node执行的js代码有问题，也就是ts打包以后有问题，我这里项目使用esbuild打包的，和tsc、webpack打包同理，也可以直接在源码中找到其中clone-deep@0.2.4引用forOwn有问题，这里commonjs 导入编译成module导入是有问题的，所以会报utils.forOwn is not a function，但是查看clone-deep@4.0.1源码就会发现它不再依赖forOwn包了，所以这里不会报错

另外，如果你用的是pnpm运行的项目，那这里就有个大坑了，因为现在很多项目对依赖管理使用的是npm或者yarn，如果对人为对依赖管理不是很清晰，就会导致很多幽灵依赖的问题出现，如果你使用pnpm安装这个工具，你就会发现需要额外安装kind-of和is-plain-object，明明你没有主动依赖这两个插件，也需要你主动去安装这个插件

最后贴两张clone-deep@0.2.4和clone-deep@4.0.1源码图片

Juaoie · 2024-05-23T07:20:53Z

clone-deep包是来自puppeteer-extra-plugin-stealth的依赖

Juaoie · 2024-05-29T03:29:12Z

在clone-deep@0.2.4中使用了lazy-cache，但是lazy-cache是专门为 CommonJS 模块系统设计的 npm 插件，所以在es模块中无法直接使用

Juaoie · 2024-05-31T03:30:00Z

pnpm解决方案：在package.json文件中加入下面代码

npm解决方案：在package.json文件中加入下面代码

yarn解决方案也是类似，覆盖clone-deep为最新版本就可以了

LeiG added issue: bug report A bug has been reported needs triage labels Jul 30, 2022

Hanaasagi mentioned this issue Jul 27, 2023

Implement utils.forOwn oven-sh/bun#3823

Closed

Juaoie mentioned this issue May 22, 2024

TypeError: utils.isObject is not a function #891

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug] TypeError: utils.forOwn is not a function #690

[Bug] TypeError: utils.forOwn is not a function #690

LeiG commented Jul 30, 2022

zkschmitz commented Jul 31, 2022

LeiG commented Aug 1, 2022

cdotte commented Sep 20, 2022

Randdalf commented Oct 25, 2022 •

edited

Loading

ledesmablt commented Oct 29, 2022

gwak2837 commented Dec 17, 2022

dbjpanda commented Jul 21, 2023

AdrKacz commented Sep 25, 2023

jeanhdev commented Sep 25, 2023 •

edited

Loading

AdrKacz commented Sep 25, 2023 •

edited

Loading

sh1kxrv commented Oct 19, 2023

shakkky commented Nov 24, 2023

AdrKacz commented Nov 25, 2023

Juaoie commented May 22, 2024

Juaoie commented May 23, 2024

Juaoie commented May 29, 2024

Juaoie commented May 31, 2024 •

edited

Loading

[Bug] TypeError: utils.forOwn is not a function #690

[Bug] TypeError: utils.forOwn is not a function #690

Comments

LeiG commented Jul 30, 2022

zkschmitz commented Jul 31, 2022

LeiG commented Aug 1, 2022

cdotte commented Sep 20, 2022

Randdalf commented Oct 25, 2022 • edited Loading

ledesmablt commented Oct 29, 2022

gwak2837 commented Dec 17, 2022

dbjpanda commented Jul 21, 2023

AdrKacz commented Sep 25, 2023

jeanhdev commented Sep 25, 2023 • edited Loading

AdrKacz commented Sep 25, 2023 • edited Loading

sh1kxrv commented Oct 19, 2023

shakkky commented Nov 24, 2023

AdrKacz commented Nov 25, 2023

Juaoie commented May 22, 2024

Juaoie commented May 23, 2024

Juaoie commented May 29, 2024

Juaoie commented May 31, 2024 • edited Loading

Randdalf commented Oct 25, 2022 •

edited

Loading

jeanhdev commented Sep 25, 2023 •

edited

Loading

AdrKacz commented Sep 25, 2023 •

edited

Loading

Juaoie commented May 31, 2024 •

edited

Loading