Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Puppeteer disconnects from the page when sending big files using MessageMedia.fromFilePath() (50MB+) #2117

Open
1 task done
gkp1 opened this issue Apr 4, 2023 · 25 comments · May be fixed by #2368
Open
1 task done

Puppeteer disconnects from the page when sending big files using MessageMedia.fromFilePath() (50MB+) #2117

gkp1 opened this issue Apr 4, 2023 · 25 comments · May be fixed by #2368
Labels
bug Something isn't working

Comments

@gkp1
Copy link

gkp1 commented Apr 4, 2023 •
edited
Loading

Is there an existing issue for this?

  • I have searched the existing issues

Describe the bug

How is the bug happening?

// LATEST UPDATE AND TL;DR: Generating a big file using MessageMedia.fromfilepath(path) generates the MessageMedia (suspiciously) quickly with no errors, but when you try to send this MessageMedia, the puppeteer Page Closed disconnection occurs. Browser still running, can send messages manually through the webpage, but a full process restart is required to make wawebjs puppeteer work again. The crash starts occurring at around 50 or 60MB file size, anything higher than that crashes.

Generating custom MB size files for test: https://fastest.fish/test-files (generate a 90MB file and test)

Error 1: BIG FILES:

(whatsapp-web file size limit is 100MB but files above 50MB or around 80MB+ cause crashes)

Every time we try to send a file with around 70 ~ 90MB or more, for example a file.7z format, nodejs throws a string limit error. Error: Cannot create a string longer than 0x1fffffe8 characters originates from MessageMedia.fromFilePath(path); . This string error is not always present but the second one always happens:
Then puppeteer disconnects from the running wawebjs chrome browser with the message Error: Protocol error (Runtime.callFunctionOn): Session closed. Most likely the page has been closed.
However, the chrome headless=false page does not close or crash and I can use whatsapp normally on it. It is not possible to close puppage or pupbrowser, also not possible to client.destroy() or cilent.initialize(). Full process restart required to close the browser and reopen

Error 2: MEDIUM FILES:

When sending files a little smaller like 49MB, there's no string limit error, and SOMETIMES it works and sends the file normally, but MOST TIMES (more than 50%), the whatsapp session silently breaks with the message {"name":"ProtocolError","originalMessage":"","message":"Protocol error (Runtime.callFunctionOn): Target closed."} and the session no longer works until the bot is restarted, the error repeats itself if I try to send any message after it occurred. But with headless=false you can see that the browser did not crash. Something with puppeteer or the browser broke, idk what. The bot session does not work anymore until you restart your bot, but I can still manually use the wa-web session normally through the chrome instance. Restarting makes it work again.
New cool github text editor feature returns table formatting just by typing a slash /, so I made this:

File size Works? Error Description
<= 40 MB Yes -
50 MB Sometimes 50% of the time it works and 50% breaks the session
80 MB No Target closed error
99 MB No Target closed + 0x1fffffe8 String limit

200% Relevant error logs:

Details: Click here to Open error logs 

Error sending media, trying to send as document...
Success sending as document!
Error sending media, trying to send as document...
Error: Cannot create a string longer than 0x1fffffe8 characters
    at Object.slice (node:buffer:648:37)
    at Buffer.toString (node:buffer:818:14)
    at Object.readFileSync (node:fs:504:41)
    at Function.fromFilePath (C:\node_modules\whatsapp-web.js\src\structures\MessageMedia.js:49:28)
    at C:\src\services\WbotServices\SendWhatsAppMedia.ts:27:33
    at Generator.next (<anonymous>)
    at fulfilled (C:\src\services\WbotServices\SendWhatsAppMedia.ts:5:58) {
  code: 'ERR_STRING_TOO_LONG'
}
Error: Cannot create a string longer than 0x1fffffe8 characters
    at Object.slice (node:buffer:648:37)
    at Buffer.toString (node:buffer:818:14)
    at Object.readFileSync (node:fs:504:41)
    at Function.fromFilePath (C:\node_modules\whatsapp-web.js\src\structures\MessageMedia.js:49:28)
    at C:\\src\services\WbotServices\SendWhatsAppMedia.ts:27:33
    at Generator.next (<anonymous>)
    at fulfilled (C:\\src\services\WbotServices\SendWhatsAppMedia.ts:5:58) {
  code: 'ERR_STRING_TOO_LONG'
}
Errors above sending media normally and sending media as document...
WARN [23:09:49.541] (19560):
    message: "ERR_SENDING_WAPP_MSG"
    statusCode: 400
{"name":"ProtocolError","originalMessage":"","message":"Protocol error (Runtime.callFunctionOn): Target closed."}
Error sending media, trying to send as document...
ProtocolError: Protocol error (Runtime.callFunctionOn): Target closed.
    at C:\\node_modules\puppeteer\src\common\Connection.ts:304:16
    at new Promise (<anonymous>)
    at CDPSession.send (C:\\node_modules\puppeteer\src\common\Connection.ts:300:12)  
    at ExecutionContext._evaluateInternal (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:254:44)
    at ExecutionContext.evaluate (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:140:23)
    at DOMWorld.evaluate (C:\\node_modules\puppeteer\src\common\DOMWorld.ts:174:20) {
  originalMessage: ''
}
Error: Protocol error (Runtime.callFunctionOn): Session closed. Most likely the page has been closed.
    at CDPSession.send (C:\\node_modules\puppeteer\src\common\Connection.ts:286:9)   
    at ExecutionContext._evaluateInternal (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:254:44)
    at ExecutionContext.evaluate (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:140:23)
    at DOMWorld.evaluate (C:\\node_modules\puppeteer\src\common\DOMWorld.ts:174:20)  
Errors above sending media normally and sending media as document...
WARN [23:10:29.723] (19560): Could not mark messages as read. Maybe whatsapp session disconnected? Err: 
WARN [23:10:29.919] (19560):
    message: "ERR_SENDING_WAPP_MSG"
    statusCode: 400
Error sending media, trying to send as document...
Error sending media, trying to send as document...
Error: Protocol error (Runtime.callFunctionOn): Session closed. Most likely the page has been closed.
    at CDPSession.send (C:\\node_modules\puppeteer\src\common\Connection.ts:286:9)   
    at ExecutionContext._evaluateInternal (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:254:44)
    at ExecutionContext.evaluate (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:140:23)
    at DOMWorld.evaluate (C:\\node_modules\puppeteer\src\common\DOMWorld.ts:174:20)  
Error: Protocol error (Runtime.callFunctionOn): Session closed. Most likely the page has been closed.
    at CDPSession.send (C:\\node_modules\puppeteer\src\common\Connection.ts:286:9)   
    at ExecutionContext._evaluateInternal (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:254:44)
    at ExecutionContext.evaluate (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:140:23)
    at DOMWorld.evaluate (C:\\node_modules\puppeteer\src\common\DOMWorld.ts:174:20)  
Errors above sending media normally and sending media as document...
Error: Protocol error (Runtime.callFunctionOn): Session closed. Most likely the page has been closed.
    at CDPSession.send (C:\\node_modules\puppeteer\src\common\Connection.ts:286:9)   
    at ExecutionContext._evaluateInternal (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:254:44)
    at ExecutionContext.evaluate (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:140:23)
    at DOMWorld.evaluate (C:\\node_modules\puppeteer\src\common\DOMWorld.ts:174:20)  
Error: Protocol error (Runtime.callFunctionOn): Session closed. Most likely the page has been closed.
    at CDPSession.send (C:\\node_modules\puppeteer\src\common\Connection.ts:286:9)   
    at ExecutionContext._evaluateInternal (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:254:44)
    at ExecutionContext.evaluate (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:140:23)
    at DOMWorld.evaluate (C:\\node_modules\puppeteer\src\common\DOMWorld.ts:174:20)  
Errors above sending media normally and sending media as document...
{}
WARN [23:10:55.986] (19560):
    message: "ERR_SENDING_WAPP_MSG"
    statusCode: 400
WARN [23:10:55.993] (19560): Could not mark messages as read. Maybe whatsapp session disconnected? Err: 
{}
WARN [23:10:59.713] (19560):
    message: "ERR_SENDING_WAPP_MSG"
    statusCode: 400
WARN [23:10:59.717] (19560): Could not mark messages as read. Maybe whatsapp session disconnected? Err: 
[INFO] 23:11:04 Restarting: \WbotServices\SendWhatsAppMedia.ts has been modified
INFO [23:11:11.424] (17904): Servidor iniciado na porta: 8080
INFO [23:11:14.531] (17904): Client Connected
INFO [23:11:28.522] (17904): Session: Normal AUTHENTICATED
INFO [23:11:28.620] (17904): Session: Normal READY
{"name":"ProtocolError","originalMessage":"","message":"Protocol error (Runtime.callFunctionOn): Target closed."}
Error sending media normally, trying to send as document...
ProtocolError: Protocol error (Runtime.callFunctionOn): Target closed.
    at C:\\node_modules\puppeteer\src\common\Connection.ts:304:16
    at new Promise (<anonymous>)
    at CDPSession.send (C:\\node_modules\puppeteer\src\common\Connection.ts:300:12)  
    at ExecutionContext._evaluateInternal (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:254:44)
    at ExecutionContext.evaluate (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:140:23)
    at DOMWorld.evaluate (C:\\node_modules\puppeteer\src\common\DOMWorld.ts:174:20) {
  originalMessage: ''
}
Error: Protocol error (Runtime.callFunctionOn): Session closed. Most likely the page has been closed.
    at CDPSession.send (C:\\node_modules\puppeteer\src\common\Connection.ts:286:9)   
    at ExecutionContext._evaluateInternal (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:254:44)
    at ExecutionContext.evaluate (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:140:23)
    at DOMWorld.evaluate (C:\\node_modules\puppeteer\src\common\DOMWorld.ts:174:20)  
Errors above sending media normally and sending media as document...
WARN [23:19:12.024] (17904): Could not mark messages as read. Maybe whatsapp session disconnected? Err: 
WARN [23:19:12.417] (17904):
    message: "ERR_SENDING_WAPP_MSG"
    statusCode: 400
{}
WARN [23:19:26.218] (17904):
    message: "ERR_SENDING_WAPP_MSG"
    statusCode: 400
WARN [23:19:26.226] (17904): Could not mark messages as read. Maybe whatsapp session disconnected? Err: 
{}
WARN [23:19:32.033] (17904):
    message: "ERR_SENDING_WAPP_MSG"
    statusCode: 400
WARN [23:19:32.038] (17904): Could not mark messages as read. Maybe whatsapp session disconnected? Err: 
[INFO] 23:19:39 Restarting: WbotServices\SendWhatsAppMedia.ts has been modified
INFO [23:19:42.550] (20716): Servidor iniciado na porta: 8080
INFO [23:19:58.571] (20716): Session: Normal AUTHENTICATED
INFO [23:19:58.665] (20716): Session: Normal READY
{"name":"ProtocolError","originalMessage":"","message":"Protocol error (Runtime.callFunctionOn): Target closed."}
Error sending media normally, trying to send as document...
ProtocolError: Protocol error (Runtime.callFunctionOn): Target closed.
    at C:\\node_modules\puppeteer\src\common\Connection.ts:304:16
    at new Promise (<anonymous>)
    at CDPSession.send (C:\\node_modules\puppeteer\src\common\Connection.ts:300:12)  
    at ExecutionContext._evaluateInternal (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:254:44)
    at ExecutionContext.evaluate (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:140:23)
    at DOMWorld.evaluate (C:\\node_modules\puppeteer\src\common\DOMWorld.ts:174:20) {
  originalMessage: ''
}
Error: Protocol error (Runtime.callFunctionOn): Session closed. Most likely the page has been closed.
    at CDPSession.send (C:\\node_modules\puppeteer\src\common\Connection.ts:286:9)   
    at ExecutionContext._evaluateInternal (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:254:44)
    at ExecutionContext.evaluate (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:140:23)
    at DOMWorld.evaluate (C:\\node_modules\puppeteer\src\common\DOMWorld.ts:174:20)  
Errors above sending media normally and sending media as document...

Expected behavior

Expected it to send files up to 100MB as whatsapp web allows.

However, on files tested with 49MB it works 50% of the time and 50% of the time it silently breaks your session without closing the browser, so you need to manually restart.

On files tested with 80MB to 100MB or even 102MB which is past web.whatsapp file limit, nodejs will throw string limit error and the same puppeteer error will happen, session silently crashes but browser stays running, requiring full api restart.

Steps to Reproduce the Bug or Issue

Put a 50MB, 90Mb or something and 101MB+ file in your app's folder and try to send them

Full function below

Relevant Code

Full test function

(tries sendAsDocument:true if normal media-sending fails)

Details: Click here to see code 

const tryToSendMediaNormallyOrDocument = async (messageMedia: MessageMedia) => {
    try {
      const sentMsg = await client.sendMessage(message.from, messageMedia, {
        sendAudioAsVoice: true,
        sendMediaAsDocument: false,
      });
      if (sentMsg) {
        console.log("Success sending file normally!");
        await message.reply("Success sending file normally!");
        return sentMsg;
      }
    } catch (err) {
      console.log(err);
      console.log("failed to send file normally! TRYING TO SEND AS DOCUMENT...");
      try {
        const sentMsgAsDocument = await client.sendMessage(message.from, messageMedia, {
          sendAudioAsVoice: true,
          sendMediaAsDocument: true,
        });
        if (sentMsgAsDocument ) {
          console.log("Success sending media as document!");
          await message.reply("Success sending file as document! (2nd try)");
          return sentMsgAsDocument ;
        }
      } catch (error) {
        console.log(error);
        console.log("failed to send media as document. both tries failed!");
        await message.reply("Both sendMediaAsDocument TRUE and FALSE failed.");
        await message.reply(err);
        await message.reply(error);
        return false;
      }
    }
  };
  if (message.body== "sf" || message.body == "sendfile") {
    const media = { path: ["./YOUR/FILE/PATHS/HERE.exe","./YOUR/FILE/PATHS/HERE.exe","./YOUR/FILE/PATHS/HERE.exe","./YOUR/FILE/PATHS/HERE.exe","./YOUR/FILE/PATHS/HERE.exe","./YOUR/FILE/PATHS/HERE.exe","./YOUR/FILE/PATHS/HERE.exe","./YOUR/FILE/PATHS/HERE.exe", "./PUBLIC/FILES/DOC.DOCX", "./DUCKVIDEO.mp4"] };
    let msgMedia: MessageMedia;
    media.path.every(async (path, index) => {
      msgMedia = await MessageMedia.fromFilePath(path);
      const sendingWorked = await tryToSendMediaNormallyOrDocument(msgMedia);
      if (sendingWorked) {
        console.log(`Sending message worked! ${index + 1}/${media.path.length}`);
      } else {
        console.log(`Sending message did not work! ${index + 1}/${media.path.length}`);
        message.reply(`Sending message did not work! ${index + 1}/${media.path.length}`);
      }
      await sleepRandom(1000, 2000);
    });

Browser Type

Google Chrome

WhatsApp Account Type

Standard

Does your WhatsApp account have multidevice enabled?

Yes, I am using Multi Device

Environment

OS: [e.g. Mac, Windows, Linux, Docker + Ubuntu 18, etc] win11, will later test with ubuntu 22.x
Phone OS: [e.g. Android, iOS] android
whatsapp-web.js version [e.g. 1.2.3] 1.19.5
WhatsApp Web version [run await client.getWWebVersion()]: 2.2314.7
Node.js Version [e.g. 1.2.3] v19.0.0

Additional context

No response
@gkp1 gkp1 added the bug Something isn't working label Apr 4, 2023
@gkp1 gkp1 changed the title Puppeteer silently breaks when trying to send big media (50MB+) Puppeteer silently breaks WA session when trying to send big media (50MB+) Apr 4, 2023
@amosayomide05
Copy link

It can only send 50mb document or video

I am affected by that bug too

@matricce
Copy link
Contributor

matricce commented Apr 5, 2023

Try this:
main...matricce:whatsapp-web.js:fix/sendBigFiles

@amosayomide05
Copy link

Try this:
main...matricce:whatsapp-web.js:fix/sendBigFiles

Please what do it do

Can it send big files?

@matricce
Copy link
Contributor

matricce commented Apr 6, 2023

Try this:
main...matricce:whatsapp-web.js:fix/sendBigFiles

Please what do it do

Can it send big files?

maybe..
fix: 'Protocol error (Runtime.callFunctionOn): Target closed.' when sending files larger than 80MB

@gkp1 gkp1 changed the title Puppeteer silently breaks WA session when trying to send big media (50MB+) WA session silently breaks when trying to send big media (50MB+) Apr 7, 2023
@matricce
Copy link
Contributor

Are you using the setRequestInterception method of Puppeteer (https://pptr.dev/api/puppeteer.page.setrequestinterception) in any part of the project?

@amosayomide05
Copy link

Are you using the setRequestInterception method of Puppeteer (https://pptr.dev/api/puppeteer.page.setrequestinterception) in any part of the project?

Nope

@matricce
Copy link
Contributor

Are you using the setRequestInterception method of Puppeteer (https://pptr.dev/api/puppeteer.page.setrequestinterception) in any part of the project?

Nope

I had the same problem of sending files larger than 50MB. The reason was that in my fork of the library, I was using the setRequestInterception method with a value of true. However, the behavior was a little different, and the process became "stuck". If I tried to send a file manually through WhatsApp Web, it kept loading infinitely.

@gkp1
Copy link
Author

gkp1 commented Apr 13, 2023

Are you using the setRequestInterception method of Puppeteer (https://pptr.dev/api/puppeteer.page.setrequestinterception) in any part of the project?

Nope. I still haven't tried your fix. Will test soon. Thanks

@EdenGottlieb
Copy link

@gkp1 Did you manage to solve the issue reliably?
Thanks!

@amosayomide05
Copy link

@gkp1 Did you manage to solve the issue reliably?
Thanks!

Nope

@pierrecorsini
Copy link
Contributor

Try this: main...matricce:whatsapp-web.js:fix/sendBigFiles

This fixed the issue for me !

@pierrecorsini
Copy link
Contributor

@matricce Thanks for the commit ! But sadly I dont see a PR with this commit ;)

@gkp1
Copy link
Author

gkp1 commented Jul 7, 2023 •
edited
Loading

Update: I am unable to test MATRICCE proposed solution as I'm not sure where to add the changes in the most recent Client.js? 😢

What happens is puppeteer disconnects from the page when sending big files (90MB for example). Even though the browser keeps running. It thinks page is closed. It is still possible to manually send messages through the running chrome browser whatsapp web, but puppeteer is no longer connected to it.

It is not possible to client.destroy() or client.initialize() or close pupPage or pupBrowser after this, a full process restart is required.

const knownBigFilesPuppeteerDisconnectionErrors = /Most likely the page has been closed\.|Protocol error \(Runtime\.callFunctionOn\): Session closed\./;

process.on("unhandledRejection", async (reason: any) => {
  const reasonString = String(reason); // Convert reason to a string
  console.log(`[unhandledRejection] ${reasonString}`);
  if (knownBrowserCrashErrorsRegex.test(reasonString)) {
    // log.info(`Client crashed. Destroying...`);
    // await activeClient.destroy(); // awaits eternally
    //await activeClient.pupPage.close(); // eternal error loop spam (don't run this)
    //await activeClient.pupBrowser.close(); // does nothing
    console.log("Client crashed. Restarting...");

 //pm2 restart myBot

  }
});

@pedroslopez @matricce @PurpShell is it possible to fix this issue on latest Client.js commit? I can test it for you. But sadly I have no idea how to implement a fix like matricce did. 😿

Atm the lib is limited to only sending ~40MB files

Whatsapp web 2.2328.5
Lib version pedroslopez/whatsapp-web.js#main (github)

@gkp1 gkp1 changed the title WA session silently breaks when trying to send big media (50MB+) Puppeteer disconnects from the page when trying to send big media (50MB+) Jul 7, 2023
@tofers
Copy link
Contributor

tofers commented Jul 8, 2023

#2129 Update the puppeteer. It works for me and large files no longer cause a crash

@matricce
Copy link
Contributor

#2129 Update the puppeteer. It works for me and large files no longer cause a crash

I tried it here, but even doing tests with different versions of whatsapp, I couldn't make it work..

lib updated with main branch, chrome updated, node v18, puppeteer v18.. v20.. but I still couldn't send an pdf with ~80MBs

I don't know how you got it just by updating puppeteer and node.. 😿

@gkp1
Copy link
Author

gkp1 commented Jul 12, 2023 •
edited
Loading

#2129 Update the puppeteer. It works for me and large files no longer cause a crash

#2129 Update the puppeteer. It works for me and large files no longer cause a crash

I tried it here, but even doing tests with different versions of whatsapp, I couldn't make it work..

lib updated with main branch, chrome updated, node v18, puppeteer v18.. v20.. but I still couldn't send an pdf with ~80MBs

I don't know how you got it just by updating puppeteer and node.. 😿

@tofers Same thing, using OVERRIDES to replace puppeteer@13 with latest puppeteer@20.8.1, same error. I'm on Node 18.16.0 and using chrome .exe to run

Not sure if it's because it automatically uses puppeteer-core instead of puppeteer. Not sure if it's intended or if i should 'force' it to use 'puppeteer' instead of 'puppeteer-core'. But all I did was update Puppeteer 13 to 20.

Tried using new headless config " headless: 'new' " which v20 recommended, didn't impact anything.

FULL ERROR MESSAGE WITH PUPP@20.8.1 

 sending file of size 90MB...
0|StickersBot  | TargetCloseError: Protocol error (Runtime.callFunctionOn): Target closed
0|StickersBot  |     at CallbackRegistry.clear (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\Connection.js:153:36)
0|StickersBot  |     at CDPSessionImpl._onClosed (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\Connection.js:468:70)
0|StickersBot  |     at Connection._Connection_onClose (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\Connection.js:338:17)
0|StickersBot  |     at WebSocket.<anonymous> (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\NodeWebSocketTransport.js:67:30)
0|StickersBot  |     at callListener (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\ws@8.13.0\node_modules\ws\lib\event-target.js:290:14)
0|StickersBot  |     at WebSocket.onClose (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\ws@8.13.0\node_modules\ws\lib\event-target.js:220:9)
0|StickersBot  |     at WebSocket.emit (node:events:513:28)
0|StickersBot  |     at WebSocket.emitClose (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\ws@8.13.0\node_modules\ws\lib\websocket.js:258:10)
0|StickersBot  |     at Socket.socketOnClose (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\ws@8.13.0\node_modules\ws\lib\websocket.js:1264:15)
0|StickersBot  |     at Socket.emit (node:events:513:28)
0|StickersBot  |     at TCP.<anonymous> (node:net:322:12)
0|StickersBot  | failed to send file normally! TRYING TO SEND AS DOCUMENT...
0|StickersBot  | TargetCloseError: Protocol error (Runtime.callFunctionOn): Session closed. Most likely the page has been closed.
0|StickersBot  |     at CDPSessionImpl.send (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\Connection.js:429:35)
0|StickersBot  |     at ExecutionContext._ExecutionContext_evaluate (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\ExecutionContext.js:244:46)
0|StickersBot  |     at async ExecutionContext.evaluate (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\ExecutionContext.js:145:16)
0|StickersBot  |     at async Client.sendMessage (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\github.com+pedroslopez+whatsapp-web.js@8fc6e6098c88e7280074e4a9bdaabedc0646589e\node_modules\whatsapp-web.js\src\Client.js:815:28)
0|StickersBot  | failed to send media as document. both tries failed!
0|StickersBot  | TargetCloseError: Protocol error (Runtime.callFunctionOn): Target closed
0|StickersBot  |     at CallbackRegistry.clear (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\Connection.js:153:36)
0|StickersBot  |     at CDPSessionImpl._onClosed (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\Connection.js:468:70)
0|StickersBot  |     at Connection._Connection_onClose (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\Connection.js:338:17)
0|StickersBot  |     at WebSocket.<anonymous> (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\NodeWebSocketTransport.js:67:30)
0|StickersBot  |     at callListener (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\ws@8.13.0\node_modules\ws\lib\event-target.js:290:14)
0|StickersBot  |     at WebSocket.onClose (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\ws@8.13.0\node_modules\ws\lib\event-target.js:220:9)
0|StickersBot  |     at WebSocket.emit (node:events:513:28)
0|StickersBot  |     at WebSocket.emitClose (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\ws@8.13.0\node_modules\ws\lib\websocket.js:258:10)
0|StickersBot  |     at Socket.socketOnClose (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\ws@8.13.0\node_modules\ws\lib\websocket.js:1264:15)
0|StickersBot  |     at Socket.emit (node:events:513:28)
0|StickersBot  |     at TCP.<anonymous> (node:net:322:12)
0|StickersBot  | TargetCloseError: Protocol error (Runtime.callFunctionOn): Session closed. Most likely the page has been closed.
0|StickersBot  |     at CDPSessionImpl.send (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\Connection.js:429:35)
0|StickersBot  |     at ExecutionContext._ExecutionContext_evaluate (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\ExecutionContext.js:244:46)
0|StickersBot  |     at async ExecutionContext.evaluate (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\ExecutionContext.js:145:16)
0|StickersBot  |     at async Client.sendMessage (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\github.com+pedroslopez+whatsapp-web.js@8fc6e6098c88e7280074e4a9bdaabedc0646589e\node_modules\whatsapp-web.js\src\Client.js:815:28)
0|StickersBot  | [10:49:21.161] ERROR (16408): [unhandledRejection] TargetCloseError: Protocol error (Runtime.callFunctionOn): Target closed
0|StickersBot  | [10:49:21.162] ERROR (16408): [unhandledRejection] TargetCloseError: Protocol error (Runtime.callFunctionOn): Session closed. Most likely the page has been closed.

If anyone wants to test, this is how you do it.

Put this overrides in package.json, delete package-lock or pnpm-lock and node_modules and run npm i or pnpm i:

if using NPM: 

{
[...]
  "dependencies": { 
    ....
    "whatsapp-web.js": "github:pedroslopez/whatsapp-web.js#main"
  },
  "devDependencies": {
    ....  
},
    "overrides": {
      "puppeteer": "latest"
    }
}

if using PNPM: 

{
[...]
  "dependencies": { 
    ....
    "whatsapp-web.js": "github:pedroslopez/whatsapp-web.js#main"
  },
  "devDependencies": {
    ....  
},
  "pnpm": {
    "overrides": {
      "puppeteer": "latest"
    }
  }
}

for overriding only the specific whatsapp-web.js puppeteer instead of any puppeteer from all dependencies in the project, use: (probably not needed)

"overrides": {
"whatsapp-web.js":{
      "puppeteer": "latest"
}
    }

@tofers
Copy link
Contributor

tofers commented Jul 13, 2023

@matricce @gkp1

I have also tested it on clients.

It sends mp4 videos up to 64 MB.
And 100 MB files (I didn't test more than that due to server load).

How do you send the files?

@gkp1
Copy link
Author

gkp1 commented Jul 13, 2023 •
edited
Loading

@tofers I try to send both normally and as document, same happens with any file format, using MessageMedia.fromFilePath(). The issue is probably with fromFilePath()? It succesfully sends anything up to around 50MB, supports videos and audios whatever, if video format is not supported, it sends the video as file normally, I can send .bin files of 20MB, 40MB, but 50MB or anything higher crashes

const sentMsg = await client.sendMessage(msg.from, messageMedia, {
        sendAudioAsVoice: true,
        sendMediaAsDocument: false,
      });
const sentMsgAsDocument = await client.sendMessage(msg.from, messageMedia, {
        sendAudioAsVoice: true,
        sendMediaAsDocument: true,
      });

@tofers
Copy link
Contributor

tofers commented Jul 13, 2023

And if you remove these options ?

const sentMsg = await client.sendMessage(msg.from, messageMedia);

I don't use fromFilePath, as all files come as a link (e.g. link to s3 storage).

I download the file and send it to you already

let fileBase64 = await axios.get(filePath, {responseType: 'arraybuffer'})
let returnedB64 = fileBase64.data.toString('base64')
let attachmentData = new MessageMedia(mime_type, returnedB64, filename);

@gkp1
Copy link
Author

gkp1 commented Jul 13, 2023 •
edited
Loading

@tofers Same thing removing the {options}. The crash isn't caused directly by fromFilePath(), but it crashes when sending the MessageMedia generated by fromFilePath(). I will rename the issue to clarify this

// send a 90MB file
log.info(`Creating MessageMedia.fromFilePath size ${arg}...`);
    const msgMedia = await MessageMedia.fromFilePath(`./storage/${arg}.bin`); // --> generates the MessageMedia in 100ms
    log.info(`sending file of size ${arg}...`);
    tryToSendMediaNormallyOrDocument(client, message, msgMedia); // crashes
  
 [15:26:46.297] INFO (15456): Creating MessageMedia.fromFilePath size 90MB...
[15:26:46.415] INFO (15456): sending file of size 90MB...
0|StickersBot  | TargetCloseError: Protocol error (Runtime.callFunctionOn): Target closed
0|StickersBot  |     at CallbackRegistry.clear (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\Connection.js:153:36)
0|StickersBot  |     at CDPSessionImpl._onClosed (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\Connection.js:468:70)
0|StickersBot  |     at Connection._Connection_onClose (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\Connection.js:338:17)
0|StickersBot  |     at WebSocket.<anonymous> (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\NodeWebSocketTransport.js:67:30)
0|StickersBot  |     at callListener (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\ws@8.13.0\node_modules\ws\lib\event-target.js:290:14)
0|StickersBot  |     at WebSocket.onClose (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\ws@8.13.0\node_modules\ws\lib\event-target.js:220:9)
0|StickersBot  |     at WebSocket.emit (node:events:513:28)
0|StickersBot  |     at WebSocket.emitClose (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\ws@8.13.0\node_modules\ws\lib\websocket.js:258:10)
0|StickersBot  |     at Socket.socketOnClose (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\ws@8.13.0\node_modules\ws\lib\websocket.js:1264:15)
0|StickersBot  |     at Socket.emit (node:events:513:28)
0|StickersBot  |     at TCP.<anonymous> (node:net:322:12)
0|StickersBot  | failed to send file normally! TRYING TO SEND AS DOCUMENT...
0|StickersBot  | TargetCloseError: Protocol error (Runtime.callFunctionOn): Session closed. Most likely the page has been closed.
0|StickersBot  |     at CDPSessionImpl.send (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\Connection.js:429:35)
0|StickersBot  |     at ExecutionContext._ExecutionContext_evaluate (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\ExecutionContext.js:244:46)
0|StickersBot  |     at async ExecutionContext.evaluate (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\ExecutionContext.js:145:16)
0|StickersBot  |     at async Client.sendMessage (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\github.com+pedroslopez+whatsapp-web.js@8fc6e6098c88e7280074e4a9bdaabedc0646589e\node_modules\whatsapp-web.js\src\Client.js:815:28)
0|StickersBot  | failed to send media as document. both tries failed!
0|StickersBot  | TargetCloseError: Protocol error (Runtime.callFunctionOn): Target closed
0|StickersBot  |     at CallbackRegistry.clear (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\Connection.js:153:36)
0|StickersBot  |     at CDPSessionImpl._onClosed (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\Connection.js:468:70)
0|StickersBot  |     at Connection._Connection_onClose (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\Connection.js:338:17)
0|StickersBot  |     at WebSocket.<anonymous> (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\NodeWebSocketTransport.js:67:30)
0|StickersBot  |     at callListener (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\ws@8.13.0\node_modules\ws\lib\event-target.js:290:14)
0|StickersBot  |     at WebSocket.onClose (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\ws@8.13.0\node_modules\ws\lib\event-target.js:220:9)
0|StickersBot  |     at WebSocket.emit (node:events:513:28)
0|StickersBot  |     at WebSocket.emitClose (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\ws@8.13.0\node_modules\ws\lib\websocket.js:258:10)
0|StickersBot  |     at Socket.socketOnClose (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\ws@8.13.0\node_modules\ws\lib\websocket.js:1264:15)
0|StickersBot  |     at Socket.emit (node:events:513:28)
0|StickersBot  |     at TCP.<anonymous> (node:net:322:12)
0|StickersBot  | TargetCloseError: Protocol error (Runtime.callFunctionOn): Session closed. Most likely the page has been closed.
0|StickersBot  |     at CDPSessionImpl.send (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\Connection.js:429:35)
0|StickersBot  |     at ExecutionContext._ExecutionContext_evaluate (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\ExecutionContext.js:244:46)
0|StickersBot  |     at async ExecutionContext.evaluate (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\ExecutionContext.js:145:16)
0|StickersBot  |     at async Client.sendMessage (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\github.com+pedroslopez+whatsapp-web.js@8fc6e6098c88e7280074e4a9bdaabedc0646589e\node_modules\whatsapp-web.js\src\Client.js:815:28)
0|StickersBot  | [15:26:47.793] ERROR (15456): [unhandledRejection] TargetCloseError: Protocol error (Runtime.callFunctionOn): Target closed
0|StickersBot  | [15:26:47.794] ERROR (15456): [unhandledRejection] TargetCloseError: Protocol error (Runtime.callFunctionOn): Session closed. Most likely the page has been closed.
0|StickersBot  | [15:26:47.794] INFO (15456): Client crashed. Restarting...
0|StickersBot  | [15:26:47.794] INFO (15456): Sleeping 3000ms...
PM2            | Stopping app:StickersBot id:0
PM2            | App [StickersBot:0] exited with code [1] via signa

@gkp1 gkp1 changed the title Puppeteer disconnects from the page when trying to send big media (50MB+) Puppeteer disconnects from the page when sending big files using MessageMedia.fromFilePath() (50MB+) Jul 13, 2023
@tofers
Copy link
Contributor

tofers commented Jul 13, 2023

Can you give me the file, I'll try it

@gkp1
Copy link
Author

gkp1 commented Jul 13, 2023

Can you give me the file, I'll try it

yep it crashes with any file extension so just specify 90MB file from this generate testfiles page https://fastest.fish/test-files Custom file size download

@opssemnik
Copy link

Try this: main...matricce:whatsapp-web.js:fix/sendBigFiles

this worked! However only for files up to 100MB - which is why i believe some folks still see issues (i assume because you just split it in the middle, creating 2x50MB chunks)

However wa web now supports 1gb files as attachments, would it be possible for you to update so it splits every 50mb?

@opssemnik
Copy link

Update: I am unable to test MATRICCE proposed solution as I'm not sure where to add the changes in the most recent Client.js? 😢

What happens is puppeteer disconnects from the page when sending big files (90MB for example). Even though the browser keeps running. It thinks page is closed. It is still possible to manually send messages through the running chrome browser whatsapp web, but puppeteer is no longer connected to it.

It is not possible to client.destroy() or client.initialize() or close pupPage or pupBrowser after this, a full process restart is required.

const knownBigFilesPuppeteerDisconnectionErrors = /Most likely the page has been closed\.|Protocol error \(Runtime\.callFunctionOn\): Session closed\./;

process.on("unhandledRejection", async (reason: any) => {
  const reasonString = String(reason); // Convert reason to a string
  console.log(`[unhandledRejection] ${reasonString}`);
  if (knownBrowserCrashErrorsRegex.test(reasonString)) {
    // log.info(`Client crashed. Destroying...`);
    // await activeClient.destroy(); // awaits eternally
    //await activeClient.pupPage.close(); // eternal error loop spam (don't run this)
    //await activeClient.pupBrowser.close(); // does nothing
    console.log("Client crashed. Restarting...");

 //pm2 restart myBot

  }
});

@pedroslopez @matricce @PurpShell is it possible to fix this issue on latest Client.js commit? I can test it for you. But sadly I have no idea how to implement a fix like matricce did. 😿

Atm the lib is limited to only sending ~40MB files

Whatsapp web 2.2328.5 Lib version pedroslopez/whatsapp-web.js#main (github)

You can just open his PR to see the entire sendMessage method
https://github.com/pedroslopez/whatsapp-web.js/blob/cfacfbe7cc1957e452fc5d6dd93244589dc66353/src/Client.js#L635C1-L717C6

@opssemnik opssemnik linked a pull request Jul 28, 2023 that will close this issue
Open
3 tasks
@PwccaCode
Copy link

Has this issue been fixed?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: Add support for sending large files (up to 2GB) opssemnik/whatsapp-web.js
8 participants
@opssemnik @EdenGottlieb @tofers @matricce @pierrecorsini @PwccaCode @amosayomide05 @gkp1