Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

URI malformed exception that is not handled #869

Closed
Khady opened this issue Sep 25, 2017 · 1 comment
Closed

URI malformed exception that is not handled #869

Khady opened this issue Sep 25, 2017 · 1 comment
Assignees
@aslushnikov

Comments

@Khady
Copy link

Khady commented Sep 25, 2017

I have an issue that I think is related to #668

I load a page in a loop, creating a new page each time and saving a screenshot as a result. After a few iterations, an exception is raised and not handled because of a malformed URI.

I tried to print all the url that are requested and I see some stuff like:

data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 16 48'%3E%3Cpath fill='%2378777A' d='M13 3.4L8 9.1 3 3.4l-1.2 1L7.5 11h1l5.7-6.6z'/%3E%3Cpath fill='%231D1D1D' d='M13 19.4l-5 5.7-5-5.7-1.2 1L7.5 27h1l5.7-6.6z'/%3E%3Cpath fill='%23FF$
https://logx.optimizely.com/log/event
data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' width='16' height='48' viewBox='0 0 16 48'%3E%3Cpath fill='%2378777A' d='M15.7 3.5c-.5.2-1.1.4-1.7.5.6-.4 1.1-1 1.4-1.6-.6.3-1.3.6-2 .7-.6-.6-1.4-1-2.3-1-1.7 0-3 1.3-3 3 0 .2 0 .5.1.7-2.6-.2-4.8-1$
data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' width='16' height='48' viewBox='0 0 16 48'%3E%3Cpath fill='%2378777A' d='M8.8 15V8.5H11l.3-2.5H8.8V4.4c0-.7.2-1.2 1.2-1.2h1.3V.9c-.2 0-1-.1-1.9-.1C7.5.8 6 2 6 4.2V6H4v2.5h2V15h2.8z'/%3E%3Cpath fil$
data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' width='16' height='48' viewBox='0 0 16 48'%3E%3Cpath d='M8 9L1 3v9a1 1 0 0 0 1.08 1h11.84A1 1 0 0 0 15 12V3z' fill='%2378777a'/%3E%3Cpath fill='%2378777a' d='M13.38 3H2.62L8 7.5 13.38 3z'/%3E%3Cpa$
data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' width='16' height='48' viewBox='0 0 16 48'%3E%3Cpath fill='%23DC4E41' d='M10.3 23c.1.4.1.7.1 1.1 0 3-2 5.2-5.1 5.2C2.4 29.3 0 26.9 0 24s2.4-5.3 5.3-5.3c1.4 0 2.6.5 3.6 1.4l-1.5 1.5c-.6-.5-1.3-.8-2$
data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' width='16' height='48' viewBox='0 0 16 48'%3E%3Cpath d='M4.9 5.5V14H2V5.5h2.9zM5.1 3c0 .4-.1.8-.4 1.1-.4.2-.7.3-1.2.3s-.9-.2-1.2-.4-.4-.7-.4-1c0-.4.1-.8.4-1.1s.7-.4 1.2-.4.9.1 1.1.4.5.6.5 1.1zM15 $
data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' width='16' height='48' viewBox='0 0 16 48'%3E%3Cpath fill='%2378777A' d='M14.7 7.3c0-.4-.2-.7-.4-1-.2-.3-.6-.5-.9-.6-.1 0-.3-.1-.5-.1-.3 0-.7.1-1 .3-.1 0-.1.1-.2.2h-.1c-1-.6-2.1-.8-3.1-.9 0-.4 0-.$
data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' width='16' height='48' viewBox='0 0 16 48'%3E%3Cpath fill='%2378777A' d='M14.2 1.8c-.5-.5-1.2-.8-2-.8s-1.5.3-2 .8l-2 2c-.9.9-1 2.3-.4 3.4l-.6.6c-.5-.3-1-.4-1.4-.4-.8 0-1.5.3-2 .8l-2 2c-1.1 1.1-1.1$
data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' width='32' height='32' viewBox='0 0 32 32'%3E%3Cpath fill='%231D1D1D' d='M16 20l6.5-6.5L21 12l-5 5-5-5-1.5 1.5z'/%3E%3C/svg%3E
data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' width='16' height='48' viewBox='0 0 16 48'%3E%3Cpath fill='%2378777A' d='M4 8l6.5 6.5L12 13 7 8l5-5-1.5-1.5z'/%3E%3Cpath fill='%231D1D1D' d='M4 24l6.5 6.5L12 29l-5-5 5-5-1.5-1.5z'/%3E%3Cpath fill=$
data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' width='16' height='48' viewBox='0 0 16 48'%3E%3Cpath fill='%2378777A' d='M12 8L5.5 1.5 4 3l5 5-5 5 1.5 1.5z'/%3E%3Cpath fill='%231D1D1D' d='M12 24l-6.5-6.5L4 19l5 5-5 5 1.5 1.5z'/%3E%3Cpath fill='$
https://platform.instagram.com/en_US/embeds.js
http://secure-au.imrworldwide.com/v60.js
http://fairfax-a.p.adnxs.com/ut/v3
http://acdn.adnxs.com/mediation/v2/mediation.js
http://acdn.adnxs.com/mediation/v2/mediation.js
http://rtax.criteo.com/delivery/rta/rta.js?netId=2187&cookieName=crtg_rta&rnd=74205018896&varName=crtg_content
https://fonts.googleapis.com/css?family=Merriweather:700%7CMontserrat
http://ads.fairfax.com.au/TechOps/uncached/ffxContainerTag.js?addiv=yes&astaddetails=yes&titanwrapper=no&titandebugging=no&iasmastheads=yes&iasallure=no&an_dssf=no&cachebuster=41074065&publisher_id=888233&creative_id=62275311&creative_size=620x164&width=620&height=164&re$
http://pixel.adsafeprotected.com/jload?anId=925294&campId=desktop%7Carticle%7C620x164%7C1&pubId=1549568&chanId=888233.2534826.10056106&placementId=3744151&pubOrder=$%7BIO_ID%7D&pubCreative=62275311&custom=sport&custom2=nrl&custom3=news
data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' width='16' height='48' viewBox='0 0 16 48'%3E%3Cpath fill='%2378777A' d='M4 14V2l10 6-10 6z'/%3E%3Cpath fill='%231D1D1D' d='M4 30V18l10 6-10 6z'/%3E%3Cpath fill='%23FCFCFC' d='M4 46V34l10 6-10 6z'$
http://ib.adnxs.com/ut/v3
http://acdn.adnxs.com/mediation/v2/mediation.js
http://acdn.adnxs.com/mediation/v2/mediation.js
data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' width='450' height='48' viewBox='125 1287 450 48'%3E%3Cpath fill='%23FFF' d='M243.1 1302.1l-4.1 2.7 1.4 1.2v15.9c0 1.4-1.2 2.4-2.8 2.4h-.7v-19.8l-3.5-2.3-4.4 2.9.2.4s.8-.5 1-.6c0 0 1.6 1.5 1.6 2.9$
data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' width='32' height='97' viewBox='0 0 32 97'%3E%3Cpath fill='%23666' d='M10.5 24.7c0 .9-.3 1.7-1 2.3-.6.6-1.4 1-2.3 1s-1.6-.3-2.2-1-1-1.4-1-2.3.3-1.7 1-2.3 1.4-1 2.3-1 1.7.3 2.3 1c.6.6.9 1.4.9 2.3zm$
data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' width='80' height='45' viewBox='-59 1288 80 45'%3E%3Ccircle fill='%23999' cx='-36.7' cy='1310.3' r='22.3'/%3E%3Cpath d='M-54.8 1314.9h1.6v-4.6h2.9v-1.2h-2.9v-3.1h3.6v-1.2h-5.2v10.1zm8.4-7.3c-.7 0-$
data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' width='450' height='48' viewBox='0 1 450 48'%3E%3Cpath d='M225.7 11.1c.6 0 1.1.4 1.4.9v-1.6l-.8.1.2-.7H225l.2.7-.8-.1V12c.2-.5.7-.9 1.3-.9zm-30 26.5zm0 0zm0 .1c0-.1 0-.1 0 0zm0 0c0-.1 0-.1 0 0zm2.$
data:image/svg+xml;charset=utf-8,%3Csvg width='262' height='32' viewBox='0 0 262 32' xmlns='http://www.w3.org/2000/svg'%3E%3Cpath d='M13.082 26.962c-1.848 0-3.735-.796-5.083-2.586v-7.243c1.116-2.23 3.272-3.183 5.158-3.183 3.466 0 5.66 3.103 5.66 6.446 0 3.145-1.848 6.566$
data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' width='450' height='48' viewBox='0 0 450 48'%3E%3Cpath d='M64.3 39.6L62.5 41c-.6 0-1.4 0-1.8-.6-.3-.6-.4-1.4-.4-2.1v-15l3.5-3.6-.8-.6-3.1 2.6-5.1-5.3-6.1 5.6-.1-16.8c0-.6.1-1.2.5-1.6.8-.7 2.3-.8 3$
data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' width='255' height='48' viewBox='0 0 255 48'%3E%3Cpath fill='%23154A93' d='M197.6 35.9h1.7v-1.8h-1.7v1.8zm8.3-1.4c-.3.3-.8.5-1.5.5-.4 0-.7-.1-1-.2-.3-.2-.5-.4-.7-.6l-.4-.9-.1-1 .1-1.1.4-.9c.2-.3.4$
http://www.fairfaxstatic.com.au/dtm/7d5ea80d054b96730162d0905d59678c4d2bf30c/scripts/satellite-57e1d41464746d3238007df6.js
http://www.fairfaxstatic.com.au/dtm/7d5ea80d054b96730162d0905d59678c4d2bf30c/scripts/satellite-5993a03b64746d0ba501d96f.js
http://www.fairfaxstatic.com.au/dtm/7d5ea80d054b96730162d0905d59678c4d2bf30c/scripts/satellite-54fd356b6266390016940000.js

I wonder if it could be the source of the problem.

Steps to reproduce

Tell us about your environment:

What steps will reproduce the problem?

'use strict';

const puppeteer = require('puppeteer');

(async() => {
  process.on("unhandledRejection", (reason, p) => {
    console.error("Unhandled Rejection at: Promise", p, "reason:", reason);
    // application specific logging, throwing an error, or other logic here
    cont = false;
  });
  const args = [
    "--disable-gpu",
    "--disable-setuid-sandbox",
    "--force-device-scale-factor",
    "--ignore-certificate-errors",
    "--no-sandbox",
  ];
  const options = {
    args,
    headless: true,
    ignoreHTTPSErrors: true,
  };
  const browser = await puppeteer.launch(options);
  let cont = true;
  let n = 0
  while (cont) {
    console.log(n);
    n++
    try {
      const page = await browser.newPage();
      await page.setRequestInterceptionEnabled(true);
      page.on("request", (request) => {
        if (request.resourceType === "Image") {
          request.abort();
        } else {
          request.continue();
        }
      });
      await page.goto('http://www.smh.com.au/rugby-league/league-news/the-year-in-discord-part-i-20140105-30bhz.html');
      await page.screenshot({ path: 'example.png' });
      await page.close();
    } catch (error) {
      console.error("exception", error);
    }
  }
  await browser.close();

})();
  1. save this file as minimal.js
  2. run nodejs minimal.js
  3. wait...
  4. relaunch after some time if it doesn't crash

What is the expected result?

The program should run forever

What happens instead?

Note that the number of successful loop before to get the error is not always the same. I guess that it depends on the behavior of the website.

$ nodejs src/minimal.js 
0
1
2
3
Unhandled Rejection at: Promise Promise {
  <rejected> URIError: URI malformed
    at decodeURI (<anonymous>)
    at generateRequestHash (/home/louis/Code/test/v2/test_server/node_modules/puppeteer/lib/NetworkManager.js:399:10)
    at NetworkManager._onRequestIntercepted (/home/louis/Code/test/v2/test_server/node_modules/puppeteer/lib/NetworkManager.js:139:25)
    at emitOne (events.js:115:13)
    at Session.emit (events.js:210:7)
    at Session._onMessage (/home/louis/Code/test/v2/test_server/node_modules/puppeteer/lib/Connection.js:199:12)
    at Connection._onMessage (/home/louis/Code/test/v2/test_server/node_modules/puppeteer/lib/Connection.js:98:19)
    at emitOne (events.js:115:13)
    at WebSocket.emit (events.js:210:7)
    at Receiver._receiver.onmessage (/home/louis/Code/test/v2/test_server/node_modules/ws/lib/WebSocket.js:143:47) } reason: URIError: URI malformed
    at decodeURI (<anonymous>)
    at generateRequestHash (/home/louis/Code/test/v2/test_server/node_modules/puppeteer/lib/NetworkManager.js:399:10)
    at NetworkManager._onRequestIntercepted (/home/louis/Code/test/v2/test_server/node_modules/puppeteer/lib/NetworkManager.js:139:25)
    at emitOne (events.js:115:13)
    at Session.emit (events.js:210:7)
    at Session._onMessage (/home/louis/Code/test/v2/test_server/node_modules/puppeteer/lib/Connection.js:199:12)
    at Connection._onMessage (/home/louis/Code/test/v2/test_server/node_modules/puppeteer/lib/Connection.js:98:19)
    at emitOne (events.js:115:13)
    at WebSocket.emit (events.js:210:7)
    at Receiver._receiver.onmessage (/home/louis/Code/test/v2/test_server/node_modules/ws/lib/WebSocket.js:143:47)
Unhandled Rejection at: Promise Promise {
  <rejected> URIError: URI malformed
    at decodeURI (<anonymous>)
    at generateRequestHash (/home/louis/Code/test/v2/test_server/node_modules/puppeteer/lib/NetworkManager.js:399:10)
    at NetworkManager._onRequestWillBeSent (/home/louis/Code/test/v2/test_server/node_modules/puppeteer/lib/NetworkManager.js:181:27)
    at emitOne (events.js:115:13)
    at Session.emit (events.js:210:7)
    at Session._onMessage (/home/louis/Code/test/v2/test_server/node_modules/puppeteer/lib/Connection.js:199:12)
    at Connection._onMessage (/home/louis/Code/test/v2/test_server/node_modules/puppeteer/lib/Connection.js:98:19)
    at emitOne (events.js:115:13)
    at WebSocket.emit (events.js:210:7)
    at Receiver._receiver.onmessage (/home/louis/Code/test/v2/test_server/node_modules/ws/lib/WebSocket.js:143:47) } reason: URIError: URI malformed
    at decodeURI (<anonymous>)
    at generateRequestHash (/home/louis/Code/test/v2/test_server/node_modules/puppeteer/lib/NetworkManager.js:399:10)
    at NetworkManager._onRequestWillBeSent (/home/louis/Code/test/v2/test_server/node_modules/puppeteer/lib/NetworkManager.js:181:27)
    at emitOne (events.js:115:13)
    at Session.emit (events.js:210:7)
    at Session._onMessage (/home/louis/Code/test/v2/test_server/node_modules/puppeteer/lib/Connection.js:199:12)
    at Connection._onMessage (/home/louis/Code/test/v2/test_server/node_modules/puppeteer/lib/Connection.js:98:19)
    at emitOne (events.js:115:13)
    at WebSocket.emit (events.js:210:7)
    at Receiver._receiver.onmessage (/home/louis/Code/test/v2/test_server/node_modules/ws/lib/WebSocket.js:143:47)
@Khady Khady mentioned this issue Sep 25, 2017
Closed
@Khady
Copy link
Author

Khady commented Sep 25, 2017

I tried the patch from #668 and I can't reproduce the error anymore with it. So I really think that it is required to sanitize (automatically or with a hook?) the urls that are requested from the pages that are navigated to.

@aslushnikov aslushnikov self-assigned this Sep 25, 2017
aslushnikov added a commit to aslushnikov/puppeteer that referenced this issue Sep 26, 2017
@aslushnikov
[fix] Do not attempt to normalize malformed URLs.
04311aa 
This patch avoids throwing 'url malformed' error during generating
request hash for request interception.

Fixes puppeteer#869.
@aslushnikov aslushnikov mentioned this issue Sep 26, 2017
Merged
aslushnikov added a commit that referenced this issue Sep 28, 2017
@aslushnikov
fix(Network): Do not attempt to normalize malformed URLs. (#875)
cfece34 
This patch avoids throwing 'url malformed' error during generating
request hash for request interception.

Fixes #869.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@aslushnikov aslushnikov

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@aslushnikov @Khady