ConsoleMessage type and text not interoperable between Chrome and Firefox

[foolip]

Steps to reproduce

Tell us about your environment:

• Puppeteer version: 5.2.0
• Platform / OS version: macOS 10.15.5
• Node.js version: v10.16.3

What steps will reproduce the problem?

Please include code that reproduces the issue.

'use strict';

const puppeteer = require('puppeteer');

async function main() {
  for (const product of ['chrome', 'firefox']) {
    const browser = await puppeteer.launch({product});
    const page = await browser.newPage();
    await page.goto('about:blank');
    const msgPromise = new Promise((resolve) => {
      page.on('console', resolve);
    });
    await page.evaluate('console.log("message")');
    const msg = await msgPromise;
    console.log({
      product,
      type: msg.type(),
      text: msg.text(),
      args: msg.args(),
    });
    await browser.close();
  }
}

main();

Output from running the above is:

{ product: 'chrome',
  type: 'log',
  text: 'message',
  args:
   [ JSHandle {
       _disposed: false,
       _context: [ExecutionContext],
       _client: [CDPSession],
       _remoteObject: [Object] } ] }
{ product: 'firefox',
  type: 'verbose',
  text: [ 'message' ],
  args: [] }

What is the expected result?

The msg.type() and msg.text() should be the same for Chrome and Firefox, to make it possible to use console messages without switching on which product is under test.

What happens instead?

• Chrome: msg.type() is "log" and msg.text() is "message".
• Firefox: msg.type() is "verbose" and msg.text() is an array with a single item "message"

msg.args() is also different, but I don't know what it should be, and am not using it myself.

[christian-bromann]

The location method also shows different return values. Firefox is missing columnNumber here. I added a PR to increase the test coverage around this area a bit.

[jackfranklin]

@mjzffr this might be a FF issue? I'm not 100% - should we also consider Puppeteer itself translating Chrome/FF consoles into an interface that is consistent regardless of browser?

[mjzffr]

There are a number of issues here. tl;dr is that Firefox doesn't implement all the CDP events yet + there may be some discrepancies between Blink and Gecko.

In the Chrome case, the sample script is reporting data based on

{"method":"Runtime.consoleAPICalled","params":{"type":"log","args":[{"type":"string","value":"message"}],"executionContextId":3,"timestamp":1.596131012139123e+12,"stackTrace":{"callFrames":[{"functionName":"","scriptId":"4","url":"__puppeteer_evaluation_script__","lineNumber":0,"columnNumber":8}]}},"sessionId":"F24311E58CE491073C2839A39B2E8C83"}

Firefox doesn't yet implement Runtime.consoleAPICalled (https://bugzilla.mozilla.org/show_bug.cgi?id=1549526).

In the Firefox case, the sample script is getting its data from Log.entryAdded, which is only partly implemented. Indeed, no columnNumber is provided yet and the type of text is wrong. (I've filed https://bugzilla.mozilla.org/show_bug.cgi?id=1656314 about that).

{"sessionId":1,"method":"Log.entryAdded","params":{"entry":{"source":"javascript","level":"verbose","text":["message"],"url":"__puppeteer_evaluation_script__","lineNumber":1,"timestamp":1596131013412}}}

I'd have to do more digging to say more about type and args but we've previously noted (on log entries in general) Gecko has different types of logs that don’t conform with Chrome’s, so I suspect that clients like Puppeteer might have to do some translation and/or logging will be unified somehow in WebDriver bidi.

[whimboo]

@foolip I implemented Runtime.consoleAPICalled a while ago and it landed in Firefox 85.

Would you mind having a look again if everything satisfies your needs?

[stale]

We're marking this issue as unconfirmed because it has not had recent activity and we weren't able to confirm it yet. It will be closed if no further activity occurs within the next 30 days.

[whimboo]

Just checked and there is no difference anymore between Chrome and Firefox. Both output the following:

{
  product: 'test',
  type: 'log',
  text: 'message',
  args: [ JSHandle {} ]
}

[foolip]

Thanks @whimboo!