[BUG] no console logging output if using custom reporter

[mstepin-edt]

Context:

• Playwright Version: 1.27.1
• Operating System: Ubuntu (WSL)
• Node.js version: 19.0.0
• Browser: Chrome

When adding a custom reporter, the console.log output is no longer displayed in the console.

NOTE: The console stdOut attachment to the html reporter still shows the desired output.

Consider the following setup:

// playwright.config.js

            reporter: [
              ['html', {open: 'never'}],
              ['./lib/utils/customReporters/myCustomReporter.js']
            ],

Custom reporter:

//  ./lib/utils/customReporters/myCustomReporter.js
// @ts-nocheck

/** @implements {import('@playwright/test/reporter').Reporter} */
class MyReporter {
    onBegin(config, suite) {
        console.log('this text IS shown in the console')
        someOtherFunction()
    }
    onStepEnd(test, result, step) {
        console.log('this text IS ALSO shown in console')
        someOtherFunctionTwo()
    }
}

module.exports = MyReporter

Now consider this test:

// ./tests/myTest.spec.js
const { test } = require('@playwright/test')

test('Some interesting test title', async ( {}, testInfo ) => {
    console.log('this text is NOT shown in the console')
    await someReallyCoolFunction()
})

// ./func/someFunctionFile.js
async function someReallyCoolFunction() {
     console.log('this text is NOT shown in the console')
}

As per code snippets above, only the console logging from the custom reporter is output to the actual console.

When commenting out the code in playwright.config.js for the custom reporter, all lines are output in the console as expected.

[dgozman]

@mstepin-edt There are two options to make console.log visible with custom reporter:

1. Handle onStdOut/onStdErr and print to the terminal in your custom reporter. This is what default reporters do for you behind the scenes.
2. Return false from printsToStdio(). In this case, Playwright will add some terminal output, including console.log output.

Let me know if this helps.

[mstepin-edt]

@dgozman I did spot those before I logged the bug report, and added them to my reporter. That said, it seems extremely counter-intuitive for a standalone reporter class to influence terminal output and have to explicitly proxy the output stream, else risk affecting existing functionality. There probably aren't all too many use cases where one might want to completely suppress terminal output, and should there be a need to suppress terminal output, it makes more sense for that to be a config option rather than this convoluted reporter path. The reporter needs to be a standalone listener -> ie. "you'll receive the stdOut and stdErr events, do with it what you will, but we'll continue doing what we've always done until now".
As you said, the default reporters under the covers continue to output to terminal, so why should they stop outputting just because there is a new reporter in town?

[dgozman]

@mstepin-edt The reporters API is designed so that you can replace default stdio reporter with anything. For example, you can use a json reporter that outputs a json dump to the stdout. I think we can improve documentation though.

[mstepin-edt]

@dgozman agreed, if at the very least some improved documentation comes out of this discussion, that would be great. I still stand by my above comment in that a new standalone module such as a reporter should be a consumer and not a proxy for existing behaviour.

[luixo]

Hello,
I found out that all the debug data I may want to look through is discarded when using a custom reporter which includes console.logs from tests itself as well as output from --list cli option.

This does not make sense if I want to keep the tests progress while printing some custom reporter extra info to console in the end of all tests.

I believe the quick fix (though it will break the compatibility) might be to force existence of printsToStdio function.
The proper one would be to let custom reporters use some kind of logger manager that will let them log whatever they report in the end / while tests are running without (or with, if needed) conflicting with default reporter.