Incorrect BDD Parallel execution background fail/success

[andremoah]

First of all, thank you for this amazing project. I have been using it in my company and we did over 1k e2e tests across multiple projects having no issues.

Since our integration tests suite grew considerably, we wanted to run them in parallel to avoid pipelines running for hours.

What are you trying to achieve?

Running BDD features in parallel and getting the correct amount of fails and success.

What do you get instead?

In the picture below, all the tests passed, but after checking the report I saw that there were 3 failed tests all of them in the background steps.

Running the tests with npx codeceptjs run-workers 4 --reporter mocha-multi --verbose I get the following output:

Details

• CodeceptJS version: 3.0.6
• NodeJS Version: 14.15.1
• Operating System: Linux / OSX
• puppeteer Version: 5.5.0
• Configuration file:

exports.config = {
    gherkin: {
        features: './features/*.feature',
		steps: [...]
    },
    helpers: {
        Mochawesome: {
            disableScreenshots: false
        },
        Puppeteer: {
            chrome: {
                args: ['--start-maximized', '--no-sandbox', '--disable-web-security', '--disable-setuid-sandbox', '--proxy-server="direct://"', '--proxy-bypass-list=*', '--disable-dev-shm-usage', '--shm-size=1gb'],
                defaultViewport: {
                    height: 625,
                    width: 1280
                }
            },
            show: true,
            // eslint-disable-next-line no-process-env
            url: '...'
        },
    },
    hooks: [],
    include: {
        // eslint-disable-next-line id-length
        I: './steps_file.js'
    },
    mocha: {
        bail: true,
        reporterOptions: {
            'codeceptjs-cli-reporter': {
                options: {
                    steps: true,
                    verbose: false,
                },
                stdout: '-',
            },
            mochawesome: {
                options: {
                    consoleReporter: false,
                    html: false,
                    overwrite: false,
                    reportDir: './',
                    reportFilename: 'e2e-report.json',
                    timestamp: true
                },
                stdout: '-'
            }
        }
    },
    name: 'cockpit-integration',
    output: './screenshots/out',
    plugins: {
        retryFailedStep: {
            enabled: true
        },
        screenshotOnFail: {
            enabled: false
        },
        stepByStepReport: {
            deleteSuccessful: true,
            enabled: false,
            fullPageScreenshots: false
        },
        tryTo: {
            enabled: true,
            registerGlobal: true
        },
    },
    tests: './*.test.js',
    timeout: 999999999999999
}

Edit 27/04

Digging deep into the code I traced the problem to codeceptjs/lib/command/workers/runTests.js

When sendToParentThread, there is a method called simplifyTest but it only handles the err object to be sent if there is a property err in the test object passed. The issue happens when an actual err is passed with a funcion in it, and the test contains no err property.

postMessage documentation gives a clear definition about what can or cannot be sent to a worker:

postMessage accept only value or JavaScript object handled by the structured clone algorithm, which includes cyclical references.

Looking at the structured clone algorithm, it accepts:

All primitive types (However, not symbols), Boolean object, String object, Date, RegExp (The lastIndex field is not preserved.), Blob, File, FileList, ArrayBuffer, ArrayBufferView (This basically means all typed arrays like Int32Array etc.), ImageBitmap, ImageData, Array, Object (This just includes plain objects (e.g. from object literals)), Map, Set

What it does not accept

Error and Function objects cannot be duplicated by the structured clone algorithm; attempting to do so will throw a DATA_CLONE_ERR exception.

Trying to fix the problem and adding the handler for err objects passed like this works partially and we start getting tests that fail in the background hooks.

I would love to make a PR with this change but I understand this is probably a core change that would probably have some other implications that I can't anticipate

[DavertMik]

I am sorry i didnt have time earlier to look into this. Sure, any PR is welcome, better to start discussion as a part of PR, it helps to dive into context