Additional options declared in the Nightwatch config file does not get passed to the cucumber runner

[luke-btn]

Describe the bug

I'm having an issue where when declaring additional options to cucumber runners, they do not seem to be getting passed to the cucumber runner. I've tried retry and format, but they do not seem to work.

While debugging, I can verify, the options I have declared in the configuration file are being picked up in the options variable at https://github.com/nightwatchjs/nightwatch/blob/main/lib/runner/test-runners/cucumber.js#L93, but the additionalOptions variable is empty after buildArgvValue() has been run at https://github.com/nightwatchjs/nightwatch/blob/main/lib/runner/test-runners/cucumber.js#L100

Forcefully changing the additionalOptions variable to the string 'retry=1' results in the expected behaviour with the scenario being retried once.

Expected behaviour: When specifying e.g. retry: 1 as an option when declaring a cucumber.js runner, if there was a failure in my test run, for the failing scenario to be retried once.

Sample test

N/A Affects any test that runs with a cucumber runner

Run with command

$ nightwatch test/sampleTest.js --your-other-arguments-here

Verbose output

Output

npx nightwatch --env chrome.local



ℹ Connected to ChromeDriver on port 4444 (1452ms).
  Using: chrome (100.0.4896.127) on MAC OS X.

  ℹ Loaded url <***>in 3766ms
✔ Element <.header-text> was visible after 22 milliseconds.
DEPRECATED: the assertion .containsText() has been deprecated and will be removed from future versions. Use assert.textContains().
✔ Testing if element <.header-text> contains text 'Welcome' (17ms)
..  ✖ Testing if the URL contains '<***>' in 5000ms - expected "contains '<***>'" but got: "<***>" (5279ms)
    at Object.verifyLoggedIn (***)
    at World.<anonymous> (***) 

F-.

ℹ Connected to ChromeDriver on port 4444 (1262ms).
  Using: chrome (100.0.4896.127) on MAC OS X.

  ℹ Loaded url <***> in 6081ms
✔ Element <.header-text> was visible after 20 milliseconds.
DEPRECATED: the assertion .containsText() has been deprecated and will be removed from future versions. Use assert.textContains().
✔ Testing if element <.header-text> contains text 'Welcome' (15ms)
..✔ Element <//h1//span[contains(text(), "Open returns")]> was visible after 545 milliseconds.
..

Failures:

1) Scenario: Successful log in to the SVR application # features/login.feature:3
   ✔ Before # node_modules/nightwatch/cucumber-js/_setup_cucumber_runner.js:6
   ✔ Given I have navigated to the SVR login page # step-definitions/login.js:4
   ✔ When I enter my correct login credentials # step-definitions/login.js:23
   ✖ Then I see I have logged into the correct environment # step-definitions/login.js:60
       Testing if the URL contains '<***>' in 5000ms - expected "contains '<***>'" but got: "<***>" (5279ms)
           at Proxy.<anonymous> (<***>)
           at Object.verifyLoggedIn (<***>)
           at World.<anonymous> (<***>)
   - Then I see I have logged into the correct environment again # step-definitions/login.js:74
   ✔ After # node_modules/nightwatch/cucumber-js/_setup_cucumber_runner.js:66

2 scenarios (1 failed, 1 passed)
7 steps (1 failed, 1 skipped, 5 passed)
0m29.174s (executing steps: 0m29.161s)

Configuration

The env with the retry config is chrome.local

nightwatch.json

const Services = {}; loadServices();

//  _   _  _         _      _                     _          _
// | \ | |(_)       | |    | |                   | |        | |
// |  \| | _   __ _ | |__  | |_ __      __  __ _ | |_   ___ | |__
// | . ` || | / _` || '_ \ | __|\ \ /\ / / / _` || __| / __|| '_ \
// | |\  || || (_| || | | || |_  \ V  V / | (_| || |_ | (__ | | | |
// \_| \_/|_| \__, ||_| |_| \__|  \_/\_/   \__,_| \__| \___||_| |_|
//             __/ |
//            |___/

module.exports = {
  src_folders: ['step-definitions'],

  page_objects_path: '',

  custom_commands_path: '',

  custom_assertions_path: '',

  globals_path: '',

  webdriver: {},

  test_runner: {
    type: 'cucumber',
    options: {
      feature_path: 'features',
      additional_config: '',
      parallel: 2
    }

  },

  test_settings: {
    default: {
      disable_error_log: false,
      launch_url: 'https://nightwatchjs.org',

      screenshots: {
        enabled: false,
        path: 'screens',
        on_failure: true
      },

      desiredCapabilities: {
        browserName: 'chrome'
      },

      webdriver: {
        start_process: true,
        server_path: ''
      }
    },

    safari: {
      desiredCapabilities: {
        browserName: 'safari',
        alwaysMatch: {
          acceptInsecureCerts: false
        }
      },
      webdriver: {
        start_process: true,
        server_path: ''
      }
    },

    firefox: {
      desiredCapabilities: {
        browserName: 'firefox',
        alwaysMatch: {
          acceptInsecureCerts: true,
          'moz:firefoxOptions': {
            args: [
              // '-headless',
              // '-verbose'
            ]
          }
        }
      },
      webdriver: {
        start_process: true,
        server_path: '',
        cli_args: [
          // very verbose geckodriver logs
          // '-vv'
        ]
      }
    },

    chrome: {
      desiredCapabilities: {
        browserName: 'chrome',
        'goog:chromeOptions': {
          w3c: true,
          args: [
            //'--no-sandbox',
            //'--ignore-certificate-errors',
            //'--allow-insecure-localhost',
            //'--headless'
          ]
        }
      },

      webdriver: {
        start_process: true,
        server_path: '',
        cli_args: [
          // --verbose
        ]
      }
    },

    edge: {
      desiredCapabilities: {
        browserName: 'MicrosoftEdge',
        'ms:edgeOptions': {
          w3c: true,
          args: [
            //'--headless'
          ]
        }
      },

      webdriver: {
        start_process: true,
        server_path: '',
        cli_args: [
          // --verbose
        ]
      }
    },

    'cucumber-js': {
      src_folders: ['step-definitions'],

      test_runner: {
        type: 'cucumber',

        options: {
          feature_path: 'features',
        }
      }
    },

    'chrome.standalone': {
      src_folders: ['step-definitions'],

      test_runner: {
        type: 'cucumber',

        options: {
          feature_path: 'features'
        }
      },
      selenium: {
        host: 'standalone-chrome',
        port: 4444
      },
      desiredCapabilities: {
        "browserName": "chrome",
        "chromeOptions" : {
        "w3c": false,
        "args" : ["--no-sandbox"],
        "prefs" : {
            "download" : {
            "default_directory": "/tmp",
            "prompt_for_download": false
        },
            "profile": {
                "default_content_setting_values" : { automatic_downloads: 1 }
            }
         }
      },
       "acceptSslCerts": true
      },
      disable_error_log: true,
      webdriver: {
        timeout_options: {
          timeout: 15000,
          retry_attempts: 3
        },
        keep_alive: true,
        start_process: false
      }
    },

    'chrome.local': {
      src_folders: ['step-definitions'],

      test_runner: {
        type: 'cucumber',

        options: {
          feature_path: 'features',
          retry: 1,
        }
      },
      selenium: {
        host: 'localhost',
        port: 4444
      },
      desiredCapabilities: {
        "browserName": "chrome",
        "chromeOptions" : {
        "w3c": false,
        "args" : ["--no-sandbox"],
        "prefs" : {
            "download" : {
            "default_directory": "/tmp",
            "prompt_for_download": false
        },
            "profile": {
                "default_content_setting_values" : { automatic_downloads: 1 }
            }
         }
      },
       "acceptSslCerts": true
      },
      disable_error_log: true,
      webdriver: {
        timeout_options: {
          timeout: 15000,
          retry_attempts: 3
        },
        keep_alive: true,
        start_process: true
      }
    },

    browserstack: {
      selenium: {
        host: 'hub-cloud.browserstack.com',
        port: 443
      },
      desiredCapabilities: {
        'bstack:options': {
          userName: '${BROWSERSTACK_USER}',
          accessKey: '${BROWSERSTACK_KEY}'
        }
      },

      disable_error_log: true,
      webdriver: {
        timeout_options: {
          timeout: 15000,
          retry_attempts: 3
        },
        keep_alive: true,
        start_process: false
      }
    },

    'browserstack.local': {
      extends: 'browserstack',
      desiredCapabilities: {
        'browserstack.local': true
      }
    },

    'browserstack.chrome': {
      extends: 'browserstack',
      desiredCapabilities: {
        browserName: 'chrome',
        chromeOptions: {
          w3c: true
        }
      }
    },

    'browserstack.firefox': {
      extends: 'browserstack',
      desiredCapabilities: {
        browserName: 'firefox'
      }
    },

    'browserstack.ie': {
      extends: 'browserstack',
      desiredCapabilities: {
        browserName: 'internet explorer',
        browserVersion: '11.0'
      }
    },

    'browserstack.safari': {
      extends: 'browserstack',
      desiredCapabilities: {
        browserName: 'safari'
      }
    },

    'browserstack.local_chrome': {
      extends: 'browserstack.local',
      desiredCapabilities: {
        browserName: 'chrome'
      }
    },

    'browserstack.local_firefox': {
      extends: 'browserstack.local',
      desiredCapabilities: {
        browserName: 'firefox'
      }
    },
    selenium_server: {
      selenium: {
        start_process: true,
        port: 4444,
        server_path: (Services.seleniumServer ? Services.seleniumServer.path : ''),
        cli_args: {
          'webdriver.gecko.driver': (Services.geckodriver ? Services.geckodriver.path : ''),
          'webdriver.chrome.driver': (Services.chromedriver ? Services.chromedriver.path : '')
        }
      }
    },

    'selenium.chrome': {
      extends: 'selenium_server',
      desiredCapabilities: {
        browserName: 'chrome',
        chromeOptions: {
          w3c: true
        }
      }
    },

    'selenium.firefox': {
      extends: 'selenium_server',
      desiredCapabilities: {
        browserName: 'firefox',
        'moz:firefoxOptions': {
          args: [
            // '-headless',
            // '-verbose'
          ]
        }
      }
    }
  }
};

function loadServices() {
  try {
    Services.seleniumServer = require('selenium-server');
  } catch (err) {}

  try {
    Services.chromedriver = require('chromedriver');
  } catch (err) {}

  try {
    Services.geckodriver = require('geckodriver');
  } catch (err) {}
}

Your Environment

Executable	Version
nightwatch --version	2.1.3
npm --version	6.14.15
node --version	v14.18.1
cucumber js	8.0.0-rc.2

| OS | Version |
| macOS Montery | 12.3 |