Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cli: use kebab-case for --retry-tag-filter arg #1293

Merged
merged 14 commits into from
Mar 17, 2020
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ Please see [CONTRIBUTING.md](https://github.com/cucumber/cucumber/blob/master/CO
#### Bug fixes

* don't execute BeforeAll and AfterAll hooks when in dry-run
* support correct case for `--retry-tag-filter` CLI argument

### [6.0.5](https://github.com/cucumber/cucumber-js/compare/v6.0.4...v6.0.5) (2019-11-13)

Expand Down
2 changes: 1 addition & 1 deletion docs/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,7 @@ By default, cucumber-js runs the entire suite and reports all the failures. This
## Retry failing tests

Use `--retry <int>` to rerun tests that have been failing. This can be very helpful for flaky tests.
To only retry failing tests in a subset of test use `--retryTagFilter <EXPRESSION>` (use the same as in Use [Tags](#tags))
To only retry failing tests in a subset of test use `--retry-tag-filter <EXPRESSION>` (use the same as in Use [Tags](#tags))

## Transpilation

Expand Down
45 changes: 33 additions & 12 deletions features/retry.feature
Original file line number Diff line number Diff line change
@@ -1,17 +1,38 @@
Feature: Retry flaky tests

Using the `--retry` flag will retry failing tests for the specified number of times
Additionally using the `--retryTagFilter` flag will re-run only tests matching the tag expression
Additionally using the `--retry-tag-filter` flag will re-run only tests matching the tag expression

@spawn
Scenario: running Cucumber JS with --retryTagFilter but no positive --retry will fail
When I run cucumber-js with `--retryTagFilter @flaky`
Scenario: running Cucumber JS with --retry-tag-filter but no positive --retry will fail
When I run cucumber-js with `--retry-tag-filter @flaky`
Then the error output contains the text:
"""
Error: a positive --retry count must be specified when setting --retryTagFilter
Error: a positive --retry count must be specified when setting --retry-tag-filter
"""
And it fails

@spawn
Scenario: running Cucumber JS with --retryTagFilter in camel case will result in a warning
Given a file named "features/a.feature" with:
"""
Feature:
Scenario:
Given a step
"""
Given a file named "features/step_definitions/cucumber_steps.js" with:
"""
const {Given} = require('cucumber')

Given(/^a step$/, function() {})
"""
When I run cucumber-js with `--retry 1 --retryTagFilter @flaky`
Then the error output contains the text:
"""
the argument --retryTagFilter is deprecated and will be removed in a future release; please use --retry-tag-filter
"""
But it passes

Scenario: running Cucumber JS with negative --retry will fail
When I run cucumber-js with `--retry -1`
Then the error output contains the text:
Expand Down Expand Up @@ -322,7 +343,7 @@ Feature: Retry flaky tests
"""
And it fails

Scenario: retrying a flaky test matching --retryTagFilter will eventually make it pass
Scenario: retrying a flaky test matching --retry-tag-filter will eventually make it pass
Given a file named "features/a.feature" with:
"""
Feature:
Expand All @@ -344,12 +365,12 @@ Feature: Retry flaky tests
throw 'fail'
})
"""
When I run cucumber-js with `--retry 1 --retryTagFilter '@flaky'`
When I run cucumber-js with `--retry 1 --retry-tag-filter '@flaky'`
Then scenario "Flaky" attempt 0 step "Given a flaky step" has status "failed"
Then scenario "Flaky" attempt 1 step "Given a flaky step" has status "passed"
And it passes

Scenario: a flaky test not matching --retryTagFilter won't re-run and just fail
Scenario: a flaky test not matching --retry-tag-filter won't re-run and just fail
Given a file named "features/a.feature" with:
"""
Feature:
Expand All @@ -371,11 +392,11 @@ Feature: Retry flaky tests
throw 'fail'
})
"""
When I run cucumber-js with `--retry 1 --retryTagFilter '@not_flaky'`
When I run cucumber-js with `--retry 1 --retry-tag-filter '@not_flaky'`
Then scenario "Flaky" step "Given a flaky step" has status "failed"
And it fails

Scenario: retrying a flaky test matching --retryTagFilter will eventually make it pass but not-matching will not be retried (AND operator between tags)
Scenario: retrying a flaky test matching --retry-tag-filter will eventually make it pass but not-matching will not be retried (AND operator between tags)
Given a file named "features/a.feature" with:
"""
Feature:
Expand Down Expand Up @@ -410,13 +431,13 @@ Feature: Retry flaky tests
throw 'fail'
})
"""
When I run cucumber-js with `--retry 1 --retryTagFilter '@flaky and @anOtherTag'`
When I run cucumber-js with `--retry 1 --retry-tag-filter '@flaky and @anOtherTag'`
Then scenario "Flaky" attempt 0 step "Given a flaky step" has status "failed"
Then scenario "Flaky" attempt 1 step "Given a flaky step" has status "passed"
And scenario "Also Flaky" step "Given an other flaky step" has status "failed"
And it fails

Scenario: retrying a flaky test matching --retryTagFilter will eventually make it pass but not-matching will not be retried (OR operator between tags)
Scenario: retrying a flaky test matching --retry-tag-filter will eventually make it pass but not-matching will not be retried (OR operator between tags)
Given a file named "features/a.feature" with:
"""
Feature:
Expand Down Expand Up @@ -464,7 +485,7 @@ Feature: Retry flaky tests
throw 'fail'
})
"""
When I run cucumber-js with `--retry 1 --retryTagFilter '@anOtherTag or @oneMoreTag'`
When I run cucumber-js with `--retry 1 --retry-tag-filter '@anOtherTag or @oneMoreTag'`
Then scenario "Flaky" attempt 0 step "Given a flaky step" has status "failed"
And scenario "Flaky" attempt 1 step "Given a flaky step" has status "passed"
And scenario "Also Flaky" attempt 0 step "Given an other flaky step" has status "failed"
Expand Down
12 changes: 10 additions & 2 deletions src/cli/argv_parser.ts
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ const ArgvParser = {
validateRetryOptions(options: IParsedArgvOptions): void {
if (options.retryTagFilter !== '' && options.retry === 0) {
throw new Error(
'a positive --retry count must be specified when setting --retryTagFilter'
'a positive --retry count must be specified when setting --retry-tag-filter'
)
}
},
Expand Down Expand Up @@ -187,7 +187,7 @@ const ArgvParser = {
0
)
.option(
'--retryTagFilter <EXPRESSION>',
'--retryTagFilter, --retry-tag-filter <EXPRESSION>',
`only retries the features or scenarios with tags matching the expression (repeatable).
This option requires '--retry' to be specified.`,
ArgvParser.mergeTags,
Expand Down Expand Up @@ -223,6 +223,14 @@ const ArgvParser = {
args: program.args,
}
},

lint(fullArgv: string[]): void {
if (fullArgv.includes('--retryTagFilter')) {
console.warn(
'the argument --retryTagFilter is deprecated and will be removed in a future release; please use --retry-tag-filter'
)
}
},
}

export default ArgvParser
1 change: 1 addition & 0 deletions src/cli/configuration_builder.ts
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ export default class ConfigurationBuilder {
constructor({ argv, cwd }: INewConfigurationBuilderOptions) {
this.cwd = cwd

ArgvParser.lint(argv)
const parsedArgv = ArgvParser.parse(argv)
this.args = parsedArgv.args
this.options = parsedArgv.options
Expand Down
11 changes: 9 additions & 2 deletions src/cli/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ import Gherkin from 'gherkin'
import { ISupportCodeLibrary } from '../support_code_library_builder/types'
import { IParsedArgvFormatOptions } from './argv_parser'
import { WriteStream } from 'fs'

const { incrementing, uuid } = IdGenerator

export interface ICliRunResult {
Expand Down Expand Up @@ -64,8 +65,14 @@ export default class Cli {
}

async getConfiguration(): Promise<IConfiguration> {
const fullArgv = await getExpandedArgv({ argv: this.argv, cwd: this.cwd })
return ConfigurationBuilder.build({ argv: fullArgv, cwd: this.cwd })
const fullArgv = await getExpandedArgv({
argv: this.argv,
cwd: this.cwd,
})
return ConfigurationBuilder.build({
argv: fullArgv,
cwd: this.cwd,
})
}

async initializeFormatters({
Expand Down