Steps highlighting shows "Undefined step" warning on Fedora Linux

[Kolobamanacas]

👓 What did you see?

"Undefined step" error occurs when creating scenario with data tables for each step of given/when/then. Also params are not highlighted properly.

✅ What did you expect to see?

No error. All params to be highlighted.

📦 Which tool/library version are you using?

Fedora Linux 36
Kernel Version 6.0.5-200.fc36.x86_64 (64-bit)
KDE Plasma 5.25.5

Visual Studio Code 1.73.0 | Dark+ (default dark)
Cucumber 1.5.1
TypeScript 4.6.2
jest 27.5.1
jest-cucumber 3.0.1

🔬 How could we reproduce it?

1. Install stable release of Fedora 36.
2. Install Visual Studio Code 1.73.0.
3. Install Cucumber 1.5.1 extension.
4. Set path to your feature folder.
5. Create a file "do-stuff.feature" in that folder.
6. Put The following code in file:

Feature: Do CCC stuff

  Scenario: Get "<AAA>" for given "<BBB>", "<CCC>" and "<DDD>"
    Given Record with "<BBB>" and "<EEE>"
      | BBB    | EEE    |
      | Value1 | Value2 |
    When Run command "do-stuff --DDD <DDD> --CCC <CCC>"
      | DDD    | CCC    |
      | Value3 | Value4 |
    Then "CCC" table contains record with "<CCC>"
      | CCC    |
      | Value4 |
    And "CCC-sub" table contains record with "<DDD>", "<CCC>", "<BBB>" and "<AAA>"
      | DDD    | CCC    | BBB    | AAA    |
      | Value3 | Value4 | Value1 | Value5 |

7. See the error. All keywords of given/when/then/and will be highlighted with:

Undefined step: Record with "BBB" and "EEE"

📚 Any additional context?

I've noticed that if I add "Examples:" section, then all errors are gone and params are highlighted correctly. Even when examples section is empty.

[aslakhellesoy]

From your use of <AAA> etc it looks like your intention is to use an Examples: section underneath.

Could you please share the expressions of your step definitions as well?

[The-BDD-Coach]

I also see this error; here I have three uses of 'Given these books in the catalog':

Feature: Library book searches and book delivery

    Scenario: The catalog can be searched by author's name.
    Given these books in the catalog

    Scenario Outline: Author name searches match if the searched string appears anywhere in the author's name.
    Given these books in the catalog

    Examples:

    Scenario: Author name searches are case insensitive.
    Given these books in the catalog

    Examples:

• The first use of 'Given these books in the catalog' does not have an Examples table; the VSCode plugin underlines it to indicate that there isn't a matching step definition.
• The second use is a Scenario Outline with an Examples table, and the line is NOT underlined.
• The third use is a Scenario (which shouldn't have an Examples table), but I added an Examples table and the line is NOT underlined.

Here are my step definitions:

  @given(/these books in the catalog/)
  public givenTheseBooksInTheCatalog(table: any) {
    this._scenarioContext.catalog = new Catalog([]);
    const creationHelperMethods = new CreationHelperMethods();
    creationHelperMethods.ReadBooksAndAddToCatalog(this._scenarioContext.catalog, table);
  }

[schwaerz]

Same here. As soon as I am adding Examples: the yellow underlineing vanishes.
However, even when underlined, the goto step definition works, as well as the autocomplete.

Maybe I should mention that those steps are parameterized.
Using Version 1.7.0 of the plugin.

[kieran-ryan]

"Undefined step" error occurs when creating scenario with data tables for each step of given/when/then. Also params are not highlighted properly.

@Kolobamanacas, delimited parameters (< >) are intended as a template for scenario outline tables and are thus only highlighted in the presence of an examples table (specifically the Examples: keyword). Would you be able to provide a step definition for one of the steps you have outlined above to understand usage? See gherkin reference on scenario outlines:

Scenario outlines allow us to more concisely express these scenarios through the use of a template with < >-delimited parameters:

@Kolobamanacas, @The-BDD-Coach and @schwaerz, diagnostics (underlining) for undefined steps is currently unsupported with Scenario Outlines (see cucumber/language-service#149 for some of the challenges to resolve that). Thus, when the Examples: keyword is introduced, underlining of undefined steps is disabled.

However, even when underlined, the goto step definition works, as well as the autocomplete.

@schwaerz, completions will appear for gherkin steps included in existing feature files, even if there is no matching step definition. However, would be interested to see an example where a step is marked 'undefined', but the 'go to step definition' functionality is working; would you be able to provide one?

[kieran-ryan]

Original comment on No error. and all subsequent comments relate to lack of support for highlighting undefined steps with Scenario Outlines (inclusion of Examples:) - covered by #90.

In relation to All params to be highlighted., no MRE provided with step definitions so whether intention was to use a Scenario Outline is presently indeterminate.

From your use of <AAA> etc it looks like your intention is to use an Examples: section underneath.

Could you please share the expressions of your step definitions as well?

Issue will be reopened if further detail provided that necessitates a fix or feature.

[The-BDD-Coach]

@kieran-ryan you said "subsequent comments relate to lack of support for highlighting undefined steps with Scenario Outlines"; all of my steps are defined. I can run the requirements - all of them - and they pass. I can do this with Behave and with Pytest-BDD-NG; I see the same behavior:

• Step definitions are marked as undefined unless I add 'Examples:' to the Scenario.
• As soon as I type the ':' the yellow squiggly underlines disappear.
• 'Go to step definition' doesn't work in either a Scenario or a Scenario Outline.
• I see this behavior for plain text step definitions:

@given(u'these books in the catalog')
def step_impl(context):
    context.catalog = Catalog()
    read_books_and_add_to_catalog(context.catalog, context.table)

Or for Pytest-BDD-NG:

@given("these books in the catalog")
def these_books_in_the_catalog(step: PickleStep, catalog: Catalog):
    catalog.add_books_to_catalog(step.data_table)

• I also see this behavior for step definitions that use Regular Expressions or Cucumber Expressions, even when the Gherkin does not contain any references to an Examples table:
  When a name search is performed for Stephen

@when("a (?P<search_type>name|title|ISBN) search is performed for (?P<search_term>.+)")
def a_SEARCH_TYPE_is_performed_for_SEARCH_TERM(context, search_type, search_term):
    # TODO: This doesn't support Title or ISBN searches
    context.search_results = context.catalog.search_by_author_name(search_term)

My VSCode 'about' shows:
Version: 1.87.2
Commit: 863d2581ecda6849923a2118d93a088b0745d9d6
Date: 2024-03-08T15:20:57.520Z
Electron: 27.3.2
ElectronBuildId: 26836302
Chromium: 118.0.5993.159
Node.js: 18.17.1
V8: 11.8.172.18-electron.0
OS: Darwin arm64 23.4.0

and the V1.8.1 Cucumber plugin.

[kieran-ryan]

@kieran-ryan you said "subsequent comments relate to lack of support for highlighting undefined steps with Scenario Outlines"; all of my steps are defined. I can run the requirements - all of them - and they pass. I can do this with Behave and with Pytest-BDD-NG; I see the same behavior:

• Step definitions are marked as undefined unless I add 'Examples:' to the Scenario.
• As soon as I type the ':' the yellow squiggly underlines disappear.

Yes, the comments on this issue relate to a lack of feature support by the Cucumber Language Service for highlighting whether a step is marked undefined with Scenario Outlines (contains Examples:). This is captured under existing issues linked above (cucumber/language-service/issues/149, #90, cucumber/monaco#60). See defensive diagnostics code which returns early if inside a Scenario Outline - instead of progressing to check whether the step is undefined.

The new and separate details relating to Python step definitions can occur for a number of reasons:

• Using explicit unicode strings (u"") - support has been implemented in the Language Service (Python u-string patterns language-service#173) but will be unavailable in the VSCode extension until a release has propogated to it. Suggest to remove the u prefix if not using Python 2 - Behave still suggests this prefix as it is compatible with Python 2 - though it is unrequired in Python 3 and will be removed (array: Add 'w' type and deprecate 'u' type. python/cpython#80480). Thus, steps matching your first pattern will be considered 'undefined' - though still execute.
• Using Cucumber Expressions optionals e.g. "I have {int} cuke(s)" to match "I have 1 cuke" or "I have 5 cukes" (Cucumber Expressions with optionals interpreted as Regular Expressions language-service#191) as brackets are incorrectly used to consider the pattern as a Regular Expression rather than a Cucumber Expression
• Using a name keyword inside Python code that isn't a Cucumber Expressions parameter type as it causes an unhandled exception with the Language Service while trying to locate parameter types - and thus raises an exception if ordinary code has a partial match with the schema of a parameter type (Cannot index Python glue files containing 'name' keyword arguments language-service#181)
• Using named parameters inside step definition patterns without an associated parameter type of the same name. For example, with {text} in the Behave step parameters example, the Language Service would look for a parameter type called text and not find one and this pattern would thus not be used for matching. The mitigation is to remove the named parameter from the pattern only and leave it blank ({}) - which is treated the same in Cucumber Expressions (matches anything).
• In general, there are numerous matchers for different Python frameworks - and accurately detecting and covering every possible variant is quite a challenge; particularly for lesser used frameworks and matchers

Your second pattern is detectable (see below) - but this can be impacted if some of the issues above are present elsewhere in that module or if the extension configuration settings are not checking the location in which you are storing step definitions.

The earlier typescript example will fail as the Language Service looks for the capitalised variants of the decorators (@Given, @When, @Then) - as used in the Cucumber documentation - though not sure which framework version is being used with this step definition exactly; and as such won't be detected.

  @given(/these books in the catalog/)
  public givenTheseBooksInTheCatalog(table: any) {
    this._scenarioContext.catalog = new Catalog([]);
    const creationHelperMethods = new CreationHelperMethods();
    creationHelperMethods.ReadBooksAndAddToCatalog(this._scenarioContext.catalog, table);
  }

Doing what I can to find and fix these issues. PRs welcome.

[kieran-ryan]

@The-BDD-Coach, with the latest release of the VSCode extension (v1.10.0), you may be interested in the following changes for Python support:

• Step definitions with Behave's generic @step decorator are detected
• Step definition patterns prefixed with an explicit unicode character (u"") are supported
• An error state is resolved where partial matches with keyword arguments used by parameter types prevents parsing step definitions or parameter types from glue files e.g. if name keyword argument was used in any Python code in that module without a regexp keyword argument
• Generated step definitions no longer generate with redundant commas and spaces

Looking forward to any feedback 👍

[The-BDD-Coach]

@kieran-ryan I installed the 1.10.0 VSCode extension but am still seeing significant problems:

1. Most (I think All) steps in Scenarios show as undefined, even for plain text matchers
2. Steps in Scenario Outlines show as defined, but 'Go to definition' doesn't work; it just says 'No definition for (the word the cursor is on)'.

This is true for both Behave and Pytest-BDD-NG - and all of the Behave step definitions worked in the past.

If you have any suggestions on the best way to troubleshoot this I would be glad hear them.

[kieran-ryan]

@The-BDD-Coach, would it be possible to create a small minimal reproducible example with a repository containing the step definitions and environment setup? We can then analyse what might be causing the issue.

What version of Visual Studio Code are you using? A copy of the extension output logs (Output -> Cucumber Language Server) would also be great.

This is true for both Behave and Pytest-BDD-NG - and all of the Behave step definitions worked in the past.

From 'Version History' in the Visual Studio Marketplace, if you install an older version of the extension, does the issue persist?

[The-BDD-Coach]

That is a good suggestion; I will work on that.

[kieran-ryan]

2. Steps in Scenario Outlines show as defined, but 'Go to definition' doesn't work; it just says 'No definition for (the word the cursor is on)'.

Please note that marking steps as 'undefined' within Scenario Outlines is presently unsupported. Therefore, rather than being considered 'defined', diagnostics (underlining) and matching the steps against definitions is disabled with Scenario Outlines at this time.

Yes, the comments on this issue relate to a lack of feature support by the Cucumber Language Service for highlighting whether a step is marked undefined with Scenario Outlines (contains Examples:). This is captured under existing issues linked above (cucumber/language-service/issues/149, #90, cucumber/monaco#60). See defensive diagnostics code which returns early if inside a Scenario Outline - instead of progressing to check whether the step is undefined.

[The-BDD-Coach]

@kieran-ryan I created https://github.com/The-BDD-Coach/Step-Definition-Bug/tree/main, and will open a new defect for tracking this.

[The-BDD-Coach]

@kieran-ryan I opened #226 and attached the Cucumber Language Server logs. The logs are interesting:
[Info - 10:19:21 AM] * Found 0 feature file(s) in ["src/test//.feature","features//.feature","tests/**/.feature","specs//.feature"]
[Info - 10:19:21 AM] * Found 0 steps in those feature files
[Info - 10:19:21 AM] * Found 0 glue file(s) in ["specs//.cs","features//.js","features//*.jsx","features//.php","features//.py","features//.rs","features//.rb","features/**/.ts","features//.tsx","features//_test.go","src/test/**/.java","tests//.py","tests//.rs"]
[Info - 10:19:21 AM] * Found 0 parameter types in those glue files
[Info - 10:19:21 AM] * Found 0 step definitions in those glue files