getTestCasesFromFilesystem seems to be inaccessible

[binarymist]

I've posted a message on the Slack, @Raymond also posted there on Feb 29.

I've searched for related issues and found #1220 which is how I used to do it.

I've just upgraded from 5.1.0 to 7.0.0-rc.0, I've read the Changelog. It appears that there is no longer a way to access cucumber.getTestCasesFromFilesystem, although the routine that does the work still appears to be in dist/cucumber.js.
Is there a new approach for getting the test cases from filesystem that makes getTestCasesFromFilesystem redundant?

Thanks.

[binarymist]

I've consolidated the needed code, although it's still tied to an old version of gherkin.

[davidjgoss]

Hi @binarymist, this area has been refactored, the loading of features from the file system is now handled by the Gherkin library.

Could you expand a little on your use case - e.g. is it some reporting you want to do as part of the run? There'll be a way, just want to understand your requirement.

[binarymist]

Hi @davidjgoss... Thanks for you response.

Yeah, so at this stage we (the back-end) send back the test plan (the concatenated feature file contents that comply with the tags) to the purpleteam CLI that will run, the plan is printed to screen. Does that make sense and is it enough information? Basically the getTestCasesFromFilesystem was exactly what we needed and were using... until we upgraded.

The consuming code is here.

[binarymist]

Hi @davidjgoss ... Any update? Would love to be able to close this (https://gitlab.com/purpleteam-labs/purpleteam/-/issues/25) issue.
Thanks.

[aslakhellesoy]

@binarymist is there anything preventing you from using the @cucumber/gherkin module directly to parse Gherkin documents?

[binarymist]

HI @aslakhellesoy and thanks for your response. I only just read this, didn't get any notification. Let me investigate your response, I've added it to our backlog to do and report back. Thanks.

[binarymist]

Hi @aslakhellesoy .

Looking at the README, it's unclear if @cucumber/gherkin provides the same functionality that was removed?

The Usage example has an options object but it doesn't appear to be used?

How do consumers of fromPaths from @cucumber/gherkin provide options { cwd, eventBroadcaster, featureDefaultLanguage, featurePaths, pickleFilter } that are required to retreive only the specific test cases that match the options?

Thanks.

[binarymist]

Does the new: gherkin.fromPaths(['features/hello.feature']) provide the same functionality as the removed getTestCasesFromFilesystem? As I've mentioned, by the looks of the missing options with the newer approach, the answer is no. Is this correct?

[aslakhellesoy]

Hi @binarymist

As far as I can tell, the getTestCasesFromFilesystem function returns an array of { pickle, uri } tuples.

The gherkin.fromPaths function will give you a readable stream of cucumber messages. The messages in this stream will be either Source, GherkinDocument or Pickle. You can use options to get just pickles if you want.

To me it sounds like that should be a simple replacement.

BTW, you're using a deprecated version of gherkin. We've moved to @cucumber/gherkin now.

[binarymist]

Thanks for your response @aslakhellesoy !

I had high hopes for your response, but...

TypeError: gherkin.fromPaths is not a function:

I tried both @cucumber/gherkin and the supposedly deprecated gherkin packages with the same result.

So I:

git clone git@github.com:cucumber/gherkin-javascript.git && cd gherkin-javascript
egrep -iR "fromPaths"
// Results in: README.md:const stream = gherkin.fromPaths(['features/hello.feature'])
// So... the fromPaths doesn't exist
// But... the README usage says to use fromPaths???

Also I'm assuming the README.md Usage is incorrect in it's require statement?
It should be:

cosnt gherkin = require(@cucumber/gherkin);

and then use some function other than the non-existent fromPaths?

Looks like @mikesalvia also had trouble finding it. This led me to your PR which after going through those changes just added more confusion, as these changes don't appear to be to the gherkin-javascript repo which we're supposed to be now using.

The gherkin-javascript repo also appears to be missing a CHANGELOG, so I'm finding it difficult to work out how to access the fromPaths function? that doesn't appear to exist?

BTW, you're using a deprecated version of gherkin. We've moved to @cucumber/gherkin now.

Yes, I've mentioned this here, although looking at the public package details now... there is no deprecation notice. Can you explain what's happening?

I'm also confused as to why the so called deprecated gherkin package refers to the same repository as the non deprecated @cucumber/gherkin package? I thought possibly they were being deployed from different branches, but in looking through the branches, I didn't see any obvious deployment branches. Can you also please shed some light on this?

So how exactly does one go about accessing this gherkin.fromPaths function in the @cucumber/gherkin ?

Thanks.

[aslakhellesoy]

Sorry this is so hard to find. It moved to @cucumber/gherkin-streams in version 17.0.0. More details in (cucumber/common#1331, cucumber/common#1332 and cucumber/common#1333)

You should be able to use it like this:

import GherkinStreams from '@cucumber/gherkin/dist/src/stream/GherkinStreams'
import { IdGenerator } from '@cucumber/messages'

const gherkinOptions = {
  defaultDialect: 'en',
  newId: IdGenerator.uuid(),
}
const gherkinEnvelopeStream = GherkinStreams.fromPaths(paths, gherkinOptions)

The gherkin-javascript repo also appears to be missing a CHANGELOG,

We're maintaining multiple language versions of gherkin, and the changelog is here.

there is no deprecation notice. Can you explain what's happening?

We moved all libraries to the @cucumber namespace last year, and deprecated cucumber at that time. We simply forgot to deprecate the other packages. I've created cucumber/common#1421 to track this.

I'm also confused as to why the so called deprecated gherkin package refers to the same repository as the non deprecated @cucumber/gherkin package?

We're maintaining a lot of multi-platform libraries in the cucumber/cucumber monorepo, and the location of the source code didn't change when we started releasing modules under the @cucumber namespace.

[binarymist]

Sorry this is so hard to find. It moved to @cucumber/gherkin-streams in version 17.0.0.

There is no package by the name of @cucumber/gherkin-streams publicly available in NPM.

The closest is @cucumber/gherkin which leads to repo: https://github.com/cucumber/gherkin-javascript which we've alredy been through and not found fromPaths

The only other place that looks like it may be is @cucumber/gherkin-utils or @cucumber/cucumber which both lead to the same repository https://github.com/cucumber/cucumber

In looking at the CHANGELOG.md it appears to be unreleased rather than in version 17.0.0?

Apon cloning of https://github.com/cucumber/cucumber-js I get a directory gherkin-streams which doesn't show in the github repository. Turns out fromPaths is actually in the cucumber/gherkin-streams directory of this repo. How does that work?

So I then npm install @cucumber/gherkin latest version (17.0.2) rather than 7.0.0-rc.0 and lo and behold, fromPaths is in there.

Am I correct in guessing that fromPaths will soon be turning up in a new NPM package called @cucumber/gherkin-streams when the next version 17.0.3 of @cucumber/gherkin is released/published?

So I can get the sources, pickles and gherkinDocuments but I don't see a way to filter the sources on pickleFilter?

The pickleFilter contains the details of my configured cucumber CLI.

As far as I can tell, the getTestCasesFromFilesystem function returns an array of { pickle, uri } tuples.

Yes... filtered by the pickleFilter which in my case lists the active tags

To me it sounds like that should be a simple replacement.

I've worked out how to do the replecement, but it still appears as though the same functionality can not be replicated with fromPaths

Have I missed some vital piece of the puzzle?

There doesn't seem to be any association in the cucumber messsages between what pickleFilter provides (tag filter, name filter, etc) and the actual source or at worst a file URI

[binarymist]

Almost there I think, I'll update this once coded and tested

[binarymist]

Hi @aslakhellesoy.

Can you please answer the outstanding questions from here, specifically:

Am I correct in guessing that fromPaths will soon be turning up in a new NPM package called @cucumber/gherkin-streams when the next version 17.0.3 of @cucumber/gherkin is released/published?

I've done the code changes, but I'm still struggling with:

To me it sounds like that should be a simple replacement.

As I mentioned here:

I've worked out how to do the replecement, but it still appears as though the same functionality can not be replicated with fromPaths

I've written some tests and some additional scenarios that should pass. The tests fail due to removed functionality (The algorithm from the removed cucumber.getTestCasesFromFilesystem that used to satisfy the tests I've written) and their isn't much point in implementing the additional test scenarios until we know we can make them pass.

Where is the code/algorithm that used to be used to do the filtering of the feature file URIs based on the tag expressions... Now that cucumber.getTestCasesFromFilesystem is gone?

Thanks.

[yopasa94]

I faced this issue some days ago and I replaced with 'parseGherkinMessageStream'

//This will return all the test id 
parseGherkinMessageStream({
      cwd: '',
      eventBroadcaster,
      eventDataCollector,
      gherkinMessageStream: gherkinMsgStream,
      order: 'defined',
      pickleFilter: new PickleFilter({
        cwd: '',
        tagExpression: cucumberTags,
      }),
    });
//returns list of scenarios as 'testsCases'

//Find all the information related to the scenarios
      tests = testCases.map((result) => {
        const pickeTest = eventDataCollector.getPickle(result);
        const gherkinDoc = eventDataCollector.getGherkinDocument(pickeTest.uri);
        const lineNumber = formatterHelpers.PickleParser.getPickleLocation({
          gherkinDocument: gherkinDoc,
          pickle: pickeTest,
        }).line;
// return desired information from the test
        return { uri: '../../' + pickeTest.uri, lineNumber };
      });

[binarymist]

Thanks @yopasa94 . A few questions from your implementation if you would be so kind?

I assume the parseGherkinMessageStream is in @cucumber/cucumber/lib/cli/helper.js?

What are the following properties being passed as the object arguement to parseGherkinMessageStream, did you define them, if so how and where, as I'll need to replicate them:

• eventDataCollector
• gherkinMessageStream
• formatterHelpers

[binarymist]

Any feedback @yopasa94 I searched your github and the internet but was unable to find your code?

[aurelien-reeves]

As you have noticed, there have been a major refactoring of the internals of cucumber in the past few month.
Also, @cucumber/gherkin-streams has been released publicly and provide the fromPaths method that will help parsing given .feature documents.

This is not a transparent refactoring: the new API relies on messages.

@binarymist it would be easier for us to help with your issue with more explanation regarding your problem. Could you please give us a concise summary of what you are facing, and a minimal reproducible example?

[binarymist]

There has been a lot of water under this bridge since I first posted this issue, so being concise may be difficult. I'll do my best though.

Rephrasing

As per my very first message in this issue on 2 Dec 2020, cucumber.getTestCasesFromFilesystem did "exactly" what I and others needed it to do.

What did I and others need it to do?

For us (purpleteam-labs) the main file involved (now and when it was doing what we needed it to do (last year)) in the Cucumber invocations is /src/api/app/models/app.js of the app-scanner.

In the past getActiveTestCases did exactly what we needed it to do.

We needed it to getActiveTestCasesFromFilesystem. We still need this functionality.

We provide the paths to our feature files, actually the cucumber Cli did and still does this.

The pickleFilter provided the details around what an active test case was, which I think was and is based on cucumber.tagExpression.

Simply put, we need the active test cases as a string based on the tags we initialise the Cucumber CLI with. Why? Because users of the system need a test plan, they need to know which feature files are going to be run when they start the test.

Where are we now?

Currently getActiveFeatureFileUris passes these tests.

Currently getActiveFeatureFileUris does not pass these tests (I mentioned this in this post).

The names of the tests should explain what they are supposed to do, but currently do not.

All you need to do is compare the passing tests with the failing tests to see where the shortfall is.

Thoughts, question, comments

So far I've setup everything as @aslakhellesoy has suggested.

Do you recognise anything workable from @yopasa94 solution?

Basically what we're doing here is reimplementing functionality that has been removed but needed by myself and others.

Hopefully this is sumarised well enough to understand?

Thanks.

[aurelien-reeves]

Did you explore gherkinStream.fromPaths? What is the situation with this?

Also, the suggestion from @yopasa94 was pretty interesting. Did you had the opportunity to take a look? parseGherkinMessageStream is a helper method in cucumber-js:

cucumber-js/src/cli/helpers.ts

Line 46 in 24ff9dc

export async function parseGherkinMessageStream({

You give too many references to your own code base. Please give minimum reproducible snippets here instead.

[binarymist]

Did you explore gherkinStream.fromPaths? What is the situation with this?

Yes. I've just provided a link to it again. Here it is again https://github.com/purpleteam-labs/purpleteam-app-scanner/blob/06c5148c9f080388164a74ccf990e8ae18f7316f/src/api/app/models/app.js#L160. This is what's under test.

Also, the suggestion from @yopasa94 was pretty interesting. Did you had the opportunity to take a look?

Yes I thought so to, but sadly who knows where it is, and there are still unanswerd questions needed to take it further. Where is it? My comments on it here, here and here.

parseGherkinMessageStream is a helper method in cucumber-js:

I thought it might be, as I mentioned here.

@aurelien-reeves: Are you actually reading my comments, as you keep asking questions I've already answerd?

You give too many references to your own code base. Please give minimum reproducible snippets here instead.

Ok, I can do this, but all I'll be doing is copy, pasting what I've already linked to. Then if you want to run it, you'll have to do even more work than simply clone the repo, npm install -> npm test. The links I've provided to the unit tests and SUT are "minimum reproducible snippets"

Thanks.

[aurelien-reeves]

@aurelien-reeves: Are you actually reading my comments, as you keep asking questions I've already answerd?

I missed that one. But yes, I do read your responses.

You keep posting links to your code base. I won't go there. There is too many code. I won't take the time to understand how all your code base works.

Also I don't ask for you to copy paste your code here. I ask you for a minimal reproducible example of your issues / what you want to accomplish.

I do my best do keep helping you. Please try to simplify as much as possible your questions.

[binarymist]

• Clone: https://github.com/purpleteam-labs/purpleteam-app-scanner_cucumber-1489-repro
• npm install
• npm test

[aurelien-reeves]

Suggestion: purpleteam-labs/purpleteam-app-scanner_cucumber-1489-repro#1

All test are passing now

[binarymist]

Thanks for that @aurelien-reeves.

This appears to asume that the envelopes array will always be in a specific order, that order being:

1. if there is a gherkinDocument push it. Then push the gherkinDocuments pickles in order from top down
2. if there is another gherkinDocument push it. Then push the gherkinDocuments pickles in order from top down
3. and so on

Is this assumption true?

If so great, it's so simple.

I also didn't know about the pickleFilter.matches, it's basically doing all/most of the work.

[binarymist]

Just looked into fromPaths and it appears that it's recursive pipeSequentially does exactly this.

[binarymist]

Big thanks @aurelien-reeves for getting this over the line!