New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

I would like to see Cucumber as supported framework for e2e testing. #46

Open
asnov opened this Issue Mar 19, 2015 · 31 comments

Comments

Projects
None yet
10 participants
@asnov

asnov commented Mar 19, 2015

I would like to see Cucumber as supported framework for e2e testing.
Thank you, guys!

@samhatoum

This comment has been minimized.

Show comment
Hide comment
@samhatoum

samhatoum Jan 17, 2016

I would also like to see this.

I am one of the maintainers of Chimp which allows you to run Cucumber with end-to-end tests. I'd be happy to collaborate with you @ArtemGovorov to get this working

Cucumber.js has a command-line runner and it outputs a JSON report file. I'm guessing we'd need to add it as a runner somehow and transform the output to something that wallaby can parse.

Any pointers on how I can start to mess with it?

samhatoum commented Jan 17, 2016

I would also like to see this.

I am one of the maintainers of Chimp which allows you to run Cucumber with end-to-end tests. I'd be happy to collaborate with you @ArtemGovorov to get this working

Cucumber.js has a command-line runner and it outputs a JSON report file. I'm guessing we'd need to add it as a runner somehow and transform the output to something that wallaby can parse.

Any pointers on how I can start to mess with it?

@ArtemGovorov

This comment has been minimized.

Show comment
Hide comment
@ArtemGovorov

ArtemGovorov Jan 18, 2016

Member

Hi @samhatoum, Chimp looks awesome, thanks for sharing.

Regarding the integration, I'm not familiar much with how Chimp/Cucumber works, so I'll try to describe what would be required from wallaby side.

A new runner most certainly is required (and that part of the core is not yet available for external extending unfortunately).

Regarding the output, wallaby.js needs quite a few things:

  • Apart from the runner, the JavaScript that Chimp/Cucumber executes has to instrumented by wallaby core (to collect the code coverage data).
  • Wallaby also needs to hook into the testing framework (by adding a reporter), so that it knows when a test/suite is completed, able to filter what tests to skip/execute, etc.

By instrumenting files and adding the reporter, wallaby.js is able to get the information it needs. With our PhantomJs runner, we create a web socket connection using which wallaby.js sends the collected data to its core (server).

Member

ArtemGovorov commented Jan 18, 2016

Hi @samhatoum, Chimp looks awesome, thanks for sharing.

Regarding the integration, I'm not familiar much with how Chimp/Cucumber works, so I'll try to describe what would be required from wallaby side.

A new runner most certainly is required (and that part of the core is not yet available for external extending unfortunately).

Regarding the output, wallaby.js needs quite a few things:

  • Apart from the runner, the JavaScript that Chimp/Cucumber executes has to instrumented by wallaby core (to collect the code coverage data).
  • Wallaby also needs to hook into the testing framework (by adding a reporter), so that it knows when a test/suite is completed, able to filter what tests to skip/execute, etc.

By instrumenting files and adding the reporter, wallaby.js is able to get the information it needs. With our PhantomJs runner, we create a web socket connection using which wallaby.js sends the collected data to its core (server).

@samhatoum

This comment has been minimized.

Show comment
Hide comment
@samhatoum

samhatoum Jan 22, 2016

Thank you @ArtemGovorov

I'll try to improvise on each point to say how I can see it being done:

  • The instrumentation can be done prior to a Cucumber.js execution
  • When Cucumber.js is complete, it generates a json file, which we can transform to a wallaby-compatible report
  • The next Cucumber.js run can be passed only the filtered features to run

Typically Cucumber.js runs in a node environment (it's also possible to run on the client, but not a typical practice).

It sounds like the reporter is where the work would be. Can I help in any way?

samhatoum commented Jan 22, 2016

Thank you @ArtemGovorov

I'll try to improvise on each point to say how I can see it being done:

  • The instrumentation can be done prior to a Cucumber.js execution
  • When Cucumber.js is complete, it generates a json file, which we can transform to a wallaby-compatible report
  • The next Cucumber.js run can be passed only the filtered features to run

Typically Cucumber.js runs in a node environment (it's also possible to run on the client, but not a typical practice).

It sounds like the reporter is where the work would be. Can I help in any way?

@ArtemGovorov

This comment has been minimized.

Show comment
Hide comment
@ArtemGovorov

ArtemGovorov Jan 28, 2016

Member

The instrumentation can be done prior to a Cucumber.js execution

Yes.

When Cucumber.js is complete, it generates a json file, which we can transform to a wallaby-compatible report

This is the hard part. Wallaby needs quite a few data bits from the test execution host, ideally in real-time just as tests are executing.

Thanks, I may need some help with the reporter, I first should find some time and have a deeper look into Cucumber.js to better understand how it works, how to make it send what wallaby needs and generally what's the best approach of integrating with it.

Member

ArtemGovorov commented Jan 28, 2016

The instrumentation can be done prior to a Cucumber.js execution

Yes.

When Cucumber.js is complete, it generates a json file, which we can transform to a wallaby-compatible report

This is the hard part. Wallaby needs quite a few data bits from the test execution host, ideally in real-time just as tests are executing.

Thanks, I may need some help with the reporter, I first should find some time and have a deeper look into Cucumber.js to better understand how it works, how to make it send what wallaby needs and generally what's the best approach of integrating with it.

@samhatoum

This comment has been minimized.

Show comment
Hide comment
@samhatoum

samhatoum Jan 28, 2016

This is the hard part. Wallaby needs quite a few data bits from the test execution host, ideally in real-time just as tests are executing.

Look no further than the StepResult event. This can give you realtime results as step are executed happen.

To use it with Wallaby.js, you'd have to create a file like step-result.js file, and put this in it:

module.exports = function() {
  this.StepResult(function(event){
    var stepResult = event.getPayloadItem('stepResult');
    var step = stepResult.getStep();
    //console.log(stepResult); // be sure to check this out
    //console.log(step);  // be sure to check this out

    console.log('\n - - - - - STEP RESULT - - - - - ');
    console.log('stepResult.isFailed - -', stepResult.isFailed());
    console.log('stepResult.isPending - -', stepResult.isPending());
    console.log('stepResult.isSkipped - -', stepResult.isSkipped());
    console.log('stepResult.isSuccessful - -', stepResult.isSuccessful());
    console.log('stepResult.isUndefined - -', stepResult.isUndefined());
    console.log('stepResult.getDuration - -', stepResult.getDuration());

    console.log('\n - - - - - STEP - - - - - ');
    console.log('step.setPreviousStep - -', step.setPreviousStep());
    console.log('step.isHidden - -', step.isHidden());
    console.log('step.isOutlineStep - -', step.isOutlineStep());
    console.log('step.getKeyword - -', step.getKeyword());
    console.log('step.getName - -', step.getName());
    console.log('step.hasUri - -', step.hasUri());
    console.log('step.getUri - -', step.getUri());
    console.log('step.getLine - -', step.getLine());
    console.log('step.getPreviousStep - -', step.getPreviousStep());
    console.log('step.hasPreviousStep - -', step.hasPreviousStep());
    console.log('step.getAttachment - -', step.getAttachment());
    console.log('step.getAttachmentContents - -', step.getAttachmentContents());
    console.log('step.getDocString - -', step.getDocString());
    console.log('step.getDataTable - -', step.getDataTable());
    console.log('step.hasAttachment - -', step.hasAttachment());
    console.log('step.hasDocString - -', step.hasDocString());
    console.log('step.hasDataTable - -', step.hasDataTable());
    console.log('step.ensureDataTableIsAttached - -', step.ensureDataTableIsAttached());
    console.log('step.isOutcomeStep - -', step.isOutcomeStep());
    console.log('step.isEventStep - -', step.isEventStep());
    console.log('step.hasOutcomeStepKeyword - -', step.hasOutcomeStepKeyword());
    console.log('step.hasEventStepKeyword - -', step.hasEventStepKeyword());
    console.log('step.isRepeatingOutcomeStep - -', step.isRepeatingOutcomeStep());
    console.log('step.isRepeatingEventStep - -', step.isRepeatingEventStep());
    console.log('step.hasRepeatStepKeyword - -', step.hasRepeatStepKeyword());
    console.log('step.isPrecededByOutcomeStep - -', step.isPrecededByOutcomeStep());
    console.log('step.isPrecededByEventStep - -', step.isPrecededByEventStep());
    console.log('step.getStepDefinition - -', step.getStepDefinition());
  });
}

And then when you run cucumberjs from Wallaby, just pass it an additional file using -r <path to>/step-result.js.

That should do it!

samhatoum commented Jan 28, 2016

This is the hard part. Wallaby needs quite a few data bits from the test execution host, ideally in real-time just as tests are executing.

Look no further than the StepResult event. This can give you realtime results as step are executed happen.

To use it with Wallaby.js, you'd have to create a file like step-result.js file, and put this in it:

module.exports = function() {
  this.StepResult(function(event){
    var stepResult = event.getPayloadItem('stepResult');
    var step = stepResult.getStep();
    //console.log(stepResult); // be sure to check this out
    //console.log(step);  // be sure to check this out

    console.log('\n - - - - - STEP RESULT - - - - - ');
    console.log('stepResult.isFailed - -', stepResult.isFailed());
    console.log('stepResult.isPending - -', stepResult.isPending());
    console.log('stepResult.isSkipped - -', stepResult.isSkipped());
    console.log('stepResult.isSuccessful - -', stepResult.isSuccessful());
    console.log('stepResult.isUndefined - -', stepResult.isUndefined());
    console.log('stepResult.getDuration - -', stepResult.getDuration());

    console.log('\n - - - - - STEP - - - - - ');
    console.log('step.setPreviousStep - -', step.setPreviousStep());
    console.log('step.isHidden - -', step.isHidden());
    console.log('step.isOutlineStep - -', step.isOutlineStep());
    console.log('step.getKeyword - -', step.getKeyword());
    console.log('step.getName - -', step.getName());
    console.log('step.hasUri - -', step.hasUri());
    console.log('step.getUri - -', step.getUri());
    console.log('step.getLine - -', step.getLine());
    console.log('step.getPreviousStep - -', step.getPreviousStep());
    console.log('step.hasPreviousStep - -', step.hasPreviousStep());
    console.log('step.getAttachment - -', step.getAttachment());
    console.log('step.getAttachmentContents - -', step.getAttachmentContents());
    console.log('step.getDocString - -', step.getDocString());
    console.log('step.getDataTable - -', step.getDataTable());
    console.log('step.hasAttachment - -', step.hasAttachment());
    console.log('step.hasDocString - -', step.hasDocString());
    console.log('step.hasDataTable - -', step.hasDataTable());
    console.log('step.ensureDataTableIsAttached - -', step.ensureDataTableIsAttached());
    console.log('step.isOutcomeStep - -', step.isOutcomeStep());
    console.log('step.isEventStep - -', step.isEventStep());
    console.log('step.hasOutcomeStepKeyword - -', step.hasOutcomeStepKeyword());
    console.log('step.hasEventStepKeyword - -', step.hasEventStepKeyword());
    console.log('step.isRepeatingOutcomeStep - -', step.isRepeatingOutcomeStep());
    console.log('step.isRepeatingEventStep - -', step.isRepeatingEventStep());
    console.log('step.hasRepeatStepKeyword - -', step.hasRepeatStepKeyword());
    console.log('step.isPrecededByOutcomeStep - -', step.isPrecededByOutcomeStep());
    console.log('step.isPrecededByEventStep - -', step.isPrecededByEventStep());
    console.log('step.getStepDefinition - -', step.getStepDefinition());
  });
}

And then when you run cucumberjs from Wallaby, just pass it an additional file using -r <path to>/step-result.js.

That should do it!

@ArtemGovorov

This comment has been minimized.

Show comment
Hide comment
@ArtemGovorov

ArtemGovorov Jan 28, 2016

Member

Thanks Sam, will look into the StepResult.

Member

ArtemGovorov commented Jan 28, 2016

Thanks Sam, will look into the StepResult.

@timReynolds

This comment has been minimized.

Show comment
Hide comment
@timReynolds

timReynolds Feb 9, 2016

Plus one comments are super annoying but I'd be interested in seeing Cucumber support for wallaby.

timReynolds commented Feb 9, 2016

Plus one comments are super annoying but I'd be interested in seeing Cucumber support for wallaby.

@eggyal

This comment has been minimized.

Show comment
Hide comment
@eggyal

eggyal Feb 22, 2016

I'd also like to see support for Cucumber, or else to understand how one can get Wallaby working with Yadda (which ostensibly overlays Gherkin-style BDD support atop one's test framework, including Jasmine and Mocha)—any thoughts?

eggyal commented Feb 22, 2016

I'd also like to see support for Cucumber, or else to understand how one can get Wallaby working with Yadda (which ostensibly overlays Gherkin-style BDD support atop one's test framework, including Jasmine and Mocha)—any thoughts?

@ArtemGovorov

This comment has been minimized.

Show comment
Hide comment
@ArtemGovorov

ArtemGovorov Feb 22, 2016

Member

@eggyal if Yadda is based on Cucumber, then this issue is definitely a prerequisite before supporting it.

Member

ArtemGovorov commented Feb 22, 2016

@eggyal if Yadda is based on Cucumber, then this issue is definitely a prerequisite before supporting it.

@eggyal

This comment has been minimized.

Show comment
Hide comment
@eggyal

eggyal Feb 22, 2016

@ArtemGovorov: No, it doesn't depend on Cucumber. AIUI, your existing test framework (Jasmine, Mocha, whatever) uses Yadda to parse Gherkin-like feature files in order to identify which test functions from your spec files should be run (and with what arguments).

The problem I've had is that it's only really practical with a single entry-point, and I think this would break Wallaby's caching?

eggyal commented Feb 22, 2016

@ArtemGovorov: No, it doesn't depend on Cucumber. AIUI, your existing test framework (Jasmine, Mocha, whatever) uses Yadda to parse Gherkin-like feature files in order to identify which test functions from your spec files should be run (and with what arguments).

The problem I've had is that it's only really practical with a single entry-point, and I think this would break Wallaby's caching?

@ArtemGovorov

This comment has been minimized.

Show comment
Hide comment
@ArtemGovorov

ArtemGovorov Feb 22, 2016

Member

@eggyal Got it, thanks.

it's only really practical with a single-entry point, and I think this would break Wallaby's caching?

Not sure about caching, but can definitely see that a few of main wallaby features, such as running tests in parallel and incrementally executing tests, will not work if the framework expects a single-entry point for executing specs.

Member

ArtemGovorov commented Feb 22, 2016

@eggyal Got it, thanks.

it's only really practical with a single-entry point, and I think this would break Wallaby's caching?

Not sure about caching, but can definitely see that a few of main wallaby features, such as running tests in parallel and incrementally executing tests, will not work if the framework expects a single-entry point for executing specs.

@eggyal

This comment has been minimized.

Show comment
Hide comment
@eggyal

eggyal Feb 22, 2016

Yes, one would invoke mocha ./yadda-entry-point.js or whatever.

So I guess Yadda's resolution of which spec functions to invoke would have to take place within Wallaby... is there a plugin API or somesuch that I can harness?

eggyal commented Feb 22, 2016

Yes, one would invoke mocha ./yadda-entry-point.js or whatever.

So I guess Yadda's resolution of which spec functions to invoke would have to take place within Wallaby... is there a plugin API or somesuch that I can harness?

@ArtemGovorov

This comment has been minimized.

Show comment
Hide comment
@ArtemGovorov

ArtemGovorov Feb 22, 2016

Member

@eggyal There's no (public) API at the moment to perform such integration.

Member

ArtemGovorov commented Feb 22, 2016

@eggyal There's no (public) API at the moment to perform such integration.

@chrisns

This comment has been minimized.

Show comment
Hide comment
@chrisns

chrisns Mar 27, 2016

In terms of the yadda stuff I put this example together so at least the wallaby hackery/work can live somewhere.
https://github.com/chrisns/yadda/tree/wallaby/examples/mocha-wallaby
I was trying to see if I could fool wallaby into working, turns out its too clever :)
Tests run beautifully the first time, I was super excited, in fact I changed the feature file a couple of times and it was all working, then it for no discernible reason stopped and started showing various irrelevant errors.
Obviously coverage + inline beautiful wallaby goodness in the .feature file are probably a little far fetched for now at least, but it would be wicked to be able to execute the tests.

chrisns commented Mar 27, 2016

In terms of the yadda stuff I put this example together so at least the wallaby hackery/work can live somewhere.
https://github.com/chrisns/yadda/tree/wallaby/examples/mocha-wallaby
I was trying to see if I could fool wallaby into working, turns out its too clever :)
Tests run beautifully the first time, I was super excited, in fact I changed the feature file a couple of times and it was all working, then it for no discernible reason stopped and started showing various irrelevant errors.
Obviously coverage + inline beautiful wallaby goodness in the .feature file are probably a little far fetched for now at least, but it would be wicked to be able to execute the tests.

@celador

This comment has been minimized.

Show comment
Hide comment
@celador

celador Apr 26, 2016

@ArtemGovorov Any idea for when/if this might happen? or when "the core may be available for external extending". Wallaby is really nice, but Chimp.js is more full featured. I bought Wallaby, and will likely use it for writing js unit tests, but having Chimp.js/Cucumber support would be a killer feature. Wallaby will be a hard sell to my team without Chimp/Cucumber support. I work on a team with 40+ developers and it would be an easy sell to the team if it had Chimp support. We'd certainly purchase a license for each developer if we could have Chimp(Chrome, Gherkin, Webdriver.io API) + Wallaby(super fast testing and code coverage in the IDE) . Please, please, please work with @samhatoum to see if you can get this working.

celador commented Apr 26, 2016

@ArtemGovorov Any idea for when/if this might happen? or when "the core may be available for external extending". Wallaby is really nice, but Chimp.js is more full featured. I bought Wallaby, and will likely use it for writing js unit tests, but having Chimp.js/Cucumber support would be a killer feature. Wallaby will be a hard sell to my team without Chimp/Cucumber support. I work on a team with 40+ developers and it would be an easy sell to the team if it had Chimp support. We'd certainly purchase a license for each developer if we could have Chimp(Chrome, Gherkin, Webdriver.io API) + Wallaby(super fast testing and code coverage in the IDE) . Please, please, please work with @samhatoum to see if you can get this working.

@ArtemGovorov

This comment has been minimized.

Show comment
Hide comment
@ArtemGovorov

ArtemGovorov Apr 27, 2016

Member

@celador I have done some estimates and looked into Cucumber/Chimp to better understand our options back in Feb, there are a few things that need to be implemented in wallaby.js core before implementing the support. Some of them are pretty big. Nevertheless, I'd like to add Chimp/Cucumber support at some point, but can't make any promises when it's going to happen.

Member

ArtemGovorov commented Apr 27, 2016

@celador I have done some estimates and looked into Cucumber/Chimp to better understand our options back in Feb, there are a few things that need to be implemented in wallaby.js core before implementing the support. Some of them are pretty big. Nevertheless, I'd like to add Chimp/Cucumber support at some point, but can't make any promises when it's going to happen.

@samhatoum

This comment has been minimized.

Show comment
Hide comment
@samhatoum

samhatoum Apr 27, 2016

So Chimp is a test runner, and Wallaby is a test runner. A test runner running a test runner is a bad idea!

Saying that, Chimp has a session manager that makes end-to-end a pleasurable experience by reusing open sessions. There may be some value in extracting the session manager into a separate module, which can then be imported into a Wallaby + Cucumber.js configuration - but I think I'll only be able to do that once we see the basic Wallaby + Cucumber combo.

@ArtemGovorov let me know if you need anything when you finally get around to doing this

samhatoum commented Apr 27, 2016

So Chimp is a test runner, and Wallaby is a test runner. A test runner running a test runner is a bad idea!

Saying that, Chimp has a session manager that makes end-to-end a pleasurable experience by reusing open sessions. There may be some value in extracting the session manager into a separate module, which can then be imported into a Wallaby + Cucumber.js configuration - but I think I'll only be able to do that once we see the basic Wallaby + Cucumber combo.

@ArtemGovorov let me know if you need anything when you finally get around to doing this

@samhatoum

This comment has been minimized.

Show comment
Hide comment
@samhatoum

samhatoum Jun 23, 2016

Hey @ArtemGovorov. I know how busy life gets and how product backlogs work. Just wondering if you have any rough outlook on when you might start attacking Cucumber.js support. And as always, here to help if you need it as I've done some major digging into Cucumber.js.

Thanks!

samhatoum commented Jun 23, 2016

Hey @ArtemGovorov. I know how busy life gets and how product backlogs work. Just wondering if you have any rough outlook on when you might start attacking Cucumber.js support. And as always, here to help if you need it as I've done some major digging into Cucumber.js.

Thanks!

@ArtemGovorov

This comment has been minimized.

Show comment
Hide comment
@ArtemGovorov

ArtemGovorov Jun 23, 2016

Member

Hey @samhatoum, can't share any plans about the feature support yet. The team is busy at the moment rushing towards releasing this big feature. After releasing that one and clearing up a few smaller long wanted/waiting ones, we'll revisit the backlog, and of course will post some update here if anything changes.

Member

ArtemGovorov commented Jun 23, 2016

Hey @samhatoum, can't share any plans about the feature support yet. The team is busy at the moment rushing towards releasing this big feature. After releasing that one and clearing up a few smaller long wanted/waiting ones, we'll revisit the backlog, and of course will post some update here if anything changes.

@samhatoum

This comment has been minimized.

Show comment
Hide comment
@samhatoum

samhatoum Jun 24, 2016

@ArtemGovorov that looks incredible!! I love it and can't wait to try it.

samhatoum commented Jun 24, 2016

@ArtemGovorov that looks incredible!! I love it and can't wait to try it.

@eggyal

This comment has been minimized.

Show comment
Hide comment
@eggyal

eggyal Jul 27, 2016

Wallaby App is indeed very cool, but the benefit it actually provides for us (over monitoring test results in-editor) doesn't come anywhere close to what we'd get from having Cucumber support.

I appreciate that there are other priorities, but it nevertheless saddens me that (15 months after it was first requested) there's still no indication this feature is even scheduled to be worked upon any time soon. Perhaps I've just become too used to how rapidly open-source tools iterate, or else how easy it is to add custom functionality where there are external APIs for plugins/modules!

I'll keep praying for this feature to appear sooooooooon!

eggyal commented Jul 27, 2016

Wallaby App is indeed very cool, but the benefit it actually provides for us (over monitoring test results in-editor) doesn't come anywhere close to what we'd get from having Cucumber support.

I appreciate that there are other priorities, but it nevertheless saddens me that (15 months after it was first requested) there's still no indication this feature is even scheduled to be worked upon any time soon. Perhaps I've just become too used to how rapidly open-source tools iterate, or else how easy it is to add custom functionality where there are external APIs for plugins/modules!

I'll keep praying for this feature to appear sooooooooon!

@eggyal

This comment has been minimized.

Show comment
Hide comment
@eggyal

eggyal Jul 27, 2016

Maybe https://github.com/cucumber/microcuke (an official, minimal—only 500 SLOC, JS reference implementation) could be useful?

eggyal commented Jul 27, 2016

Maybe https://github.com/cucumber/microcuke (an official, minimal—only 500 SLOC, JS reference implementation) could be useful?

@tedvanderveen

This comment has been minimized.

Show comment
Hide comment
@tedvanderveen

tedvanderveen Oct 11, 2016

@ArtemGovorov can you please share status on this enhancement request?

tedvanderveen commented Oct 11, 2016

@ArtemGovorov can you please share status on this enhancement request?

@ArtemGovorov

This comment has been minimized.

Show comment
Hide comment
@ArtemGovorov

ArtemGovorov Oct 11, 2016

Member

@tedvanderveen The status hasn't really changed, but I'll post an update here when it does.

Member

ArtemGovorov commented Oct 11, 2016

@tedvanderveen The status hasn't really changed, but I'll post an update here when it does.

@jfstephe

This comment has been minimized.

Show comment
Hide comment
@jfstephe

jfstephe Mar 10, 2017

For me, test coverage is not a great measure of quality (not that people are necessarily saying it is) but perhaps at best an indication of the quality of code. All the arguments about this aside, if you are running cucumber tests and non-cuke tests if you can only report the test coverage on part of the tests you are running that makes this less than useful for me. Was quite excited about this tool until I saw this was not supported.

ETA on this feature? 1 week? 3 months? 6 months? 12 months? Never? Can you give us a ball-park idea?

jfstephe commented Mar 10, 2017

For me, test coverage is not a great measure of quality (not that people are necessarily saying it is) but perhaps at best an indication of the quality of code. All the arguments about this aside, if you are running cucumber tests and non-cuke tests if you can only report the test coverage on part of the tests you are running that makes this less than useful for me. Was quite excited about this tool until I saw this was not supported.

ETA on this feature? 1 week? 3 months? 6 months? 12 months? Never? Can you give us a ball-park idea?

@ArtemGovorov

This comment has been minimized.

Show comment
Hide comment
@ArtemGovorov

ArtemGovorov Mar 11, 2017

Member

@jfstephe

ETA on this feature? 1 week? 3 months? 6 months? 12 months? Never? Can you give us a ball-park idea?

Unfortunately I can't share ETA at the moment. It is highly unlikely that we add the support within the next 3 months.

Member

ArtemGovorov commented Mar 11, 2017

@jfstephe

ETA on this feature? 1 week? 3 months? 6 months? 12 months? Never? Can you give us a ball-park idea?

Unfortunately I can't share ETA at the moment. It is highly unlikely that we add the support within the next 3 months.

@samhatoum

This comment has been minimized.

Show comment
Hide comment
@samhatoum

samhatoum Jun 22, 2017

@ArtemGovorov @charlierudolph is the maintainer of CucumberJS. I'm hoping this intro could accelerate this ticket :)

samhatoum commented Jun 22, 2017

@ArtemGovorov @charlierudolph is the maintainer of CucumberJS. I'm hoping this intro could accelerate this ticket :)

@eggyal

This comment has been minimized.

Show comment
Hide comment
@eggyal

eggyal Jun 26, 2017

FWIW, I see that #1144 was linked in the context of Cucumber being for E2E testing. That's not really correct, at all. Cucumber is merely an engine for running Gherkin files, which are themselves a way of specifying tests in natural language—those tests could be unit tests, integration tests, E2E tests or indeed anything at all.

Certainly it's true that the overhead of writing tests in natural language is typically reserved for cases where they are reviewed by a non-technical audience, e.g. acceptance tests (which are usually only E2E)—but there is nothing that requires that to be the case.

It may be that this issue was lumped together with E2E testing frameworks for brevity, but I thought it worth setting the record straight lest a misunderstanding or incorrect assumption have caused it to be given lower priority than it might otherwise have received.

eggyal commented Jun 26, 2017

FWIW, I see that #1144 was linked in the context of Cucumber being for E2E testing. That's not really correct, at all. Cucumber is merely an engine for running Gherkin files, which are themselves a way of specifying tests in natural language—those tests could be unit tests, integration tests, E2E tests or indeed anything at all.

Certainly it's true that the overhead of writing tests in natural language is typically reserved for cases where they are reviewed by a non-technical audience, e.g. acceptance tests (which are usually only E2E)—but there is nothing that requires that to be the case.

It may be that this issue was lumped together with E2E testing frameworks for brevity, but I thought it worth setting the record straight lest a misunderstanding or incorrect assumption have caused it to be given lower priority than it might otherwise have received.

@samhatoum

This comment has been minimized.

Show comment
Hide comment
@samhatoum

samhatoum Aug 13, 2017

I'm hawking this issue as Wallaby support for CucumberJS would make me jump for joy.

So I second the comment from @eggyal. At Xolv.io we don't use Cucumber for e2e testing at all, we use it to do drive development of the system. If wallaby is to run tests within the browser/node context, this is the same use case for Cucumber as it is for mocha/jasmine/tape. From wallaby's perspective, Cucumber should be treated just as another test framework.

I also want to reiterate that Chimp has nothing to do with this implementation.

More interestingly, a new development in CucumberJS 3.0 is that it now has an event-protocol for reporters and other plugins.

See here: https://docs.cucumber.io/event-protocol/

and here: https://github.com/cucumber/cucumber-js/blob/master/src/formatter/event_protocol_formatter.js

@ArtemGovorov I think this might give you the realtime-ness you're looking for in a decoupled fashion

samhatoum commented Aug 13, 2017

I'm hawking this issue as Wallaby support for CucumberJS would make me jump for joy.

So I second the comment from @eggyal. At Xolv.io we don't use Cucumber for e2e testing at all, we use it to do drive development of the system. If wallaby is to run tests within the browser/node context, this is the same use case for Cucumber as it is for mocha/jasmine/tape. From wallaby's perspective, Cucumber should be treated just as another test framework.

I also want to reiterate that Chimp has nothing to do with this implementation.

More interestingly, a new development in CucumberJS 3.0 is that it now has an event-protocol for reporters and other plugins.

See here: https://docs.cucumber.io/event-protocol/

and here: https://github.com/cucumber/cucumber-js/blob/master/src/formatter/event_protocol_formatter.js

@ArtemGovorov I think this might give you the realtime-ness you're looking for in a decoupled fashion

@FlippieCoetser

This comment has been minimized.

Show comment
Hide comment
@FlippieCoetser

FlippieCoetser Jul 21, 2018

@ArtemGovorov any new feedback regarding this issue? WallabyJS is great having it support CucumberJS would be even better.

FlippieCoetser commented Jul 21, 2018

@ArtemGovorov any new feedback regarding this issue? WallabyJS is great having it support CucumberJS would be even better.

@eggyal

This comment has been minimized.

Show comment
Hide comment
@eggyal

eggyal Jul 26, 2018

@FlippieCoetser: Don't bother, it's been over three years with absolutely zero progress. Look elsewhere.

eggyal commented Jul 26, 2018

@FlippieCoetser: Don't bother, it's been over three years with absolutely zero progress. Look elsewhere.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment