Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] BDD implementation with Playwright test runner #11975

Open
ShanmukhaGajula opened this issue Feb 9, 2022 · 163 comments
Open

[Feature] BDD implementation with Playwright test runner #11975

ShanmukhaGajula opened this issue Feb 9, 2022 · 163 comments

Comments

@ShanmukhaGajula
Copy link

ShanmukhaGajula commented Feb 9, 2022

Our team has a case to use the BDD test, Playwright has any plan to add their own BDD implementation using the playwright test runner? That would be great if to have.

This can help many people who want to use BDD along with Playwright. That's where cucumber became popular 😉

@unlikelyzero

This comment was marked as off-topic.

@saranyakakarlapudi
Copy link

BDD implementation with Playwright test runner is what am also looking for..

@keshavmunagala
Copy link

It would be great to have BDD with Playwright

@ShanmukhaGajula
Copy link
Author

ShanmukhaGajula commented Feb 9, 2022

@1996phani would https://codecept.io/playwright/ serve your needs?

@unlikelyzero, The package you're mentioning has been deprecated.

I am looking for something kind of Native support from the Playwright

@pavelfeldman
Copy link
Member

Let's collect upvotes. Please upvote the OP!

@CodeStrikerr
Copy link

Yeah BBD is a must thing!

@soundaryakk
Copy link

It would be helpful ,if we have BDD along with Playwright.

@kvsrk432
Copy link

BDD support is a perk

@siddharththakurji
Copy link

bdd is a good feature to have.. i am using in selenium java.. i need it in playwright to migrate

@Vinaymanchala
Copy link

I'm looking for bdd too!

@Geetha2604
Copy link

Good to have BDD with playwright

@dimmetasowjanya
Copy link

Having BDD with Playwright will be helpful

@unlikelyzero
Copy link
Contributor

@1996phani would https://codecept.io/playwright/ serve your needs?

@unlikelyzero, The package you're mentioning has been deprecated.

I am looking for something kind of Native support from the Playwright

@ShanmukhaGajula where does it say that?

@ShanmukhaGajula
Copy link
Author

@1996phani would https://codecept.io/playwright/ serve your needs?

@unlikelyzero, The package you're mentioning has been deprecated.
I am looking for something kind of Native support from the Playwright

@ShanmukhaGajula where does it say that?

I see a pop-up yesterday when I opened that link. I think they've removed now. However, I see playwright test runner and reporting is very good and hence requesting Playwright team for that feature within playwright itself.

@abhirammocharla
Copy link

BDD is always good to have

@PranayEtikala
Copy link

Yes, Looking for BDD to implement playwright in our project.

@Akhila4062
Copy link

BDD is a must-have feature for us too! It really helps.

@Mohan2736
Copy link

Hope playwright will add this BDD feature..

@agarwalu
Copy link

i was also looking for same.. BDD will really add value

@marko-simic
Copy link

I'm looking for BDD too!

@IsabelFrancis
Copy link

Looking forward to have BDD feature with Playwright,

@KalyanPuppala36
Copy link

It would be great to have BDD with Playwright

@ghost
Copy link

ghost commented Feb 11, 2022

It would be helpful ,if we have BDD along with Playwright.

@NagaSivaSankar45
Copy link

It would be nice to have BDD along with Playwright.

@LoneyR001
Copy link

BDD with Playwright! That sounds great. Looking forward to it.

@pbusam
Copy link

pbusam commented Feb 11, 2022

It would be good to have the BDD feature, looking forward for it

@macroking
Copy link

@pavelfeldman Looks like 90% of the comments and upvotes are well planned, just digging through the user profile, most of the users just opened GitHub account to upvote this request.

May be one particular group of users really need this!

@ahnpnl
Copy link

ahnpnl commented Dec 13, 2023

This issue is actually a major blocker for my team to switch to Playwright

@alescinskis
Copy link

This issue is actually a major blocker for my team to switch to Playwright

I would recommend taking a look at https://github.com/vitalets/playwright-bdd as lack of bdd was a deal breaker for me, but then I've stumbled upon this project.

@LANDG-MartinP
Copy link

https://github.com/vitalets/playwright-bdd looks interesting but I wonder if there are any limitations? I know, for example, it only recently introduced support for Hooks?

Does it generate the Playwright HTML report? Can it be used to fully shard tests for parallel execution (test by test, not file by file aka cucumber.js)

@odeatomas
Copy link

odeatomas commented Dec 14, 2023 via email

@jakedavis219
Copy link

I've put together a small framework using playwright and the implementation off bdd/gherkin to organise tests if people want to take a look: https://github.com/jakedavis219/playwright-bdd-typescript-framework

This is the closest I could get without any BDD features being offered by playwright.

@nokrashy-pwc
Copy link

We need this ASAP please!

@asdfsafd
Copy link

No problem using the playwright test lib with the Cucumber test runner. It's the playwright test runner features we want to benefit from using our Gherkin Scenarios.

@tcydik
Copy link

tcydik commented Mar 15, 2024

https://github.com/vitalets/playwright-bdd looks interesting but I wonder if there are any limitations? I know, for example, it only recently introduced support for Hooks?

Does it generate the Playwright HTML report? Can it be used to fully shard tests for parallel execution (test by test, not file by file aka cucumber.js)

This library went a long way since this post. Have a look at it again.

@MarcHendo
Copy link

Currently using a combination of typescript, playwright and cucumber js. So far its working well, but native support would be great. If it could somehow be integrated into the visual code extension it would take playwright to the next level.

@acmetro
Copy link

acmetro commented May 14, 2024

Native support of BDD would be amazing!

@RichardCariven
Copy link

Native support of BDD would be amazing!

https://github.com/vitalets/playwright-bdd we are implementing that within our organisation and it is far far far better then any Playwright/Cucumber solution we have tried.

being able to use Fixtures and the playwright Config opens up so many more opportunities to make our behave tests far more flexible and simpler to maintain and run.

@Tallyb
Copy link

Tallyb commented May 16, 2024

I was not aware of this: https://pypi.org/project/pytest-bdd/ can be an inspiration.

@MartinB134
Copy link

MartinB134 commented May 16, 2024 via email

@LahiruMadhawa2020
Copy link

LahiruMadhawa2020 commented May 27, 2024

Eagerly waiting on this feature for a long time. Hoping to see it in an early future release.

@Xotabu4
Copy link

Xotabu4 commented Jun 10, 2024

The last remaining person who was working on the Cucumber project has been laid off:

https://mattwynne.net/new-beginning

@gasparnagy
Copy link

The last remaining person who was working on the Cucumber project has been laid off:

https://mattwynne.net/new-beginning

This is misleading. Matt has been laid off from SmartBear, but the Cucumber project moves on. All cucumber flavors have regular releases.

@nilPPanda
Copy link

Is this under development then? Also interested on this. Thanks!

@anemer
Copy link

anemer commented Jun 27, 2024

Hello,

True question: what do you mean by "BDD implementation"?

Do you mean something that make Playwright Test runner able to:

  • link steps written in feature files (in Gherkin style) to steps implemented in Playwright (Typescript version?) functions? Like Cucumber.js
  • manage a isolated context by scenario (as World notion in Cucumber),
  • run those tests/scenarios?

Second question: why?

IMHO :

  • it's possible to "do" BDD with any tool.
  • It's already possible to "do" it with Playwright Test (in particular with its notion of test.step() that gives clear reports...).
  • BDD is not a matter of tools. Because BDD != Gherkin != Cucumber 😉

Those articles explain this better than me:

@nilPPanda
Copy link

nilPPanda commented Jun 27, 2024

Hello,

True question: what do you mean by "BDD implementation"?

Do you mean something that make Playwright Test runner able to:

  • link steps written in feature files (in Gherkin style) to steps implemented in Playwright (Typescript version?) functions? Like Cucumber.js
  • manage a isolated context by scenario (as World notion in Cucumber),
  • run those tests/scenarios?

Second question: why?

IMHO :

  • it's possible to "do" BDD with any tool.
  • It's already possible to "do" it with Playwright Test (in particular with its notion of test.step() that gives clear reports...).
  • BDD is not a matter of tools. Because BDD != Gherkin != Cucumber 😉

Those articles explain this better than me:

I honestly think that when people is asking about BDD integration, at the end its asking about cucumber step system integration and still use capabilities of playwright runner.

@asdfsafd
Copy link

We would like to use the playwright test runner with our Cucumber feature files. Using the playwright lib with the Cucumber test runner is working but we would love to be able to use all the playwright tools, too.

@jan-molak
Copy link
Contributor

  • it's possible to "do" BDD with any tool.
  • It's already possible to "do" it with Playwright Test (in particular with its notion of test.step() that gives clear reports...).
  • BDD is not a matter of tools. Because BDD != Gherkin != Cucumber 😉

@anemer - I couldn't agree more. BDD is about collaboration and not the tools you use.

I honestly think that when people is asking about BDD integration, at the end its asking about cucumber step system integration and still use capabilities of playwright runner.

@nilPPanda - I think you're right. From what I'm reading in this thread, it feels like the desire to "do BDD with Playwright" comes down to three needs:

  1. Having test reports that explain what happened in the test scenario in a human-readable format, so that they could be shared with the business sponsors akin to Cucumber reports (Playwright reports are great, but they're optimised for developers, not business partners)
  2. Having an abstraction on top of the raw Playwright APIs, so that when you read the code it's easy to understand what's going on in business domain terms
  3. Being able to still use the excellent Playwright Test tools and the VSCode / JetBrains IDE plugin

So let's think through that:

  1. Plugging Playwright into Cucumber is reasonably easy - you'd to it the same way you'd plug in Selenium or any other web integration tool. I gave examples of how this could be accomplished in my previous comment [Feature] BDD implementation with Playwright test runner #11975 (comment)
  2. Designing good quality abstractions in automated tests can be really challenging, as challenging as designing good quality abstractions in production code. Cucumber step definitions mechanism provides an easy way to introduce basic abstractions that aggregate a bunch of low-level Playwright API calls and give them a nice human-readable description. However, Cucumber step definitions are designed to be simple mappings that don't support code reuse or composition (e.g. you can't call a Cucumber step from another Cucumber step). I completely agree with what @anemer said that developers could use test.step() and build their own abstractions around it. However, building test automation frameworks from scratch requires significant experience and time investment. Most teams don't feel like they can afford this as they need to focus on delivering features.
  3. This one's tricky, too. Understandably, the VisualStudio Code extension for Playwright, just like the JetBrains plugin for Playwright, is optimised to support Playwright Test runner. Making them support Cucumber would require a significant time and effort investment.

What if you really need Cucumber?

So let's go back to @anemer's original question: "do you need Cucumber with Playwright?"

Some teams simply like Cucumber and Gherkin. Some teams use Cucumber because it integrates with their internal reporting, documentation, or auditing systems. Some teams need to write their executable specifications in a language other than English, and Cucumber offers an easy way to introduce a translation layer.

If this describes your situation, then you'll probably want to stick with Cucumber as a test runner and use Playwright as a web integration tool.

You can, however, introduce Serenity/JS into this mix to improve the reporting, code design, and integration capabilities of your framework and avoid having to build everything from scratch.

Check out this integration guide and this Serenity/JS Cucumber Playwright project template

What if you don't need Cucumber but want great reporting and test design?

Now, what if you really want the capabilities and developer experience offered by Playwright Test, great reporting, business-friendly abstractions, but don't want to spend years developing a test automation framework from scratch?

What if I told you that you might not need Cucumber at all, even if your team does BDD 🤔

Have a look at my previous comment at #11975 (comment) explaining an alternative approach of using Serenity/JS with Playwright Test as a business-friendly abstraction layer.

You might also want to check out:

@gasparnagy
Copy link

I think this feature request is really about to make the Playwright runner extensible in a way that other sub-frameworks can plug in, so that you can use other frameworks (Cucumber, Serenity/JS, etc) to provide test source for the Playwright runner, so that you can use all the benefits that comes form the Playwright runner, but describe your tests in a feature file for example.

This is the approach that WebdriverIO uses, see https://webdriver.io/docs/frameworks/#using-cucumber.

As many of you have mentioned, you can live and do BDD without this (can use the Cucumber runner, have a BDD-like own naming convention, etc), but still, this would add quite some benefits.

@jan-molak
Copy link
Contributor

I think this feature request is really about to make the Playwright runner extensible in a way that other sub-frameworks can plug in, so that you can use other frameworks (Cucumber, Serenity/JS, etc) to provide test source for the Playwright runner, so that you can use all the benefits that comes form the Playwright runner, but describe your tests in a feature file for example.

This is the approach that WebdriverIO uses, see https://webdriver.io/docs/frameworks/#using-cucumber.

I agree; it would be amazing if Playwright Test supported a similar mechanism.
There are some interesting challenges here, though. Particularly around supporting parallel execution and making Cucumber compatible with Playwright Test's async dependency injection mechanism (given Cucumber's synchronous nature of the World object -cucumber/cucumber-js#2156).

I think closer integration would be great, and I'd love to collaborate on that; it would, however, require a lot of effort and the maintainers of the various frameworks working together.

@MartinB134
Copy link

MartinB134 commented Jun 28, 2024 via email

@jan-molak
Copy link
Contributor

jan-molak commented Jun 28, 2024

Correct me if I am wrong, but I think we are talking about two different things here.

That's a great point. I think that when saying "Playwright" many people in this thread mean the "full Playwright experience for Node.js", so the combination of all the various Playwright tools for Node.js: VisualStudio Playwright Test Extension, Playwright Test runner, Playwright UI Mode and Playwright web integration library.

Based on my current understanding, VisualStudio Code Playwright Test Extension supports only the Playwright Test runner, and Playwright Test runner is available only for Node.js.

And so, the way I interpret the desire expressed in this thread for Playwright to support Cucumber is that "we'd like Playwright Test Extension, Playwright Test runner, and Playwright UI mode to support running and debugging Cucumber.js .feature files just as well as they support running and debugging Playwright Test spec files".

If my interpretation is correct, this feels like a big ask and a significant development effort. For example, VS Code extension would need to integrate with or replicate the behaviour of the Cucumber extension for VS Code, Playwright Test runner would need to integrate with Cucumber.js, Cucumber.js would need to be made compatible with Playwright's fixtures mechanism, to mention but a few key points.

You can use nose2, behave, pytest-pdd with playwright if you "just" import the playwright module and make the page object available to your test files and test functions.

Yes, this is what I was referring to in - #11975 (comment)
You can use Playwright web integration library for Node.js with Cucumber.js, Mocha, Jasmine, Vitest, or any other JavaScript test runner, the exact same way you'd do it Python - you'd import it just like any other library.

However, not using Playwright Test means you can no longer use the VisualStudio Code Playwright Test Extension, or the Playwright UI Mode.

The approach I recommend to teams using Playwright for Node.js is to use Playwright Test runner, take advantage of Playwright Test extension for VS Code or the Playwright plugin for JetBrains IDEs, and use Serenity/JS Playwright Test integration to introduce business domain language and abstractions and provide more advanced test reporting features.

@vitalets
Copy link
Contributor

I'll just add that with playwright-bdd you can use Playwright VSCode extension showing spec files that are very close to feature files. Yes, they are not feature files, but you have all tooling you need: run/debug specific scenario, get step with error, view each step execution time, etc.
It was the main reason why we generate Playwright spec files from feature files - to let developers use Playwright tooling as much as possible:

image

@pramothvm
Copy link

Hi Friends. Do you have the same sample repo project for PlayWright with BDD frame work java. Is there a mail id to reach out to get hep reg this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests