New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Behat 3 Support #62
Comments
@jeffreyguo @srizzling Made a start on this, but its quite complicated - they've changed pretty much everything, without upgrading docs. |
http://docs.behat.org/en/v3.0/ These are less technical docs to follow as well. Would need a task to migrate the regex definitions to use the new twig. Although, from what I know its BC. |
Another important document is the changelog https://github.com/Behat/Behat/blob/master/CHANGES.md |
@srizzling I've found those guides are way too highlevel to describe the actual changes required. |
So I have tried to start this but my composer complains
This seems to me that |
Should be fine, wanna commit that to the |
I will till I ensure its what its required composer took a long time install everything :/ |
Brain Dumping for my sake :P: ModuleContextClassGuesser is how we are able to write tests for separate modules by specifying via SpecficationLocator requires two methods to be implemented:
Subtasks Required to make this extension work:
|
Not much of a licence but given this module is based off Symfony quite heavily what are requirements? https://github.com/Behat/Symfony2Extension/blob/master/LICENSE |
Sorry for the spam :P. @chillu original issue comment points to changes made to Symfonys behat upgrade. It seems that commit is quite old and a few changes have been made since then. The changes described are enough to get a working copy of behat 3 tho. Renaming Extension.php: This is an interesting change the reason says "to benefit from the behat shortcuts". Don't really know what that means... unclear benefits? |
@chillu change your minkselenium2driver to 1.2.*-dev |
Is it 1.2.*`dev it worked as is currently but I guess locking it down is easier :P |
There's no stable release for 1.2 I'm afraid, but 1.1.1 (latest stable) is against the old mink 1.5 API. |
Are there any plans for this task to be completed any time soon or are you planning to stay with Behat 2 for the foreseeable future? |
I think for 4.0 we should probably upgrade behat. But, it's not currently on the official roadmap, so it'll have to wait until someone finds time to finish it. ;) |
Do you have any estimate how big of a task this is for a reasonably good programmer? Are we talking 10 hours or 100 hours or more than that? |
I've spent a few hours on it, and it still looked pretty complicated - the whole Behat internals changed. Gut feel would be 2-3 days of work. |
+1 for codeception instead :P |
Codeception does make a lot of sense if developers are writing the tests (which is almost always the case with a SilverStripe module)... |
Codeception makes more sense, especially if SS 4.0 is going to be using PSR-4. Not sure if thats the case, but it should be ;) |
It will be using PSR4 I believe |
@dhensby as we talked about on Gitter I'm pretty interested in codeception right now and would be willing to put time in getting it up and running for SS. @chillu thoughts / feelings for that against behat? I realise there is a huge amount of work that has gone into the behat framework and testing for SS and don't want to step in and change something that you have internal knowledge or resource for |
I am 100% for codeception over separated unit tests/behat tests. It provides a more consistent and simple framework for testing - it's just a big time investment at the moment and I couldn't see a nice way to splitting tests across modules like we do at the moment. Though I only looked into it for half-an-hour about a year ago. |
I'd be interested to hear success stories of Codeception first before making that call. It sounds a bit like throwing out the baby with the bathwater: Behat is slow and complex, so Codeception must automatically be better because it looks more like PHP ;)
That being said, I'm not "married" to Behat ;) The missing upgrade path and its overarchitected internal structure has annoyed me quite a bit already. I'm just quite aware that we have many other challenges in core in the next months, and this seems like a distraction. Most of the code in the SS-specific Behat feature files would be quite easy to transfer to Codeception though (e.g. dealing with Select2, CMS ajax behaviour), since it's all just Selenium interactions anyway. So happy to be convinced if somebody feels like throwing together a proof of concept. |
My view is that if someone out there wanted to do a complete port of the CMS or Framework Behat test suite to Codeception, I'd support its merge, but I wouldn't want to see the tests of a single module sit in a state of being half Behat and half Codeception. I'm not sure what @chillu and the other core committers think of that? Last time I spoke to @dhensby he was keen on Codeception. Of course, as PoC it would probably be best to start with 1 or 2 tests to see how it looks. I tend to agree with Ingo that Behat/Codeception isn't the biggest pain point, but this is open-source: people can scratch whichever itch frustrates them the most. |
Just a little support for Behat here. |
Laravel has just brought out a nice lightweight browser testing library as well: https://laravel.com/docs/master/dusk. It runs without a Java/Selenium dependency by default, but you can opt-in to that for additional browser support. Doesn't support Cucumber, but I don't think that's necessarily a blocker. We'll likely need to get some movement here soon, since the Symfony 2.x dependencies in Behat 2.x are holding us back in other areas where dependencies overlap (e.g. Symfony YAML) |
If the amount of work is 2-3 days to upgrade I'll look at taking it on in a side project. |
@chillu this is the new part of the loader; Instead of having a single This means we can use composable configs, and the root config doesn't need to have all per-module configs setup. |
Can you please review this loader to see if this is heading in the right track? |
Updated: All framework tests are now passing in behat 3 :) moving onto cms |
most cms tests green, a few failed. |
Not bad. |
getting closer. |
CMS tests green, :D |
framework, asset-admin, and cms tests are all done. Just need to finish siteconfig tests and ready to PR. |
PR at #152 |
All merged and green. |
WIP under https://github.com/silverstripe-labs/silverstripe-behat-extension/tree/experiments/behat-3-upgrade
Requires CMS and Framework branches as well through
composer.json
:Since the extension was heavily based on the Symfony2Extension, tracking changes required to get that extension working will provide at least an indication what needs to change (https://github.com/Behat/Symfony2Extension). Specifically this commit: Behat/Symfony2Extension@65c80f2
Also useful are the Magento Extension upgrade and the Drupal Extension upgrade (1, 2, 3)
Tasks (incomplete):
ModuleContextClassGuesser
to implementSpecificationLocator
(and use inExtension->load()
)InitProcessor
to implementClassGenerator
(and use inExtension->load()
)InitProcessor
core.xml
to PHP inExtension->configure()
(unclear benefits?)Step\Given
namespaces withCall\Given
throughout all tests (cms, framework, modules)The text was updated successfully, but these errors were encountered: