With my limited knowledge, here's what I'm thinking:
From my perspective, the most important things are unit tests for important classes (which we'll have to go looking for) and acceptance tests (because those cover the other things).
I see there was a help wanted. If you'd like me to contribute with this I'd love to create the tests for the php side of things (laravel, eloquent, user creation etc).
Woah, I just played around with Codeception, and acceptance testing for the whole stack amounts to a whole lot of work. Therefore, I'd say we focus on the following for now:
We probably should add a few acceptance tests for the critical routes (such as registration and posting maybe), but limit it to a few...
What do you think?
Sounds good to me. I'll work on setting up a front-end testing framework, you focus on the backend? :)
What about acceptance testing the JSON-API? That would be easier than acceptance testing our whole stack, and is quite important since our app isn't the only one using the JSON-API.
@Luceos Sorry, didn't see your comment above. I'm sure @franzliedke would greatly appreciate your help with backend testing!
Yeah, I'll have a look at how we can do JSON-API testing.
@Luceos If you ever feel bored enough to help with tests, you could look at testing some of the central base classes, e.g. the Action class. ;)
I'll take a look at that asap @franzliedke , i have more experience with travis though, so you've decided for codeception, I'll read up into that then.
Nah, I won't use Codeception for now, that's only for the acceptance tests. For unit testing, we use PhpSpec. Take a look at our first (and only) test. :)
I'm happy to contribute here. I have years of testing experience and really loving working with PHPspec. In fact, I'd be very happy to push forward with and champion the test strategy, beginning with the core library and feature set.
One thing to add here - the dependencies and uses of Laravel in the project prevent phpspec from being used on a number of classes. I tried writing some specs for the database setting repository, and a couple of handlers, it was literally impossible. The lack of dependency injection (such as injecting the event dispatcher instead of using event()) is causing some problems.
Imho this highlights a possible design problem, and certainly a code smell. Having used those functions on other projects, they end up causing more problems than they're worth.
Happy to close this one now? I'll continue to help and push forward with the tests.
I think we still need to sort the frontend
Haven't looke at the frontend yet, but if it's written in a modular fashion happy to move ahead with that as well, using jasmine.
Thanks for your work so far @kirkbushell, we're really lucky to have you pushing this forward!
That would be amazing if you lead the charge with the front-end tests too. I have virtually no experience with testing so I don't know how it will fare, but as far as I'm concerned it's pretty modular (split up into components/classes/helper functions). However, some of them rely on an app global... will that be problematic?
In any case, please take a quick look at the front-end codebase when you have a chance and let me know what you think :)
Yup, anything global will be an issue for testing :)
I'll check it out once I get a handle on the server-side testing. Still a lot to do there. Also, we'll need to include tests and how to write them/what's expected as part o the contribution documentation.
Moving this back to beta.4 because we still have to sort out the frontend. :)
Thinking about front-end testing... I'm in way over my head, but here are some ideas:
import Button from 'flarum/components/Button'
Another possibility for testing components: https://github.com/StephanHoyer/mithril-query
I like what Ember.JS suggests for components: integration tests. Example: https://github.com/openHPI/ember-form-components/blob/master/tests/integration/components/editable-field-test.js
Basically, you render a component, pass in any data you want, and then query the DOM for any changes you want. It's a reasonable and sensile way to test components work as expected; we still need to find a good setup for full acceptance tests, though...
I don't have any knowledge in regards to browser JS testing, though, so I can't make any recommendations on how to achieve that.