Decide on testing strategy #245

Open
tobscure opened this Issue Aug 27, 2015 · 19 comments

Projects

None yet

4 participants

@tobscure
Member

With my limited knowledge, here's what I'm thinking:

  • Unit tests for backend classes. I think phpspec is the preferred tool here – we already have one spec thanks to @franzliedke.
  • Functional/integration tests for the JSON-API. (Not sure what tool would be best for this?)
  • Functional/integration tests for the extension API. (Ditto?)
  • Unit tests for front-end utils, components, helpers, and models. (Probably using QUnit?)
  • Acceptance tests for the whole thing.

Thoughts?

@franzliedke
Member

From my perspective, the most important things are unit tests for important classes (which we'll have to go looking for) and acceptance tests (because those cover the other things).

@tobscure tobscure added Help Wanted and removed help wanted labels Aug 28, 2015
@franzliedke franzliedke added this to the 0.1.0-beta.3 milestone Sep 3, 2015
@Luceos
Member
Luceos commented Sep 5, 2015

I see there was a help wanted. If you'd like me to contribute with this I'd love to create the tests for the php side of things (laravel, eloquent, user creation etc).

@justjavac justjavac referenced this issue in justjavac/Flarum Sep 7, 2015
Open

Flarum v0.1.0 开发路线图 #3

10 of 53 tasks complete
@tobscure tobscure added the Backend label Sep 16, 2015
@franzliedke
Member

Woah, I just played around with Codeception, and acceptance testing for the whole stack amounts to a whole lot of work. Therefore, I'd say we focus on the following for now:

  • more PhpSpec unit tests
  • integration (?) tests for the front-end components, one-by-one (cause that's where the magic is at)

We probably should add a few acceptance tests for the critical routes (such as registration and posting maybe), but limit it to a few...

What do you think?

@tobscure
Member

Sounds good to me. I'll work on setting up a front-end testing framework, you focus on the backend? :)

@tobscure
Member

What about acceptance testing the JSON-API? That would be easier than acceptance testing our whole stack, and is quite important since our app isn't the only one using the JSON-API.

@Luceos Sorry, didn't see your comment above. I'm sure @franzliedke would greatly appreciate your help with backend testing!

@franzliedke
Member

Yeah, I'll have a look at how we can do JSON-API testing.

@Luceos If you ever feel bored enough to help with tests, you could look at testing some of the central base classes, e.g. the Action class. ;)

@Luceos
Member
Luceos commented Sep 17, 2015

I'll take a look at that asap @franzliedke , i have more experience with travis though, so you've decided for codeception, I'll read up into that then.

@franzliedke
Member

Nah, I won't use Codeception for now, that's only for the acceptance tests. For unit testing, we use PhpSpec. Take a look at our first (and only) test. :)

@kirkbushell
Contributor

I'm happy to contribute here. I have years of testing experience and really loving working with PHPspec. In fact, I'd be very happy to push forward with and champion the test strategy, beginning with the core library and feature set.

@kirkbushell
Contributor

One thing to add here - the dependencies and uses of Laravel in the project prevent phpspec from being used on a number of classes. I tried writing some specs for the database setting repository, and a couple of handlers, it was literally impossible. The lack of dependency injection (such as injecting the event dispatcher instead of using event()) is causing some problems.

Imho this highlights a possible design problem, and certainly a code smell. Having used those functions on other projects, they end up causing more problems than they're worth.

@kirkbushell
Contributor

Happy to close this one now? I'll continue to help and push forward with the tests.

@franzliedke
Member

I think we still need to sort the frontend

@kirkbushell
Contributor

Haven't looke at the frontend yet, but if it's written in a modular fashion happy to move ahead with that as well, using jasmine.

@tobscure
Member

Thanks for your work so far @kirkbushell, we're really lucky to have you pushing this forward!

That would be amazing if you lead the charge with the front-end tests too. I have virtually no experience with testing so I don't know how it will fare, but as far as I'm concerned it's pretty modular (split up into components/classes/helper functions). However, some of them rely on an app global... will that be problematic?

In any case, please take a quick look at the front-end codebase when you have a chance and let me know what you think :)

@kirkbushell
Contributor

Yup, anything global will be an issue for testing :)

I'll check it out once I get a handle on the server-side testing. Still a lot to do there. Also, we'll need to include tests and how to write them/what's expected as part o the contribution documentation.

@franzliedke
Member

Moving this back to beta.4 because we still have to sort out the frontend. :)

@tobscure
Member

Thinking about front-end testing... I'm in way over my head, but here are some ideas:

  • Write tests in ES6 using Jasmine
    • e.g. import Button from 'flarum/components/Button'
    • Render the component to a staging area via Mithril and make sure DOM output matches component configuration
    • Fake interactions and make sure they behave as expected
    • Spy on app global/its properties, watching for the correct calls
  • Compile tests into ES5 in the same way as the main app, output to the dist folder
  • Set up Karma to run dist/tests.js against dist/app.js in PhantomJS or Chrome or whatever

@kirkbushell input?

@tobscure
Member

Another possibility for testing components: https://github.com/StephanHoyer/mithril-query

@franzliedke
Member

I like what Ember.JS suggests for components: integration tests. Example: https://github.com/openHPI/ember-form-components/blob/master/tests/integration/components/editable-field-test.js

Basically, you render a component, pass in any data you want, and then query the DOM for any changes you want. It's a reasonable and sensile way to test components work as expected; we still need to find a good setup for full acceptance tests, though...

I don't have any knowledge in regards to browser JS testing, though, so I can't make any recommendations on how to achieve that.

@Luceos Luceos removed the Help Wanted label Feb 18, 2016
This was referenced Feb 24, 2016
@tobscure tobscure modified the milestone: 0.1.0-beta.7, 0.1.0-beta.6 Aug 23, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment