New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set up frontend testing framework #245

Open
tobscure opened this Issue Aug 27, 2015 · 20 comments

Comments

Projects
None yet
4 participants
@tobscure
Member

tobscure commented Aug 27, 2015

With my limited knowledge, here's what I'm thinking:

  • Unit tests for backend classes. I think phpspec is the preferred tool here – we already have one spec thanks to @franzliedke.
  • Functional/integration tests for the JSON-API. (Not sure what tool would be best for this?)
  • Functional/integration tests for the extension API. (Ditto?)
  • Unit tests for front-end utils, components, helpers, and models. (Probably using QUnit?)
  • Acceptance tests for the whole thing.

Thoughts?

@franzliedke

This comment has been minimized.

Show comment
Hide comment
@franzliedke

franzliedke Aug 27, 2015

Member

From my perspective, the most important things are unit tests for important classes (which we'll have to go looking for) and acceptance tests (because those cover the other things).

Member

franzliedke commented Aug 27, 2015

From my perspective, the most important things are unit tests for important classes (which we'll have to go looking for) and acceptance tests (because those cover the other things).

@luceos

This comment has been minimized.

Show comment
Hide comment
@luceos

luceos Sep 5, 2015

Member

I see there was a help wanted. If you'd like me to contribute with this I'd love to create the tests for the php side of things (laravel, eloquent, user creation etc).

Member

luceos commented Sep 5, 2015

I see there was a help wanted. If you'd like me to contribute with this I'd love to create the tests for the php side of things (laravel, eloquent, user creation etc).

@justjavac justjavac referenced this issue Sep 7, 2015

Open

Flarum v0.1.0 开发路线图 #3

18 of 53 tasks complete

@tobscure tobscure added the Backend label Sep 16, 2015

@franzliedke

This comment has been minimized.

Show comment
Hide comment
@franzliedke

franzliedke Sep 17, 2015

Member

Woah, I just played around with Codeception, and acceptance testing for the whole stack amounts to a whole lot of work. Therefore, I'd say we focus on the following for now:

  • more PhpSpec unit tests
  • integration (?) tests for the front-end components, one-by-one (cause that's where the magic is at)

We probably should add a few acceptance tests for the critical routes (such as registration and posting maybe), but limit it to a few...

What do you think?

Member

franzliedke commented Sep 17, 2015

Woah, I just played around with Codeception, and acceptance testing for the whole stack amounts to a whole lot of work. Therefore, I'd say we focus on the following for now:

  • more PhpSpec unit tests
  • integration (?) tests for the front-end components, one-by-one (cause that's where the magic is at)

We probably should add a few acceptance tests for the critical routes (such as registration and posting maybe), but limit it to a few...

What do you think?

@tobscure

This comment has been minimized.

Show comment
Hide comment
@tobscure

tobscure Sep 17, 2015

Member

Sounds good to me. I'll work on setting up a front-end testing framework, you focus on the backend? :)

Member

tobscure commented Sep 17, 2015

Sounds good to me. I'll work on setting up a front-end testing framework, you focus on the backend? :)

@tobscure

This comment has been minimized.

Show comment
Hide comment
@tobscure

tobscure Sep 17, 2015

Member

What about acceptance testing the JSON-API? That would be easier than acceptance testing our whole stack, and is quite important since our app isn't the only one using the JSON-API.

@luceos Sorry, didn't see your comment above. I'm sure @franzliedke would greatly appreciate your help with backend testing!

Member

tobscure commented Sep 17, 2015

What about acceptance testing the JSON-API? That would be easier than acceptance testing our whole stack, and is quite important since our app isn't the only one using the JSON-API.

@luceos Sorry, didn't see your comment above. I'm sure @franzliedke would greatly appreciate your help with backend testing!

@franzliedke

This comment has been minimized.

Show comment
Hide comment
@franzliedke

franzliedke Sep 17, 2015

Member

Yeah, I'll have a look at how we can do JSON-API testing.

@luceos If you ever feel bored enough to help with tests, you could look at testing some of the central base classes, e.g. the Action class. ;)

Member

franzliedke commented Sep 17, 2015

Yeah, I'll have a look at how we can do JSON-API testing.

@luceos If you ever feel bored enough to help with tests, you could look at testing some of the central base classes, e.g. the Action class. ;)

@luceos

This comment has been minimized.

Show comment
Hide comment
@luceos

luceos Sep 17, 2015

Member

I'll take a look at that asap @franzliedke , i have more experience with travis though, so you've decided for codeception, I'll read up into that then.

Member

luceos commented Sep 17, 2015

I'll take a look at that asap @franzliedke , i have more experience with travis though, so you've decided for codeception, I'll read up into that then.

@franzliedke

This comment has been minimized.

Show comment
Hide comment
@franzliedke

franzliedke Sep 17, 2015

Member

Nah, I won't use Codeception for now, that's only for the acceptance tests. For unit testing, we use PhpSpec. Take a look at our first (and only) test. :)

Member

franzliedke commented Sep 17, 2015

Nah, I won't use Codeception for now, that's only for the acceptance tests. For unit testing, we use PhpSpec. Take a look at our first (and only) test. :)

@kirkbushell

This comment has been minimized.

Show comment
Hide comment
@kirkbushell

kirkbushell Sep 25, 2015

Contributor

I'm happy to contribute here. I have years of testing experience and really loving working with PHPspec. In fact, I'd be very happy to push forward with and champion the test strategy, beginning with the core library and feature set.

Contributor

kirkbushell commented Sep 25, 2015

I'm happy to contribute here. I have years of testing experience and really loving working with PHPspec. In fact, I'd be very happy to push forward with and champion the test strategy, beginning with the core library and feature set.

@kirkbushell

This comment has been minimized.

Show comment
Hide comment
@kirkbushell

kirkbushell Sep 26, 2015

Contributor

One thing to add here - the dependencies and uses of Laravel in the project prevent phpspec from being used on a number of classes. I tried writing some specs for the database setting repository, and a couple of handlers, it was literally impossible. The lack of dependency injection (such as injecting the event dispatcher instead of using event()) is causing some problems.

Imho this highlights a possible design problem, and certainly a code smell. Having used those functions on other projects, they end up causing more problems than they're worth.

Contributor

kirkbushell commented Sep 26, 2015

One thing to add here - the dependencies and uses of Laravel in the project prevent phpspec from being used on a number of classes. I tried writing some specs for the database setting repository, and a couple of handlers, it was literally impossible. The lack of dependency injection (such as injecting the event dispatcher instead of using event()) is causing some problems.

Imho this highlights a possible design problem, and certainly a code smell. Having used those functions on other projects, they end up causing more problems than they're worth.

@kirkbushell

This comment has been minimized.

Show comment
Hide comment
@kirkbushell

kirkbushell Sep 28, 2015

Contributor

Happy to close this one now? I'll continue to help and push forward with the tests.

Contributor

kirkbushell commented Sep 28, 2015

Happy to close this one now? I'll continue to help and push forward with the tests.

@franzliedke

This comment has been minimized.

Show comment
Hide comment
@franzliedke

franzliedke Sep 28, 2015

Member

I think we still need to sort the frontend

Member

franzliedke commented Sep 28, 2015

I think we still need to sort the frontend

@kirkbushell

This comment has been minimized.

Show comment
Hide comment
@kirkbushell

kirkbushell Sep 28, 2015

Contributor

Haven't looke at the frontend yet, but if it's written in a modular fashion happy to move ahead with that as well, using jasmine.

Contributor

kirkbushell commented Sep 28, 2015

Haven't looke at the frontend yet, but if it's written in a modular fashion happy to move ahead with that as well, using jasmine.

@tobscure

This comment has been minimized.

Show comment
Hide comment
@tobscure

tobscure Sep 29, 2015

Member

Thanks for your work so far @kirkbushell, we're really lucky to have you pushing this forward!

That would be amazing if you lead the charge with the front-end tests too. I have virtually no experience with testing so I don't know how it will fare, but as far as I'm concerned it's pretty modular (split up into components/classes/helper functions). However, some of them rely on an app global... will that be problematic?

In any case, please take a quick look at the front-end codebase when you have a chance and let me know what you think :)

Member

tobscure commented Sep 29, 2015

Thanks for your work so far @kirkbushell, we're really lucky to have you pushing this forward!

That would be amazing if you lead the charge with the front-end tests too. I have virtually no experience with testing so I don't know how it will fare, but as far as I'm concerned it's pretty modular (split up into components/classes/helper functions). However, some of them rely on an app global... will that be problematic?

In any case, please take a quick look at the front-end codebase when you have a chance and let me know what you think :)

@kirkbushell

This comment has been minimized.

Show comment
Hide comment
@kirkbushell

kirkbushell Sep 29, 2015

Contributor

Yup, anything global will be an issue for testing :)

I'll check it out once I get a handle on the server-side testing. Still a lot to do there. Also, we'll need to include tests and how to write them/what's expected as part o the contribution documentation.

Contributor

kirkbushell commented Sep 29, 2015

Yup, anything global will be an issue for testing :)

I'll check it out once I get a handle on the server-side testing. Still a lot to do there. Also, we'll need to include tests and how to write them/what's expected as part o the contribution documentation.

@franzliedke

This comment has been minimized.

Show comment
Hide comment
@franzliedke

franzliedke Oct 23, 2015

Member

Moving this back to beta.4 because we still have to sort out the frontend. :)

Member

franzliedke commented Oct 23, 2015

Moving this back to beta.4 because we still have to sort out the frontend. :)

@franzliedke franzliedke modified the milestones: 0.1.0-beta.4, 0.1.0-beta.3 Oct 23, 2015

@tobscure

This comment has been minimized.

Show comment
Hide comment
@tobscure

tobscure Oct 31, 2015

Member

Thinking about front-end testing... I'm in way over my head, but here are some ideas:

  • Write tests in ES6 using Jasmine
    • e.g. import Button from 'flarum/components/Button'
    • Render the component to a staging area via Mithril and make sure DOM output matches component configuration
    • Fake interactions and make sure they behave as expected
    • Spy on app global/its properties, watching for the correct calls
  • Compile tests into ES5 in the same way as the main app, output to the dist folder
  • Set up Karma to run dist/tests.js against dist/app.js in PhantomJS or Chrome or whatever

@kirkbushell input?

Member

tobscure commented Oct 31, 2015

Thinking about front-end testing... I'm in way over my head, but here are some ideas:

  • Write tests in ES6 using Jasmine
    • e.g. import Button from 'flarum/components/Button'
    • Render the component to a staging area via Mithril and make sure DOM output matches component configuration
    • Fake interactions and make sure they behave as expected
    • Spy on app global/its properties, watching for the correct calls
  • Compile tests into ES5 in the same way as the main app, output to the dist folder
  • Set up Karma to run dist/tests.js against dist/app.js in PhantomJS or Chrome or whatever

@kirkbushell input?

@tobscure

This comment has been minimized.

Show comment
Hide comment
@tobscure

tobscure Oct 31, 2015

Member

Another possibility for testing components: https://github.com/StephanHoyer/mithril-query

Member

tobscure commented Oct 31, 2015

Another possibility for testing components: https://github.com/StephanHoyer/mithril-query

@franzliedke

This comment has been minimized.

Show comment
Hide comment
@franzliedke

franzliedke Oct 31, 2015

Member

I like what Ember.JS suggests for components: integration tests. Example: https://github.com/openHPI/ember-form-components/blob/master/tests/integration/components/editable-field-test.js

Basically, you render a component, pass in any data you want, and then query the DOM for any changes you want. It's a reasonable and sensile way to test components work as expected; we still need to find a good setup for full acceptance tests, though...

I don't have any knowledge in regards to browser JS testing, though, so I can't make any recommendations on how to achieve that.

Member

franzliedke commented Oct 31, 2015

I like what Ember.JS suggests for components: integration tests. Example: https://github.com/openHPI/ember-form-components/blob/master/tests/integration/components/editable-field-test.js

Basically, you render a component, pass in any data you want, and then query the DOM for any changes you want. It's a reasonable and sensile way to test components work as expected; we still need to find a good setup for full acceptance tests, though...

I don't have any knowledge in regards to browser JS testing, though, so I can't make any recommendations on how to achieve that.

@franzliedke franzliedke modified the milestones: 0.1.0-beta.5, 0.1.0-beta.6 Jan 7, 2016

@luceos luceos removed the Help Wanted label Feb 18, 2016

@tobscure tobscure modified the milestones: 0.1.0-beta.7, 0.1.0-beta.6 Aug 23, 2016

@luceos

This comment has been minimized.

Show comment
Hide comment
@luceos

luceos Apr 7, 2017

Member

Revisiting this, I'm sure that Laravel Dusk can be a big life saver here for the Frontend testing. We would need Laravel 5.4 though.

Member

luceos commented Apr 7, 2017

Revisiting this, I'm sure that Laravel Dusk can be a big life saver here for the Frontend testing. We would need Laravel 5.4 though.

@tobscure tobscure removed this from the 0.1.0-beta.7 milestone Jul 22, 2017

@tobscure tobscure changed the title from Decide on testing strategy to Set up frontend testing framework Jun 16, 2018

@franzliedke franzliedke removed the Backend label Jul 21, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment