Skip to content

Loading…

Allow plugins to bundle unit tests files #1673

Closed
robocoder opened this Issue · 12 comments

3 participants

@robocoder

Currently it is not possible for plugins to have tests. plugin developers have to put the tests in the piwik/tests/PHPUnit/* folders. This is not practical because the tests cannot be bundled with the plugin.

Goal: give Piwik developers the ability to bundle tests in their plugins. This will provide community with ability to build plugins with high quality and QA standards.

Tasks:

  • Allow plugins to bundle tests in their tests/* directories. Especially unit tests, and integration tests
  • Developers can execute their plugin tests
  • Add basic doc to plugins developer documentation #testing
  • Add console command to create test file and to execute tests for a specific plugin

Out of scope:

  • moving tests from PHPUnit/Plugins/* to the respective plugins/*/tests/ directory is not desired and out of scope.
  • supporting Benchmark, UI/screenshot tests is out of scope
  • Run these tests in Travis CI covered in #4280
@mattab
Piwik Open Source Analytics member

I think it would be ideal to implement this using an Ant task running on our Hudson server.

@robocoder

re: comment:1 That's possible. I'll need to secure the CI environment a bit more than currently (e.g., suhosin, open basedir restriction, etc), and limit integration testing to some pre-approved set of third-party plugins.

The build job would iterate through each plugin in the repository:

  • install Piwik trunk
  • download and install a third-party plugin from the repository
  • check for errors
  • activate plugin
  • check for errors
  • run plugin-specific unit tests
@mattab
Piwik Open Source Analytics member

From #2017: WP has some interesting plugins for checking themes and plugins that we can learn from.

  • Theme Authenticity Checker
  • Exploit Scanner
  • Theme Check

Lots of possibilities: we could run a test before activating a plugin, test plugins submitted to the repository, svn commit hook, or via jenkins.

@tsteur
Piwik Open Source Analytics member

In 53fc1b9: refs #1673 allow plugins to bundle tests

@tsteur
Piwik Open Source Analytics member

In bce736f: refs #1673 move it to correct testsuite

@tsteur
Piwik Open Source Analytics member

In 793c3f8: refs #1673 added a simple test to example plugin, added test generator

@tsteur
Piwik Open Source Analytics member

Test generator: done.
./console generate:test

Executing a test using
./console tests:run ExamplePlugin

@tsteur
Piwik Open Source Analytics member

TBD: do we need different options to generate StandardTest (Unit Test), DB Test, Integration Test?

@tsteur
Piwik Open Source Analytics member

In 9d48059: refs #1673 always add plugins group

@tsteur
Piwik Open Source Analytics member

In a1c6c2e: refs #1673 automatically attach "Test" to testname if needed, added optional option to define type of test

@mattab
Piwik Open Source Analytics member

In 6156407: Refs #1673 Also allow plugin tests in the /Test folder

@robocoder robocoder added this to the 2.0 - Piwik 2.0 milestone
@tsteur tsteur was assigned by robocoder
@sabl0r sabl0r pushed a commit to sabl0r/piwik that referenced this issue
@tsteur tsteur refs #1673 allow plugins to bundle tests 53fc1b9
@sabl0r sabl0r pushed a commit to sabl0r/piwik that referenced this issue
@tsteur tsteur refs #1673 move it to correct testsuite bce736f
@sabl0r sabl0r pushed a commit to sabl0r/piwik that referenced this issue
@tsteur tsteur refs #1673 always add plugins group 9d48059
@sabl0r sabl0r pushed a commit to sabl0r/piwik that referenced this issue
@tsteur tsteur refs #1673 automatically attach "Test" to testname if needed, added o…
…ptional option to define type of test
a1c6c2e
@sabl0r sabl0r pushed a commit to sabl0r/piwik that referenced this issue
@mattab mattab Refs #1673 Also allow plugin tests in the /Test folder 6156407
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.