Continuous Integration server improvements

[mattab]

Improvement ideas

• Run code coverage during the tests and produce reports: Generate Code Coverage of Piwik test suite #4224
• Implement new Webtests to automatically test the installation, update, login. See Write Installer/Updater screenshot tests #2935
• Run our piwik.js javascript tests on continuous integration server Run Piwik.js javascript tests on continuous integration #4225
• automated API doc generation for piwik.js tracker
  • jsphpdoc - does some text transformations and feeds the output to phpDocumentor; since it doesn't parse the JS, it doesn't recognize some of the syntactic sugar to define methods. The result is missing the public methods, but includes many private methods.
  • jsdoc-toolkit - uses rhino/java; despite failing miserably (initial output has only three functions: cvt_hex, piwik_log, and piwik_track), it's probably the more promising of the two given its feature set (and usage by other projects). It appears to we have to add more tags as hints...

Created as a followup / summary of the 2 year old ticket #1470

More information about piwik QA: http://piwik.org/qa/

[mattab]

In the last 9 months we have made many changes and improvements to the QA environment. We are using Travis and testing on 3 different php versions, mysqli + PDO Mysql, we have unit tests, integration tests, plugin tests, and also Javascript tracker tests, and Angular JS tests (just starting those ones slowly!).

Well done to the team for their dedication to testing... it's so nice when we manage to develop in TDD style!

More info at http://piwik.org/qa/
And our travis builds: https://travis-ci.org/piwik/piwik