Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Setup Jenkins to run PHPUnit test suite #3445

Closed
mattab opened this Issue · 27 comments

3 participants

@mattab
Owner

Jenkins was much needed for running smooth operations at Piwik. without jenkins we have to spend a lot of time running tests manually.

It would be fantastic to setup Jenkins as before, that would run the PHPUnit test suite.

Also would be great to generate the PHP docs for: PHP tracker class. They havent been generated in a while and are missing the last new functions.

We do not need more things for now, but this would be VERY useful.

@robocoder
Collaborator

Jenkins is running on dev5 (on Debian Squeeze).

Planned/ToDo:

  • install vagrant
  • setup Piwik VM as a slave
  • configure jobs
@mattab
Owner

how is it going, have you made some progress?
No problem if you can't finish, but please send us asap the list of tasks left do to setup Jenkins on dev5, so we can organize the work :)

it's going to be awesome to have jenkins back!

@robocoder
Collaborator

dev5:

  • a precise32 VM is running as a Jenkins slave; (precise64 is not compatible with this host)
  • ToDo:
    • migrate users and jobs from dev6
    • fix /etc/init.d/vboxweb-service (it isn't automatically starting vboxwebsrv)

Travis-CI:

@sgiehl
Collaborator

I just migrated the jenkins user to the new server and fixed /etc/init.d/vboxweb-service by creating the config in /etc/default/virtualbox. Now the jenkins slave node is running.

@sgiehl
Collaborator

@vipsoft: Do you have time to migrate/configure the jobs? I don't know exactly how to do that using the slave.

@robocoder
Collaborator

(In [7459]) refs #3445 - migrate Piwik CI job

@robocoder
Collaborator

I bumped the VM's max_connections = 200 for MySQL because some tests aren't closing connections.

ToDo (these items can be tracked in another ticket):

  • migrate the remaining jobs from dev6 (i.e., docs, git mirroring, nightly snapshots)
  • add a daily build that runs code coverage if we can workaround the zend_mm_heap corrupted crash
@sgiehl
Collaborator

(In [7461]) refs #3445 write junit log to make it possible to display test results
within the jenkins job

@sgiehl
Collaborator

(In [7462]) refs #3445 moved junit log creation to phpunit.xml, otherwise skipped
and incomplete tests can't be logged

@robocoder
Collaborator

I migrated the git mirroring jobs.

I'm rebuilding the VM. vboxmanage is powerful, and equally dangerous without a backup. =P

@mattab
Owner

it looks like builds are working on Jenkins!! wouhou... awesome!

Speaking of backups, is it possible to make an image of the full box to be able to restore it if anything happens? I'm not familiar with such techniques... but because Jenkins is such a key service for us, I'm curious as to how we could have this extra security?

Do you know why emails are not sent to the piwik-builds mailing list from dev5?

@mattab
Owner

The console output in dev5.piwik.org:8080/job/Piwik.CI/70/console says
[exec] The Xdebug extension is not loaded. No code coverage will be generated.

Once enabled we could get code coverage reports! :)

@robocoder
Collaborator

@SteveG I see you're configuring build-ci-full.xml. When you get around to running phpunit with code coverage, you'll have to load the xdebug extension manually. (It's disabled by default so that the CI-on-commit is faster.) Example:

    <target name="test-mysqli" depends="process-build-resources-mysqli">
        <echo>MYSQLI unit tests started</echo>

        <exec executable="php" dir="${basedir}/PHPUnit" failonerror="true" failifexecutionfails="true">
            <arg line="-d zend_extension=/usr/lib/php5/20090626/xdebug.so
                       /usr/bin/phpunit
                       --log-junit ${basedir}/build/logs/junit.xml
                       --coverage-html ${basedir}/build/coverage
                       --configuration ${source}/app/phpunit.xml.dist"/>
        </exec>

        <echo>MYSQLI unit tests finished</echo>
    </target>
@robocoder
Collaborator

@matt I reconfigured exim on dev5. Outbound email should now work.

@mattab
Owner

Great emails now work!

Also test coverage reports work: http://dev5.piwik.org:8080/job/Piwik.Full/cloverphp/ - Kuddos Stefan

I think they still dont run for Integration tests but hopefully you find the problem.

@vipsoft is it OK now to make qa.piwik.org point to dev5?

@robocoder
Collaborator

Sure, you can update the dns as long as we can still reach dev6.

I've installed Apache on dev5 so we can serve out generated docs on port 80. I hope to move those jobs this week.

re: integration tests and code coverage: coverage is only collectable for the phpunit process.

@mattab
Owner

I updated the CNAME to dev5

These URLs linked from the piwik QA page are 404:

Thanks for restoring phpdocs, let me know the new URL i'll update QA page and the tracking API doc.

re: integration tests and code coverage: coverage is only collectable for the phpunit process.
we're lucky, because the integration tests call the API internally so the code coverage will be collectible :) -- except for piwik.php tracker requests

@sgiehl
Collaborator

I tried to activate the integration tests for full build. the build time went up to about 2 hours. Without the integration tests it runs about 15 minutes. But there are still problems with some tests and xdebug.

@mattab
Owner

Replying to SteveG:

I tried to activate the integration tests for full build. the build time went up to about 2 hours. Without the integration tests it runs about 15 minutes. But there are still problems with some tests and xdebug.

What did you mean, it seems integration tests are enabled in mysqli and pdo-mysql and takes 36 minutes total.
http://qa.piwik.org:8080/job/Piwik.CI/178/console

Maybe it would work if the "nightly" build had xdebug enabled and would generate code coverage once a day only? or does it fail completely with xdebug enabled?

@sgiehl
Collaborator

The tests are already sometimes failing without running the integration tests.
See http://qa.piwik.org:8080/job/Piwik.Full/164/console - Result 139 should be a segmentation fault. Running the tests within the vm on console has sometimes the same result. But sadly it doesn't seem to be the same causing it each time.

Another problem sometimes occures while generating the codecoverage as html: http://qa.piwik.org:8080/job/Piwik.Full/185/console

@mattab
Owner

I moved the "Code coverage" feature request to: #3606 since it appears we are faving at least 2 critical core php bugs.

I think this task is done, but pending one last item: http://qa.piwik.org:8080/phpdocs/ is 404 - @vipsoft is it possible to restore the phpdocs?

Then we can close the ticket -- good times!

@robocoder
Collaborator

Yeah, I've seen similar (and other) problems at work when generating code coverage. We were stuck on phpunit 3.7.1 for a while because an autoloader refactoring in phpunit was triggering a php bug (segfault).

The php-xdebug package from Canonical is 2.1.0. We can try asking the Ubuntu folks to backport 2.1.4. (Preferably we won't manage our own custom builds.) Alternately, upgrade the VM from 12.04 to 12.10 (which would give us php 5.4.6 and xdebug 2.2.1).

Yes, I'll take a look at phpdocs soon.

@mattab
Owner

The php-xdebug package from Canonical is 2.1.0. We can try asking the Ubuntu folks to backport 2.1.4. (Preferably we won't manage our own custom builds.) Alternately, upgrade the VM from 12.04 to 12.10 (which would give us php 5.4.6 and xdebug 2.2.1).

+1 for upgrading the VM

Yes, I'll take a look at phpdocs soon.

Very appreciated

@mattab
Owner

I just saw this related ticket #3054

@mattab
Owner

(In [7654]) Refs #3445 Adding doc for --testdox

@mattab
Owner

(In [7655]) Refs #3445 try again

@mattab
Owner

now working! awesome...

@mattab mattab added this to the 1.11 - Piwik 1.11 milestone
@robocoder robocoder was assigned by mattab
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.