Configure CI system

[andresriancho]

This task has the objective of configuring a CI system to build w3af each time a new push is done. Once moth has been migrated to fabric (#107) it will be easier to integrate it into any CI system.

There are various SaaS Jenkins:

• https://travis-ci.org/
• https://buildhive.cloudbees.com/

[righettod]

Hi,

I use CloudBees for my Github project and CloudBees provide free account
for Open Source project. It's a very good service and is use Jenkins for
CI, Jenkins is very very popular in CI in Java World....

Overview for my project :
https://righettod.ci.cloudbees.com/job/HibernateValidatorSecurityContribs/

Cordialement, Best regards,
Dominique Righetto
dominique.righetto@gmail.com
dominique.righetto@owasp.org
Twitter: @righettod
GPG: 0x323D19BA
http://righettod.github.com
"No trees were killed to send this message, but a large number of electrons
were terribly inconvenienced."

On Thu, Feb 7, 2013 at 12:11 AM, andresriancho notifications@github.comwrote:

This task has the objective of configuring a CI system to build w3af each
time a new push is done. Once moth has been migrated to fabric (#107#107)
it will be easier to integrate it into any CI system.

There are various SaaS Jenkins http://jenkins-ci.org/:

• https://travis-ci.org/

• https://buildhive.cloudbees.com/

  —
  Reply to this email directly or view it on GitHubhttps://github.com/Configure CI system #116.

[andresriancho]

I've noticed that cloudbees is a little bit more java-focused than travis, have you tested cloudbees with something non-java? Python maybe? Lets continue this talk in the mailing list.

[dmitris]

YUI3 uses Travis for a massive test suite - https://github.com/yui/yui3, a few of Yahoo projects as well (Mojito, Arrow) - https://github.com/yahoo/mojito, https://github.com/yahoo/arrow.

[andresriancho]

Will take a look at them, thanks for the links!

[andresriancho]

Decided to build my own Jenkins. Latest error in pip install:

In file included from src/lxml/lxml.etree.c:257:0:
/opt/bitnami/apps/jenkins/jenkins_home/workspace/w3af/.pyenv/build/lxml/src/lxml/includes/etree_defs.h:9:31: fatal error: libxml/xmlversion.h: No such file or directory
compilation terminated.
error: command 'gcc' failed with exit status 1

[andresriancho]

TODO:

• Have a working build for everything (from smoke tests to GUI)
• Store all Jenkins configuration in Github. Make sure no private information is leaked! [This document on backups] might help (https://wiki.jenkins-ci.org/display/JENKINS/Administering+Jenkins)
• Add wiki page on w3af project to document all the required steps for managing and using Jenkins
• Tests that run with internet need to run at the end
• Should tests have dependencies? B requires A, so when running B, A runs first and B runs only if A was successful
• I want to specify branch and commit for a build
• moth AMI should be started when the 'w3af - moth' job starts
• Configure sentinela to shutdown the VM after 45min of inactivity (no new jobs launched)
• Make sure my jenkins deploy script uses python-keyring
• Shorten commands with $nose_smoke = 'nosetests .... -A "smoke..."' ; $nose_smoke core/
• Install Embeddable Build Status Plugin

[righettod]

Andres,

You have issue with Travis ?

If you want to use Jenkins, we can perhaps try to use CloudBees Open Source project offering first in order to avoid extra maintenance coming with the hosting of your own Jenkins ?

What do you think ?

[andresriancho]

• travis-ci.org has too many limitations on what you can do, amount of time your tests can run, etc.
• Haven't tested cloudbees, but I expect the same thing to happen.

It's a shame that there's no good "paid travis ci" (it seems to be in beta?) http://beta.travis-ci.com/

Hosting my own jenkins is cheap at ec2 (turn off when not needed) and gives me more flexibility

[andresriancho]

PS: I would be more than happy if travis would be more flexible for advanced configurations and avoid me the hassle to configure my own jenkins

[andresriancho]

Code can be found here:
https://github.com/andresriancho/w3af-qa/tree/master/jenkins

[andresriancho]

Testing build badge:

[andresriancho]

Most likely going to use this for spawning Xvfb
https://wiki.jenkins-ci.org/display/JENKINS/Spawning+processes+from+build

[andresriancho]

Interesting! Changing branches for build jobs.
http://julianhigman.com/blog/2012/02/22/building-github-branches-with-jenkins/

[andresriancho]

Closing this task because I migrated all the comments and stuff to individual tasks and grouped everything in a milestone.

https://github.com/andresriancho/w3af/issues?milestone=6&state=open