Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Set up Jenkins for rootpy and root_numpy Continuous Integration #106

Closed
cdeil opened this Issue Nov 30, 2012 · 14 comments

Comments

Projects
None yet
4 participants
Owner

cdeil commented Nov 30, 2012

We have travis-ci, but it only runs tests for Ubuntu 32 bit.
We also want to test other Linuxes and Mac OS X.

We currently have two Macs available, one running OS X 10.4 and one 10.7, for the 10.7 machine I have virtualbox and vmware fusion installed, so we can run any number of Linuxes in virtual machines (VMs).

The current plan is to run a http://jenkins-ci.org either on the rootpy.org webserver or on the 10.7 machine, and use the others as build slaves, but if it's easier we could also run Jenkins on the 10.7 machine.

Stuff we have to figure out:

  • How to set up the 10.7 machine so that it is reachable at a fixed URL
  • How to run Jenkins such that it's web interface is accessible somewhere on rootpy.org
  • How to manage SSH keys so that our machines remain secure.
  • How to wake up, run tests and shut down VMs from Jenkins
  • How to trigger Jenkins on github commits (and ideally pull requests)
  • How to put the built sphinx docs from each commit on rootpy.org (but only if the build succeeded I suppose)

Any help on this is welcome, I've never done this.
We can just use this ticket to discuss the issues that come up.
I have time tomorrow and would like to get this up and running quickly.

References for myself:

@ghost ghost assigned cdeil Nov 30, 2012

Member

ekfriis commented Nov 30, 2012

Hi all,

I've just wasted a weekend (but think of my future efficiency!!!) setting
up jenkins for my CMS analysis framework:

https://github.com/uwcms/FinalStateAnalysis

(note my awesome build status badges in the readme, that was today's
contribution to science)

So anyway, I can definitely help with "How to trigger Jenkins on github
commits (and ideally pull requests)", as I've just gone through the
excercise.

Evan

On Fri, Nov 30, 2012 at 3:19 PM, Christoph Deil notifications@github.comwrote:

We have travis-ci, but it only runs tests for Ubuntu 32 bit.
We also want to test other Linuxes and Mac OS X.

We currently have two Macs available, one running OS X 10.4 and one 10.7,
for the 10.7 machine I have virtualbox and vmware fusion installed, so we
can run any number of Linuxes in virtual machines (VMs).

The current plan is to run a http://jenkins-ci.org either on the
rootpy.org webserver or on the 10.7 machine, and use the others as build
slaves, but if it's easier we could also run Jenkins on the 10.7 machine.

Stuff we have to figure out:

  • How to set up the 10.7 machine so that it is reachable at a fixed URL
  • How to run Jenkins such that it's web interface is accessible
    somewhere on rootpy.org
  • How to manage SSH keys so that our machines remain secure.
  • How to wake up, run tests and shut down VMs from Jenkins
  • How to trigger Jenkins on github commits (and ideally pull requests)
  • How to put the built sphinx docs from each commit on rootpy.org (but
    only if the build succeeded I suppose)

Any help on this is welcome, I've never done this.
We can just use this ticket to discuss the issues that come up.
I have time tomorrow and would like to get this up and running quickly.

References:

Owner

cdeil commented Nov 30, 2012

@ekfriis Great.
I am currently running Jenkins on my Macbook. I have Noel's 10.4 Mac as a build slave and can execute builds there.
Now I'd like to start a Jenkins on the rootpy.org server, but I'm not quite sure how to get started...
Google chat?

Owner

cdeil commented Dec 1, 2012

Another task we should schedule for the Jenkins is generating per-commit or daily coverage reports and put those on rootpy.org. Here's some info:
#19 (comment)

Owner

cdeil commented Dec 3, 2012

I've made a wiki page describing how I think the rootpy.org website and continuous integration could work:
https://github.com/rootpy/rootpy/wiki/Webserver-and-Continuous-Integration

@ndawe and others: what do you think?

Maybe we can do a Google hangout this week to get this off the ground quickly?

Owner

cdeil commented Dec 12, 2012

There's two things we should change about make test-coverage:

  • Don't run this one very slow test that takes minutes
  • Sky external code like pyparsing (I couldn't figure out how to do that yet, so if someone knows, please comment here or make a PR)
Owner

pwaller commented Feb 21, 2013

@cdeil status?

Owner

cdeil commented Feb 21, 2013

We have a Mac running OS X 10.7 and Jenkins here:
http://mac-ci.mooo.com:8080

What remains to be figured out / done:

  • configure the rootpy and root_numpy build matrix.
  • trigger on github commits (and pull requests if possible)
  • send email notifications. Either to the committter, or to me and I file a github issue.

I do plan to do this in the next two weeks, please leave this ticket open for now.

@ekfriis or anyone else: if you have time to do it, great, just go ahead.

Member

ekfriis commented Feb 21, 2013

I tried to pick up where I left things, but after I ran a new build, the
jenkins interface has stopped responding. What a flake.

On Thu, Feb 21, 2013 at 12:41 PM, Christoph Deil
notifications@github.comwrote:

We have a Mac running OS X 10.7 and Jenkins here:
http://mac-ci.mooo.com:8080

What remains to be figured out / done:

  • configure the rootpy and root_numpy build matrix.
  • trigger on github commits (and pull requests if possible)
  • send email notifications. Either to the committter, or to me and I
    file a github issue.

I do plan to do this in the next two weeks, please leave this ticket open
for now.

@ekfriis https://github.com/ekfriis or anyone else: if you have time to
do it, great, just go ahead.


Reply to this email directly or view it on GitHubhttps://github.com/rootpy/rootpy/issues/106#issuecomment-13884881.

Owner

cdeil commented Feb 21, 2013

@ekfriis Sorry for that.

The Jenkins process hangs at 100% CPU.
The log at /var/log/jenkins/jenkins.log end with

Exception in thread "Executor #0 for master" Exception in thread "Jenkins cron thread"

Google suggests this might be related: https://issues.jenkins-ci.org/browse/JENKINS-12781

Do you know how I can shut down and restart Jenkins from the command line?

An extra complication is that when installing Jenkins I didn't have to set a password, so now I can't log in as the Jenkins user ... hmmm, I'm new to Jenkins and don't have time right now ... I'll try to figure it out tonight.

Alternatively, @ekfriis I think you have an admin account, so if you know how to fix this, go ahead.

Member

ekfriis commented Feb 21, 2013

How was it installed? In my (baby) Jenkins installation I just have a tmux
instance where I ran it via "java ...." and leave it running, so it is
trivial to restart. Is it setup as a service on the mac?

On Thu, Feb 21, 2013 at 1:21 PM, Christoph Deil notifications@github.comwrote:

@ekfriis https://github.com/ekfriis Sorry for that.

The Jenkins process hangs at 100% CPU.
The log at /var/log/jenkins/jenkins.log end with

Exception in thread "Executor #0 for master" Exception in thread "Jenkins cron thread"

Google suggests this might be related:
https://issues.jenkins-ci.org/browse/JENKINS-12781

Do you know how I can shut down and restart Jenkins from the command line.
An extra complication is that when installing Jenkins I didn't have to set
a password, so now I can't log in as the Jenkins user ... hmmm, I'm new to
Jenkins and don't have time right now ... I'll try to figure it out tonight.
Alternatively, @ekfriis https://github.com/ekfriis I think you have an
admin account, so if you know how to fix this, go ahead.


Reply to this email directly or view it on GitHubhttps://github.com/rootpy/rootpy/issues/106#issuecomment-13886172.

Owner

cdeil commented Feb 21, 2013

I downloaded the Jenkins Mac installer and double-clicked.
This asked me for the Mac admin password and created a new Jenkins user that ran this command:

jenkins           81   0.0  4.6  2953156 289124   ??  Ss    3Dec12 160:31.36 /usr/bin/java -jar /Applications/Jenkins/jenkins.war

I never logged in as the Jenkins user.

Does this mean it is setup as a service?

Since there's not much useful configuration there, maybe we should just start over?
As which user did you run the java -jar Jenkins.war command? Using sudo?

Member

ekfriis commented Feb 21, 2013

Not sure, I think so. When I run it on my local (UW) machine I just run it
as my normal user:

JENKINS_HOME=/scratch/efriis/jenkins java -jar jenkins.war
--argumentsRealm.passwd.bucky=XXXXX --argumentsRealm.roles.bucky=admin

where bucky is then the admin user name. I don't think this is the best
practice, but it is stable enough to me.

Evan

On Thu, Feb 21, 2013 at 1:41 PM, Christoph Deil notifications@github.comwrote:

I downloaded the Jenkins Mac installer and double-clicked.
This asked me for the Mac admin password and created a new Jenkins user
that ran this command:

jenkins 81 0.0 4.6 2953156 289124 ?? Ss 3Dec12 160:31.36 /usr/bin/java -jar /Applications/Jenkins/jenkins.war

I never logged in as the Jenkins user.

Does this mean it is setup as a service?

Since there's not much useful configuration there, maybe we should just
start over?
As which user did you run the java -jar Jenkins.war command? Using sudo?


Reply to this email directly or view it on GitHubhttps://github.com/rootpy/rootpy/issues/106#issuecomment-13886774.

Owner

cdeil commented Apr 7, 2013

@ndawe ndawe changed the title from Set up Jenkins for roopy and root_numpy Continuous Integration to Set up Jenkins for rootpy and root_numpy Continuous Integration May 26, 2015

Owner

ndawe commented May 17, 2016

Closing in favour of Travis-CI

@ndawe ndawe closed this May 17, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment