Bump Jenkins to version 1.580.3 and enable Travis container-based infrastructure

[whimboo]

The number of failures increased dramatically yesterday. Since then each Travis run is permanent red. Nothing on our side has been changed in that area. So I wonder if that could be caused by slow loading configuration pages by Jenkins and that the elements are not there when the check is made. Here both the submit-button and the main-panel showing this behavior.

https://github.com/mozilla/mozmill-ci/blob/master/test/configuration/save_config.py#L42

Maybe we can increase the timeout for driver.get() to be larger than 5s which seems to be the default.

@davehunt do you have another idea?

[davehunt]

Where do you see a default timeout of 5 seconds? There's an implicit timeout for finding elements, which is set to 10 seconds. This doesn't wait for those elements to be visible though. We could probably make this more robust by removing the implicit wait and using explicit waits. You haven't provided any examples of tracebacks though, so I'm only building on your speculation here.

[whimboo]

Then let it be 10s. I thought it uses the same implicit value as marionette. ;) Haven't actually used the clock to measure it. An example you can find here:

https://travis-ci.org/mozilla/mozmill-ci/builds/89242155#L502

[whimboo]

Oh, now I see the 10s implicit wait setter in the test file. Haven't checked that yet.

[whimboo]

I was trying a bit in PR #686 and as it looks like we only have a blank screen. The screenshot I got is wonderful white:

So I assume that we fail to start Jenkins or PhantomJS is not working correctly?

[whimboo]

Indeed, Jenkins is not getting started. There is no Jenkins is fully up and running line in that log. Due to that it would totally explain this failure.

[whimboo]

So the problem here is a very slow download of the jenkins.war. As of now we have 56kB/s, so downloading the build will take much longer than the 60s sleep we have to wait for Jenkins being up and running.

[whimboo]

So what I wanna do is:

• Use mozdownload for downloading the war file so we do not run into a timeout due to no output to stdout
• Let Jenkins run as a service and rename start.py to jenkins.py or similar and let it start/stop the service
• Check if using the container based infrastructure of Travis would allow us to cache the downloaded jar file

[davehunt]

Caching the war file is a good idea, and easy to do with Travis.

[davehunt]

I wonder how many people use a CI system to download and start another CI system. 😀

[whimboo]

Interestingly this problem seems to only happen for the stable version 1.580.1 but not 1.580.3. Maybe its a good time to upgrade to the latest patch of that LTS.

[whimboo]

Without caching the duration of a testrun takes about 3:45min. With caching enabled and having the war file + pip in cache it takes only 2:40. Lets see if I can get rid of the extra 60s sleep by polling for Jenkins being up and running.

[whimboo]

I think getting rid of the 60s sleep is better covered once the run_tests.sh script has been converted to a Python script.

[whimboo]

PR #686 has been merged. So this will go live in the next days.

[whimboo]

This landed already on production some days ago and works fine.