Convert ondemand update tests from Mozmill to Marionette #765

Closed
whimboo opened this Issue Mar 10, 2016 · 11 comments

Projects

None yet

3 participants

@whimboo
Contributor
whimboo commented Mar 10, 2016

As we have noticed today our ondemand update tests are busted for Firefox 46.0b1 due to JS engine changes. That means we on longer can make use of them. See details in bug 1255566.

A transition to the new framework has to be made now. When I do that, I will keep a copy of the current Jenkins jobs, so that softvision can still run update tests for Firefox releases older than 38.0 like esr31.

@whimboo whimboo added the must-have label Mar 10, 2016
@whimboo whimboo self-assigned this Mar 10, 2016
@KaiRo-at
Contributor

I personally think we can stop testing builds older than 38.0 right now, you don't need to do extra work for that.

@whimboo
Contributor
whimboo commented Mar 10, 2016

Softvision is still running update tests for esr31.* to esr38.*. And this will still be the case for the next two major releases. If you really think that I can remove those jobs I will do it.

@KaiRo-at
Contributor

Given how long it's been, I think they should stop running those and only care about ESR 38.0 and higher. Don't do extra work to still support 31, esp. as those show intermittent errors all the time anyhow.

@whimboo
Contributor
whimboo commented Mar 10, 2016

I would only have to rename the jobs, thats all. I think that I will keep them for now. They can be removed once esr31 is deprecated.

@whimboo
Contributor
whimboo commented Mar 10, 2016

Current status is that I have kept the old mozmill-based jobs and added a suffix_mozmill to them. For marionette based update tests I can already run the ondemand_update job manually. Here a result.

Tomorrow I will have to rewrite the ondemand_trigger job which will actually create those ondemand_update jobs.

@FlorinMezei

So it sounds like we can still run updates from ESR 31... we'll still do that then until we're done with ESR 38 (which should be after the next cycle).

If it helps not supporting them, note that I have no problem dropping them for the next ESR 38.

@whimboo
Contributor
whimboo commented Mar 11, 2016

Yes, when we remove any esr31 trace from Mozmill-CI I will also delete the ondemand_update tests for Mozmill. Until then you can still use the trigger_ondemand_mozmill job.

@whimboo
Contributor
whimboo commented Mar 11, 2016

Given that we report to Treeherder I would like to not have the problem with the wrongly specified target-build-id anymore. So I would like to see a target-version entry in the config file. The trigger script will then automatically determine the build id and revision of the build per platform.

@whimboo
Contributor
whimboo commented Mar 11, 2016

Trigger script is working. Here the output: https://gist.github.com/whimboo/79ae3e1b1d7ffd641773

@whimboo whimboo added a commit to whimboo/mozmill-ci that referenced this issue Mar 11, 2016
@whimboo whimboo Convert ondemand update tests from Mozmill to Marionette (#765) 068e8b8
@whimboo whimboo added a commit to whimboo/mozmill-ci that referenced this issue Mar 11, 2016
@whimboo whimboo Convert ondemand update tests from Mozmill to Marionette (#765) ce1b000
@whimboo whimboo added a commit to whimboo/mozmill-ci that referenced this issue Mar 11, 2016
@whimboo whimboo Convert ondemand update tests from Mozmill to Marionette (#765) 2c79477
@whimboo whimboo added a commit to whimboo/mozmill-ci that referenced this issue Mar 11, 2016
@whimboo whimboo Convert ondemand update tests from Mozmill to Marionette (#765) 409a0a9
@whimboo whimboo added a commit to whimboo/mozmill-ci that referenced this issue Mar 11, 2016
@whimboo whimboo Remove unused ondemand_functional jobs (#765) f71b4f4
@whimboo
Contributor
whimboo commented Mar 11, 2016

I already pushed the PR #766 to the master branch and will make it available on staging for a complete test. If all goes well I will publish to production afterward.

@whimboo whimboo added a commit to whimboo/mozmill-ci that referenced this issue Mar 11, 2016
@whimboo whimboo Fix broken Jenkins patches for staging and production (#765) 0232d5f
@whimboo
Contributor
whimboo commented Mar 15, 2016

This is done.

@whimboo whimboo closed this Mar 15, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment