FNPRMS is missing results 4/9-4/12

[mcomella]

The results are NA for these days (and hasn't yet run on the 13th).

---

After a brief investigation, it looks like we're unable to download the build:

Downloading apk.
Done downloading apk.
Trying again to download the nightly apk (error 22).
...

And the reason is because the fennec-nightly builds don't exist for those days: https://firefox-ci-tc.services.mozilla.com/tasks/index/project.mobile.fenix.v2.fennec-nightly.2020.04

Since the nightly builds are also missing, perhaps it's because there was a compilation error on master that prevented these builds from running.

[mcomella]

After reading the #fenix-team slack backscroll from late last week, I discovered Google Play is backlogged on our Nightly builds so we disabled the builds. This impacts performance testing because we can't measure the changes making it in every night. I suggested that we continue building Nightly but disable the upload to GP instead so we can resume performance testing.

Waiting on a response from the Fenix team.

[liuche]

@JohanLorenzo from the GPS email thread, it looks like one of the options is re-enabling the Nightly builds but publishing them to archive.mo without pushing to the Play Store. Is there a bug to track that work? That option helps solve some of our QA/perf problems in the short-term - and in the meantime, Relman is going to see how long it actually takes a Nightly build to get approved, but we want to continue building our Nightlies.

[mcomella]

I found https://bugzilla.mozilla.org/show_bug.cgi?id=1628413 as an upstream issue.

[JohanLorenzo]

Thanks for raising this issue! I'm going to make the right changes in bug 1628413

[JohanLorenzo]

Good news: fenix nightlies are back - on their regular schedule, they're just not shipped to Google Play anymore.

(I just landed a way to manually send them to GP. This way, we have control over what gets sent to GP).

Please let me know if you see any issue!

[mcomella]

I confirmed that our perf test results are back and it seems there haven't been any regressions (though it's within noise limits so we won't know for a few days). Thanks a bunch JohanLorenzo! Closing as there are no more actions to do here.

[bdekoz]

I see apk's again on 2020-04-14, but none for 10, 11, 12, 13? Will these be back-filled?

…

On Wed, Apr 15, 2020 at 1:31 PM Michael Comella ***@***.***> wrote: I confirmed that our perf test results are back and it seems there haven't been any regressions (though it's within noise limits so we won't know for a few days). Thanks a bunch JohanLorenzo! Closing as there are no more actions to do here. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#100 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAHTQP6HJNTC2CMZVF222HTRMYKQZANCNFSM4MHJDSOA> .

[mcomella]

As we discussed during sync today, we don't think it's necessary to backfill because we haven't seen a regression.

[mcomella]

As we discussed during sync today,

Sorry, I hope this didn't come off passive aggressively :) – I wanted to reference the discussion we had in sync for extra context to my response.