Test Pilot Stage and Dev installed add-ons are older than Production add-on based on the version number

[SoftVision-CosminMuntean]

[Notes]:

• All the versions seems to be messed up because production add-on seems to be the newest (2.0.2) while the stage one seems to be the oldest (1.1.4) and the dev server is in between (2.0.0).

[Affected versions]:

• Firefox 55.0.3

[Affected Platforms]:

• All Windows
• All Mac
• All Linux

[Prerequisites]:

• Create 3 new Firefox profiles, one for each of the Test Pilot servers.
• Add the "testpilot.env" string type pref in "about:config" page on all 3 profiles with the server name value (dev/stage/production).
• Add the "extensions.webapi.testing" boolean type pref in "about:config" page with "true" value on all 3 profiles.
• Install Test Pilot corresponding server add-on on all 3 profiles

[Steps to reproduce]:

1. Start all 3 above created Firefox profiles in 3 separate instances.
2. Navigate to "about:addons" page on all 3.
3. Observe the Test Pilot version on each profile.

[Expected result]:

• Usually the dev server has the highest version while production has the lowest and stage is in between.

[Actual results]:

• Test Pilot stage website installs an older add-on version (v1.1.4) than Dev (2.0.0) while Production version is higher than both of them (2.0.2).

[Additional Notes]:

• Here is a screen recording with the issue: https://goo.gl/BX8Adz

[lmorchard]

I can reproduce this and will look into it today

[lmorchard]

Possibly a lead: b7d0e75#diff-01dd223246b3d82a911250aceaac85b5L95

We stopped uploading *.xpi and *.rdf (the add-on files) since we moved to Mozilla Extensions signing on production. This probably needs to be re-enabled for dev & stage

Also, tried to switch dev & stage to use local unsigned build of addon (which we're no longer uploading): e60e84c

[SoftVision-CosminMuntean]

After the latest changes that landed, I have retest this issue using all the 5 prefs and it seems that the behavior is changed. But I am not sure if it's the correct one:

Test Pilot dev server:
The add-on is successfully installed and has the 2.0.4 version.

Test Pilot stage server:
The add-on is installed but still has the 1.1.4 version. However, if you navigate to "about:addons" and manually check for updates, Test Pilot add-on is being updated . But the update is only to 2.0.2 version which is currently on production, and not to the dev version 2.0.4

I'm not sure if it's ok to wait for an auto-update when you install TxP from scratch.

Test Pilot production server:
The add-on is successfully installed and has the 2.0.2 version, just like the one from stage server.

I'm not sure if this will affect the production deploy, but like I mentioned in the issue, usually the production version is lower then stage and dev versions. Now on stage and production we have the same version (2.0.2).

[lmorchard]

Test Pilot dev server: The add-on is successfully installed and has the 2.0.4 version.

👍

Test Pilot stage server: The add-on is installed but still has the 1.1.4 version.

There was a change to the staging deployment process merged into ops' configuration scripts. But, I'm not sure whether that itself deploys automatically, or if we have to wait for an ops deploy cycle. (/cc @relud ?)

However, if you navigate to "about:addons" and manually check for updates, Test Pilot add-on is being updated . But the update is only to 2.0.2 version which is currently on production, and not to the dev version 2.0.4

For what it's worth, updates are always served up from production. This is not affected by the location from which you installed the add-on. The production update URL is baked into the add-on and can't really vary between environments.

Test Pilot production server: The add-on is successfully installed and has the 2.0.2 version, just like the one from stage server.

This is as expected, because we haven't pushed to production yet.

[lmorchard]

I can confirm that stage still has the old version of the add-on though 😞

$ wget https://testpilot.stage.mozaws.net/static/addon/addon.xpi
$ unzip addon.xpi
$ grep em:version install.rdf
          <em:version>1.1.4-dev-22b26fa</em:version>

[relud]

deploys made after a merge pick up changes to ops' deploy scripts, so the deploy yesterday passed the stage argument to deploy.sh

[lmorchard]

We just did some troubleshooting and I think we've got this resolved on stage!

$ wget https://testpilot.stage.mozaws.net/static/addon/addon.xpi
$ unzip addon.xpi
$ grep em:version install.rdf
    <em:version>2.0.4-dev-05dbd9e</em:version>