automatically generate update manifest for self-hosted updates

[Azeirah]

It would be extremely helpful to have a command to generate an updateManifest.json within this web-ext tool, just like how jpm is able to do it. Right now, I'm using a custom script, but I feel it should be part of web-ext.

Is there any reason why web-ext doesn't automatically generate an updateManifest.json file during the sign task like jpm does it?

[kumar303]

Right now, I'm using a custom script

Why do you need a script for this? Is it helpful because it pre-populates the extension ID? I'd be curious to look at this because I can't think of anything else that is possible to generate automatically.

Just to provide more context for other readers, updates for WebExtensions are different from SDK based add-ons (made with jpm). This article explains how updates work. The first step is to add an update_url to the WebExtension manifest.

[Azeirah]

Why do you need a script for this?

I'm using a custom script because I want to automate the process from git push, on my webextension to automatically deploy the new version on my website if the whole pipeline -- test, build.. and then deploy -- is passed.

I am still working on this. The custom script is indeed based on the article you mentioned.

Am I wrong in assuming that it's possible to automate this process?

1. Add update_url to manifest.json
2. Create initial updateManifest.json file, and upload it to my webserver
3. Whenever I git push, check if version in manifest.json has changed
4. If so, fetch the old updateManifest.json
5. Add the details of the new version to the new updateManifest.json (which should be handled by web-ext, not by my custom script, in my opinion)
6. Upload the new updateManifest.json to my web server

All variables, such as the extension ID, web server URL, will be defined within the pipeline itself.

To clarify, the script takes three variables as input; the old updateManifest.json, the extension ID, and the URL where the newly generated XPI will be placed.

[kumar303]

Ah, thanks for the details. I understand this better.

Yes, I think the build command should produce updateManifest.json if the manifest includes update_url.

I don't think web-ext should try to push updateManifest.json to a remote server though. The details for how to do that are going to vary from server to server so it seems out of scope.

[kumar303]

Note that the sign command calls build automatically

[kumar303]

It may be easier to start this as a standalone command like web-ext update-info (or something)

[Azeirah]

I don't think web-ext should try to push updateManifest.json to a remote server though.

Oh no I completely agree, that'd be out-of-scope.

It may be easier to start this as a standalone command like web-ext update-info (or something)

I agree, because it differs from cfx's and jpm's commands in that it needs the old updateManifest.json file to work. Otherwise, I'd say include it in the build step.

[kumar303]

it needs the old updateManifest.json file to work

This should be straight forward because we can decide on a standard file name like web-ext-update-manifest.json or something. It could be overidden by an option too. I imagine developers would commit this file to their repo. The command could start by looking for the file and appending versions to it if it exists.

[Azeirah]

I imagine developers would commit this file to their repo

Not sure, if I understand the update page correctly, you can use a single web-ext-update-manifest.json file for multiple addons. It'd get complicated if you wanted to use this single file in multiple repositories, where the easiest solution would be a git submodule with only this file inside of it.

An object containing one entry for each add-on to be updated. For each such entry, the name of the property must be the add-on's ID, and the value must be an object describing the add-on and its updates.

I think it's a better idea to just let the user decide where to get this file from, be it that it's stored inside the repo, or downloaded from some webserver. And just pass the path of this file to the command

web-ext generateUpdateManifest [extensionID] [updatedXPIURL] [oldUpdateManifest.json]

[kumar303]

you can use a single web-ext-update-manifest.json file for multiple addons

Crap, so you can. That makes it a bit more complicated. I suppose the command will need to take an option then, like:

web-ext update-manifest --existing-manifest ~/my-shared-update-manifest.json

[Azeirah]

I don't think web-ext should try to push updateManifest.json to a remote server though.

Now I understand where you were coming from. The update_link property in the updateManifest should point to the newly generated XPI file, right?

{
    "addons": {
        "someAddonExtensionId": {
            "updates": [
                {
                    "version": "1.1",
                    "update_link": "https://example.com/plugins/plugin-version-1.1.xpi"
                }
            ]
        }
    }
}

The https://example.com/plugins/ needs to be filled in, no? That's what I meant with updatedXPIURL and URL where the newly generated XPI will be placed. in my earlier comments.

This shouldn't handle uploading, only fill in the url in the updateManifest.

[kumar303]

Yes, so, we would need to specify the update_link URL on the command line in some way like:

web-ext ... --update-link="https://example.com/plugins/${xpiFileName}"

I thought of something else. The script could parse applications.gecko.update_url from the manifest and if it's a valid URL, it could pull the existing update manifest JSON from that URL and append new versions to that.

When I think about this more, it might make sense to only bake this into the sign command since that's the only thing that produces a signed XPI for self-distribution anyway.

[Azeirah]

I thought of something else. The script could parse applications.gecko.update_url from the manifest and if it's a valid URL, it could pull the existing update manifest JSON from that URL and append new versions to that.

Hadn't thought of that. What would the script do if the manifest doesn't have the applications.gecko.update_url property? Throw an error? Or should it tell the user to provide the optional parameter --existing-manifest?