Ability to automatically update add-ons

[nvaccessAuto]

Original report

Reported by nvdakor on 2013-05-08 07:08
Hi,
With the launch of the official NvDA add-ons page at addons.nvda-project.org, it became easier to search add-ons from one central location. Although this helps with downloading add-ons for the first time, a user who is running an add-on may find that he or she is running an older version of an add-on. So just like automatic NVDA update feature, there could be a feature where newer versions of add-ons would be downloaded automatically and prompt the user to install it without having to visit the addons site. Thanks.

Remaining work

update channel selection

• Ability to decide the default update channel for all add-ons, and the preferred update channel on a per add-on basis.
• Per-addon update channel selection is done from within the add-on store update tab using a submenu of the actions menu
  • "update channel" submenu items: Default, same channel, any, do not update, stable, beta&dev, beta, dev
• Anything but same, which is the current default, should allow auto updating from external add-ons

Add-on store panel settings

• Combo-box to change the default update channel for all add-ons.
  • Combo-box items: same (default), any, do not update, stable, beta&dev, beta, dev
• Checkbox to include incompatible add-ons when checking for update
• Convert the update system / combobox so users can decide between:
  • notify on updates (default)
  • fully automatic update
  • disabled

[nvaccessAuto]

Comment 1 by blindbhavya on 2014-09-07 08:27
This proposal would be very useful if implemented.
Could you (the devs James Sir) share your thoughts on this?

[nvaccessAuto]

Comment 3 by bhavyashah on 2015-01-01 07:44
Hi,
Please add #4762 to 'blocking'.

[nvaccessAuto]

Comment 5 by fastfinge on 2015-02-19 16:17
As far as I know, we haven't yet had security issues with an addon. But I'm wondering what the security implications of people running out-of-date addons are? Especially as NVDA needs administrator access. This is another reason why, personally, I think having the ability to check for addon updates from within NVDA is kind of important. Even if the addon itself could specify a URL to check for updates, rather than forcing everything through the centralized NVDA addon website, that might be enough to get people updating old addons when, not if, a serious security hole in an addon is discovered. In it's simplest form, it could be a file with a version number, download URL, and file hash. I don't think addons are currently signed via any kind of private key system at the moment, and I'm not sure they urgently need to be. Maybe this isn't a big deal, though, if addons are sandboxed in some way I don't know about?

[nvaccessAuto]

Comment 6 by jteh on 2015-02-20 01:17
First and most important, NVDA does not require administrator access. You should absolutely not be running NVDA with administrator privileges. You also should not disable UAC.

For what it's worth, no one is disputing that add-on updates would be nice. It's just a matter of someone having the time to do it, including integration with the add-ons repository.

[nvaccessAuto]

Comment 7 by nvdakor on 2015-04-26 05:54
Hi,
After some investigation, it appears it might be doable. The biggest challenge is the mechanism to integrate with the add-ons repository. Recording the URL and then downloading and instaling the add-on is easy (we can borrow ideas from update check's UpdateDownloader class); the issue is how to differentiate between current and the newer add-on (hashing may work).

The big picture could be the following:

1. Add an update button in the add-ons manager if the add-on manifest specifies an update URL. This could be either the repurposed URL field from the current manifest format, or it could be a new field just for this purpose.
2. When the update button is clicked, a routine similar to update check routine from NVDA core would run, attempting to compare hashes and version and so on, which will result in the update UI similar to that of NVDA core.
3. Once the update is downloaded, the add-on installation UI should come up, asking users if they wish to install the new add-on.

Issues to be resolved:

• How can we calculate the first hash for our add-ons?
• How should the web server which hosts add-ons website communicate with NvDA core as to comparing hashes, add-on version and so on?

I think anyone can implement the client side, and Mesar knows more about the server side unless if Jamie has access to that server. We also need to hear from add-on authors so we can prepare our add-ons to take advantage of this idea in the future.
Thanks.

[nvaccessAuto]

Comment 8 by jteh (in reply to comment 7) on 2015-04-27 03:50
Replying to nvdakor:

Recording the URL and then downloading and instaling the add-on is easy (we can borrow ideas from update check's UpdateDownloader class);

Ideally, that code should be refactored so that it can be reused rather than duplicating it. I started work on this a while ago (#3504), but never got around to finishing it.

the issue is how to differentiate between current and the newer add-on (hashing may work).

Keep it simple. Just compare the old version with the new version. If it's different, consider it an update. This is all we do for NVDA updates. That doesn't account for someone running a versio that is newer than is available on the site, but if that happens, the user clearly is a developer and/or is working with one.

1. Add an update button in the add-ons manager

We should also consider automatic updates.

if the add-on manifest specifies an update URL.

I'm leaning towards restricting this functionality to the official add-ons repository, which means we don't need a full update URL, just some key indicating that this can do updates.

• How can we calculate the first hash for our add-ons?

See above. We don't need hashes for version identification. We do need hashes for verifying file integrity, but those can be generated automatically.

Also, I don't think this needs to magically start working for existing add-ons. I think it's acceptable to require that users download a new version of the add-on initially to have it start updating.

• How should the web server which hosts add-ons website communicate with NvDA core as to comparing hashes, add-on version and so on?

We could do something similar to the current NVDA update check server script. The bigger issue is actually having a database on the server of add-ons and versions. Currently, as I understand it, version information is only listed in the Markdown for the website, which is not something we can reliably use.

I think anyone can implement the client side, and Mesar knows more about the server side unless if Jamie has access to that server.

The entire translation and add-ons system is now all running on the NV Access server. Mesar is pretty busy with other things these days and I haven't seen much from him lately, so I'm now handling everything related to this (unless Mesar wants to step in). I have a pretty good understanding of some parts of the system, though am still learning my way around others.

[nvaccessAuto]

Comment 9 by nvdakor (in reply to comment 8) on 2015-04-27 03:59
Replying to jteh:

Replying to nvdakor:

Recording the URL and then downloading and instaling the add-on is easy (we can borrow ideas from update check's UpdateDownloader class);

Ideally, that code should be refactored so that it can be reused rather than duplicating it. I started work on this a while ago (#3504), but never got around to finishing it.

If you've started on this, then I think we should wait until #3504 is in place before returning to this ticket. Besides, update downloader and installer GUI has issues - for instance, the GUI for update downloader doesn't close automatically once an update has been downloaded and multiple update progress GUI may appear after a failed update is attempted.

1. Add an update button in the add-ons manager

We should also consider automatic updates.

if the add-on manifest specifies an update URL.

I'm leaning towards restricting this functionality to the official add-ons repository, which means we don't need a full update URL, just some key indicating that this can do updates.

Understood.

Also, I don't think this needs to magically start working for existing add-ons. I think it's acceptable to require that users download a new version of the add-on initially to have it start updating.

Sure. I'd say we the authors need at least three months time to adjust to the new routines once this ticket has been implemented; whatever major version of the add-on we were working on, that version or the version after that can implement needed changes, similar to what happened last fall when we started implementing add-on help functionality.
Thanks.

[nvaccessAuto]

Comment 10 by nvdakor on 2015-10-02 01:40
Hi,
Coming back to this ticket...
Looking at urllib and NvDA Core's update routines ghave me an idea as to how to implement client side, but for version 1, it'll require user intervention.
The proposal is as follows:

1. Add-ons should specify the download URL (perhaps a new key). This can be done easily, as many add-ons come with readme files written in Markdown that specifies URL's for stable and development versions.
2. When update button is clicked, NVDA will gather the add-on manifest for the selected add-on, which will return add-on version, URL and so on. Perhaps a call to locate the add-on installer size should be specified for dev versions.
3. NVDA will open the URL via urllib.urlopen, parse the URL and look at content length header. Thankfully, because of the add-on file naming convention in use, it is easy to parse the address looking for the version information (a couple calls to strip function).
4. If the version on the server is newer than the installed add-on, assume that a newer version is available and present the update dialog (borrow NVDA Core's update presentation dialog, rewording parts of it to refer to add-ons).
5. If a development version is being updated, check the content length against the size of the add-on installer for the currently installed add-on (initially the client side is 0), and if they are different, assume that a new version is available and act accordingly.
6. For steps 4 or 5, if an update is downloaded, save the content length for future update lookup.
7. Until Split out downloader used by updateCheck so it can be used elsewhere #3504 is implemented, NVDA will call os.startfile.
   I'd be happy to write a proof of concept add-on that demonstrates this algorithm.
   Thanks.

[nvaccessAuto]

Comment 11 by jteh (in reply to comment 10) on 2015-10-02 08:29
Replying to nvdakor:

1. Add-ons should specify the download URL (perhaps a new key).

I'm still thinking we should restrict this to the official add-on repository. Also, there are actually two URLs required in your proposed system: the URL for the manifest and the URL for the add-on itself.

1. When update button is clicked, NVDA will gather the add-on manifest for the selected add-on, which will return add-on version, URL and so on.

As noted in comment:8, I think this is better done with a server script similar to the current update check server. This centralises things and allows for tracking of statistics (though that's of course not required for the first round).

1. If a development version is being updated, check the content length against the size of the add-on installer for the currently installed add-on (initially the client side is 0), and if they are different, assume that a new version is available and act accordingly.

This is going to be fragile. Honestly, I think dev versions should just increment the version number (like NVDA snapshots do) if users really are supposed to update.

Thanks.