Add-on store #13985
Initial design for addon store. - A new dialog ("add-on store") can be launched from the 'tools' menu. - Data is pulled from the live Add-on API and shown in the Dialog. - The dialog presents a filterable sortable list with basic add-on data. - An add-on can be selected and its description / further details are presented. - When filtering, care is taken to preserve the selection. Known issues: - The current version of NVDA (alpha 2022.2) is not known to the add-on store yet. - Fetching add-on listings for it produces a 400 error. This could be resolved with Allow 'dev' version of add-on addon-datastore#40 - The first time the dialog is opened, there is a noticeable delay, the dialog creation is blocked by downloading the add-on data. This is then cached in memory, there is no way to refresh the data, NVDA must be restarted. Will be resolved in a future commit. - The list currently shows all add-ons. There is no way to select between stable / beta / all. Planned for resolution. - Due to inheriting from SettingsDialog (for other functionality, such as single instance etc) there are ok / cancel buttons. A single button "done" may be more appropriate.
- In order to be able to test the NVDA add-on store implementation locally, developers must be able to provide test data for the store. - The add-on store is fetching data from the server every time it is opened, this may overload the server. - There was a delay when opening the add-on store Description of change: Introduce a (json) file cache, which includes: - the string (data) returned by the API - the datetime when the data was fetched (stored as an ISO date time string) With this cache file: - Testing can be achieved by overriding the cached data. - Data is cached every 6 hours. - The cache can be used (almost) instantly. Testing strategy: - Ran NVDA without an internet connection. Led to ConnectionError. - Ran NVDA with a version that returns no data. Cache is used instead. - Ensured that when the cache has a time stamp less than 6 hour old there is no delay to use it. - Ensured that the cache can be modified locally, the change is reflected in the GUI. - When the cache is older than 6 hours, there is a short delay before data is populated into the GUI. Known issues: - The cache does not include the NVDA version that was used when fetching. This may mean that an invalid cache could be used for up to 6 hours after updating NVDA. - When the cache is older than 6 hours, there is a short delay before data is populated into the GUI.
- The add-on store needs a way to allow installation. - The actions should happen in the background, steps like downloading may take significant time. - Progress should be discoverable. Description of change: - Introduces a way to associate actions with the GUI. - To cater to power users a context menu has been added to list items (available add-ons) - To ensure discoverability, buttons for actions are added to the details. Note There is currently an "update" action, which only logs. It is included just as a demonstration / proof of concept. There are many rough edges in this implementation, but since this is already a sizable change, and many further changes are more likely to self contained it is a good checkpoint. Testing strategy: - Used the links in the add-on store data to download several *.nvda-addon files - The manifests were modified (in various ways) to create different installation outcomes. - The python "simpleHttpServer" module was used to srt. - The links to the URL for add-ons in the cached addon-store file were pointed to localhost. - Used 'Clumsy' tool to simulate slow connections, adding latency to the download process. Known issues: These will be worked on separately: - The status of installed add-ons isn't reflected in the list, installed add-ons with a matching version should be omitte". There is a problem here, an add-on manifest does not include a numerical version, only a version name string. Ordering based on version number with the add-on store will not be possible for side-loaded add-ons. - Pressing enter on an addon should open the context menu. - Pressing enter on a link (in the details) should open the URL - When closing the dialog (after inste blocked by add-ons being installed (consider NVDA exit / os shutdown), and progress shown. - Task management required for download / install steps. Install steps for multiple add-ons should be serialized. Download can happen in parallel. - Ok / Cancel should be eplaced wih a 'Done' button - Add-ons optionally have an install_task which runs post installation, prior to restart. This can be disruptive when installing multiple add-ons from the store. Consider making this occur post restart.
- When an addon was already installed (matching ID and version) it should not have been listed as available (to download) - When the addon was installed, but there is a newer version available, it should not have been listed as "update available". - When the addon isn't compatible, the status column should say so. This shouldn't occur if the add-on submission process works correctly, however, to catch bugs with that system it is worth showing. - Adds typing and deeper docs to the addonHandler and addonVersioning. This needs to be understood for the version / name / id comparisons with addonStore data. - Set status of add-ons to "installed" / "enabled" / "disabled" / "update available" / "incompatible" - Don't show add-ons that are "enabled" (I.E. Installed and running). - Don't show add-ons that are "disabled" - Check for compatibility, show incompatible as status. - Used the links in the add-on store data to download several *.nvda-addon files - The manifests were modified (in various ways) to create different installation outcomes. - The python "simpleHttpServer" module was used to serve the add-ons. - The links to the URL for add-ons in the cached addon-store file were pointed to localhost. - Used 'Clumsy' tool to simulate slow connections, adding latency to the download process. - Modified the cached addon-store file to test different add-on versions being available (to test updates available) - Installed an add-on to ensure that they aren't listed when "running". - Disabled add-on to ensure that they aren't listed when "disabled". - Modified the addon-store file to test showing addon compatibility in status. - Does not properly handle add-on updates (from the store). The numerical version information from the add-on needs to be stored somewhere. Information for side loaded add-ons will be shown as "update available" if the version doesn't match, even if the add-on store version is older.
Installing add-ons must happen on the main thread. They must be installed one by one. Downloads may happen concurrently, moving off the main thread allows NVDA to stay responsive. Description of change: Use a ThreadPoolExecutor to enable downloading in the background. This will allow scaling up the number of threads as required. Provided an abstraction to make cancellation/tracking of complete simpler. Addons are installed (if download has completed) when the user tries to exit the dialog.
It's a very good news to see this topic arrive in NVDA's repo!
The PR is not in draft. Is it already testable?
I get the following error:
@CyrilleB79 theoretically it should be testable.
However, I had previously been testing it before the NVDA version change, and with a prior datastore cache this error doesn't cause a major issue. I'll look into improving the robustness in this case.
There is a known issue to allow support for "in development" NVDA versions on the API side.
For testing purposes, you can safely override
def _getCurrentApiVersionForURL() -> str: #currentVersion = addonAPIVersion.CURRENT currentVersion = (2022, 2, 0)
Thanks for the feedback.
Our next phase will be looking at the updating use-cases more carefully. Hopefully this will address these concerns.
Can you get me a log or some information about the dialog? I haven't noticed this, and don't know what it could be.
Note you can access "actions" for the selected add-on via the application/context menu. I intend to open this menu with the enter key is press while an add-on is selected. This is still a todo item. When we are confident about the full list of actions we can also add in accelerator keys.
Is the concern about how long it takes to get back to the list, perhaps an accelerator key (for the list) would be sufficient?
Yes, this is a concern. It isn't always obvious which add-on is presenting a dialog. We intent to move the install tasks running to after NVDA is restarted, which will make this worse. This might resolve itself through add-on updates. Let's give it some time, if the problem still exists when we get to that stage we can look into providing more information about which add-on install is completing.
I see, STR:
I think It's debatable where focus should be shifted. Given this is the standard Windows behavior, I think there should be a pretty strong rational for changing it. Note that when there are other buttons in this group, focus will shift to one of those buttons.
At this stage, this is a low priority usability concern but may be worth considering further later.
@feerrenrut From the discussion on the PR it seems you're planning to make the install tasks execute after NVDA is restarted, rather than, as it is now, during the add-on install. This is going to make most usages of install tasks pretty difficult and make them much less useful. I don't want to go into specific examples here to avoid getting off topic, and also because you gave no reasoning for the change, so it is hard to discuss pros and cons. Could you create a separate issue / discussion outlining what, exactly, you intend to modify and why, so that add-on and core developers can be informed in advance and check if this is going to meet their needs?