Improve flexibility of appModules discovery by allowing to map arbitr…

…ary appModule to a given executable. (#13366)

Related to #13364

Summary of the issue:
Currently when NVDA tries to load an appModule for a given program it just looks for a Python module of the same name as the program's executable. This has its limitations in particular:

Some names are incompatible with the Python's import system (for example see App module handler: handle executable names with dots as part of the file name #5323 where we resorted to replacing dots with underscores before attempting the import).
When a single apModule should be used for a multiple executable's we need as many appModules as there are executable's (this also often causes linter failures as these alias modules just star imports everything from the main module).
Even if the given module name does not contain symbols which are incompatible with importlib they may contain characters which are invalid in ordinary import statements (for one example see Fix crashes in 64-bit builds of Notepad++ 8.3 and later #13364 where the apModule is called "notepad++"). Since "+" is invalid in Python's import statement add-on developers cannot import this module from nvdaBuiltin which means that it cannot be extended in an add-on.
Description of how this pull request fixes the issue:
This PR introduces a mapping of executable names to appModules which is consulted before the given module is imported. It also adds a convenience functions for add-on developers which can be used to register or unregister a specific module for a given program. All alias apModules currently present in the source are marked as deprecated and application's for which they were loaded are mapped to the right appModule using the new map.
Since we cannot remove the old alias app modules care has been taken not to use them - they' re kept only for add-ons developers.

devDocs/developerGuide.t2t

@@ -195,8 +195,9 @@ The following few sections will talk separately about App Modules and Global Plu
 After this point, discussion is again more general.
 
 ++ Basics of an App Module ++
-App Module files have a .py extension, and are named the same as either the main executable of the application for which you wish them to be used or the package inside a host executable.
+App Module files have a .py extension, and in most cases should be named the same as either the main executable of the application for which you wish them to be used or the package inside a host executable.
 For example, an App Module for notepad would be called notepad.py, as notepad's main executable is called notepad.exe.
+If you want to use a single App Module for multiple executables, or the name of the executable conflicts with the standard Python import rules read [Associating App Modules with an executable #AssociatingAppModule].
 For apps hosted inside host executables, see the section on app modules for hosted apps.
 
 App Module files must be placed in the appModules subdirectory of an add-on, or of the scratchpad directory of the NVDA user configuration directory.
@@ -208,6 +209,30 @@ This will all be covered in depth later.
 NVDA loads an App Module for an application as soon as it notices the application is running.
 The App Module is unloaded once the application is closed or when NVDA is exiting.
 
+++ Associating App Modules with an executable ++[AssociatingAppModule]
+As explained above, sometimes the default way of associating an App Module with an application is not flexible enough. Examples include:
+- You want to use a single App Module for various binaries (perhaps both stable and preview versions of the application should have the same accessibility enhancements)
+- The executable file is named in a way which conflicts with the Python naming rules. I.e. for an application named "time", naming the App Module "time.py" would conflict with the built-in module from the standard library
+-
+
+In such cases you can distribute a small global plugin along with your App Module which maps it to the executable.
+For example to map the App Module named "time_app_mod" to the "time" executable the plugin may be written as follows:
+```
+import appModuleHandler
+import globalPluginHandler
+
+
+class GlobalPlugin(globalPluginHandler.GlobalPlugin):
+
+	def __init__(self, *args, **kwargs):
+		super().__init__(*args, **kwargs)
+		appModuleHandler.registerExecutableWithAppModule("time", "time_app_mod")
+
+	def terminate(self, *args, **kwargs):
+		super().terminate(*args, **kwargs)
+		appModuleHandler.unregisterExecutable("time")
+```
+
 ++ Example 1: An App Module that Beeps on Focus Change Events ++[Example1]
 The following example App Module makes NVDA beep each time the focus changes within the notepad application.
 This example shows you the basic layout of an App Module.

devDocs/technicalDesignOverview.md

@@ -181,6 +181,8 @@ An app module provides support specific to an application for these cases.
 An app module is derived from the `appModuleHandler.AppModule` base class.
 App modules receive events for all [NVDA objects](#nvda-objects) in the application and can bind scripts which can be executed anywhere in that application.
 They can also implement their own NVDA objects for use within the application.
+Usually the App Module should be named the same as the executable for which it should be loaded.
+In cases where this is problematic (one App Module should support multiple applications, the binary is named in a way which conflicts with the Python import system) you can add an entry to the `appModules.EXECUTABLE_NAMES_TO_APP_MODS` where the binary name is the key and the name of the App Module is the value.
 
 #### Global Plugins
 Aside from application specific customisation using [app modules](#app-modules), it is also possible to extend NVDA on a global level.

source/appModuleHandler.py

@@ -15,10 +15,15 @@
 import ctypes.wintypes
 import os
 import sys
+from types import ModuleType
 from typing import (
 	Dict,
+	List,
 	Optional,
+	Tuple,
+	Union,
 )
+import zipimport
 
 import winVersion
 import pkgutil
@@ -38,11 +43,13 @@
 import extensionPoints
 from fileUtils import getFileVersionInfo
 
+_KNOWN_IMPORTERS_T = Union[importlib.machinery.FileFinder, zipimport.zipimporter]
 # Dictionary of processID:appModule pairs used to hold the currently running modules
 runningTable: Dict[int, AppModule] = {}
 #: The process ID of NVDA itself.
 NVDAProcessID=None
-_importers=None
+_CORE_APP_MODULES_PATH: os.PathLike = appModules.__path__[0]
+_importers: Optional[List[_KNOWN_IMPORTERS_T]] = None
 _getAppModuleLock=threading.RLock()
 #: Notifies when another application is taking foreground.
 #: This allows components to react upon application switches.
@@ -51,6 +58,13 @@
 post_appSwitch = extensionPoints.Action()
 
 
+_executableNamesToAppModsAddons: Dict[str, str] = dict()
+"""AppModules registered with a given binary by add-ons are placed here.
+We cannot use l{appModules.EXECUTABLE_NAMES_TO_APP_MODS} for modules included in add-ons,
+since appModules in add-ons should take precedence over the one bundled in NVDA.
+"""
+
+
 class processEntry32W(ctypes.Structure):
 	_fields_ = [
 		("dwSize",ctypes.wintypes.DWORD),
@@ -65,14 +79,121 @@ class processEntry32W(ctypes.Structure):
 		("szExeFile", ctypes.c_wchar * 260)
 	]
 
-def getAppNameFromProcessID(processID,includeExt=False):
+
+def _warnDeprecatedAliasAppModule() -> None:
+	"""This function should be executed at the top level of an alias App Module,
+	to log a deprecation warning when the module is imported.
+	"""
+	import inspect
+	# Determine the name of the module inside which this function is executed by using introspection.
+	# Since the current frame belongs to the calling function inside `appModuleHandler`
+	# we need to retrieve the file name from the preceding frame which belongs to the module in which this
+	# function is executed.
+	currModName = os.path.splitext(os.path.basename(inspect.stack()[1].filename))[0]
+	try:
+		replacementModName = appModules.EXECUTABLE_NAMES_TO_APP_MODS[currModName]
+	except KeyError:
+		raise RuntimeError("This function can be executed only inside an alias App Module.") from None
+	else:
+		log.warning(
+			(
+				f"Importing from appModules.{currModName} is deprecated,"
+				f" you should import from appModules.{replacementModName}."
+			)
+		)
+
+
+def registerExecutableWithAppModule(executableName: str, appModName: str) -> None:
+	"""Registers appModule to be used for a given executable.
+	"""
+	_executableNamesToAppModsAddons[executableName] = appModName
+
+
+def unregisterExecutable(executableName: str) -> None:
+	"""Removes the executable of a given name from the mapping of applications to appModules.
+	"""
+	try:
+		del _executableNamesToAppModsAddons[executableName]
+	except KeyError:
+		log.error(f"Executable {executableName} was not previously registered.")
+
+
+def _getPathFromImporter(importer: _KNOWN_IMPORTERS_T) -> os.PathLike:
+	try:  # Standard `FileFinder` instance
+		return importer.path
+	except AttributeError:
+		try:  # Special case for `zipimporter`
+			return os.path.normpath(os.path.join(importer.archive, importer.prefix))
+		except AttributeError:
+			raise TypeError(f"Cannot retrieve path from {repr(importer)}") from None
+
+
+def _getPossibleAppModuleNamesForExecutable(executableName: str) -> Tuple[str, ...]:
+	"""Returns list of the appModule names for a given executable.
+	The names in the tuple are placed in order in which import of these aliases should be attempted that is:
+	- The alias registered by add-ons if any add-on registered an appModule for the executable
+	- Just the name of the executable to cover a standard appModule named the same as the executable
+	- The alias from `appModules.EXECUTABLE_NAMES_TO_APP_MODS` if it exists.
+	"""
+	return tuple(
+		aliasName for aliasName in (
+			_executableNamesToAppModsAddons.get(executableName),
+			executableName,
+			appModules.EXECUTABLE_NAMES_TO_APP_MODS.get(executableName)
+		) if aliasName is not None
+	)
+
+
+def doesAppModuleExist(name: str, ignoreDeprecatedAliases: bool = False) -> bool:
+	"""Returns c{True} if App Module with a given name exists, c{False} otherwise.
+	:param ignoreDeprecatedAliases: used for backward compatibility, so that by default alias modules
+	are not excluded.
+	"""
+	for importer in _importers:
+		modExists = importer.find_module(f"appModules.{name}")
+		if modExists:
+			# While the module has been found it is possible tis is just a deprecated alias.
+			# Before PR #13366 the only possibility to map a single app module to multiple executables
+			# was to create a alias app module and import everything from the main module into it.
+			# Now the preferred solution is to add an entry into `appModules.EXECUTABLE_NAMES_TO_APP_MODS`,
+			# but old alias modules have to stay to preserve backwards compatibility.
+			# We cannot import the alias module since they show a deprecation warning on import.
+			# To determine if the module should be imported or not we check if:
+			# - it is placed in the core appModules package, and
+			# - it has an alias defined in `appModules.EXECUTABLE_NAMES_TO_APP_MODS`.
+			# If both of these are true the module should not be imported in core.
+			if (
+				ignoreDeprecatedAliases
+				and name in appModules.EXECUTABLE_NAMES_TO_APP_MODS
+				and _getPathFromImporter(importer) == _CORE_APP_MODULES_PATH
+			):
+				continue
+			return True
+	return False  # None of the aliases exists
+
+
+def _importAppModuleForExecutable(executableName: str) -> Optional[ModuleType]:
+	"""Import and return appModule for a given executable or `None` if there is no module.
+	"""
+	for possibleModName in _getPossibleAppModuleNamesForExecutable(executableName):
+		# First, check whether the module exists.
+		# We need to do this separately to exclude alias modules,
+		# and because even though an ImportError is raised when a module can't be found,
+		# it might also be raised for other reasons.
+		if doesAppModuleExist(possibleModName, ignoreDeprecatedAliases=True):
+			return importlib.import_module(
+				f"appModules.{possibleModName}",
+				package="appModules"
+			)
+	return None  # Module not found
+
+
+def getAppNameFromProcessID(processID: int, includeExt: bool = False) -> str:
 	"""Finds out the application name of the given process.
 	@param processID: the ID of the process handle of the application you wish to get the name of.
-	@type processID: int
-	@param includeExt: C{True} to include the extension of the application's executable filename, C{False} to exclude it.
-	@type window: bool
+	@param includeExt: C{True} to include the extension of the application's executable filename,
+	C{False} to exclude it.
 	@returns: application name
-	@rtype: str
 	"""
 	if processID==NVDAProcessID:
 		return "nvda.exe" if includeExt else "nvda"
@@ -95,20 +216,20 @@ def getAppNameFromProcessID(processID,includeExt=False):
 	# This might be an executable which hosts multiple apps.
 	# Try querying the app module for the name of the app being hosted.
 	try:
-		mod = importlib.import_module("appModules.%s" % appName, package="appModules")
-		return mod.getAppNameFromHost(processID)
-	except (ImportError, AttributeError, LookupError):
+		return _importAppModuleForExecutable(appName).getAppNameFromHost(processID)
+	except (AttributeError, LookupError):
 		pass
 	return appName
 
+
 def getAppModuleForNVDAObject(obj):
 	if not isinstance(obj,NVDAObjects.NVDAObject):
 		return
 	return getAppModuleFromProcessID(obj.processID)
 
 
 def getAppModuleFromProcessID(processID: int) -> AppModule:
-	"""Finds the appModule that is for the given process ID. The module is also cached for later retreavals.
+	"""Finds the appModule that is for the given process ID. The module is also cached for later retrievals.
 	@param processID: The ID of the process for which you wish to find the appModule.
 	@returns: the appModule
 	"""
@@ -153,39 +274,36 @@ def cleanup():
 		except:
 			log.exception("Error terminating app module %r" % deadMod)
 
-def doesAppModuleExist(name):
-	return any(importer.find_module("appModules.%s" % name) for importer in _importers)
 
-def fetchAppModule(processID,appName):
+def fetchAppModule(processID: int, appName: str) -> AppModule:
 	"""Returns an appModule found in the appModules directory, for the given application name.
 	@param processID: process ID for it to be associated with
-	@type processID: integer
 	@param appName: the application name for which an appModule should be found.
-	@type appName: str
-	@returns: the appModule, or None if not found
-	@rtype: AppModule
-	"""  
-	# First, check whether the module exists.
-	# We need to do this separately because even though an ImportError is raised when a module can't be found, it might also be raised for other reasons.
+	@returns: the appModule.
+	"""
 	modName = appName
 
-	if doesAppModuleExist(modName):
-		try:
-			return importlib.import_module("appModules.%s" % modName, package="appModules").AppModule(processID, appName)
-		except:
-			log.exception(f"error in appModule {modName!r}")
-			import ui
-			import speech.priorities
-			ui.message(
-				# Translators: This is presented when errors are found in an appModule
-				# (example output: error in appModule explorer).
-				_("Error in appModule %s") % modName,
-				speechPriority=speech.priorities.Spri.NOW
-			)
+	try:
+		importedMod = _importAppModuleForExecutable(modName)
+		if importedMod is not None:
+			return importedMod.AppModule(processID, appName)
+		# Broad except since we do not know
+		# what exceptions may be thrown during import / construction of the App Module.
+	except Exception:
+		log.exception(f"error in appModule {modName!r}")
+		import ui
+		import speech.priorities
+		ui.message(
+			# Translators: This is presented when errors are found in an appModule
+			# (example output: error in appModule explorer).
+			_("Error in appModule %s") % modName,
+			speechPriority=speech.priorities.Spri.NOW
+		)
 
 	# Use the base AppModule.
 	return AppModule(processID, appName)
 
+
 def reloadAppModules():
 	"""Reloads running appModules.
 	especially, it clears the cache of running appModules and deletes them from sys.modules.
@@ -311,7 +429,11 @@ class AppModule(baseObject.ScriptableObject):
 	Each app module should be a Python module or a package in the appModules package
 	named according to the executable it supports;
 	e.g. explorer.py for the explorer.exe application or firefox/__init__.py for firefox.exe.
-	It should containa  C{AppModule} class which inherits from this base class.
+	If the name of the executable is not compatible with the Python's import system
+	i.e. contains some special characters such as "." or "+" you can name the module however you like
+	and then map the executable name to the module name
+	by adding an entry to `appModules.EXECUTABLE_NAMES_TO_APP_MODS` dictionary.
+	It should contain a C{AppModule} class which inherits from this base class.
 	App modules can implement and bind gestures to scripts.
 	These bindings will only take effect while an object in the associated application has focus.
 	See L{ScriptableObject} for details.

source/appModules/__init__.py

@@ -0,0 +1,68 @@
+# A part of NonVisual Desktop Access (NVDA)
+# Copyright (C) 2009-2022 NV Access Limited, Łukasz Golonka
+# This file may be used under the terms of the GNU General Public License, version 2 or later.
+# For more details see: https://www.gnu.org/licenses/gpl-2.0.html
+
+from typing import Dict
+
+
+EXECUTABLE_NAMES_TO_APP_MODS: Dict[str, str] = {
+	# Azure Data Studio (both stable and Insiders versions) should use module for Visual Studio Code
+	"azuredatastudio": "code",
+	"azuredatastudio-insiders": "code",
+	# Windows 11 calculator should use module for the Windows 10 one.
+	"calculatorapp": "calculator",
+	# The Insider version of Visual Studio Code should use the module for the stable version.
+	"code - insiders": "code",
+	# commsapps is an alias for the Windows 10 mail and calendar.
+	"commsapps": "hxmail",
+	# DBeaver is based on Eclipse and should use its appModule.
+	"dbeaver": "eclipse",
+	# Preview version of the Adobe Digital Editions should use the module for the stable version.
+	"digitaleditionspreview": "digitaleditions",
+	# Esybraille should use module for esysuite.
+	"esybraille": "esysuite",
+	# hxoutlook is an alias for Windows 10 mail in Creators update.
+	"hxoutlook": "hxmail",
+	# 64-bit versions of Miranda IM should use module for the 32-bit executable.
+	"miranda64": "miranda32",
+	# Various incarnations of Media Player Classic.
+	"mpc-hc": "mplayerc",
+	"mpc-hc64": "mplayerc",
+	# The binary file for Notepad++ is named `notepad++` which makes its appModule not importable
+	# (Python's import statement cannot deal with `+` in the file name).
+	# Therefore the module is named `notepadPlusPlus` and mapped to the right binary below.
+	"notepad++": "notepadPlusPlus",
+	# searchapp is an alias for searchui in Windows 10 build 18965 and later.
+	"searchapp": "searchui",
+	# Windows search in Windows 11.
+	"searchhost": "searchui",
+	# Spring Tool Suite is based on Eclipse and should use its appModule.
+	"springtoolsuite4": "eclipse",
+	"sts": "eclipse",
+	# Various versions of Teamtalk.
+	"teamtalk3": "teamtalk4classic",
+	# App module for Windows 10/11 Modern Keyboard aka new touch keyboard panel
+	# should use Composable Shell modern keyboard app module
+	"textinputhost": "windowsinternal_composableshell_experiences_textinput_inputapp",
+	# Total Commander X64 should use the module for the 32-bit version.
+	"totalcmd64": "totalcmd",
+	# The calculator on Windows Server and LTSB versions of Windows 10
+	# should use the module for the desktop calculator from the earlier Windows releases.
+	"win32calc": "calc",
+	# Windows Mail should use module for Outlook Express.
+	"winmail": "msimn",
+	# Zend Eclipse PHP Developer Tools is based on Eclipse and should use its appModule.
+	"zend-eclipse-php": "eclipse",
+	# Zend Studio is based on Eclipse and should use its appModule.
+	"zendstudio": "eclipse",
+}
+
+"""Maps names of the executables to the names of the appModule which should be loaded for the given program.
+Note that this map is used only for appModules included in NVDA
+and appModules registered by add-ons are placed in a different one.
+This mapping is needed since:
+- Names of some programs are incompatible with the Python's import system (they contain a dot or a plus)
+- Sometimes it is necessary to map one module to multiple executables - this map saves us from adding multiple
+ appModules in such cases.
+"""

source/appModules/azuredatastudio-insiders.py

@@ -9,3 +9,5 @@
 
 # Ignoring Flake8 imported but unused error since appModuleHandler yet uses the import.
 from .azuredatastudio import AppModule  # noqa: F401
+from appModuleHandler import _warnDeprecatedAliasAppModule
+_warnDeprecatedAliasAppModule()