Concise and static Python imports

[Helveg]

Removed all forms of dynamic module importing and from X import *. If the need for dynamic modules arises again they can be defined as a lazy loaded module such as spatial with some extra indications for the user when missing optional modules are accessed:

>>> nest.optional.feature()
ImportError: This functionality requires the optional `optional` module. See <nest-docs> to install it.

PR summary

• Replaced ast/compile based dynamic importing by importlib.
  • ast/compile sidesteps import machinery.
  • Added mechanism for optional modules _lazy_module_property("feature", optional=True, optional_hint="Please install feature.")
• Removed hl_api
• Replaced from X import * calls with explicit imports.
• Simplified test imports/discovery, removed gratuitous test helper file.

[Helveg]

@jougs I hope you don't mind I went ahead and removed hl_api.py prematurely, the code in it would only confuse developers. I checked the codebase and it wasn't used anywhere except for a couple of times as nest.hl_api.serializable, I think also #2157 changes those to just nest.serializable.

The change could have affected users that do funky things with deep imports, but I just added nest as nest.hl_api so they can safely continue importing from there until we clean that up in 4.0. I'm actually all for removing it altogether because I doubt many people actually use these deep imports.

[jougs]

No worries! I'm always in favor of cleaning things up and not adding backwards-compatibility-cruft.

As this PR won't make it into 3.1, I suggest, you create another small PR, adding a sentence to the release notes that the hl_api namespace will go away with 3.2.

[Helveg]

(side note on #2157 to @jougs and in general also for @heplesser and others related to the modules)

All of our problems with the reference to the old nest module seem to be disguised circular import errors that due to a bug in Python makes it pick up the unreplaced module object that it compiled and executed in order to even get to the point in __init__.py where we try to replace the sys.modules object, but then because of circular imports it silently doesn't finish making that object and the old one probably dangles around somewhere..

I think the codebase should be thoroughly vetted for code that gets executed at import time and also the nest submodules should not import nest itself at import time, but the sibling modules that actually provide the logic they're looking for. i.e. from .spatial import thing_in_spatial not from . import thing_in_spatial. I haven't checked how often that occurs but it relies on the fact that everything is shortcut unto nest on import. This is only done for user convenience and developers should never use these shortcuts internally as they cause circular imports when nest is loading.

[jougs]

Other than my few comments (that were likely triggered by my ignorance about how pytest works) I'm fine with this. Thanks for the nice contribution :-)

[jougs]

Thanks for the nice contribution and the fruitful discussion.

[heplesser]

@Helveg Looks generally good to me, just two small suggestions.