Feat entrypoints

[althonos]

Hi ! This PR attempts to switch the fs.opener micro-API to the setuptools entry points mecanism.

Rationale

By the words of Kenneth Reitz, 180° turns are encouraged. While I was actively developing fs.sshfs and fs.proxy lately, I learned a lot about setuptools and Python packages distribution in general (since instead of using a plain setup.py as I used to, I decided to switch to a purely declarative setup using the setup.cfg file). In particular, that:

1. setuptools supported native namespace packages through a single configuration value in setup.py
2. pkg_resources is bundled with setuptools, and since your requirements include setuptools, we can safely rely on it instead of pkgutil which is not as powerful.
3. EntryPoints are cool, and I put more confidence in them than in the hackish loader I implemented.

Proposed edits

Unfortunately, for ＃1, it's useless in our case since native namespace packages cannot have an __init__.py, meaning their namespace cannot be overloaded (goodbye, fs.open_fs) !

For ＃2, I replaced pkgutil namespace definitions with pkg_resources one. Not a lot changed though, but since the pkg_resources is more recent, I put more faith in it.

But for ＃3, there were actual things to enhance: openers. I edited the Registry to load openers dynamically using pkg_resources to load entry points of the fs.opener group. This triggers two major changes with extensions:

• the registry does not need to register filesystems anymore
• external packages can define entry points without putting anything in the fs or fs.opener namespace
• opener code is only imported when needed (this makes the filesystem / opener separation quite obsolete, since the opener is only loaded when needed, so a class could derive both from FS and Opener and still only be imported when the open_fs requires it to).

Attaching an opener class to an protocol is now done declaratively in the setup.py file:

setup(
    ...
    entry_points={'fs.opener': [
        'awe = fs.opener.awesomefs:AwesomeOpener',
        'cool = fs.opener.coolfs:CoolOpener',
    ]}
    ...
)

or even in a setup.cfg file:

[options.entry_points]
fs.opener =
	awe = fs.opener.awesomefs:AwesomeOpener
	cool = fs.opener.coolfs:CoolOpener

and then entry points are loaded at runtime:

import fs
with fs.open_fs('awe://...') as awe_fs:
   # ... #

Perks

• Openers from anywhere: opener now do not need to be implemented in the fs.opener module, which in turns mean libraries do not require to enter the fs namespace at all to declare an opener !
• Even lazier loading: fs.open_fs was able to load filesystem submodules only when needed to, but the fs.opener package was exhaustively imported, causing some overhead. With this declarative style, only fs.opener.{base, errors, registry} are imported, and even the opener declaration is imported on-the-go !
• Registry not really needed anymore: Since nothing has to be stored internally by the registry, it could be rewritten just as a module with plain functions
• Support for zipped extensions: opener can be loaded even from egg files (although, who uses eggs ?)

Drawbacks

• Loading order: I have no idea in what order the entry points will be loaded if two extensions declare the same opener protocol. Previously, it was the last loaded, so potentially the last module in lexicographic order installed in fs.opener.

[coveralls]

Coverage decreased (-0.8%) to 99.205% when pulling 59b12f1 on althonos:feat-entrypoints into 44260e7 on PyFilesystem:master.

[coveralls]

Coverage decreased (-0.8%) to 99.204% when pulling 9882545 on althonos:feat-entrypoints into 44260e7 on PyFilesystem:master.

[coveralls]

Coverage remained the same at 100.0% when pulling 632f9b2 on althonos:feat-entrypoints into 44260e7 on PyFilesystem:master.

[wgaylord]

This sounds cool!

[willmcgugan]

Hi @althonos

I'm not entirely familiar with the magic that is going on here. The previous opener mechanism had the benefit that it was quite transparent . But it does sound like letting setuptools do the work is a good idea, and the lazy loading is nice. So 👍 for the PR.

Would you mind updating the docs you wrote?

Will

[althonos]

I edited the Opener section in the doc to include information about entry points, as well as some links to useful resources, and a boilerplate setup.py.

[coveralls]

Coverage remained the same at 100.0% when pulling e036d5c on althonos:feat-entrypoints into 44260e7 on PyFilesystem:master.

[willmcgugan]

Sorry for the delay. I think I grok entry points now. Is it good to merge?

[althonos]

Wonderful. Sorry I didn't answer earlier. I'll submit a PR to update the documentation with my FS extensions once you create a new release and I make sure everything works fine.

[willmcgugan]

Did you get the email I sent you earlier? I sent you some ideas re namespace packages and openers...