DeprecationWarning: pkg_resources is deprecated as an API

[simonw]

Got this running tests against Python 3.11.

../../../.local/share/virtualenvs/datasette-big-local-6Yn-280V/lib/python3.11/site-packages/datasette/app.py:14: in <module>
    import pkg_resources
../../../.local/share/virtualenvs/datasette-big-local-6Yn-280V/lib/python3.11/site-packages/pkg_resources/__init__.py:121: in <module>
    warnings.warn("pkg_resources is deprecated as an API", DeprecationWarning)
E   DeprecationWarning: pkg_resources is deprecated as an API

I ran with pytest -Werror --pdb -x to get the debugger for that warning, but it turned out searching the code worked better. It's used in these two places:

datasette/datasette/plugins.py

Lines 43 to 50 in 5890a20

	if pkg_resources.resource_isdir(plugin.__name__, "static"):
	static_path = pkg_resources.resource_filename(
	plugin.__name__, "static"
	)
	if pkg_resources.resource_isdir(plugin.__name__, "templates"):
	templates_path = pkg_resources.resource_filename(
	plugin.__name__, "templates"
	)

datasette/datasette/app.py

Line 1037 in 5890a20

info["pysqlite3"] = pkg_resources.get_distribution(package).version

[simonw]

I ran this prompt against ChatGPT with the Browsing alpha:

                if pkg_resources.resource_isdir(plugin.__name__, "static"):
                    static_path = pkg_resources.resource_filename(
                        plugin.__name__, "static"
                    )
                if pkg_resources.resource_isdir(plugin.__name__, "templates"):
                    templates_path = pkg_resources.resource_filename(
                        plugin.__name__, "templates"
                    )

This code gives a deprecation warning in Python 3.11 - fix it

It looked up the fix for me:

And suggested:

import importlib.resources

# Replace pkg_resources.resource_isdir with importlib.resources.files().is_file()
if importlib.resources.files(plugin.__name__).joinpath("static").is_file():
    static_path = importlib.resources.as_file(
        importlib.resources.files(plugin.__name__).joinpath("static")
    )
if importlib.resources.files(plugin.__name__).joinpath("templates").is_file():
    templates_path = importlib.resources.as_file(
        importlib.resources.files(plugin.__name__).joinpath("templates")
    )

This looks wrong to me - I would expect something like is_directory() not is_file() for telling if static/ is a directory.

[simonw]

Another prompt:

How to fix this:

pkg_resources.get_distribution(package).version

Response:

import importlib.metadata

# Get the version number of the specified package
package_version = importlib.metadata.version(package)

That seems to work:

>>> import importlib.metadata
>>> importlib.metadata.version("datasette")
'0.64.2'
>>> importlib.metadata.version("pluggy")
'1.0.0'
>>> importlib.metadata.version("not-a-package")
...
importlib.metadata.PackageNotFoundError: No package metadata was found for not-a-package

[simonw]

This looks wrong to me - I would expect something like is_directory() not is_file() for telling if static/ is a directory.

I was right about that:

>>> importlib.resources.files('datasette_graphql')
PosixPath('/Users/simon/.local/share/virtualenvs/datasette-big-local-6Yn-280V/lib/python3.11/site-packages/datasette_graphql')
>>> importlib.resources.files('datasette_graphql').joinpath("static")
PosixPath('/Users/simon/.local/share/virtualenvs/datasette-big-local-6Yn-280V/lib/python3.11/site-packages/datasette_graphql/static')
>>> p = importlib.resources.files('datasette_graphql').joinpath("static")
>>> p
PosixPath('/Users/simon/.local/share/virtualenvs/datasette-big-local-6Yn-280V/lib/python3.11/site-packages/datasette_graphql/static')
>>> p.is_
p.is_absolute()     p.is_char_device()  p.is_fifo()         p.is_mount()        p.is_reserved()     p.is_symlink()     
p.is_block_device() p.is_dir()          p.is_file()         p.is_relative_to(   p.is_socket()      
>>> p.is_dir()
True
>>> p.is_file()
False

[simonw]

Frustrating that this warning doesn't show up in the Datasette test suite itself. It shows up in plugin test suites that run this test:

@pytest.mark.asyncio
async def test_plugin_is_installed():
    datasette = Datasette(memory=True)
    response = await datasette.client.get("/-/plugins.json")
    assert response.status_code == 200
    installed_plugins = {p["name"] for p in response.json()}
    assert "datasette-chronicle" in installed_plugins

If you run that test inside Datasette core installed_plugins is an empty set, which presumably is why the warning doesn't get triggered there.

[simonw]

Weird, I still can't get the warning to show even with this:

@pytest.mark.asyncio
async def test_plugin_is_installed():
    datasette = Datasette(memory=True)

    class DummyPlugin:
        __name__ = "DummyPlugin"

        @hookimpl
        def actors_from_ids(self, datasette, actor_ids):
            return {}

    try:
        pm.register(DummyPlugin(), name="DummyPlugin")
        response = await datasette.client.get("/-/plugins.json")
        assert response.status_code == 200
        installed_plugins = {p["name"] for p in response.json()}
        assert "DummyPlugin" in installed_plugins

    finally:
        pm.unregister(name="ReturnNothingPlugin")

[simonw]

Here's the exception it uses:

>>> importlib.metadata.version("datasette")
'1.0a6'
>>> importlib.metadata.version("datasette2")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/homebrew/Caskroom/miniconda/base/lib/python3.10/importlib/metadata/__init__.py", line 996, in version
    return distribution(distribution_name).version
  File "/opt/homebrew/Caskroom/miniconda/base/lib/python3.10/importlib/metadata/__init__.py", line 969, in distribution
    return Distribution.from_name(distribution_name)
  File "/opt/homebrew/Caskroom/miniconda/base/lib/python3.10/importlib/metadata/__init__.py", line 548, in from_name
    raise PackageNotFoundError(name)
importlib.metadata.PackageNotFoundError: No package metadata was found for datasette2

[simonw]

Now I need to switch out pkg_resources in plugins.py:

datasette/datasette/plugins.py

Lines 33 to 74 in 852f501

	if DATASETTE_LOAD_PLUGINS is not None:
	for package_name in [
	name for name in DATASETTE_LOAD_PLUGINS.split(",") if name.strip()
	]:
	try:
	distribution = pkg_resources.get_distribution(package_name)
	entry_map = distribution.get_entry_map()
	if "datasette" in entry_map:
	for plugin_name, entry_point in entry_map["datasette"].items():
	mod = entry_point.load()
	pm.register(mod, name=entry_point.name)
	# Ensure name can be found in plugin_to_distinfo later:
	pm._plugin_distinfo.append((mod, distribution))
	except pkg_resources.DistributionNotFound:
	sys.stderr.write("Plugin {} could not be found\n".format(package_name))
	# Load default plugins
	for plugin in DEFAULT_PLUGINS:
	mod = importlib.import_module(plugin)
	pm.register(mod, plugin)
	def get_plugins():
	plugins = []
	plugin_to_distinfo = dict(pm.list_plugin_distinfo())
	for plugin in pm.get_plugins():
	static_path = None
	templates_path = None
	if plugin.__name__ not in DEFAULT_PLUGINS:
	try:
	if pkg_resources.resource_isdir(plugin.__name__, "static"):
	static_path = pkg_resources.resource_filename(
	plugin.__name__, "static"
	)
	if pkg_resources.resource_isdir(plugin.__name__, "templates"):
	templates_path = pkg_resources.resource_filename(
	plugin.__name__, "templates"
	)
	except (KeyError, ImportError):
	# Caused by --plugins_dir= plugins - KeyError/ImportError thrown in Py3.5
	pass

[simonw]

The importlib.metadata.entry_points() function is pretty interesting:

>>> import importlib.metadata
>>> from pprint import pprint
>>> pprint(importlib.metadata.entry_points())
{'babel.checkers': [EntryPoint(name='num_plurals', value='babel.messages.checkers:num_plurals', group='babel.checkers'),
                    EntryPoint(name='python_format', value='babel.messages.checkers:python_format', group='babel.checkers')],
 'babel.extractors': [EntryPoint(name='jinja2', value='jinja2.ext:babel_extract[i18n]', group='babel.extractors'),
                      EntryPoint(name='ignore', value='babel.messages.extract:extract_nothing', group='babel.extractors'),
                      EntryPoint(name='javascript', value='babel.messages.extract:extract_javascript', group='babel.extractors'),
                      EntryPoint(name='python', value='babel.messages.extract:extract_python', group='babel.extractors')],
 'console_scripts': [EntryPoint(name='datasette', value='datasette.cli:cli', group='console_scripts'),
                     EntryPoint(name='normalizer', value='charset_normalizer.cli.normalizer:cli_detect', group='console_scripts'),
                     EntryPoint(name='pypprint', value='pprintpp:console', group='console_scripts'),
                     EntryPoint(name='cog', value='cogapp:main', group='console_scripts'),
                     EntryPoint(name='icdiff', value='icdiff:start', group='console_scripts'),
                     EntryPoint(name='pycodestyle', value='pycodestyle:_main', group='console_scripts'),
                     EntryPoint(name='sphinx-autobuild', value='sphinx_autobuild.__main__:main', group='console_scripts'),
                     EntryPoint(name='sphinx-apidoc', value='sphinx.ext.apidoc:main', group='console_scripts'),
                     EntryPoint(name='sphinx-autogen', value='sphinx.ext.autosummary.generate:main', group='console_scripts'),
                     EntryPoint(name='sphinx-build', value='sphinx.cmd.build:main', group='console_scripts'),
                     EntryPoint(name='sphinx-quickstart', value='sphinx.cmd.quickstart:main', group='console_scripts'),
                     EntryPoint(name='sphinx-to-sqlite', value='sphinx_to_sqlite.cli:cli', group='console_scripts'),
                     EntryPoint(name='pybabel', value='babel.messages.frontend:main', group='console_scripts'),
                     EntryPoint(name='docutils', value='docutils.__main__:main', group='console_scripts'),
                     EntryPoint(name='isort', value='isort.main:main', group='console_scripts'),
                     EntryPoint(name='isort-identify-imports', value='isort.main:identify_imports_main', group='console_scripts'),
                     EntryPoint(name='hupper', value='hupper.cli:main', group='console_scripts'),
                     EntryPoint(name='sqlite-utils', value='sqlite_utils.cli:cli', group='console_scripts'),
                     EntryPoint(name='py.test', value='pytest:console_main', group='console_scripts'),
                     EntryPoint(name='pytest', value='pytest:console_main', group='console_scripts'),
                     EntryPoint(name='pyflakes', value='pyflakes.api:main', group='console_scripts'),
                     EntryPoint(name='livereload', value='livereload.cli:main', group='console_scripts'),
                     EntryPoint(name='uvicorn', value='uvicorn.main:main', group='console_scripts'),
                     EntryPoint(name='httpx', value='httpx:main', group='console_scripts'),
                     EntryPoint(name='flake8', value='flake8.main.cli:main', group='console_scripts'),
                     EntryPoint(name='blacken-docs', value='blacken_docs:main', group='console_scripts'),
                     EntryPoint(name='pip', value='pip._internal.cli.main:main', group='console_scripts'),
                     EntryPoint(name='pip3', value='pip._internal.cli.main:main', group='console_scripts'),
                     EntryPoint(name='pip3.10', value='pip._internal.cli.main:main', group='console_scripts'),
                     EntryPoint(name='wheel', value='wheel.cli:main', group='console_scripts'),
                     EntryPoint(name='pygmentize', value='pygments.cmdline:main', group='console_scripts'),
                     EntryPoint(name='black', value='black:patched_main', group='console_scripts'),
                     EntryPoint(name='blackd', value='blackd:patched_main [d]', group='console_scripts'),
                     EntryPoint(name='codespell', value='codespell_lib:_script_main', group='console_scripts'),
                     EntryPoint(name='tabulate', value='tabulate:_main', group='console_scripts')],
 'datasette': [EntryPoint(name='debug_permissions', value='datasette_debug_permissions', group='datasette'),
               EntryPoint(name='codespaces', value='datasette_codespaces', group='datasette'),
               EntryPoint(name='vega', value='datasette_vega', group='datasette'),
               EntryPoint(name='x_forwarded_host', value='datasette_x_forwarded_host', group='datasette'),
               EntryPoint(name='json_html', value='datasette_json_html', group='datasette'),
               EntryPoint(name='datasette_write_ui', value='datasette_write_ui', group='datasette'),
               EntryPoint(name='pretty_json', value='datasette_pretty_json', group='datasette'),
               EntryPoint(name='graphql', value='datasette_graphql', group='datasette')],
 'distutils.commands': [EntryPoint(name='compile_catalog', value='babel.messages.frontend:compile_catalog', group='distutils.commands'),
                        EntryPoint(name='extract_messages', value='babel.messages.frontend:extract_messages', group='distutils.commands'),
                        EntryPoint(name='init_catalog', value='babel.messages.frontend:init_catalog', group='distutils.commands'),
                        EntryPoint(name='update_catalog', value='babel.messages.frontend:update_catalog', group='distutils.commands'),
                        EntryPoint(name='isort', value='isort.setuptools_commands:ISortCommand', group='distutils.commands'),
                        EntryPoint(name='alias', value='setuptools.command.alias:alias', group='distutils.commands'),
                        EntryPoint(name='bdist_egg', value='setuptools.command.bdist_egg:bdist_egg', group='distutils.commands'),
                        EntryPoint(name='bdist_rpm', value='setuptools.command.bdist_rpm:bdist_rpm', group='distutils.commands'),
                        EntryPoint(name='build', value='setuptools.command.build:build', group='distutils.commands'),
                        EntryPoint(name='build_clib', value='setuptools.command.build_clib:build_clib', group='distutils.commands'),
                        EntryPoint(name='build_ext', value='setuptools.command.build_ext:build_ext', group='distutils.commands'),
                        EntryPoint(name='build_py', value='setuptools.command.build_py:build_py', group='distutils.commands'),
                        EntryPoint(name='develop', value='setuptools.command.develop:develop', group='distutils.commands'),
                        EntryPoint(name='dist_info', value='setuptools.command.dist_info:dist_info', group='distutils.commands'),
                        EntryPoint(name='easy_install', value='setuptools.command.easy_install:easy_install', group='distutils.commands'),
                        EntryPoint(name='editable_wheel', value='setuptools.command.editable_wheel:editable_wheel', group='distutils.commands'),
                        EntryPoint(name='egg_info', value='setuptools.command.egg_info:egg_info', group='distutils.commands'),
                        EntryPoint(name='install', value='setuptools.command.install:install', group='distutils.commands'),
                        EntryPoint(name='install_egg_info', value='setuptools.command.install_egg_info:install_egg_info', group='distutils.commands'),
                        EntryPoint(name='install_lib', value='setuptools.command.install_lib:install_lib', group='distutils.commands'),
                        EntryPoint(name='install_scripts', value='setuptools.command.install_scripts:install_scripts', group='distutils.commands'),
                        EntryPoint(name='rotate', value='setuptools.command.rotate:rotate', group='distutils.commands'),
                        EntryPoint(name='saveopts', value='setuptools.command.saveopts:saveopts', group='distutils.commands'),
                        EntryPoint(name='sdist', value='setuptools.command.sdist:sdist', group='distutils.commands'),
                        EntryPoint(name='setopt', value='setuptools.command.setopt:setopt', group='distutils.commands'),
                        EntryPoint(name='test', value='setuptools.command.test:test', group='distutils.commands'),
                        EntryPoint(name='upload_docs', value='setuptools.command.upload_docs:upload_docs', group='distutils.commands'),
                        EntryPoint(name='bdist_wheel', value='wheel.bdist_wheel:bdist_wheel', group='distutils.commands')],
 'distutils.setup_keywords': [EntryPoint(name='message_extractors', value='babel.messages.frontend:check_message_extractors', group='distutils.setup_keywords'),
                              EntryPoint(name='cffi_modules', value='cffi.setuptools_ext:cffi_modules', group='distutils.setup_keywords'),
                              EntryPoint(name='dependency_links', value='setuptools.dist:assert_string_list', group='distutils.setup_keywords'),
                              EntryPoint(name='eager_resources', value='setuptools.dist:assert_string_list', group='distutils.setup_keywords'),
                              EntryPoint(name='entry_points', value='setuptools.dist:check_entry_points', group='distutils.setup_keywords'),
                              EntryPoint(name='exclude_package_data', value='setuptools.dist:check_package_data', group='distutils.setup_keywords'),
                              EntryPoint(name='extras_require', value='setuptools.dist:check_extras', group='distutils.setup_keywords'),
                              EntryPoint(name='include_package_data', value='setuptools.dist:assert_bool', group='distutils.setup_keywords'),
                              EntryPoint(name='install_requires', value='setuptools.dist:check_requirements', group='distutils.setup_keywords'),
                              EntryPoint(name='namespace_packages', value='setuptools.dist:check_nsp', group='distutils.setup_keywords'),
                              EntryPoint(name='package_data', value='setuptools.dist:check_package_data', group='distutils.setup_keywords'),
                              EntryPoint(name='packages', value='setuptools.dist:check_packages', group='distutils.setup_keywords'),
                              EntryPoint(name='python_requires', value='setuptools.dist:check_specifier', group='distutils.setup_keywords'),
                              EntryPoint(name='setup_requires', value='setuptools.dist:check_requirements', group='distutils.setup_keywords'),
                              EntryPoint(name='test_loader', value='setuptools.dist:check_importable', group='distutils.setup_keywords'),
                              EntryPoint(name='test_runner', value='setuptools.dist:check_importable', group='distutils.setup_keywords'),
                              EntryPoint(name='test_suite', value='setuptools.dist:check_test_suite', group='distutils.setup_keywords'),
                              EntryPoint(name='tests_require', value='setuptools.dist:check_requirements', group='distutils.setup_keywords'),
                              EntryPoint(name='use_2to3', value='setuptools.dist:invalid_unless_false', group='distutils.setup_keywords'),
                              EntryPoint(name='zip_safe', value='setuptools.dist:assert_bool', group='distutils.setup_keywords')],
 'egg_info.writers': [EntryPoint(name='PKG-INFO', value='setuptools.command.egg_info:write_pkg_info', group='egg_info.writers'),
                      EntryPoint(name='dependency_links.txt', value='setuptools.command.egg_info:overwrite_arg', group='egg_info.writers'),
                      EntryPoint(name='depends.txt', value='setuptools.command.egg_info:warn_depends_obsolete', group='egg_info.writers'),
                      EntryPoint(name='eager_resources.txt', value='setuptools.command.egg_info:overwrite_arg', group='egg_info.writers'),
                      EntryPoint(name='entry_points.txt', value='setuptools.command.egg_info:write_entries', group='egg_info.writers'),
                      EntryPoint(name='namespace_packages.txt', value='setuptools.command.egg_info:overwrite_arg', group='egg_info.writers'),
                      EntryPoint(name='requires.txt', value='setuptools.command.egg_info:write_requirements', group='egg_info.writers'),
                      EntryPoint(name='top_level.txt', value='setuptools.command.egg_info:write_toplevel_names', group='egg_info.writers')],
 'flake8.extension': [EntryPoint(name='C90', value='mccabe:McCabeChecker', group='flake8.extension'),
                      EntryPoint(name='E', value='flake8.plugins.pycodestyle:pycodestyle_logical', group='flake8.extension'),
                      EntryPoint(name='F', value='flake8.plugins.pyflakes:FlakesChecker', group='flake8.extension'),
                      EntryPoint(name='W', value='flake8.plugins.pycodestyle:pycodestyle_physical', group='flake8.extension')],
 'flake8.report': [EntryPoint(name='default', value='flake8.formatting.default:Default', group='flake8.report'),
                   EntryPoint(name='pylint', value='flake8.formatting.default:Pylint', group='flake8.report'),
                   EntryPoint(name='quiet-filename', value='flake8.formatting.default:FilenameOnly', group='flake8.report'),
                   EntryPoint(name='quiet-nothing', value='flake8.formatting.default:Nothing', group='flake8.report')],
 'pylama.linter': [EntryPoint(name='isort', value='isort.pylama_isort:Linter', group='pylama.linter')],
 'pytest11': [EntryPoint(name='icdiff', value='pytest_icdiff', group='pytest11'),
              EntryPoint(name='asyncio', value='pytest_asyncio.plugin', group='pytest11'),
              EntryPoint(name='xdist', value='xdist.plugin', group='pytest11'),
              EntryPoint(name='xdist.looponfail', value='xdist.looponfail', group='pytest11'),
              EntryPoint(name='timeout', value='pytest_timeout', group='pytest11'),
              EntryPoint(name='anyio', value='anyio.pytest_plugin', group='pytest11')],
 'setuptools.finalize_distribution_options': [EntryPoint(name='keywords', value='setuptools.dist:Distribution._finalize_setup_keywords', group='setuptools.finalize_distribution_options'),
                                              EntryPoint(name='parent_finalize', value='setuptools.dist:_Distribution.finalize_options', group='setuptools.finalize_distribution_options')],
 'sphinx.html_themes': [EntryPoint(name='alabaster', value='alabaster', group='sphinx.html_themes'),
                        EntryPoint(name='basic-ng', value='sphinx_basic_ng', group='sphinx.html_themes'),
                        EntryPoint(name='furo', value='furo', group='sphinx.html_themes')],
 'sqlite_utils': [EntryPoint(name='hello_world', value='sqlite_utils_hello_world', group='sqlite_utils')]}

[simonw]

Just found this migration guide: https://importlib-metadata.readthedocs.io/en/latest/migration.html

[simonw]

This broke in Python 3.8:

if plugin.__name__ not in DEFAULT_PLUGINS:
    try:
        if (importlib.resources.files(plugin.__name__) / "static").is_dir():
E                   AttributeError: module 'importlib.resources' has no attribute 'files'

[simonw]

Confirmed: https://docs.python.org/3/library/importlib.resources.html#importlib.resources.files

importlib.resources.files(package)
[...]
New in version 3.9.

[simonw]

I think I can fix this using https://importlib-resources.readthedocs.io/en/latest/using.html - maybe as a dependency only installed if the Python version is less than 3.9.

[simonw]

I'll imitate certbot:

https://github.com/certbot/certbot/blob/694c758db7fcd8410b5dadcd136c61b3eb028fdc/certbot-ci/setup.py#L9

    'importlib_resources>=1.3.1; python_version < "3.9"',

Looks like 1.3 is the minimum version needed for compatibility with the 3.9 standard library, according to https://github.com/python/importlib_resources/blob/main/README.rst#compatibility

https://github.com/certbot/certbot/blob/694c758db7fcd8410b5dadcd136c61b3eb028fdc/certbot/certbot/_internal/constants.py#L13C29-L16

if sys.version_info >= (3, 9):  # pragma: no cover
    import importlib.resources as importlib_resources
else:  # pragma: no cover
    import importlib_resources

[simonw]

That passed on 3.8 but should have failed: https://github.com/simonw/datasette/actions/runs/6266341481/job/17017099801 - the "Test DATASETTE_LOAD_PLUGINS" test shows errors but did not fail the CI run.

[simonw]

In my Python 3.8 environment I ran:

datasette install datasette-init datasette-json-html

And now datasette plugins produces this error:

  File "/Users/simon/Dropbox/Development/datasette/datasette/cli.py", line 192, in plugins
    click.echo(json.dumps(app._plugins(all=all), indent=4))
  File "/Users/simon/Dropbox/Development/datasette/datasette/app.py", line 1136, in _plugins
    ps.sort(key=lambda p: p["name"])
TypeError: '<' not supported between instances of 'NoneType' and 'NoneType'

[simonw]

In the debugger:

>>> import pdb
>>> pdb.pm()
> /Users/simon/Dropbox/Development/datasette/datasette/app.py(1136)_plugins()
-> ps.sort(key=lambda p: p["name"])
(Pdb) ps
[{'name': None, 'static_path': None, 'templates_path': None, 'hooks': ['prepare_connection', 'render_cell'], 'version': '1.0.1'}, {'name': None, 'static_path': None, 'templates_path': None, 'hooks': ['startup'], 'version': '0.2'}]

[simonw]

Relevant code:

datasette/datasette/app.py

Lines 1127 to 1146 in 80a9cd9

	def _plugins(self, request=None, all=False):
	ps = list(get_plugins())
	should_show_all = False
	if request is not None:
	should_show_all = request.args.get("all")
	else:
	should_show_all = all
	if not should_show_all:
	ps = [p for p in ps if p["name"] not in DEFAULT_PLUGINS]
	ps.sort(key=lambda p: p["name"])
	return [
	{
	"name": p["name"],
	"static": p["static_path"] is not None,
	"templates": p["templates_path"] is not None,
	"version": p.get("version"),
	"hooks": list(sorted(set(p["hooks"]))),
	}
	for p in ps
	]

[simonw]

So the problem is the get_plugins() function returning plugins with None for their name:

datasette/datasette/plugins.py

Lines 61 to 91 in 80a9cd9

	def get_plugins():
	plugins = []
	plugin_to_distinfo = dict(pm.list_plugin_distinfo())
	for plugin in pm.get_plugins():
	static_path = None
	templates_path = None
	if plugin.__name__ not in DEFAULT_PLUGINS:
	try:
	if (importlib_resources.files(plugin.__name__) / "static").is_dir():
	static_path = str(
	importlib_resources.files(plugin.__name__) / "static"
	)
	if (importlib_resources.files(plugin.__name__) / "templates").is_dir():
	templates_path = str(
	importlib_resources.files(plugin.__name__) / "templates"
	)
	except (TypeError, ModuleNotFoundError):
	# Caused by --plugins_dir= plugins
	pass
	plugin_info = {
	"name": plugin.__name__,
	"static_path": static_path,
	"templates_path": templates_path,
	"hooks": [h.name for h in pm.get_hookcallers(plugin)],
	}
	distinfo = plugin_to_distinfo.get(plugin)
	if distinfo:
	plugin_info["version"] = distinfo.version
	plugin_info["name"] = distinfo.name
	plugins.append(plugin_info)
	return plugins

[simonw]

The problem is here:

 86  	        distinfo = plugin_to_distinfo.get(plugin)
 87  	        if distinfo is None:
 88  	            breakpoint()
 89  ->	            assert False
 90  	        if distinfo.name is None:
 91  	            breakpoint()
 92  	            assert False
 93  	        if distinfo:
 94  	            plugin_info["version"] = distinfo.version
(Pdb) distinfo
(Pdb) plugin
<module 'datasette.sql_functions' from '/Users/simon/Dropbox/Development/datasette/datasette/sql_functions.py'>

That plugin_to_distinfo is missing some stuff.

[simonw]

No that's not it actually, it's something else.

Got to this point:

DATASETTE_LOAD_PLUGINS=datasette-init python -i $(which datasette) plugins

That fails and drops me into a debugger:

  File "/Users/simon/Dropbox/Development/datasette/datasette/cli.py", line 186, in plugins
    app = Datasette([], plugins_dir=plugins_dir)
  File "/Users/simon/Dropbox/Development/datasette/datasette/app.py", line 405, in __init__
    for plugin in get_plugins()
  File "/Users/simon/Dropbox/Development/datasette/datasette/plugins.py", line 89, in get_plugins
    plugin_info["name"] = distinfo.name or distinfo.project_name
AttributeError: 'PathDistribution' object has no attribute 'name'

[simonw]

Hunch: https://pypi.org/project/importlib-metadata/ may help here.

[simonw]

That does seem to fix the problem!

[simonw]

Still fails in Python 3.9: https://github.com/simonw/datasette/actions/runs/6266752548/job/17018363302

    plugin_info["name"] = distinfo.name or distinfo.project_name
AttributeError: 'PathDistribution' object has no attribute 'name'
Test failed: datasette-json-html should not have been loaded

[simonw]

Tested that locally with Python 3.9 from pyenv and it worked.

[simonw]

Tests all pass now.