Skip to content

Commit

Permalink
Convert ksconf_cmd loading using importlib.metadata
Browse files Browse the repository at this point in the history
- Update the entrypoint loading process for the 'ksconf_cmd' group to now use
  importlib.metadata or the importlib_metadata backport package.  This reduces
  code complexity, and for newer versions of python, it results in fewer total
  package dependencies.
  • Loading branch information
lowell80 committed Sep 27, 2023
1 parent 04579e8 commit 30ea18c
Show file tree
Hide file tree
Showing 6 changed files with 31 additions and 11 deletions.
8 changes: 8 additions & 0 deletions docs/source/changelog.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,19 @@ Ksconf 0.12
* Pre-commit hooks have been moved into their own `ksconf pre-commit repo <ksconf-pre-commit>`_ repository.
To allow time for migration to the new repo, the existing hooks will remain for a few release before being removed.
To migrate, simply add ``-pre-commit`` to the end of the ``repo`` field, and update ``rev`` to ``v0.12.0`` or later.


**Packaging changes:**

* Dropped hard ``lxml`` from requirements.
This is still handled automatically when using the pre-commit hooks (from the new repository).
But this may be missing.
To get access to all CLI functionality, run ``pip install ksconf[thirdparty]``,
or for the full experience use ``pip install ksconf[fully-loaded]``
* Remove the use of the ``endpoints`` python package and shift to using ``importlib.metadata`` (or the equivalent backport), as it suggested by the original author of that package.
The original necessity of this library was to workaround performance issues in ``pkg_resources`` (and the fact that it's no present in Splunk's embedded python.
This move reduces code complexity but it does mean some additional runtime dependencies on older versions of Python.
In many cases, this really isn't a new dependency, since pluggy requires it as well.


Ksconf v0.12.0 (DRAFT)
Expand Down
5 changes: 1 addition & 4 deletions ksconf/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -187,10 +187,7 @@ def build_cli_parser(do_formatter=False):
if hasattr(dist, "version"):
if hasattr(dist, "name"):
# entrypoints (required by ksconf)
distro = f"{dist.name} ({ep.entry.dist.version})"
elif hasattr(dist, "location") and hasattr(dist, "project_name"): # pragma: no cover
# Attributes per pkg_resource (currently disabled)
distro = f"{dist.project_name} ({dist.version}) @{dist.location}"
distro = f"{dist.name} ({dist.version})"

subcommands[distro].append((ep.name, ep.cmd_cls, ep.error))
if ep.cmd_cls:
Expand Down
23 changes: 19 additions & 4 deletions ksconf/commands/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -534,6 +534,7 @@ def add_file_handler(parser: ArgumentParser) -> ArgumentParser:
return parser


'''
def _get_entrypoints_lib(group, name=None):
import entrypoints
Expand All @@ -544,10 +545,21 @@ def _get_entrypoints_lib(group, name=None):
return entrypoints.get_single(group, name)
else:
return entrypoints.get_group_named(group)
'''

# TODO: Find out if Splunk ships with importlib.metadata A: No, neither is present (no surprise)


@cache
def _get_importlib_entrypoints(group, name=None) -> list:
# Using a backport library to get Python 3.10 EntryPoints.select() functionality.
# But practically, if the backport is available, use it. It's likely newer than stdlib.
try:
from importlib_metadata import entry_points
except ImportError:
from importlib.metadata import entry_points
return entry_points(group=group, name=name)

# TODO: Switch to importlib.metadata once ksconf is Python 3.8+ (Could use importlib_metadata backport)
# TODO: Find out if Splunk ships with importlib.metadata

""" Disabling this. Because the DistributionNotFound isn't thrown until the entrypoint.load()
function is called outside of our control. Going with 'entrypoints' module or local fallback,
Expand Down Expand Up @@ -578,8 +590,11 @@ def _get_fallback(group, name=None):
return entrypoints[name]


# Removed _get_pkgresources_lib as middle option
__get_entity_resolvers = [_get_entrypoints_lib, _get_fallback]
__get_entity_resolvers = [
_get_importlib_entrypoints,
# _get_entrypoints_lib,
_get_fallback,
]

if "ksconf_cmd" in os.environ.get("KSCONF_DISABLE_PLUGINS", ""): # pragma: no cover
# Only use the fallback built in mechanism. This is helpful when unittesting and building docs
Expand Down
3 changes: 2 additions & 1 deletion ksconf/setup_entrypoints.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
This is a silly hack allows for fallback mechanism when
(a) running unit tests (can happen before install)
(b) if entrypoints or pkg_resources are not available at run time (Splunk's embedded python)
(b) unexpected issues with importlib.metadata or backport
"""

from __future__ import absolute_import, unicode_literals
Expand Down Expand Up @@ -44,6 +44,7 @@ class Ep(NamedTuple):


def get_entrypoints_setup():
""" Build entry point text descriptions for ksconf packaging """
setup = {}
for (group, entries) in _entry_points.items():
setup[group] = [f"{ep.name} = {ep.module_name}:{ep.object_name}" for ep in entries]
Expand Down
1 change: 0 additions & 1 deletion requirements-app.txt
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@

# These are copied from requirements.txt
# NOTE: We copy instead of reference because this list is used by build.py.
entrypoints
splunk-sdk>=1.7.0
# 1.3.0 Drops Python 3.7 support
pluggy>=1.2.0,<1.3
Expand Down
2 changes: 1 addition & 1 deletion requirements.txt
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
entrypoints
splunk-sdk>=1.7.0
pluggy>=1.2.0
importlib-metadata>4.6; python_version < "3.10"

0 comments on commit 30ea18c

Please sign in to comment.