Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

plugin loader - cache .py files in all() #82448

Draft
wants to merge 1 commit into
base: devel
Choose a base branch
from

Conversation

s-hertel
Copy link
Contributor

SUMMARY

this is based on #32609

  • avoid calling glob.glob() when looking for py files in variable folders for each host listed in the inventory.

In addition, I added a global cache so all plugin loaders stay in sync (not sure if this is wanted), and reset the cache when new directories are added to the loader, like the other caches.

#79687 also identifies this as a hot code path.

On devel, the inventory script given in #32609 as time ansible-inventory -i inv.py --list --output /dev/null takes ~10 seconds, and on this branch ~5 seconds.

ISSUE TYPE
  • Bugfix Pull Request
  • Feature Pull Request

@webknjaz

This comment was marked as resolved.

This comment was marked as resolved.

@ansibot ansibot added bug This issue/PR relates to a bug. feature This issue/PR relates to a feature request. needs_triage Needs a first human triage before being processed. needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. stale_ci This PR has been tested by CI more than one week ago. Close and re-open this PR to get it retested. and removed needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. stale_ci This PR has been tested by CI more than one week ago. Close and re-open this PR to get it retested. labels Jan 2, 2024
@bcoca bcoca removed the needs_triage Needs a first human triage before being processed. label Jan 9, 2024
@ansibot ansibot added the stale_ci This PR has been tested by CI more than one week ago. Close and re-open this PR to get it retested. label Jan 16, 2024
@ansibot ansibot added the needs_rebase https://docs.ansible.com/ansible/devel/dev_guide/developing_rebasing.html label Jan 30, 2024
this is based on ansible#32609

* avoid calling glob.glob() when looking for py files in variable folders for each host listed in the inventory.

In addition, I added a global cache so all plugin loaders stay in sync
(not sure if this is wanted), and reset the cache when new directories
are added to the loader, like the other caches.

ansible#79687 also identifies this as a hot code path.

Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
@s-hertel s-hertel force-pushed the cache-pluginloader-all-globbing branch from e901b5b to f2c5c78 Compare January 31, 2024 14:40
@ansibot ansibot removed needs_rebase https://docs.ansible.com/ansible/devel/dev_guide/developing_rebasing.html stale_ci This PR has been tested by CI more than one week ago. Close and re-open this PR to get it retested. labels Jan 31, 2024
@ansibot ansibot added the stale_ci This PR has been tested by CI more than one week ago. Close and re-open this PR to get it retested. label Feb 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue/PR relates to a bug. feature This issue/PR relates to a feature request. stale_ci This PR has been tested by CI more than one week ago. Close and re-open this PR to get it retested.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants