Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] grp.getgrall() in user.py causing performance issue during minion connections #64888

Closed
wcbzero opened this issue Aug 1, 2023 · 4 comments
Assignees
Labels
Bug broken, incorrect, or confusing behavior Performance

Comments

@wcbzero
Copy link

wcbzero commented Aug 1, 2023

Description
On systems with a large number of non-local groups, grp.getgrall() is an expensive operation. When large numbers of minions connect this can cause performance issues, and minion connection timeouts

Setup
(Please provide relevant configs and/or SLS files (be sure to remove sensitive info. There is no general set-up of Salt.)

Please be as specific as possible and give set-up details.

  • on-prem machine
  • VM (VMWare ESXi)
  • OpenLDAP Slapd LDAP Caching Server
  • Active Directory with > 1000 groups

Steps to Reproduce the behavior

  1. Configure Slapd as a caching server for an Active Directory instance containing more than 1000 groups with Unix GID numbers.
  2. Start the salt master
  3. Have > 500 Machines connect to the machine
  4. Observe large number of slow grp.getgrall() calls

Expected behavior

Only lookups of the groups the user is a member of happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Versions Report

```Salt Version: Salt: 3006.1

Python Version:
Python: 3.8.5 (default, May 27 2021, 13:30:53)

Dependency Versions:
cffi: Not Installed
cherrypy: Not Installed
dateutil: Not Installed
docker-py: 5.0.3
gitdb: Not Installed
gitpython: Not Installed
Jinja2: 3.1.2
libgit2: Not Installed
looseversion: 1.2.0
M2Crypto: Not Installed
Mako: Not Installed
msgpack: 1.0.4
msgpack-pure: Not Installed
mysql-python: Not Installed
packaging: 23.1
pycparser: Not Installed
pycrypto: 3.15.0
pycryptodome: 3.12.0
pygit2: Not Installed
python-gnupg: Not Installed
PyYAML: 6.0
PyZMQ: 25.0.2
relenv: Not Installed
smmap: Not Installed
timelib: Not Installed
Tornado: 4.5.3
ZMQ: 4.3.4

System Versions:
dist: ubuntu 20.04.6 focal
locale: utf-8
machine: x86_64
release: 5.4.0-148-generic
system: Linux
version: Ubuntu 20.04.6 focal


</details>

@wcbzero wcbzero added Bug broken, incorrect, or confusing behavior needs-triage labels Aug 1, 2023
@welcome
Copy link

welcome bot commented Aug 1, 2023

Hi there! Welcome to the Salt Community! Thank you for making your first contribution. We have a lengthy process for issues and PRs. Someone from the Core Team will follow up as soon as possible. In the meantime, here’s some information that may help as you continue your Salt journey.
Please be sure to review our Code of Conduct. Also, check out some of our community resources including:

There are lots of ways to get involved in our community. Every month, there are around a dozen opportunities to meet with other contributors and the Salt Core team and collaborate in real time. The best way to keep track is by subscribing to the Salt Community Events Calendar.
If you have additional questions, email us at saltproject@vmware.com. We’re glad you’ve joined our community and look forward to doing awesome things with you!

@wcbzero
Copy link
Author

wcbzero commented Aug 1, 2023

I created a Patch in PR #64891

@whytewolf
Copy link
Contributor

looking into this and #64953 to try and determine the best coarse of action. right now it is looking like a revert of the code in question. but i want to talk to @nicholasmhughes first. this does not look like it will be in 3006.3.

@nicholasmhughes
Copy link
Collaborator

Fixed by #65077

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug broken, incorrect, or confusing behavior Performance
Projects
None yet
Development

No branches or pull requests

5 participants