Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[master] Fix 64260: Remove isfile check to increase speed of listing large numbers of keys #64262

Merged
merged 1 commit into from
Jun 9, 2023

Conversation

bluesliverx
Copy link
Contributor

What does this PR do?

It adds a configuration option for skipping the isfile check when enumerating keys in various situations. This speeds up minion key listing significantly in certain situations.

What issues does this PR fix or reference?

Fixes: #64260

Previous Behavior

Minion key listing and salt master startup took minutes with 10k+ minions and key storage on a slow filesystem (e.g. NAS).

New Behavior

Minion key listing is performant.

Merge requirements satisfied?

[NOTICE] Bug fixes or features added to Salt require tests.

Commits signed with GPG?

No

@bluesliverx bluesliverx requested a review from a team as a code owner May 9, 2023 14:08
@bluesliverx bluesliverx requested review from Ch3LL and removed request for a team May 9, 2023 14:08
@bluesliverx bluesliverx temporarily deployed to ci May 9, 2023 14:35 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci May 9, 2023 14:35 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci May 9, 2023 14:35 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci May 9, 2023 15:37 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci May 9, 2023 15:37 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci May 9, 2023 15:37 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci May 9, 2023 16:25 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci May 9, 2023 16:25 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci May 9, 2023 16:25 — with GitHub Actions Inactive
Ch3LL
Ch3LL previously approved these changes May 9, 2023
@Ch3LL Ch3LL requested a review from dwoz May 9, 2023 19:14
@Ch3LL
Copy link
Contributor

Ch3LL commented May 9, 2023

Looks good, but I would also like to get @dwoz 's review here

@bluesliverx bluesliverx temporarily deployed to ci May 9, 2023 19:22 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci May 9, 2023 19:22 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci May 9, 2023 19:22 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci May 9, 2023 22:17 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci May 9, 2023 22:17 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci May 9, 2023 22:33 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci May 9, 2023 23:49 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci May 9, 2023 23:49 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci May 9, 2023 23:49 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci May 10, 2023 00:19 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci May 10, 2023 00:19 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci May 10, 2023 00:19 — with GitHub Actions Inactive
@dwoz
Copy link
Contributor

dwoz commented May 12, 2023

I'm not sure I agree with adding an option for this. I think it would make sense to have this (not checking isfile) as the default. In general, I think we should default to the more performant case and only add an option for the less performant case when absolutely necessary. In this particular case, I can't see a reason for the less performant state. Let's assume salt is the only thing managing the keys and in the rare situation we run into a bad one, warn, clean it up, and move on.

@bluesliverx bluesliverx temporarily deployed to ci June 8, 2023 01:19 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci June 8, 2023 01:19 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci June 8, 2023 20:38 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci June 8, 2023 20:38 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci June 8, 2023 20:38 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci June 8, 2023 20:38 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci June 8, 2023 20:39 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci June 8, 2023 20:39 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci June 8, 2023 21:31 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci June 8, 2023 21:31 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci June 8, 2023 21:36 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci June 8, 2023 21:36 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci June 8, 2023 21:50 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci June 8, 2023 21:57 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci June 8, 2023 22:54 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci June 8, 2023 22:54 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci June 8, 2023 22:54 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci June 8, 2023 22:54 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci June 8, 2023 22:54 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci June 8, 2023 22:54 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci June 8, 2023 23:17 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci June 8, 2023 23:17 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci June 8, 2023 23:17 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci June 8, 2023 23:17 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci June 8, 2023 23:17 — with GitHub Actions Inactive
@bluesliverx bluesliverx temporarily deployed to ci June 8, 2023 23:17 — with GitHub Actions Inactive
@Ch3LL Ch3LL merged commit 96c4c44 into saltstack:master Jun 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] Minion key listing is too slow on NFS mounts
3 participants