Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[master] Move "FQDNs" grains calculation to "network" module and allow disabled them #55581

Conversation

meaksh
Copy link
Contributor

@meaksh meaksh commented Dec 10, 2019

This PR ports #52527 to master branch.


What does this PR do?

This PR fixes an issue that is currently blocking the "core" grains execution for several seconds when the some "FQDNs" cannot be calculated.

Currently, in order to calculate the fqdns grain, all the IP addresses from the minion are processed sequentially. The problem here comes from the underlying calls to socket.gethostbyaddr which can takes 5 seconds until it's released (after reaching the socket.timeout) when there is no defined "fqdn" for that IP.

In some scenarios, depending on the minion network configuration, this situation makes the fqdn grain calculation to take even more than 15 seconds. This completely breaks minion execution not only at startup time but any time the minion refreshes its grains.

This PR makes the fqdn_lookup to be executed in parallel (using threads) so, in those cases we would only wait until "socket.timeout" is reached once and prevent from blocking the minion execution for such a long time.

Besides of that, the whole "fqdn" calculation logic has been moved to the network execution module. Since the process of calculating the "fqdns" can take too much in case of networking issues, it makes more sense to move if away from the "core" grains and also introduce a new configuration parameters called enable_fqdns_grains(default True) to allow not to calculate "fqdns" as part of the "core" grains rendering.

Tests written?

Yes

Commits signed with GPG?

Yes

@meaksh meaksh requested a review from a team as a code owner December 10, 2019 11:49
@ghost ghost requested a review from Akm0d December 10, 2019 11:50
@meaksh meaksh force-pushed the master-calculate-fqdns-in-parallel-to-avoid-blockings branch from 97dae3d to 7b898f2 Compare April 7, 2020 12:40
@Ch3LL Ch3LL removed the request for review from a team April 15, 2020 14:31
@meaksh meaksh force-pushed the master-calculate-fqdns-in-parallel-to-avoid-blockings branch from bee6065 to 7bfffe6 Compare April 24, 2020 14:30
@meaksh meaksh force-pushed the master-calculate-fqdns-in-parallel-to-avoid-blockings branch from 0a69294 to dce0664 Compare May 14, 2020 12:43
@dwoz dwoz merged commit 9d97848 into saltstack:master May 18, 2020
@meaksh meaksh deleted the master-calculate-fqdns-in-parallel-to-avoid-blockings branch May 19, 2020 07:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Sodium
  
Done
Development

Successfully merging this pull request may close these issues.

None yet

4 participants