[master] Move "FQDNs" grains calculation to "network" module and allow disabled them#55581
Merged
dwoz merged 19 commits intosaltstack:masterfrom May 18, 2020
Conversation
ghost
requested a review
from 
meaksh
force-pushed
the
master-calculate-fqdns-in-parallel-to-avoid-blockings
branch
from
97dae3d to
7b898f2
Compare
meaksh
force-pushed
the
master-calculate-fqdns-in-parallel-to-avoid-blockings
branch
from
bee6065 to
7bfffe6
Compare
* Duplicate fqdns logic in module.network * Move _get_interfaces to utils.network * Reuse network.fqdns in grains.core.fqdns * Return empty list when fqdns grains is disabled Co-authored-by: Eric Siebigteroth <eric.siebigteroth@suse.de>
meaksh
force-pushed
the
master-calculate-fqdns-in-parallel-to-avoid-blockings
branch
from
0a69294 to
dce0664
Compare
dwoz
approved these changes
May 18, 2020
3 tasks
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR ports #52527 to
masterbranch.What does this PR do?
This PR fixes an issue that is currently blocking the "core" grains execution for several seconds when the some "FQDNs" cannot be calculated.
Currently, in order to calculate the
fqdnsgrain, all the IP addresses from the minion are processed sequentially. The problem here comes from the underlying calls tosocket.gethostbyaddrwhich can takes 5 seconds until it's released (after reaching thesocket.timeout) when there is no defined "fqdn" for that IP.In some scenarios, depending on the minion network configuration, this situation makes the
fqdngrain calculation to take even more than 15 seconds. This completely breaks minion execution not only at startup time but any time the minion refreshes its grains.This PR makes the
fqdn_lookupto be executed in parallel (using threads) so, in those cases we would only wait until "socket.timeout" is reached once and prevent from blocking the minion execution for such a long time.Besides of that, the whole "fqdn" calculation logic has been moved to the
networkexecution module. Since the process of calculating the "fqdns" can take too much in case of networking issues, it makes more sense to move if away from the "core" grains and also introduce a new configuration parameters calledenable_fqdns_grains(default True) to allow not to calculate "fqdns" as part of the "core" grains rendering.Tests written?
Yes
Commits signed with GPG?
Yes