Ubuntu 12.04, salt 2014.1.3-1precise1, minion dead #12172
Comments
LeTink
changed the title
|
OK, did some more digging. Turns out that only my playpens minion is karking it. Since UtahDave asked about the masters and minions config file(s) I compared the playpens file against a "plain" one, and it turns out that I had played with the default hash in the past, changing it from md5 to sha512 on the playpen. This worked OK in the past, but must have suffered a regression in 2014.1.3 ... reverting to md5 fixed the issue of the minion not wanting to start any more. |
|
Any chance you could test this on the |
|
I'd be happy to, but have to admit I don't know how to go about that? |
|
You can use salt bootstrap using flags to install from git, or you can just run |
|
OK - it would appear the regressions is also present in devel, not just in the 2014.1.3 release. After the bootstrap & canging the hash back to sha512 I still get the issues originally reported. $ salt-minion --version sudo salt-minion -l debug [DEBUG ] Loaded ldapmod as virtual ldap |
|
Awesome, thanks for testing that! We'll definitely investigate. |
|
+1, can confirm that changing the hash function back to md5 fixes this for this package. |
|
I can confirm this on fedora19 and fedora20 (salt 2014.1.4). switching back to md5 solved the problem. Switching to sha384 interestingly eliminated the error messages, but not the highstate hangs on the minion side when doing a salt-call. |
|
Thanks for the continued updates. |
|
There is a known kernel limitation for socket path length, defined in /* Structure describing the address of an AF_LOCAL (aka AF_UNIX) socket. */
struct sockaddr_un
{
__SOCKADDR_COMMON (sun_);
char sun_path[108]; /* Path name. */
};This path is limited to 107 characters, since C strings end with a null terminator. The path of the socket Salt is trying to use is 171 characters long, which is what is causing the exception. The error can be tracked down to this block of code. It looks like some work had already been done in the past to shorten the path length in the event that sha256 was being used. However, this was shortening id hash to 10 characters, when the length of an md5 hash is 32 characters. I know what needs to be done to resolve this, but we need to decide on how we're going to handle this path name. If the idea is that we want the path to be the same every time, in my opinion we should still be using a hash, but we should always shorten it to the same length. If we don't care about the socket path being the same every time, then there is no reason we can't use tempfille.mktemp() or something like that to derive a unique path. My guess is that the idea is to have a regular path name, because if they are not cleaned up when the minion stops, or the minion does not exit cleanly, then you'll end up with a bunch of sockets accumulating in the sock_dir. So, I think the best course of action is to truncate the id hash to 10 characters in all cases. |
|
After some internal discussion, the choice will indeed be to truncate the ID hash to 10 chars. |
This helps avoid the sock path being longer than the max length allowed by the kernel. See the following comment for more information: saltstack#12172 (comment)
|
Fix has been merged and unit test has been added, this fix should be in 2014.1.5. |
This helps avoid the sock path being longer than the max length allowed by the kernel. See the following comment for more information: saltstack#12172 (comment)
After the latest upgrade to salt-stack minions fail to start;
The text was updated successfully, but these errors were encountered: