-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
More granular master job targeting #13285
Conversation
Test Failed. If the failures are unrelated to your code, don't stress, a core developer will know these apart. |
I never got the go ahead on this one, is it ready for merge/review? |
This is ready for review, but I need to remove the default config change. The plan is to make the master by default send the message with the new header. And then you can enable the actual filtering on the minion side. The plan would be to change the default for that after this major release. Would it be possible to get a couple of you guys to test this out as well? Everything passes on my box, but more people testing it is almost always better :) I know that when this is disabled tests still pass, question is does it work when it is enabled. I've tested, but the CI stack has pinned master/minion versions which I can't change the configs of. |
Sounds good, you can change the configs in the test suite in a pull req to run them on Jenkins. |
@jacksontj should we close this? it is badly conflicting. |
I'm going to try to get some time this week to fix it up, I can just fix
|
Salt targeting is done using the zmq pub/sub mechanism. No filtering is done so all requests go to all minions. This is an attempt to make the requests more target-able. ZeroMQ supports filters on these sockets which are processed publisher side. So the thought is to set it to the minion id and a shared "broadcast" one. Only requests of tgt_type == list will use this. The thought is if we like this we can use other data (mine etc) to determine a more specific target on the master (such as a list) from a glob match etc. This is more or less backwards compatible, during my testing it still works but the old minion will throw an exception like: [CRITICAL] An exception occurred while polling the minion Traceback (most recent call last): File "/home/thjackso/src/salt/salt/minion.py", line 1324, in tune_in payload = self.serial.loads(self.socket.recv(zmq.NOBLOCK)) File "/home/thjackso/src/salt/salt/payload.py", line 95, in loads return msgpack.loads(msg, use_list=True) File "_unpacker.pyx", line 114, in msgpack._unpacker.unpackb (msgpack/_unpacker.cpp:114) ExtraData: unpack(b) recieved extra data. So if we decide this is a good idea we'll probably want to wrap the new master side logic in a config option, and just merge in the minion side changes for the first release. This is another attempt of saltstack#10374.
@thatch45 I just fixed the merge conflict, but I'm not sure how to change the configs for the jenkins run. I changed the default config values in this pull req, but if I remember right the Jenkins runs use a pinned version of the master and minion-- if thats the case I can't really get the CI stack to run all this code for me. If that is the case we can either manually run it somehow (tests are running locally right now, but in jenkins would be nice) or we can merge this in with the configs off then switch the pinned version to include this code so then i could make another branch which just has the config change to trigger test runs. Apologies if I lost you there ;) Maybe we should involve @s0undt3ch since he is the test master :) |
FYI the tests just all passed locally. |
Test Failed. If the failures are unrelated to your code, don't stress, a core developer will know these apart. |
Interesting, seems that all tests (besides some unrelated pylint tests) passed. With the config that I have (zmq_filtering on) this should fail the backwards compatibility tests, are we not running those? |
No, we don't have backward compat tests inline. But this does look ready to merge. |
More granular master job targeting
The defaults i set make it backwards incompatible. Just have to change the
|
Change the defaults to be backwards compatible (fix #13285)
Please DO NOT MERGE IMMEDIATELY.
Salt targeting is done using the zmq pub/sub mechanism. No filtering is done so all requests go to all minions. This is an attempt to make the requests more target-able. ZeroMQ supports filters on these sockets which are processed publisher side. So the thought is to set it to the minion id and a shared "broadcast" one. Only requests of tgt_type == list will use this. The thought is if we like this we can use other data (mine etc) to determine a more specific target on the master (such as a list) from a glob match etc.
This is more or less backwards compatible, during my testing it still works but the old minion will throw an exception like:
[CRITICAL] An exception occurred while polling the minion
Traceback (most recent call last):
File "/home/thjackso/src/salt/salt/minion.py", line 1324, in tune_in
payload = self.serial.loads(self.socket.recv(zmq.NOBLOCK))
File "/home/thjackso/src/salt/salt/payload.py", line 95, in loads
return msgpack.loads(msg, use_list=True)
File "_unpacker.pyx", line 114, in msgpack._unpacker.unpackb (msgpack/_unpacker.cpp:114)
ExtraData: unpack(b) recieved extra data.
The new minion (in this PR) will work with both configurations-- although once a minion has enabled targeting it will no longer receive old-style messages.
This is another attempt of #10374.
And exactly the same as #13154