Write lists of minions targeted by syndic masters to job cache #31670
Conversation
This allows minions to be updated when the master receives events containing lists of minions targeted by lower-level masters.
This allows the job cache to include minions matched by lower-level masters for a given job.
…r API compatibility
… API compatibility
…r API compatibility
|
Go Go Jenkins! |
2 similar comments
|
Go Go Jenkins! |
|
Go Go Jenkins! |
|
Go Go Jenkins! |
| ''' | ||
| Included for API consistency | ||
| ''' | ||
| pass |
There was a problem hiding this comment.
I'm sort of inclined to raise exceptions for these and catch and log higher up.
There was a problem hiding this comment.
The only reason that I did not is that we're not already saving the minion list for this returner.
|
(With apologies for being the person who brought up locking the first place.) After looking at this, I'm wondering if instead of locking we consider splitting this write up into multiple files and then scanning those when we retrieve the minions list. (I'm talking about the local cache of course.) The locking approach is starting to worry me.. |
cachedout
added
the
Pending-Discussion
|
@cachedout Good idea, I just pushed some changes, let me know what you think. |
Some events just contain a timestamp and a minion list, but no jid. These can safely be ignored, lest they cause supurious log messages. Also, pass the syndic's id so that it can be stored in a unique file in the job cache (in the local_cache returner).
Also fix successive updates
This is used to provide a unique filename for the minion list for a given syndic.
|
Yeah, I think this is better. Let's go with this. |
Write lists of minions targeted by syndic masters to job cache
Summary
When a master-of-masters publishes a command, and a syndic master re-publishes it to its subscribed minions, it fires an event back to the master containing the list of minions matched by the syndic master for that job.
This pull request scans these events for lists of minions, and writes them to the job cache.
Fixes #30528.
How to test
order_masters: Trueon the master-of-masters and subscribe the lower-level minion to the syndic master, and the syndic master to the master-of-masters.salt -v '*' test.pingsalt-run jobs.list_job <jid>. TheMinionslist in the return data should contain the minion ID of the lower-level minion. Prior to this pull request, the lower-level minions would not be in this return data. After completing the rest of the test steps, installing Salt from the head of2015.8on the master-of-masters and repeating steps 3 and 4 should show no lower-level minions in thejobs.list_jobreturn data.salt-minionservice on the lower-level minion, and then repeat step 3.salt-run jobs.lookup_jid <jid> missing=True. The lower-level minion should be included in the return data, and should showMinion did not return. Repeating the call tojobs.lookup_jidwithout themissing=Trueshould exclude the lower-level minion from the return data.Review notes
Of particular focus in the review of this pull request should be the following:
couchbase_returnreturner. To maintain API compatibility I needed to move the writing of minion lists into a separate function, as I did for thelocal_cachereturner, and I did not have an instance to test this.