Make AES key never hit disk on the master #19323
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
In the past we've been doing coordination of the aes key using dropfiles etc. Not only is this significantly more costly (at least 1 stat per reqserver request) it also means that the symmetric key we use to pub/req messages has been on disk. This re-works the aes key to be a multiprocessing.Array() (char array) which is shared amongst the processes. Now the dropfile is just a request for the master to rotate the key, and this means that *all* key rotation on the master will generate the appropriate event (instead of just ones who passed in sock_dir to dropfile())
|
Go Go Jenkins! |
|
Looks like this may break the pillar |
|
Go Go Jenkins! |
We should probably not put "aes" in the global opts dict, but rather as a class attribute of auth/crypticle or something
|
@thatch45 And that it did ;) It appears that the aes key is passed in the opts all over creation-- and multiprocessing.Array is not deepcopy-able (which the git external pillar does). This should fix the issue for now, but as part of my cleanup I plan on moving the AES key out of the opts dict all together. |
|
Go Go Jenkins! |
|
I agree, this is great stuff, thanks! |
|
That test failure seems unrelated ;) |
thatch45
added a commit
that referenced
this pull request
Jan 6, 2015
Make AES key never hit disk on the master
|
W00t! tests ran! |
|
Very exciting, From my local testing this compared to just the stat (not the reading of the aes key of disk) will be at least an order of magnitude faster! Very exciting |
|
This is seriously awesome. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In the past we've been doing coordination of the aes key using dropfiles etc. Not only is this significantly more costly (at least 1 stat per reqserver request) it also means that the symmetric key we use to pub/req messages has been on disk. This re-works the aes key to be a multiprocessing.Array() (char array) which is shared amongst the processes. Now the dropfile is just a request for the master to rotate the key, and this means that all key rotation on the master will generate the appropriate event (instead of just ones who passed in sock_dir to dropfile())