Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Salt Master Getting Hammered by Auth calls? #4985

Closed
s0undt3ch opened this issue May 11, 2013 · 2 comments
Closed

Salt Master Getting Hammered by Auth calls? #4985

s0undt3ch opened this issue May 11, 2013 · 2 comments
Milestone
0.16.0

Comments

@s0undt3ch
Copy link
Member

Notice the date stamps from the logs.

2013-05-11 11:43:42,427 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:42,428 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:42,428 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:42,428 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:42,455 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:42,455 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:42,456 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:42,457 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:42,483 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:42,484 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:42,484 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:42,485 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:42,512 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:42,512 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:42,512 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:42,513 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:42,538 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:42,538 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:42,539 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:42,539 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:42,566 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:42,566 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:42,567 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:42,567 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:42,593 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:42,593 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:42,594 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:42,594 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:42,639 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:42,640 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:42,640 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:42,641 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:42,668 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:42,669 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:42,670 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:42,670 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:42,700 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:42,701 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:42,701 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:42,702 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:42,755 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:42,755 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:42,756 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:42,756 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:42,785 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:42,785 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:42,786 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:42,786 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:42,821 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:42,822 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:42,823 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:42,823 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:42,853 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:42,854 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:42,854 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:42,855 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:42,883 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:42,884 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:42,884 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:42,885 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:42,939 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:42,940 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:42,940 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:42,941 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:42,977 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:42,978 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:42,979 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:42,979 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,027 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:43,027 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:43,028 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,028 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,060 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:43,061 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:43,062 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,063 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,119 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:43,120 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:43,121 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,121 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,150 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:43,151 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:43,152 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,152 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,232 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:43,233 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:43,233 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,234 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,262 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:43,263 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:43,263 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,264 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,292 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:43,292 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:43,293 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,294 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,324 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:43,325 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:43,326 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,326 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,357 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:43,357 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:43,358 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,359 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,386 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:43,386 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:43,387 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,387 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,435 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:43,436 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:43,437 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,437 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,468 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:43,468 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:43,469 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,469 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,500 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:43,501 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:43,501 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,502 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,534 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:43,534 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:43,535 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,535 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,583 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:43,584 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:43,585 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,586 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,668 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:43,669 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:43,670 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,671 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,746 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:43,747 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:43,748 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,749 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,816 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:43,817 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:43,817 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,818 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,850 [salt.master      ][INFO    ] Clear payload received with command _auth
2013-05-11 11:43:43,851 [salt.utils.verify][DEBUG   ] This salt-master instance has accepted 0 minion keys.
2013-05-11 11:43:43,851 [salt.master      ][INFO    ] Authentication request from ubuntu-13.04-amd64-buildbot-rnd1888
2013-05-11 11:43:43,852 [salt.master      ][INFO    ] Authentication failed from host ubuntu-13.04-amd64-buildbot-rnd1888, the key is in pending and needs to be accepted with salt-key -a ubuntu-13.04-amd64-buildbot-rnd1888

The master processes cpu usage is quite low, between 1-5%, so this is more a question/elucidation than a problematic bug
@thatch45
Copy link
Member

I saw this recently, I think that an SAuth loop is missing a sleep somewhere, probably has to do with the changes in timeout and safe being added.

@s0undt3ch
Copy link
Member Author

Nice! 😄

thatch45 added a commit that referenced this issue May 17, 2013
@thatch45 @basepi
fix #4985
6b63e72
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
0.16.0
Development

No branches or pull requests
2 participants
@s0undt3ch @thatch45