apiserver: rate-limit logsink receives #7474
Conversation
axw
force-pushed
the
logsink-ratelimit-2.2
branch
2 times, most recently
from
d187a52
to
e760375
Compare
api/apiclient_test.go
Outdated
| LogSink: apiserver.LogSinkConfig{ | ||
| RateLimitBurst: apiserver.DefaultLogSinkRateLimitBurst, | ||
| RateLimitRefill: apiserver.DefaultLogSinkRateLimitRefill, | ||
| }, |
There was a problem hiding this comment.
It might be nicer if LogSink was a pointer. Then if it's nil then the defaults could be used. This might be preferable to having to specify the defaults all over the place.
There was a problem hiding this comment.
I've done that in my other PR that makes the db logging parameters configurable. I'll rebase this once that has landed.
Pretty arbitrarily; 1000 log messages/s per agent seems like quite a lot already to me. I tested this with canonical-kubernetes, and it didn't cause any backing up. I'll put it in the cards to do some scale testing before we release 2.2.1. |
axw
force-pushed
the
logsink-ratelimit-2.2
branch
from
e760375
to
ec2fff3
Compare
|
|
|
Status: merge request accepted. Url: http://juju-ci.vapour.ws:8080/job/github-merge-juju |
|
Build failed: Generating tarball failed |
Update the logsink handler so that it will rate-limit receipt of log messages. Rate-limiting defaults to a burst of 1000 messages, after which we allow 1 message per millisecond. This is configurable by modifying the controller agent's agent.conf. If we find that users need to configure it regularly, we can promote the configuration to controller config.
axw
force-pushed
the
logsink-ratelimit-2.2
branch
from
ec2fff3
to
b979bca
Compare
|
|
|
Status: merge request accepted. Url: http://juju-ci.vapour.ws:8080/job/github-merge-juju |
Description of change
Update the logsink handler so that it will
rate-limit receipt of log messages. Rate-limiting
defaults to a burst of 1000 messages, after
which we allow 1 message per millisecond. This
is configurable by modifying the controller
agent's agent.conf. If we find that users
need to configure it regularly, we can promote
the configuration to controller config.
QA steps
Observe that the logs come in quickly.
LOGSINK_RATELIMIT_REFILL: 1sundervalues.Observe that the log messages come in quickly to begin with (up to 1000 messages), and then they are limited to 1 per second.
Documentation changes
Probably not until we know that people need to tweak the defaults, at which point we can add controller-config.
Bug reference
None.