Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix up pusher logging a bit #4716

Merged
merged 5 commits into from Feb 22, 2019
Merged
Diff settings

Always

Just for now

Copy path View file
@@ -0,0 +1 @@
Reduce pusher logging on startup
Copy path View file
@@ -97,6 +97,11 @@ def __init__(self, hs, pusherdict):
pusherdict['pushkey'],
)

if self.data is None:
raise PusherConfigException(

This comment has been minimized.

Copy link
@richvdh

richvdh Feb 22, 2019

Member

will we actually log any less for a PusherConfigException than for a TypeError or whatever it was?

This comment has been minimized.

Copy link
@erikjohnston

erikjohnston Feb 22, 2019

Author Member

Woops, I missed a code path where it wasn't handled.

"data can not be null for HTTP pusher"
)

if 'url' not in self.data:
raise PusherConfigException(
"'url' required in data for HTTP pusher"
Copy path View file
@@ -56,7 +56,7 @@ def create_pusher(self, pusherdict):
f = self.pusher_types.get(kind, None)
if not f:
return None
logger.info("creating %s pusher for %r", kind, pusherdict)
logger.debug("creating %s pusher for %r", kind, pusherdict)

This comment has been minimized.

Copy link
@richvdh

richvdh Feb 22, 2019

Member

I'm not entirely convinced by this. it's the only thing that gets logged during minutes of cpu-spinning, and it's not massively high-volume.

This comment has been minimized.

Copy link
@erikjohnston

erikjohnston Feb 22, 2019

Author Member

It's logging several times a millisecond with a reasonably a non-trivial dict, which feels a bit OTT. There are ~170K pushers, with a log length of ~600 characters, means about 102MB of logs

I could make it so it logs in batches the progress?

This comment has been minimized.

Copy link
@richvdh

richvdh Feb 22, 2019

Member

fair enough. let's see how it looks.

return f(self.hs, pusherdict)

def _create_email_pusher(self, _hs, pusherdict):
ProTip! Use n and p to navigate between commits in a pull request.
You can’t perform that action at this time.