Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

synapse_rate_limit_sleep_affected_hosts registered twice #13641

Closed
richvdh opened this issue Aug 26, 2022 · 2 comments · Fixed by #13649
Closed

synapse_rate_limit_sleep_affected_hosts registered twice #13641

richvdh opened this issue Aug 26, 2022 · 2 comments · Fixed by #13649
Assignees
@MadLittleMods
Labels
A-Metrics metrics, measures, stuff we put in Prometheus T-Defect Bugs, crashes, hangs, security vulnerabilities, or other reported issues. X-Regression Something broke which worked on a previous release X-Release-Blocker Must be resolved before making a release

Comments

@richvdh
Copy link
Member

richvdh commented Aug 26, 2022 •
edited by MadLittleMods

this is typically a sign that metrics will not be reported correctly:

2022-08-26 11:49:42,388 - synapse.metrics - 117 - WARNING - sentinel - synapse_rate_limit_sleep_affected_hosts already registered, reregistering
2022-08-26 11:49:42,388 - synapse.metrics - 117 - WARNING - sentinel - synapse_rate_limit_reject_affected_hosts already registered, reregistering

These things were added in #13534 #13541.
@richvdh
Copy link
Member Author

richvdh commented Aug 26, 2022

cc @MadLittleMods

@richvdh richvdh added X-Release-Blocker Must be resolved before making a release X-Regression Something broke which worked on a previous release labels Aug 26, 2022
@MadLittleMods
Copy link
Contributor

MadLittleMods commented Aug 26, 2022 •
edited

They were added in #13541 and are using the LaterGauge so we are probably creating the FederationRateLimiter twice somewhere 🤔

Probably related exactly to one of them evaluating to 0 all the time, #13541 (comment)

@MadLittleMods MadLittleMods changed the title synapse_rate_limit_sleep_affected_hosts registered twice synapse_rate_limit_sleep_affected_hosts registered twice Aug 26, 2022
@MadLittleMods MadLittleMods self-assigned this Aug 26, 2022
@MadLittleMods MadLittleMods added T-Defect Bugs, crashes, hangs, security vulnerabilities, or other reported issues. A-Metrics metrics, measures, stuff we put in Prometheus labels Aug 26, 2022
MadLittleMods added a commit that referenced this issue Aug 26, 2022
@MadLittleMods
Fix rate limit metrics registering twice and misreporting
b99829a 
Fix #13641
This was referenced Aug 26, 2022
Merged
Merged
DMRobertson pushed a commit that referenced this issue Aug 30, 2022
@MadLittleMods
Fix rate limit metrics registering twice and misreporting
8be5d7c 
Fix #13641
@MadLittleMods MadLittleMods mentioned this issue Aug 30, 2022
Closed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@MadLittleMods MadLittleMods

Labels
A-Metrics metrics, measures, stuff we put in Prometheus T-Defect Bugs, crashes, hangs, security vulnerabilities, or other reported issues. X-Regression Something broke which worked on a previous release X-Release-Blocker Must be resolved before making a release
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix rate limit metrics registering twice and misreporting matrix-org/synapse
2 participants
@MadLittleMods @richvdh