Fix rate limit metrics registering twice and misreporting

[MadLittleMods]

Fix rate limit gauge metrics registering twice and misreporting by registering once and adding a label to differentiate the instances (although we only track the federation servlets at the moment). The gauge metrics were originally added in #13541

FederationRateLimiter is being instantiated twice in Synapse:

1. UsernameAvailabilityRestServlet which uses it to rate limit people by IP
2. For the Federation servlets which FederationRateLimiter was made for

Fix #13641

Dev notes

https://github.com/matrix-org/synapse/blob/c4e29b6908ac8ae57b5e9a3e7662ad638b61e94a/docs/metrics-howto.md

docker run \
    -p 9090:9090 \
    -v /Users/eric/Documents/github/element/.configs/prometheus.yml:/etc/prometheus/prometheus.yml \
    prom/prometheus

prometheus.yml

global:
  scrape_interval:     15s # By default, scrape targets every 15 seconds.

scrape_configs:
  - job_name: "synapse"
    scrape_interval: 15s
    metrics_path: "/_synapse/metrics"
    static_configs:
      - targets: ["host.docker.internal:8008"]

Pull Request Checklist

• Pull request is based on the develop branch
• Pull request includes a changelog file. The entry should:
  • Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from EventStore to EventWorkerStore.".
  • Use markdown where necessary, mostly for code blocks.
  • End with either a period (.) or an exclamation mark (!).
  • Start with a capital letter.
  • Feel free to credit yourself, by adding a sentence "Contributed by @github_username." or "Contributed by [Your Name]." to the end of the entry.
• Pull request includes a sign off
• Code style is correct
  (run the linters)

[MadLittleMods]

I've made the label rate_limiter_name instead of name because name seemed pretty ambiguous in the context of synapse_rate_limit_sleep_affected_hosts. Trying to clarify the question "name of what?"

[MadLittleMods]

This is the same pattern as

synapse/synapse/http/request_metrics.py

Lines 110 to 142 in c4e29b6

	_in_flight_requests: Set["RequestMetrics"] = set()
	# Protects the _in_flight_requests set from concurrent access
	_in_flight_requests_lock = threading.Lock()
	def _get_in_flight_counts() -> Mapping[Tuple[str, ...], int]:
	"""Returns a count of all in flight requests by (method, server_name)"""
	# Cast to a list to prevent it changing while the Prometheus
	# thread is collecting metrics
	with _in_flight_requests_lock:
	reqs = list(_in_flight_requests)
	for rm in reqs:
	rm.update_metrics()
	# Map from (method, name) -> int, the number of in flight requests of that
	# type. The key type is Tuple[str, str], but we leave the length unspecified
	# for compatability with LaterGauge's annotations.
	counts: Dict[Tuple[str, ...], int] = {}
	for rm in reqs:
	key = (rm.method, rm.name)
	counts[key] = counts.get(key, 0) + 1
	return counts
	LaterGauge(
	"synapse_http_server_in_flight_requests_count",
	"",
	["method", "servlet"],
	_get_in_flight_counts,
	)

[MadLittleMods]

Slightly worried about context issues here again just because it's going a bit off the beaten path (please double-check) 😰

I've made metrics_name optional because I'm not concerned about tracking the UsernameAvailabilityRestServlet which is used to rate limit people by IP. I only care about tracking the federation servlets which FederationRateLimiter was made for.

We could make metrics_name required and record metrics for the username availability servlet to get rid of this complexity though.

[richvdh]

lgtm otherwise

[DMRobertson]

How about this @richvdh @MadLittleMods?

[richvdh]

@DMRobertson: sgtm. Shall we just merge that and get an RC out?

[DMRobertson]

Let's do it.

@DMRobertson: sgtm. Shall we just merge that and get an RC out?

[DMRobertson]

Given that [b077d81](https://github.com/matrix-org/synapse/pull/13649/commits/b077d815e95da208ed981338793ded9d21c2e644) only changes docstrings and that its parent passed CI, I am going to merge without CI.

[DMRobertson]

I sanity checked this by grepping for "already registered" in logs. No matches from today.

[MadLittleMods]

Thanks for the review @richvdh and @DMRobertson for making the changes to make this mergeable and confirming the duplicate registering message wasn't in the logs 🐸