Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ignore unexpected metric messages #28

Open
bmario opened this issue Dec 8, 2021 · 0 comments
Open

Ignore unexpected metric messages #28

bmario opened this issue Dec 8, 2021 · 0 comments
Labels
enhancement New feature or request

Comments

@bmario
Copy link
Member

bmario commented Dec 8, 2021

In conjunction with metricq/metricq-manager#39, we could (and should) ignore unexpected messages, as the manager and the DB are - in theory - on the same page. Thus the configuration should always be okay. And under normal operation, It works fine.

However, there is this check.

Image the following situation:

  • You remove metric M from the DB config
  • You restart the DB to apply the change, as you need to
  • During the restart, the manager reconfigures the queue, but there is at least one message for metric M in the queue
  • During the DB restart, the DB receives this one message for M
  • M is no longer in the config for the HTA::Directory, so that will raise an exception, which gets caught and ignored. Fine, but this message also creates an entry in the mapped_metrics_ entry because, for the wildcard DB config, we need to add mappings on the fly.
  • Now, you realize, you didn't mean to remove the metric M
  • So you add back the config entry for M
  • You send a reconfigure RPC to the DB
  • The DB performs the reconfiguration, but look again at the linked check from above. The metric M already has an entry in the mapped_metrics_ member. So its configuration never gets added to the HTA::Directory.
  • Now, the messages for metric M will get ignored, as there is no configuration for it in the HTA::Directory

TL;DR Meme:
The meme police didn't approve this meme, yet

@bmario bmario added the enhancement New feature or request label Dec 8, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant