Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reconciliation Loop for Prometheus Converter #869

Closed
mirondeldev opened this issue Feb 13, 2024 · 3 comments
Closed

Reconciliation Loop for Prometheus Converter #869

mirondeldev opened this issue Feb 13, 2024 · 3 comments
Labels
enhancement New feature or request

Comments

@mirondeldev
Copy link

mirondeldev commented Feb 13, 2024

Current behavior of the vmprometheusconverter rely on Prometheus CRD create and update events, e.g.:

c.ruleInf.AddEventHandler(cache.ResourceEventHandlerFuncs{
AddFunc: c.CreatePrometheusRule,
UpdateFunc: c.UpdatePrometheusRule,

In some cases (like manual or circumstantial deletion of custom resources) it leads to "orphaned" Prometheus resources.

So, the problem is VictoriaMetrics Operator does not recreate converted VM-resources after ones being deleted

Steps to reproduce

  • Create Prometheus resource, e.g. PodMonitor
    • Operator creates corresponding PodScraper resource
  • Delete created PodScraper resource and keep original Prometheus PodMonitor
    • PodScraper resource will not be created until PodMonitor will be updated or recreated

Would you please consider implementing reconciliation loop for Prometheus resources in vmprometheusconverter?

@f41gh7 f41gh7 added the enhancement New feature or request label Feb 22, 2024
@f41gh7
Copy link
Collaborator

f41gh7 commented Feb 22, 2024

Hello, it makes sense to me. And it'd be great to have an option to enable periodical reconciliation of prometheus watched objects.

f41gh7 added a commit that referenced this issue Mar 6, 2024
allow to configure resync period for prometheus CRD converter with flag controller.prometheusCRD.resyncPeriod
#869
@f41gh7
Copy link
Collaborator

f41gh7 commented Mar 6, 2024

Added new flag for period reconciliation at v0.42.2 release

It accepts duration format, -controller.prometheusCRD.resyncPeriod=10m.

By default, it's disabled.

@f41gh7 f41gh7 closed this as completed Mar 6, 2024
@mirondeldev
Copy link
Author

@f41gh7 thx, much appreciated

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants