Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide a way to insert random uuidv4 in alert definition. #4710

Open
Toriniasty opened this Issue Oct 9, 2018 · 2 comments

Comments

Projects
None yet
2 participants
@Toriniasty
Copy link

Toriniasty commented Oct 9, 2018

Proposal

Use case. Why is this important?
We have a 4 non clustered alartmanagers setup with custom written http webhook server.

We have also custom written clients(except prometheus) which submit payload via alertmanager api, but when they do we insert there uuidv4 to deduplicate the messages in our mysql instance by providing unique key on the column.

Where the setup works perfectly fine for the custom written clients we struggle a bit with alerts coming from Prometheus where we aren't able to do that since alertmanagers do not know about the message on other nodes.

Ideally we would like to see an option/variable which we would be able to put in alert definition which would be then automatically substituted by uuidv4.

@simonpasquier

This comment has been minimized.

Copy link
Member

simonpasquier commented Oct 9, 2018

Not sure that I fully understand what you're looking for so let me try to rephrase your use case. When an alert is firing in Prometheus, you want to add a label with a random value that will remain constant as long as the alert is active. If the same alert gets resolved and fires again then you want the label value to be regenerated. Is that correct?

There's no way I know that it would work out of the box and your use case seems very specific to be relevant for the community. I would recommend that you extend your webhook receiver to manage the insertion of the UUID value instead.

Finally I would recommend you to use our user mailing list where you will reach out a broader audience.

@Toriniasty

This comment has been minimized.

Copy link
Author

Toriniasty commented Oct 9, 2018

Not sure that I fully understand what you're looking for so let me try to rephrase your use case. When an alert is firing in Prometheus, you want to add a label with a random value that will remain constant as long as the alert is active. If the same alert gets resolved and fires again then you want the label value to be regenerated. Is that correct?
Yes

Problem with insertion on uuid on the webhook server is that it runs alongside each alertmanager so the uuid generated would be different on each of the alertmanagers and again alert wouldn't be deduplicated.

I spoke to someone on Freenode channel some time ago and told me that's currently not possible and asked me to open this feature request

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.