Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Uptime] Specify alert recovery context #128761

Closed
dominiqueclarke opened this issue Mar 29, 2022 · 4 comments · Fixed by #128693
Closed

[Uptime] Specify alert recovery context #128761

dominiqueclarke opened this issue Mar 29, 2022 · 4 comments · Fixed by #128693
Assignees
Labels
enhancement New value added to drive a business result Team:Uptime - DEPRECATED Synthetics & RUM sub-team of Application Observability v8.3.0

Comments

@dominiqueclarke
Copy link
Contributor

dominiqueclarke commented Mar 29, 2022

The alerting framework has recently merged a PR to provide rule executors with the ability to specify context for recovered alerts. Details are available in the PR and in the alerting README but essentially,

  1. Rule types should set doesSetRecoveryContext to true on the rule type. This allows context variables to appear in the action variable list in the UI for the recovered action group.
  2. Inside the rule executor, when the executor has finished scheduling actions for active alerts, the following code should be added:
// Call done() to gain access to recovery utils
// If `doesSetRecoveryContext` is set to `false`, getRecoveredAlerts() returns an empty list
const { getRecoveredAlerts } = alertsFactory.done();

for (const alert of getRecoveredAlerts()) {
	const alertId = alert.getId();
	alert.setContext({
		CONTEXT HERE!
	})
}

Note that currently, we expect the recovered context to match the existing typing for context variables on the rule.

Recovery defaults

Alert type Default Recovery message
Monitor status Alert for monitor {monitorName} with url {monitorUrl} from {observerLocation} has recovered
TLS Alert for TLS certificate {commonName} from issuer {issuer} has recovered
Duration Anomaly Alert for abnormal ({severity} level) response time detected on monitor {monitorId} with url {monitorUrl} at {anomalyStartTimestamp} has recovered

Related to: #126280

@dominiqueclarke dominiqueclarke added enhancement New value added to drive a business result Team:Uptime - DEPRECATED Synthetics & RUM sub-team of Application Observability labels Mar 29, 2022
@dominiqueclarke dominiqueclarke self-assigned this Apr 15, 2022
@dominiqueclarke
Copy link
Contributor Author

dominiqueclarke commented Apr 27, 2022

@colleenmcginnis @drewpost @paulb-elastic

Following #130925, for the first time we'll be able to specify default alert recovery messages for all alert types! 🥳

Could I get your thoughts on the content for these default recovery messages? Users can adjust these messages to fit their needs, but the default message will be displayed as the placeholder. Please let me know if you need any more information.

Alert type Default Recovery message
Monitor status Alert for monitor {monitorName} with url {monitorUrl} from {observerLocation} has recovered.
TLS Alert for TLS certificate {commonName} from issuer {issuer} has recovered.
Duration Anomaly Alert for abnormal ({severity} level) response time detected on {monitor} with url {monitorUrl} at {anomalyStartTimestamp} has recovered.

@colleenmcginnis
Copy link
Contributor

The format seems to be aligned with other default messages for alerts. 👍 Just one thing stands out to me: would it be helpful to use monitor {monitor} in the duration anomaly message like you did in monitor status with monitor {monitorName}? I'm not sure if monitor = monitorName, but if the string that monitor resolves to doesn't necessarily indicate that it is a monitor, then having that extra keyword might be helpful.

@paulb-elastic
Copy link
Contributor

Default messages LGTM (thanks @colleenmcginnis for picking up and asking about monitor vs monitorName)

@justinkambic
Copy link
Contributor

Post-FF testing LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New value added to drive a business result Team:Uptime - DEPRECATED Synthetics & RUM sub-team of Application Observability v8.3.0
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants