Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(api): expose alert fingerprint in the API response #1913

Merged
merged 4 commits into from
Jun 30, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
16 changes: 16 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,8 @@ see [CONFIGURATION.md](docs/CONFIGURATION.md#alertmanagers).

### Alert visualization

#### Alert groups

Alerts are displayed grouped preserving
[group_by](https://prometheus.io/docs/alerting/configuration/#route)
configuration option in Alertmanager.
Expand All @@ -55,6 +57,20 @@ and annotations that are shared between all alerts are moved to the footer.

![Example](/docs/img/alertGroup.png)

#### Inhibited alerts

Inhibited alerts (suppressed by other alerts,
[see Alertmanager docs](https://prometheus.io/docs/alerting/latest/alertmanager/#inhibition))
will have a "muted" button.

![Inhibited alert](/docs/img/inhibited.png)

Clicking on that button will bring a modal with a list of inhibiting alerts.

![Inhibiting alerts](/docs/img/inhibitedByModal.png)

#### Silence deduplication

If all alerts in a group were suppressed by the same silence then, to save
screen space, the silence will also be moved to the footer.

Expand Down
Binary file modified docs/img/alertGroup.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/img/dark.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/img/footerSilence.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/img/inhibited.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/img/inhibitedByModal.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/img/multiGrid.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/img/overview.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/img/screenshot.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/img/silenceBrowser.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions internal/alertmanager/models.go
Original file line number Diff line number Diff line change
Expand Up @@ -244,6 +244,7 @@ func (am *Alertmanager) pullAlerts(version string) error {

alert.Alertmanager = []models.AlertmanagerInstance{
{
Fingerprint: alert.Fingerprint,
Name: am.Name,
Cluster: am.ClusterName(),
State: alert.State,
Expand Down
39 changes: 39 additions & 0 deletions internal/filters/filter_fingerprint.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
package filters

import (
"fmt"

"github.com/prymitive/karma/internal/models"
)

type fingerprintFilter struct {
alertFilter
}

func (filter *fingerprintFilter) Match(alert *models.Alert, matches int) bool {
if filter.IsValid {
var isMatch bool
for _, am := range alert.Alertmanager {
m := filter.Matcher.Compare(am.Fingerprint, filter.Value)
if m {
isMatch = m
}
}
if isMatch {
filter.Hits++
}
return isMatch
}
e := fmt.Sprintf("Match() called on invalid filter %#v", filter)
panic(e)
}

func (filter *fingerprintFilter) MatchAlertmanager(am *models.AlertmanagerInstance) bool {
return filter.Matcher.Compare(am.Fingerprint, filter.Value)
}

func newFingerprintFilter() FilterT {
f := fingerprintFilter{}
f.IsAlertmanagerFilter = true
return &f
}
79 changes: 78 additions & 1 deletion internal/filters/filter_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,83 @@ var tests = []filterTest{
IsValid: false,
},

{
Expression: "@fingerprint=",
IsValid: false,
},
{
Expression: "@fingerprint==",
IsValid: false,
},
{
Expression: "@fingerprint<=active",
IsValid: false,
},
{
Expression: "@fingerprint=123",
IsValid: true,
Alert: models.Alert{},
IsMatch: false,
IsAlertmanagerMatch: true,
},
{
Expression: "@fingerprint!=123",
IsValid: true,
Alert: models.Alert{},
IsMatch: true,
IsAlertmanagerMatch: true,
},
{
Expression: "@fingerprint=1234",
IsValid: true,
Alert: models.Alert{},
Alertmanagers: []models.AlertmanagerInstance{
{Fingerprint: "1234"},
},
IsMatch: true,
IsAlertmanagerMatch: true,
},
{
Expression: "@fingerprint=~123",
IsValid: true,
Alert: models.Alert{},
Alertmanagers: []models.AlertmanagerInstance{
{Fingerprint: "01234"},
},
IsMatch: true,
IsAlertmanagerMatch: true,
},
{
Expression: "@fingerprint=abc",
IsValid: true,
Alert: models.Alert{},
Alertmanagers: []models.AlertmanagerInstance{
{Fingerprint: "12345"},
},
IsMatch: false,
IsAlertmanagerMatch: false,
},
{
Expression: "@fingerprint!=1a1",
IsValid: true,
Alert: models.Alert{},
Alertmanagers: []models.AlertmanagerInstance{
{Fingerprint: "1a1"},
},
IsMatch: false,
IsAlertmanagerMatch: false,
},
{
Expression: "@fingerprint!=cde",
IsValid: true,
Alert: models.Alert{},
Alertmanagers: []models.AlertmanagerInstance{
{Fingerprint: "abc"},
},
IsMatch: true,
IsAlertmanagerMatch: true,
},

{
Expression: "@silence_id=abcdef",
IsValid: true,
Expand Down Expand Up @@ -699,7 +776,7 @@ func TestFilters(t *testing.T) {
}
if f.GetIsValid() {
isAlertmanagerFilter := slices.StringInSlice(
[]string{"@age", "@alertmanager", "@cluster", "@state", "@silence_id", "@silence_ticket", "@silence_author"},
[]string{"@age", "@alertmanager", "@cluster", "@state", "@silence_id", "@silence_ticket", "@silence_author", "@fingerprint"},
f.GetName())
if isAlertmanagerFilter != f.GetIsAlertmanagerFilter() {
t.Errorf("[%s] GetIsAlertmanagerFilter() returned %#v while %#v was expected", ft.Expression, f.GetIsAlertmanagerFilter(), isAlertmanagerFilter)
Expand Down
6 changes: 6 additions & 0 deletions internal/filters/registry.go
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,12 @@ var AllFilters = []filterConfig{
Factory: newStateFilter,
Autocomplete: stateAutocomplete,
},
{
Label: "@fingerprint",
LabelRe: regexp.MustCompile("^@fingerprint$"),
SupportedOperators: []string{regexpOperator, equalOperator, notEqualOperator},
Factory: newFingerprintFilter,
},
{
Label: "@receiver",
LabelRe: regexp.MustCompile("^@receiver$"),
Expand Down
1 change: 1 addition & 0 deletions internal/mapper/v017/api.go
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ func groups(c *client.Alertmanager, timeout time.Duration) ([]models.AlertGroup,
}
for _, alert := range group.Alerts {
a := models.Alert{
Fingerprint: *alert.Fingerprint,
Receiver: *group.Receiver.Name,
Annotations: models.AnnotationsFromMap(alert.Annotations),
Labels: alert.Labels,
Expand Down
1 change: 1 addition & 0 deletions internal/models/alert.go
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ type Alert struct {
// those are not exposed in JSON, Alertmanager specific value will be in kept
// in the Alertmanager slice
// skip those when generating alert fingerprint too
Fingerprint string `json:"-" hash:"-"`
GeneratorURL string `json:"-" hash:"-"`
SilencedBy []string `json:"-" hash:"-"`
InhibitedBy []string `json:"-" hash:"-"`
Expand Down
5 changes: 3 additions & 2 deletions internal/models/alertmanager.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,9 @@ import "time"
// AlertmanagerInstance describes the Alertmanager instance alert was collected
// from
type AlertmanagerInstance struct {
Name string `json:"name"`
Cluster string `json:"cluster"`
Fingerprint string `json:"fingerprint"`
Name string `json:"name"`
Cluster string `json:"cluster"`
// per instance alert state
State string `json:"state"`
// timestamp collected from this instance, those on the alert itself
Expand Down
24 changes: 16 additions & 8 deletions internal/models/api_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,10 @@ import (

func TestDedupSharedMaps(t *testing.T) {
am := models.AlertmanagerInstance{
Cluster: "fakeCluster",
SilencedBy: []string{"fakeSilence1", "fakeSilence2"},
Fingerprint: "1",
Name: "am",
Cluster: "fakeCluster",
SilencedBy: []string{"fakeSilence1", "fakeSilence2"},
}
ag := models.APIAlertGroup{
AlertGroup: models.AlertGroup{
Expand Down Expand Up @@ -98,7 +100,8 @@ func TestDedupSharedMaps(t *testing.T) {
"state": "suppressed",
"alertmanager": [
{
"name": "",
"fingerprint": "1",
"name": "am",
"cluster": "fakeCluster",
"state": "",
"startsAt": "0001-01-01T00:00:00Z",
Expand All @@ -110,7 +113,8 @@ func TestDedupSharedMaps(t *testing.T) {
"inhibitedBy": null
},
{
"name": "",
"fingerprint": "1",
"name": "am",
"cluster": "fakeCluster",
"state": "",
"startsAt": "0001-01-01T00:00:00Z",
Expand All @@ -135,7 +139,8 @@ func TestDedupSharedMaps(t *testing.T) {
"state": "suppressed",
"alertmanager": [
{
"name": "",
"fingerprint": "1",
"name": "am",
"cluster": "fakeCluster",
"state": "",
"startsAt": "0001-01-01T00:00:00Z",
Expand All @@ -147,7 +152,8 @@ func TestDedupSharedMaps(t *testing.T) {
"inhibitedBy": null
},
{
"name": "",
"fingerprint": "1",
"name": "am",
"cluster": "fakeCluster",
"state": "",
"startsAt": "0001-01-01T00:00:00Z",
Expand All @@ -172,7 +178,8 @@ func TestDedupSharedMaps(t *testing.T) {
"state": "suppressed",
"alertmanager": [
{
"name": "",
"fingerprint": "1",
"name": "am",
"cluster": "fakeCluster",
"state": "",
"startsAt": "0001-01-01T00:00:00Z",
Expand All @@ -184,7 +191,8 @@ func TestDedupSharedMaps(t *testing.T) {
"inhibitedBy": null
},
{
"name": "",
"fingerprint": "1",
"name": "am",
"cluster": "fakeCluster",
"state": "",
"startsAt": "0001-01-01T00:00:00Z",
Expand Down
1 change: 1 addition & 0 deletions ui/src/Common/Query.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ const StaticLabels = Object.freeze({
AlertName: "alertname",
AlertManager: "@alertmanager",
AlertmanagerCluster: "@cluster",
Fingerprint: "@fingerprint",
Receiver: "@receiver",
State: "@state",
SilenceID: "@silence_id",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ exports[`<Alert /> matches snapshot when inhibited 1`] = `
<div style=\\"display: inline-block; max-width: 100%;\\"
class=\\" tooltip-trigger\\"
>
<span class=\\"badge badge-light components-label\\">
<span class=\\"badge badge-light components-label components-label-with-hover cursor-pointer\\">
<svg aria-hidden=\\"true\\"
focusable=\\"false\\"
data-prefix=\\"fas\\"
Expand Down
21 changes: 8 additions & 13 deletions ui/src/Components/Grid/AlertGrid/AlertGroup/Alert/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,16 +3,13 @@ import PropTypes from "prop-types";

import { useObserver } from "mobx-react-lite";

import { FontAwesomeIcon } from "@fortawesome/react-fontawesome";
import { faVolumeMute } from "@fortawesome/free-solid-svg-icons/faVolumeMute";

import { APIAlert, APIGroup } from "Models/API";
import { AlertStore } from "Stores/AlertStore";
import { SilenceFormStore } from "Stores/SilenceFormStore";
import { BorderClassMap } from "Common/Colors";
import { StaticLabels } from "Common/Query";
import { FilteringLabel } from "Components/Labels/FilteringLabel";
import { TooltipWrapper } from "Components/TooltipWrapper";
import { InhibitedByModal } from "Components/InhibitedByModal";
import { RenderNonLinkAnnotation, RenderLinkAnnotation } from "../Annotation";
import { AlertMenu } from "./AlertMenu";
import { RenderSilence } from "../Silences";
Expand All @@ -39,13 +36,15 @@ const Alert = ({

const silences = {};
let clusters = [];
let isInhibited = false;
let inhibitedBy = [];
for (const am of alert.alertmanager) {
if (!clusters.includes(am.cluster)) {
clusters.push(am.cluster);
}
if (am.inhibitedBy.length > 0) {
isInhibited = true;
for (const fingerprint of am.inhibitedBy) {
if (!inhibitedBy.includes(fingerprint)) {
inhibitedBy.push(fingerprint);
}
}
if (!silences[am.cluster]) {
silences[am.cluster] = {
Expand Down Expand Up @@ -87,12 +86,8 @@ const Alert = ({
silenceFormStore={silenceFormStore}
setIsMenuOpen={setIsMenuOpen}
/>
{isInhibited ? (
<TooltipWrapper title="This alert is inhibited by other alerts">
<span className="badge badge-light components-label">
<FontAwesomeIcon className="text-success" icon={faVolumeMute} />
</span>
</TooltipWrapper>
{inhibitedBy.length > 0 ? (
<InhibitedByModal alertStore={alertStore} fingerprints={inhibitedBy} />
) : null}
{Object.entries(alert.labels).map(([name, value]) => (
<FilteringLabel
Expand Down
17 changes: 17 additions & 0 deletions ui/src/Components/Grid/AlertGrid/AlertGroup/Alert/index.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,23 @@ describe("<Alert />", () => {
});

it("renders inhibition icon when inhibited", () => {
const alert = MockedAlert();
alert.alertmanager[0].inhibitedBy = ["123456"];
alert.alertmanager.push({
name: "ha2",
cluster: "HA",
state: "active",
startsAt: "2018-08-14T17:36:40.017867056Z",
source: "localhost/prometheus",
silencedBy: [],
inhibitedBy: ["123456"],
});
const group = MockAlertGroup({}, [alert], [], {}, {});
const tree = MountedAlert(alert, group, false, false);
expect(tree.find(".fa-volume-mute")).toHaveLength(1);
});

it("inhibition icon passes only unique fingerprints", () => {
const alert = MockedAlert();
alert.alertmanager[0].inhibitedBy = ["123456"];
const group = MockAlertGroup({}, [alert], [], {}, {});
Expand Down