Alerting: Export of alert rules in HCL format #73166

yuri-tceretian · 2023-08-10T17:12:32Z

What is this feature?
Adds export of alert rule groups in HCL format. It uses the hashicorp/hcl/v2 module to encode a struct to HCL resource. The changes in the export models are because file provisioning and terraform models are slightly different.

Demo

firefox_GUjTsGWL3K.mp4

Why do we need this feature?
To let users export resources in format that is acceptable by Grafana Terraform Provider.

Who is this feature for?
Those who use Terraform to provision alerting resources.

Which issue(s) does this PR fix?:

Related #72135

Special notes for your reviewer:
This PR implements only alert rules. Remaining resources will be addressed in the follow-up PRs.

Please check that:

It works as expected from a user's perspective.
If this is a pre-GA feature, it is behind a feature toggle.
The docs are updated, and if this is a notable improvement, it's added to our What's New doc.

soniaAguilarPeiron

LGTM from the FE side!🚀

konrad147

LGTM!

soniaAguilarPeiron · 2023-09-05T13:24:44Z

public/app/features/alerting/unified/components/export/providers.ts

+  exportFormat: 'hcl',
+};
+
+export const grafanaRuleExportProviders = {


soniaAguilarPeiron

LGTM! Nice work!

rwwiv

This accomplishes a lot with very little code added, awesome to see! Couple comments, mostly I'd love to hear how you feel about what would need to happen before this can be pulled out of alerting.

rwwiv · 2023-09-05T17:34:11Z

public/app/features/alerting/unified/components/rule-editor/AlertRuleForm.tsx

@@ -241,7 +241,7 @@ export const AlertRuleForm = ({ existing, prefill }: Props) => {
          disabled={submitState.loading}
          size="sm"
        >
-          {isCortexLokiOrRecordingRule(watch) ? 'Edit YAML' : 'View YAML'}
+          {isCortexLokiOrRecordingRule(watch) ? 'Edit YAML' : 'Export'}


This needs to be updated for the "View rule" page as well.

Tangentially, I feel like either this button or the title of the panel this opens need a new name.

rwwiv · 2023-09-05T19:05:40Z

pkg/services/ngalert/api/tooling/definitions/provisioning_alert_rules.go

+	RelativeTimeRange RelativeTimeRangeExport `json:"relativeTimeRange,omitempty" yaml:"relativeTimeRange,omitempty" hcl:"relative_time_range,block"`
+	DatasourceUID     string                  `json:"datasourceUid" yaml:"datasourceUid" hcl:"datasource_uid"`
+	Model             map[string]interface{}  `json:"model" yaml:"model"`
+	ModelString       string                  `json:"-" yaml:"-" hcl:"model"`


It's unfortunate there's not a better way to format this without a lot more work 😞 Generating a Heredoc or plain HCL wrapped in jsonencode would make the output a lot easier to read.

It will be possible when\if hashicorp/hcl#608 is merged.

public/app/features/alerting/unified/api/alertRuleApi.ts

rwwiv · 2023-09-05T19:28:02Z

pkg/services/ngalert/api/hcl/hcl_test.go

+	"github.com/stretchr/testify/require"
+)
+
+func TestEncode(t *testing.T) {


Could this test be simpler? We don't necessarily need to worry about testing gohcl.EncodeAsBlock or documenting how the field tags work, so we could reduce data to a single field.

I could not find a decent documentation about how the encoder works. So, I decided to write a test as a doc

rwwiv · 2023-09-05T22:30:20Z

pkg/services/ngalert/api/api_provisioning.go

@@ -517,3 +522,43 @@ func exportResponse(c *contextmodel.ReqContext, body definitions.AlertingFileExp
 	}
 	return r(http.StatusOK, body)
 }
+
+func exportHcl(download bool, body definitions.AlertingFileExport) response.Response {


This is probably fine for a POC but I'm worried about how we'd be able to generalize this in the future.

I think it will be possible when the approach to encoding a struct to different formats will use the same approach. I think if hashicorp/hcl#608 gets merged, we can unify this. For now, I think the separate function is more cleaner.

grobinson-grafana · 2023-09-07T10:03:29Z

pkg/services/ngalert/api/api_provisioning.go

+	}
+
+	// TODO implement support.
+	// for idx, cp := range ex.ContactPoints {


Yuri, are you planning to do this TODO before merging this feature or will it come later?

It will be addressed in a follow up PR

grobinson-grafana · 2023-09-07T10:04:15Z

pkg/services/ngalert/api/hcl/hcl.go

+
+func Encode(resources ...Resource) (data []byte, err error) {
+	defer func() {
+		if r := recover(); r != nil {


I like this, we use it in the new Alertmanager label matchers parser too.

yeah, Encoder is prone to panic :) when the struct has incorrect tags.

# Conflicts: # pkg/services/ngalert/api/tooling/definitions/provisioning_alert_rules.go # public/app/features/alerting/unified/api/alertRuleApi.ts # public/app/features/alerting/unified/components/export/providers.ts # public/app/features/alerting/unified/components/rule-editor/GrafanaRuleInspector.tsx

JacobsonMT

LGTM, let's also remember to update the swagger definition when this is all finished. Maybe with // enum: yaml,json,hcl in ExportQueryParams

* import hashicopr/hcl/v2 * add hcl package and export to HCL * annotate export structs --------- Co-authored-by: Konrad Lalik <konrad.lalik@grafana.com>

yuri-tceretian self-assigned this Aug 10, 2023

grafana-delivery-bot bot added this to the 10.2.x milestone Aug 10, 2023

grafana-pr-automation bot added the area/backend label Aug 10, 2023

yuri-tceretian force-pushed the yuri-tceretian/hcl-rule-export branch 2 times, most recently from cf2abca to 943f941 Compare August 10, 2023 17:35

grafana-pr-automation bot added the area/frontend label Aug 10, 2023

yuri-tceretian force-pushed the yuri-tceretian/hcl-rule-export branch from 943f941 to 65ada50 Compare August 10, 2023 18:30

yuri-tceretian added 4 commits August 29, 2023 14:46

import hashicopr/hcl/v2

027ad1b

add hcl package and export to HCL

2d9c0c8

annotate export structs

4f3735b

add UI

f0119e6

yuri-tceretian force-pushed the yuri-tceretian/hcl-rule-export branch from 65ada50 to f0119e6 Compare August 29, 2023 20:10

yuri-tceretian added 2 commits August 29, 2023 16:41

fixup

09a16cc

lint

1dac0da

yuri-tceretian added area/alerting Grafana Alerting area/provisioning add to changelog no-backport Skip backport of PR add to what's new labels Aug 29, 2023

yuri-tceretian marked this pull request as ready for review August 29, 2023 23:15

yuri-tceretian requested review from a team as code owners August 29, 2023 23:15

yuri-tceretian requested review from gillesdemey, VikaCep, konrad147, soniaAguilarPeiron and rwwiv and removed request for a team August 29, 2023 23:15

yuri-tceretian requested review from mildwonkey, suntala and yangkb09 August 29, 2023 23:15

yuri-tceretian added 2 commits August 29, 2023 19:27

fix formatting

fae2a88

fixup + tests

a1fe29d

soniaAguilarPeiron approved these changes Aug 30, 2023

View reviewed changes

konrad147 approved these changes Aug 30, 2023

View reviewed changes

Add support for any number of exporters

f1e5071

konrad147 requested review from soniaAguilarPeiron and konrad147 August 31, 2023 10:58

soniaAguilarPeiron reviewed Sep 5, 2023

View reviewed changes

soniaAguilarPeiron approved these changes Sep 5, 2023

View reviewed changes

rwwiv reviewed Sep 5, 2023

View reviewed changes

yuri-tceretian assigned konrad147 Sep 5, 2023

yuri-tceretian removed the request for review from konrad147 September 5, 2023 22:48

grobinson-grafana reviewed Sep 7, 2023

View reviewed changes

grobinson-grafana approved these changes Sep 7, 2023

View reviewed changes

yuri-tceretian added 2 commits September 8, 2023 15:18

use RuleExportFormats

d7440c4

JacobsonMT approved these changes Sep 11, 2023

View reviewed changes

yuri-tceretian merged commit dce4926 into main Sep 11, 2023
14 checks passed

yuri-tceretian deleted the yuri-tceretian/hcl-rule-export branch September 11, 2023 15:48

yuri-tceretian mentioned this pull request Sep 11, 2023

Alerting provisioning improvements #72135

Closed

rwwiv pushed a commit that referenced this pull request Oct 2, 2023

Alerting: Export of alert rules in HCL format (#73166)

76ddf95

* import hashicopr/hcl/v2 * add hcl package and export to HCL * annotate export structs --------- Co-authored-by: Konrad Lalik <konrad.lalik@grafana.com>

yuri-tceretian mentioned this pull request Oct 18, 2023

Docs: Add Alerting export to what's new #76775

Merged

zerok modified the milestones: 10.2.x, 10.2.0 Oct 23, 2023

dnhn mentioned this pull request Oct 24, 2023

grafana 10.2.0 Homebrew/homebrew-core#152264

Closed

BrewTestBot mentioned this pull request Oct 25, 2023

grafana 10.2.0 Homebrew/homebrew-core#152321

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Alerting: Export of alert rules in HCL format #73166

Alerting: Export of alert rules in HCL format #73166

yuri-tceretian commented Aug 10, 2023 •

edited

soniaAguilarPeiron left a comment

konrad147 left a comment

soniaAguilarPeiron Sep 5, 2023

soniaAguilarPeiron left a comment

rwwiv left a comment

rwwiv Sep 5, 2023

rwwiv Sep 5, 2023

yuri-tceretian Sep 6, 2023

rwwiv Sep 5, 2023

yuri-tceretian Sep 6, 2023

rwwiv Sep 5, 2023

yuri-tceretian Sep 6, 2023

grobinson-grafana Sep 7, 2023

yuri-tceretian Sep 7, 2023

grobinson-grafana Sep 7, 2023

yuri-tceretian Sep 7, 2023

JacobsonMT left a comment

Alerting: Export of alert rules in HCL format #73166

Alerting: Export of alert rules in HCL format #73166

Conversation

yuri-tceretian commented Aug 10, 2023 • edited

soniaAguilarPeiron left a comment

Choose a reason for hiding this comment

konrad147 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

soniaAguilarPeiron left a comment

Choose a reason for hiding this comment

rwwiv left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JacobsonMT left a comment

Choose a reason for hiding this comment

yuri-tceretian commented Aug 10, 2023 •

edited