Support env var substitution in all config.

[archfz]

I would like to use env vars in any yaml:

name: API Error
type: any
index: filebeat-*
filter:
    - query:
          query_string:
              query: 'event.dataset: "dotnet" AND log.level: "Error"'
alert:
    - "discord"
discord_webhook_url: $ELAST_ALERT_DISCORD_URL

This seems to not work currently. Am I missing something?

[jertel]

I'm not aware of rules supporting $ENV_VAR syntax. Can you provide a doc URL or something that indicates this is supported?

Moved this to a discussion as this is a question.

[archfz]

Sorry but this is not a question but a feature request. If you check how elasticsearch works with its configuration yaml s it supports env var substitution anywhere.

It would make sense to support this as it is closely related to elasticsearch.

Otherwise if you think this is not a required feature for the package, how do you propose having different discord url (or any other similar config) different per environment (and ofc I dont want to maintain multiple versions of the same config file). Also how would one protect (not commit) such values if one cannot use env vars?

[archfz]

Here is documented how for example one can use env vars in filbeat.yml (elastic beats) https://www.elastic.co/guide/en/beats/filebeat/current/using-environ-vars.html
Or here is the repo of elk docker https://github.com/deviantony/docker-elk . You can easily try out setting env vars in any of the yamls.

[archfz]

But to answer your question, indeed in the elastalert2 documentation haven't found this to be supported. I have found that in some places it might work, but don't understand why it shouldn't in all places work.

[nsano-rururu]

I haven't seen the implementation yet, so I'm not sure if it's supported, but.

In elastalert2, the yaml file uses PyYAML, so I think it is technically possible to expand the environment variables embedded in YAML at the time of loading with the function of PyYAML.

How to replace environment variable value in yaml file to be parsed using python script
https://stackoverflow.com/questions/52412297/how-to-replace-environment-variable-value-in-yaml-file-to-be-parsed-using-python
PyYAMLでYAMLに埋め込まれた環境変数をロード時に展開する
https://qiita.com/kanga/items/198a1079f9ed0b11b7e8

[nsano-rururu]

@archfz

I think that elastic's filebeat supports it because the program that reads the environment variables written in yaml supports it. Even with elastalert2, I think that you can not read the environment variables written in yaml unless the program side supports it.

[nsano-rururu]

@archfz

You found that it might work, but what is the filename and number of lines in your program? elastalert2 is a yelp/elastalert fork. I'm very sorry that there is no one who can ask how yelp made such an implementation, but if there is no description in the document, the answer is that I do not know. Maybe you can find the answer in yelp/elastalert's past issues and pull requests ...

[archfz]

I can take a look and submit a pr. You should mark this as a feature request.