Remove environment name from Airflow Alerts message username

[sarayourfriend]

Problem

Airflow alerts include the environment "production" in the username in Slack:

The message above is about the staging ES cluster. But the username of the message makes it easy to get confused and think it's for the production cluster.

Description

We have a single channel for alerts from the live Airflow instance. Any alerts going to that channel must be from the live instance and must never be from another source (outside of specific testing scenarios where this is communicated extremely clearly). The channel is exclusively for important alerts that need to be triaged.

In other words, the contract of that channel is such that the alerts are only from the live instance. This does not need to be clarified in every message from the live instance. Therefore, we can remove the "environment" name from the message username.

To fix this, remove the environment note from the username here:

openverse/catalog/dags/common/slack.py

Lines 339 to 346 in d500b77

	environment = Variable.get("ENVIRONMENT", default_var="local")
	s = SlackMessage(
	f"{username} | {environment}",
	icon_emoji,
	unfurl_links,
	unfurl_media,
	http_conn_id=http_conn_id,
	)

If we want to make sure locally created messages can under no circumstances be confused for real ones, even if they are somehow mistakenly sent to the live channel, we can modify the message body to indicate this when ENVIRONMENT == "local" or something... but I don't think this is expressly necessary.

[zackkrida]

For these messages as well, maybe we should pass the service name as the title so the full title would be something like "Airflow Alert | Elasticsearch Staging Cluster"

[sarayourfriend]

Nice! Good idea to lead with the environment name, even: "Staging ES cluster"/"Production ES cluster". Along with abbreviating ES in the title, it helps the environment not get lost or glossed over, which is really the important thing here.

[AetherUnbound]

For these kinds of alerts, we can also change the icon used for the alert itself in Slack, perhaps for even further distinction!