Create MesosExecutorRemovedRule to ease upgrade to Airflow 2.0

[turbaszek]

This issue is part of #8765

Rule

Create MesosExecutorRemovedRule which corresponds to

Removal of Mesos Executor

entry in UPDATING.md. This rule should allow users to check if their current configuration needs any adjusting
before migration to Airflow 2.0.

How to guide

To implement a new rule, create a class that inherits from airflow.upgrade.rules.base_rule.BaseRule.
It will be auto-registered and used by airflow upgrade-check command. The custom rule class has to have title,
description properties and should implement check method which returns a list of error messages in case of
incompatibility.

For example:

airflow/airflow/upgrade/rules/conn_type_is_not_nullable.py

Lines 25 to 42 in ea36166

	class ConnTypeIsNotNullableRule(BaseRule):
	title = "Connection.conn_type is not nullable"
	description = """\
	The `conn_type` column in the `connection` table must contain content. Previously, this rule was \
	enforced by application logic, but was not enforced by the database schema.
	If you made any modifications to the table directly, make sure you don't have null in the conn_type column.\
	"""
	@provide_session
	def check(self, session=None):
	invalid_connections = session.query(Connection).filter(Connection.conn_type.is_(None))
	return (
	'Connection<id={}", conn_id={}> have empty conn_type field.'.format(conn.id, conn.conn_id)
	for conn in invalid_connections
	)

Remember to open the PR against v1-10-test branch.

[RaviTezu]

Hi @turbaszek I would like to take this up. Could you please assign this to me? Thanks.

[RaviTezu]

Thanks @turbaszek
I see there's a open PR on adding a new mesos executer here #10977
Do you think we will still need this rule ? I just want to check before starting on this. Thanks.

[mik-laj]

@RaviTezu Yes. Mesos may be added after the release Airflow 2.0, as we are now focusing on this release and prioritizing the tasks that are related to this release. Adding new features that are not in the roadmap is not essential now, so it may be a bit slower. Additionally, few core contributors have experience with Mesos, so there is a chance that there will be a problem with the reviewer. That being said, we should work as planned and add a migration alert.

[RaviTezu]

Thanks for the reply @mik-laj 👍

[RaviTezu]

@mik-laj @turbaszek Just want to confirm what needs to be done:

• Add a new rule under airflow/upgrade/rules with the name something like mesos_executor_removed.py
• Include a class which inherits from BaseRule and have title and descripption and override the check method.
• Inside the check method look for mesos conf and if present then return a string saying - Mesos has been deprecated

Thanks.

[dimberman]

@turbaszek @RaviTezu can we close this ticket?

[RaviTezu]

@dimberman Yes. I think, it can be closed.

[turbaszek]

Done in #11528, thanks!