Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a Rule to Check version of Database (MySQL / Postgres / SQLite) #13850

Closed
kaxil opened this issue Jan 22, 2021 · 3 comments
Closed

Add a Rule to Check version of Database (MySQL / Postgres / SQLite) #13850

kaxil opened this issue Jan 22, 2021 · 3 comments
Assignees
Labels
area:upgrade Facilitating migration to a newer version of Airflow kind:feature Feature Requests upgrade-check upgrade-check CLI

Comments

@kaxil
Copy link
Member

kaxil commented Jan 22, 2021

Many users are not sure about the requirements of the DB versions mentioned in https://github.com/apache/airflow#requirements and run into different errors.

We should have a "Check / Rule" in our Upgrade Check Plugin to warn users if they use a version that will be not compatible with 2.0.0

This issue is part of #8765

How to guide

To implement a new rule, create a class that inherits from airflow.upgrade.rules.base_rule.BaseRule.
It will be auto-registered and used by airflow upgrade-check command. The custom rule class has to have title,
description properties and should implement check method which returns a list of error messages in case of
incompatibility.

For example:

class ConnTypeIsNotNullableRule(BaseRule):
title = "Connection.conn_type is not nullable"
description = """\
The `conn_type` column in the `connection` table must contain content. Previously, this rule was \
enforced by application logic, but was not enforced by the database schema.
If you made any modifications to the table directly, make sure you don't have null in the conn_type column.\
"""
@provide_session
def check(self, session=None):
invalid_connections = session.query(Connection).filter(Connection.conn_type.is_(None))
return (
'Connection<id={}", conn_id={}> have empty conn_type field.'.format(conn.id, conn.conn_id)
for conn in invalid_connections
)

@kaxil kaxil added kind:feature Feature Requests area:upgrade Facilitating migration to a newer version of Airflow upgrade-check upgrade-check CLI labels Jan 22, 2021
@kaxil kaxil added this to the Airflow Upgrade Check - 1.2.0 milestone Jan 22, 2021
@kaxil
Copy link
Member Author

kaxil commented Jan 22, 2021

cc @Dr-Denzy

@Dr-Denzy
Copy link
Contributor

I will take this on right away.

kaxil pushed a commit that referenced this issue Feb 3, 2021
This upgrade check inspects the version of the supported database
backend (PostgreSQL, MySQL, and SQLite) so as to verify if the version
is supported in Airflow 2.0

This is ease upgrade to Airflow 2.0

closes: #13850
@kaxil
Copy link
Member Author

kaxil commented Feb 3, 2021

Closed by #13955

@kaxil kaxil closed this as completed Feb 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:upgrade Facilitating migration to a newer version of Airflow kind:feature Feature Requests upgrade-check upgrade-check CLI
Projects
None yet
Development

No branches or pull requests

2 participants