Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DB migration: TaskInstance tables from execution_date to run_id #16302

Closed
2 of 4 tasks
kaxil opened this issue Jun 7, 2021 · 0 comments
Closed
2 of 4 tasks

DB migration: TaskInstance tables from execution_date to run_id #16302

kaxil opened this issue Jun 7, 2021 · 0 comments
Assignees
Labels
AIP-39 Timetables

Comments

@kaxil
Copy link
Member

kaxil commented Jun 7, 2021

Create a migration for the following tables to use dag_run.run_id instead of execution_date. Remove execution_date from the primary_key too.

Use SQLAlchemy's Association proxy (https://docs.sqlalchemy.org/en/13/orm/extensions/associationproxy.html) for keeping backwards compatibility for execution_date.

As part of this, also replace the usage of TI.execution_date to TI.run_id in the entire codebase.

Add a check *BEFORE any migrations are run to ensure that there are no missing DagRuns for a TaskInstance and vice-versa, and that a user is not left with a DB version "part-way" between two Airflow versions.

@kaxil kaxil changed the title DB migration: TaskInstance table from execution_date to run_id DB migration: TaskInstance tables from execution_date to run_id Jun 7, 2021
@eladkal eladkal added the AIP-39 Timetables label Jun 9, 2021
@ashb ashb self-assigned this Aug 19, 2021
@kaxil kaxil closed this as completed Sep 20, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
AIP-39 Timetables
Projects
None yet
Development

No branches or pull requests

3 participants