Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SkipMixin: Add missing session.commit() and test #10421

Merged
merged 1 commit into from
Sep 22, 2020

Conversation

yuqian90
Copy link
Contributor

@yuqian90 yuqian90 commented Aug 20, 2020

I don't know the reason, but if we omit the session.commit() that I just added, test_skip_all_except fails because task3 state is None instead of SKIPPED. I think it's an issue with sqlalchemy or sqlite and not much to do with Airflow itself.

(NOTE: The impact of this bug being fixed here is pretty minimal and hard to catch because even though this line does not set the state to SKIPPED, the newly introduced NotPreviouslySkippedDep actually causes the scheduler to skip the task that are supposed to be skipped. So to the user, this bug being fixed here had almost no impact other than in some test cases. But this session.commit() is needed in case in the future someone calls skip_all_except() and expects the state to be set to SKIPPED for the skipped tests).

@yuqian90
Copy link
Contributor Author

Hi @kaxil this bug affects 1.10.12rc3 (even though it's a pretty minor issue). I'm hoping this can be merged into both master and the upcoming 1.10.12 release.

@potiuk
Copy link
Member

potiuk commented Aug 20, 2020

The reason is this: https://github.com/apache/airflow/blob/master/airflow/models/xcom.py#L101 . The xcom_push method performs "expunge_all()" which removes all modified object from the session. Not sure why though :)

@kaxill - I believe it''s not a case for 1.10.12rc4, but we should definitely merge it if there are other changes. :)

@potiuk potiuk added this to the Airflow 1.10.12 milestone Aug 20, 2020
@yuqian90
Copy link
Contributor Author

The reason is this: https://github.com/apache/airflow/blob/master/airflow/models/xcom.py#L101 . The xcom_push method performs "expunge_all()" which removes all modified object from the session. Not sure why though :)

@kaxill - I believe it''s not a case for 1.10.12rc4, but we should definitely merge it if there are other changes. :)

I see. Thanks @potiuk for pointing out!

@kaxil
Copy link
Member

kaxil commented Aug 20, 2020

I would say this can wait for 1.10.13. If we have a major bug report (hopefully not 🤞 ) for 1.10.12rc3 -- then we can include this for rc4 -- but I seriously hope not :D

@potiuk
Copy link
Member

potiuk commented Aug 20, 2020

Marked it as 1.10.13 then :)

@kaxil kaxil merged commit 423a382 into apache:master Sep 22, 2020
kaxil pushed a commit that referenced this pull request Nov 19, 2020
kaxil pushed a commit that referenced this pull request Nov 19, 2020
cfei18 pushed a commit to cfei18/incubator-airflow that referenced this pull request Mar 5, 2021
@yuqian90 yuqian90 deleted the master_skipmixin_fix branch May 29, 2021 08:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants