Improve detection of tests vs. airflow code for internal API#41030
Merged
potiuk merged 1 commit intoapache:mainfrom Jul 25, 2024
Merged
Improve detection of tests vs. airflow code for internal API#41030potiuk merged 1 commit intoapache:mainfrom
potiuk merged 1 commit intoapache:mainfrom
Conversation
In case of database isolation, some of our tests are running a number of fixtures that allow for example to create dag runs as setup code - this code, however sometimes runs methods that are used for internal_api calls and those calls are not needed to be run via internal_api. This PR adds capability of detecting such case - by checking if any of the "callers" of the internal_api are `conftest.py` which means that this is a test fixture - and in this case, direct method call is used rather than internal API call. This way we better separate test code from "airflow" code in DB isolation tests - and internal API is only used by the tested methods and not the test code that manages setup/teardown Also orm_create_dagrun which should only be used in scheduler is unmarked as "internal_api" method - it is heavily used in the fixtures, but neither DAGFileProcessor, Triggerer nor Worker should create new DAGRuns.
ephraimbuddy
added
the
changelog:skip
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In case of database isolation, some of our tests are running a number of fixtures that allow for example to create dag runs as setup code - this code, however sometimes runs methods that are used for internal_api calls and those calls are not needed to be run via internal_api.
This PR adds capability of detecting such case - by checking if any of the "callers" of the internal_api are
conftest.pywhich means that this is a test fixture - and in this case, direct method call is used rather than internal API call.This way we better separate test code from "airflow" code in DB isolation tests - and internal API is only used by the tested methods and not the test code that manages setup/teardown
Also orm_create_dagrun which should only be used in scheduler is unmarked as "internal_api" method - it is heavily used in the fixtures, but neither DAGFileProcessor, Triggerer nor Worker should create new DAGRuns.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.