Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AIRFLOW-5580] Add base class for system test #6229

Merged
merged 2 commits into from
Oct 7, 2019

Conversation

turbaszek
Copy link
Member

Make sure you have checked all steps below.

Jira

Description

  • Here are some details about my PR, including screenshots of any UI changes:

This PR proposes base class for running system test in Airflow. The
main concepts is to create an example DAG and run it for test purpose. This
is especially important in case of integration with third party services. The change is a part of refactor of GCP system test and aims to decouple running a test and handling authentication.

Tests

  • My PR adds the following unit tests OR does not need testing for this extremely good reason:

Commits

  • My commits all reference Jira issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
    1. Subject is separated from body by a blank line
    2. Subject is limited to 50 characters (not including Jira issue reference)
    3. Subject does not end with a period
    4. Subject uses the imperative mood ("add", not "adding")
    5. Body wraps at 72 characters
    6. Body explains "what" and "why", not "how"

Documentation

  • In case of new functionality, my PR adds documentation that describes how to use it.
    • All the public functions and the classes in the PR contain docstrings that explain what it does
    • If you implement backwards incompatible changes, please leave a note in the Updating.md so we can assign it to a appropriate release

@codecov-io
Copy link

codecov-io commented Oct 1, 2019

Codecov Report

Merging #6229 into master will decrease coverage by 0.09%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff            @@
##           master    #6229     +/-   ##
=========================================
- Coverage   80.05%   79.96%   -0.1%     
=========================================
  Files         610      610             
  Lines       35264    35264             
=========================================
- Hits        28232    28200     -32     
- Misses       7032     7064     +32
Impacted Files Coverage Δ
airflow/operators/postgres_operator.py 0% <0%> (-100%) ⬇️
airflow/hooks/postgres_hook.py 94.73% <0%> (-1.76%) ⬇️
airflow/utils/sqlalchemy.py 91.52% <0%> (-1.7%) ⬇️
airflow/hooks/dbapi_hook.py 86.44% <0%> (-1.7%) ⬇️
airflow/utils/dag_processing.py 56.37% <0%> (-0.52%) ⬇️
airflow/models/taskinstance.py 93.24% <0%> (-0.51%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 76fe45e...4437fc2. Read the comment docs.

@potiuk
Copy link
Member

potiuk commented Oct 2, 2019

It should be rebased to the latest master @nuclearpinguin

@mik-laj
Copy link
Member

mik-laj commented Oct 3, 2019

@nuclearpinguin This PR contains too much commits. Can you fix it?

@turbaszek turbaszek force-pushed the add-system-test-base-class branch 2 times, most recently from 1034138 to 74e7526 Compare October 3, 2019 09:06
@turbaszek
Copy link
Member Author

@potiuk @mik-laj I have rebased the branch

dag_folder = resolve_dags_folder()
with empty_dags_directory(dag_folder):
db.upgradedb()
db.resetdb()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resetdb does upgrade, there's no need to do both

This commit proposes base class for running system test in Airflow. The
main concepts is to create an example DAG and run it for test purpose. This
is especially important in case of integration with third party services.
@potiuk potiuk merged commit 965c902 into apache:master Oct 7, 2019
potiuk pushed a commit that referenced this pull request Oct 7, 2019
* [AIRFLOW-5580] Add base class for system test

This commit proposes base class for running system test in Airflow. The
main concepts is to create an example DAG and run it for test purpose. This
is especially important in case of integration with third party services.
@turbaszek turbaszek deleted the add-system-test-base-class branch December 4, 2019 11:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants