Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AIRFLOW-5195] Move GCP Dataflow to core #5799

Merged
merged 1 commit into from
Aug 17, 2019

Conversation

turbaszek
Copy link
Member

@turbaszek turbaszek commented Aug 13, 2019

Make sure you have checked all steps below.

Jira

Description

  • Here are some details about my PR, including screenshots of any UI changes:
    This PR moves GCP Dataflow from contrib to core. For more information check AIP-21.

Tests

  • My PR adds the following unit tests OR does not need testing for this extremely good reason:

Commits

  • My commits all reference Jira issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
    1. Subject is separated from body by a blank line
    2. Subject is limited to 50 characters (not including Jira issue reference)
    3. Subject does not end with a period
    4. Subject uses the imperative mood ("add", not "adding")
    5. Body wraps at 72 characters
    6. Body explains "what" and "why", not "how"

Documentation

  • In case of new functionality, my PR adds documentation that describes how to use it.
    • All the public functions and the classes in the PR contain docstrings that explain what it does
    • If you implement backwards incompatible changes, please leave a note in the Updating.md so we can assign it to a appropriate release

Code Quality

  • Passes flake8

@turbaszek
Copy link
Member Author

Depends on #5791.

@mik-laj mik-laj added AIP-21 provider:google Google (including GCP) related issues and removed waiting-AIP labels Aug 13, 2019
@turbaszek turbaszek changed the title [AIRFLOW-5195][depend on AIRFLOW-5166] Move GCP Dataflow to core [AIRFLOW-5195][depend on AIRFLOW-5183] Move GCP Dataflow to core Aug 13, 2019
@mik-laj
Copy link
Member

mik-laj commented Aug 13, 2019

======================================================================
45) ERROR: testSuccessfulRun (tests.contrib.utils.test_mlengine_operator_utils.CreateEvaluateOpsTest)
----------------------------------------------------------------------
   Traceback (most recent call last):
    tests/contrib/utils/test_mlengine_operator_utils.py line 100 in testSuccessfulRun
      with patch('airflow.contrib.operators.dataflow_operator.'
    /usr/local/lib/python3.6/unittest/mock.py line 1247 in __enter__
      original, local = self.get_original()
    /usr/local/lib/python3.6/unittest/mock.py line 1221 in get_original
      "%s does not have the attribute %r" % (target, name)
   AttributeError: <module 'airflow.contrib.operators.dataflow_operator' from '/opt/airflow/airflow/contrib/operators/dataflow_operator.py'> does not have the attribute 'DataFlowHook'
   -------------------- >> begin captured logging << --------------------
   py.warnings: WARNING: /opt/airflow/airflow/contrib/operators/dataflow_operator.py:32: DeprecationWarning: This module is deprecated. Please use `airflow.gcp.operators.dataflow`.
     DeprecationWarning,
   
   --------------------- >> end captured logging << ---------------------

@turbaszek turbaszek force-pushed the gcp-dataflow branch 2 times, most recently from d974a91 to b0019d9 Compare August 15, 2019 06:24
This commit moves GCP Dataflow from contrib to core.
For more information check AIP-21.
@turbaszek turbaszek changed the title [AIRFLOW-5195][depend on AIRFLOW-5183] Move GCP Dataflow to core [AIRFLOW-5195] Move GCP Dataflow to core Aug 16, 2019
@kaxil kaxil merged commit b3acb3d into apache:master Aug 17, 2019
@turbaszek turbaszek deleted the gcp-dataflow branch September 19, 2019 11:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
provider:google Google (including GCP) related issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants