Skip to content

[AIRFLOW-2016] assign template_fields for Dataproc Workflow Template sub-classes#3346

Closed
mchalek wants to merge 1 commit intoapache:masterfrom
mchalek:mchalek-fix-inline-workflow-template-operator
Closed

[AIRFLOW-2016] assign template_fields for Dataproc Workflow Template sub-classes#3346
mchalek wants to merge 1 commit intoapache:masterfrom
mchalek:mchalek-fix-inline-workflow-template-operator

Conversation

@mchalek
Copy link
Contributor

@mchalek mchalek commented May 10, 2018

JIRA

In those comments I confirmed with the original submitter for that ticket, @DanSedov , that the existing code has a problem and that it is correctly addressed by this commit.

Description

  • This change moves the template_fields for the DataProc WorkflowTemplate operators out of the base operator and into the derived classes. This is necessary because the derived classes only use subsets of the templated fields, and so when task instances are instantiated for them, errors are thrown for the fields that are missing.

Tests

  • My PR is a small bug fix that will be difficult to test in unit tests, but I have tested it on our Airflow deployments and confirmed that it fixes the problem. Prior to this commit, the WorkflowTemplate operators are not usable because of this bug.

Commits

  • My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
    1. Subject is separated from body by a blank line
    2. Subject is limited to 50 characters
    3. Subject does not end with a period
    4. Subject uses the imperative mood ("add", not "adding")
    5. Body wraps at 72 characters
    6. Body explains "what" and "why", not "how"

Documentation

  • No new functionality is provided

Code Quality

  • Passes git diff upstream/master -u -- "*.py" | flake8 --diff

@DanSedov
Copy link
Contributor

+@fenglu-g Looks like I applied templates incorrectly

@asfgit asfgit closed this in 961aa05 May 10, 2018
asfgit pushed a commit that referenced this pull request May 10, 2018
sub-classes, not base class

Closes #3346 from mchalek/mchalek-fix-inline-
workflow-template-operator

(cherry picked from commit 961aa05)
Signed-off-by: Kaxil Naik <kaxilnaik@gmail.com>
aliceabe pushed a commit to aliceabe/incubator-airflow that referenced this pull request Jan 3, 2019
sub-classes, not base class

Closes apache#3346 from mchalek/mchalek-fix-inline-
workflow-template-operator
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants