[AIRFLOW-5873] KubernetesPodOperator fixes and test #6524
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
2 tasks
ddelange
force-pushed
the
patch-1
branch
25 times, most recently
from
1332ace
to
2068a35
Compare
|
Now just waiting for the master one to pass :) |
- `security_context` was missing from docs of `KubernetesPodOperator` - `KubernetesPodOperator` kwarg `in_cluster` erroneously defaults to False in comparison to `default_args.py`, also default `do_xcom_push` was overwritten to False in contradiction to `BaseOperator` - `KubernetesPodOperator` kwarg `resources` is erroneously passed to `base_operator`, instead should only go to `PodGenerator`. The two have different syntax. (both on `master` and `v1-10-test` branches) - `kubernetes/pod.py`: classes do not have `__slots__` so they would accept arbitrary values in `setattr` - Reduce amount of times the pod object is copied before execution
Codecov Report
@@ Coverage Diff @@
## master #6524 +/- ##
=========================================
Coverage ? 83.62%
=========================================
Files ? 645
Lines ? 37291
Branches ? 0
=========================================
Hits ? 31186
Misses ? 6105
Partials ? 0
Continue to review full report at Codecov.
|
potiuk
approved these changes
Nov 12, 2019
| @@ -40,6 +40,8 @@ class Resources(K8SModel): | |||
| :param limit_gpu: Limits for GPU used | |||
| :type limit_gpu: int | |||
| """ | |||
| __slots__ = ('request_memory', 'request_cpu', 'limit_memory', 'limit_cpu', 'limit_gpu') | |||
GnunuX
pushed a commit
to GnunuX/airflow
that referenced
this pull request
Nov 13, 2019
- `security_context` was missing from docs of `KubernetesPodOperator` - `KubernetesPodOperator` kwarg `in_cluster` erroneously defaults to False in comparison to `default_args.py`, also default `do_xcom_push` was overwritten to False in contradiction to `BaseOperator` - `KubernetesPodOperator` kwarg `resources` is erroneously passed to `base_operator`, instead should only go to `PodGenerator`. The two have different syntax. (both on `master` and `v1-10-test` branches) - `kubernetes/pod.py`: classes do not have `__slots__` so they would accept arbitrary values in `setattr` - Reduce amount of times the pod object is copied before execution
dimberman
pushed a commit
that referenced
this pull request
Jun 4, 2020
- `security_context` was missing from docs of `KubernetesPodOperator` - `KubernetesPodOperator` kwarg `in_cluster` erroneously defaults to False in comparison to `default_args.py`, also default `do_xcom_push` was overwritten to False in contradiction to `BaseOperator` - `KubernetesPodOperator` kwarg `resources` is erroneously passed to `base_operator`, instead should only go to `PodGenerator`. The two have different syntax. (both on `master` and `v1-10-test` branches) - `kubernetes/pod.py`: classes do not have `__slots__` so they would accept arbitrary values in `setattr` - Reduce amount of times the pod object is copied before execution (cherry picked from commit cf38ddc)
dimberman
pushed a commit
that referenced
this pull request
Jun 4, 2020
- `security_context` was missing from docs of `KubernetesPodOperator` - `KubernetesPodOperator` kwarg `in_cluster` erroneously defaults to False in comparison to `default_args.py`, also default `do_xcom_push` was overwritten to False in contradiction to `BaseOperator` - `KubernetesPodOperator` kwarg `resources` is erroneously passed to `base_operator`, instead should only go to `PodGenerator`. The two have different syntax. (both on `master` and `v1-10-test` branches) - `kubernetes/pod.py`: classes do not have `__slots__` so they would accept arbitrary values in `setattr` - Reduce amount of times the pod object is copied before execution (cherry picked from commit cf38ddc)
kaxil
pushed a commit
that referenced
this pull request
Jul 1, 2020
- `security_context` was missing from docs of `KubernetesPodOperator` - `KubernetesPodOperator` kwarg `in_cluster` erroneously defaults to False in comparison to `default_args.py`, also default `do_xcom_push` was overwritten to False in contradiction to `BaseOperator` - `KubernetesPodOperator` kwarg `resources` is erroneously passed to `base_operator`, instead should only go to `PodGenerator`. The two have different syntax. (both on `master` and `v1-10-test` branches) - `kubernetes/pod.py`: classes do not have `__slots__` so they would accept arbitrary values in `setattr` - Reduce amount of times the pod object is copied before execution (cherry picked from commit cf38ddc)
cfei18
pushed a commit
to cfei18/incubator-airflow
that referenced
this pull request
Mar 5, 2021
- `security_context` was missing from docs of `KubernetesPodOperator` - `KubernetesPodOperator` kwarg `in_cluster` erroneously defaults to False in comparison to `default_args.py`, also default `do_xcom_push` was overwritten to False in contradiction to `BaseOperator` - `KubernetesPodOperator` kwarg `resources` is erroneously passed to `base_operator`, instead should only go to `PodGenerator`. The two have different syntax. (both on `master` and `v1-10-test` branches) - `kubernetes/pod.py`: classes do not have `__slots__` so they would accept arbitrary values in `setattr` - Reduce amount of times the pod object is copied before execution (cherry picked from commit cf38ddc)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Make sure you have checked all steps below.
Jira
Description
This PR implements the same as #6523 but is adapted to
masterbranch:KubernetesPodOperatorkwargresourcesis erroneously passed tobase_operator, instead should only go toPodGenerator. The two have different syntax. (both onmasterandv1-10-testbranches)resourcespassed to PodGenerator should bek8s.V1ResourceRequirements, which is now handled inKubernetesPodOperatorkubernetes/pod.py:Resourcesdoes not have__slots__so accepts arbitrary values insetattr(not present on either branch https://github.com/apache/airflow/blame/50343040ff4679e32e01f138ead80bc4bcef4b47/airflow/contrib/operators/kubernetes_pod_operator.py#L166-L171)KubernetesPodOperatorkwargin_clustererroneously defaults to False in comparison todefault_args.py, also defaultdo_xcom_pushwas overwritten to False in contradiction toBaseOperatorv1-10-testis behindmasterwith KubernetesPodOperator fixes and refactors (will not be addressed in this PR,)move kubernetes folder one level up fromhttps://github.com/apache/airflow/blame/4dd24a2c595d4042ffe745aed947eaaea6abb652/airflow/contrib/operators/kubernetes_pod_operator.py#L21/contribfixhttps://github.com/apache/airflow/blame/4dd24a2c595d4042ffe745aed947eaaea6abb652/airflow/contrib/operators/kubernetes_pod_operator.py#L90xcom_pushtodo_xcom_pushTests
contrib/operators/test_kubernetes_pod_operator.py