AIP-39: Refactoring works #2
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Seems to work! |
uranusjr
force-pushed
the
aip-39-timetables
branch
2 times, most recently
from
347c5cb
to
a1796d3
Compare
uranusjr
changed the title
uranusjr
force-pushed
the
aip-39-timetables
branch
4 times, most recently
from
d2ab3c5
to
51ee062
Compare
uranusjr
force-pushed
the
aip-39-timetables
branch
5 times, most recently
from
5b05d39
to
186ebc4
Compare
It seems that the < 20.0 limit for gunicorn was added at some point in time without actual reason. We are already using gunicorn in 1.10 line of Airflow, so it should not be a problem to bump the version of gunicorn, especially that the 19. line is somewhat deprecated already. This change came after the discussion n apache#15570
This change adds test for the create user job and ensures changing of the uid is detected Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
Closes: apache#15374 This pull request follows apache#14776. Clearing a subdag with Downstream+Recursive does not automatically set the state of the parent dag so that the downstream parent tasks can execute.
While we generally suggest using Postgres, we should support using a non-chart-provisioned mysql database as well. closes: apache#15558
This PR builds off of and supersedes @jaydesl's work on his [PR](apache#11769) to move forward with properly following [helm's rbac best practices](https://helm.sh/docs/chart_best_practices/rbac/). This PR updates every potential pod that can be deployed to include the option to either create or use an existing service account. This is the first step towards supporting environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled without forcing such users to provide any additional permissions to the default service account in the namespace this is deployed to. closes: apache#11755 related: apache#13643 Co-authored-by: jaydesl <jay.deslauriers@gmail.com> Co-authored-by: Ian Stanton <ian@astronomer.io> Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>
Minor refactor of how the results backend connection secret is built. This moves logic out of an already long line, like it is already done in the metadata connection secret.
This PR updates changelog and bumps version of providers to be released after we reached PIP 21 compatibility. It is necessary for two reasons: 1) We need it in order to get constraints for PyPI released providers to be updated automatically (PIP 21 conflicts master airflow with few released providers 2) We want to release Airflow 2.0.3 which will be PIP 21 installable with those providers.
Following discussion here: https://lists.apache.org/thread.html/r04f228319206ab920088c89f1001482671ca06b3b6b5cc12477c76ab%40%3Cdev.airflow.apache.org%3E we update the policy of supporting Python/K8S versions.
The image tagging now is fully automated within the build dockerhub script including :<VERSION> and :latest tags.
Some of the tests were redundant in `kubernetes_tests/test_kubernetes_pod_operator_backcompat.py` which didn't test anything related to deprecated classes and were already covered by either `kubernetes_tests/test_kubernetes_pod_operator.py` or `tests/providers/cncf/kubernetes/operators/test_kubernetes_pod.py`
…15628) Some of the docs advised to use 'airflow[azure]' whereas it should be apache-airflow[azure]
When building images for production we are using docker-context-files where we build packages to install. However if those context files are not cleaned up, they unnecessary increase size and time needed to build image and they invalidate the COPY . layer of the image. This PR checks if docker-context-files folder contains just readme when Breeze build-image command is run (for cases where images are not built from docker-context-files). Inversely it also checks that there are some files in case the image is built with --install-from-docker-context-files switch. This PR also ads a --cleanup-docker-context-files switch to clean-up the folder automatically. The error mesages also help the user instructing the user what to do.
…pache#15412) Co-authored-by: Jordan Zhang <jorzhang@justin.tv>
…he#15592) * Better description of UID/GID behaviour in image and quickstart Following the discussion in apache#15579 seems that the AIRFLOW_UID/GID parameters were not clearly explained in the Docker Quick-start guide and some users could find it confusing. This PR attempts to clarify it. * fixup! Better description of UID/GID behaviour in image and quickstart
* add quotes to extraConfigMaps Add quotes to extraConfigMaps * Update values.yaml add quotes
* Make DAG Import errors collapsable Show only the first line of a DAG Import Error that will expand when clicked on * make each error its own alert box
Thest time tables (plus the @once thing) covers all existing DAG scheduling logic.
This makes things a bit simpler.
This removes duplicated logic and delegate the logic to the time table's backing schedule. The abstraction boundary is pretty wrong, but that's because those functions *are* pretty wrong to begin with. DAG.is_fixed_schedule() is removed since it is no longer used anywhere.
This removes a lot of the need to depend on TimeTable internals
uranusjr
force-pushed
the
aip-39-timetables
branch
from
186ebc4
to
439c168
Compare
|
The Workflow run is cancelling this PR. Building images for the PR has failed. Follow the workflow link to check the reason. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
For CI. This builds on apache#15397 and will be submitted as a separate PR (or multiple PRs) after that one is accepted.