Update version to V2 1 1 #16781
Closed
Update version to V2 1 1 #16781
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Modal events are triggering twice. once, with an executionDate, and again without. Now the submit function will check that an executionDate exists before doing an action (cherry picked from commit a73ba4b)
(cherry picked from commit d88d117)
apache#15899) And rather than only applying the filters for the current known remote backends, I have applied the filter to the task handler "globally" so that all task logs are filtered, even for custom remote back ends. (cherry picked from commit 7ea1b84)
This was missed in apache#15781 Since "connexion" is Apache licensed, this is "not a blocker" for 2.1.0 as mentioned in https://www.apache.org/legal/release-policy.html#license-file >When a package bundles code under several licenses, the LICENSE file MUST contain details of all these licenses. For each component which is not Apache licensed, details of the component MUST be appended to the LICENSE file. The component license itself MUST either be appended or else stored elsewhere in the package with a pointer to it from the LICENSE file, e.g. if the license is long. As "connextion" is Apache 2 Licensed, this _might_ be OK. (cherry picked from commit 180df03)
(cherry picked from commit 208b0c7)
(cherry picked from commit 0c2848c)
Templates _shouldn't_ ever be taken from untrusted user input (and it's hard to do so without just edit the dag file, at which point you can run whatever python code you like _anyway_), but this is a reasonable safety measure just in case someone does something "clever". (cherry picked from commit 4295845)
…n multiprocessing (apache#15989) Fixes apache#15938 multiprocessing.Pool is known to often become stuck. It causes celery_executor to hang randomly. This happens at least on Debian, Ubuntu using Python 3.8.7 and Python 3.8.10. The issue is reproducible by running test_send_tasks_to_celery_hang in this PR several times (with db backend set to something other than sqlite because sqlite disables some parallelization) The issue goes away once switched to concurrent.futures.ProcessPoolExecutor. In python 3.6 and earlier, ProcessPoolExecutor has no initializer argument. Fortunately, it's not needed because reset_signal is no longer needed because the signal handler now checks if the current process is the parent. (cherry picked from commit f75dd7a)
…e#15382) closes: apache#14260 related: apache#9824 When clearing task across dags using ExternalTaskMarker the dag state of the external DagRun is not set to active. So cleared tasks in the external dag will not automatically start if the DagRun is a Failed or Succeeded state. Two changes are made to fix the issue: Make clear_task_instances set DagRuns' state to dag_run_state for all the affected DagRuns. The filter for DagRun in clear_task_instances is fixed too. Previously, it made an assumption that execution_dates for all the dag_ids are the same, which is not always correct. test_external_task_marker_clear_activate is added to make sure the fix does the right thing. (cherry picked from commit 2bca8a5)
…he#16018) Co-authored-by: Felipe Lolas <felipe.lolas@bci.cl> Obtain tree_data object endpoint from meta. closes: apache#16017 (cherry picked from commit c288957)
The GCSToLocalFilesystemOperator in Google Provider <=3.0.0 had wrong import for apply_defaults. It used `from airflow.sensors.base_sensor_operator import apply_defaults` instead of `from airflow.utils.decorators import apply_defaults` When we removed apply_defaults in apache#15667, the base_sensor_operator import was removed as well which made the GCSToLocalFilestystemOperator stops working in 2.1.0 Fixes: apache#16035 (cherry picked from commit 0f8f66e)
…ache#16047) It is possible that `exc_info` can be set, but contain no exception. We shouldn't fail in this case, even if the output doesn't make sense as shown by the test (the `NoneType: None` line is the exception being logged.) (cherry picked from commit 2f77633)
Although `Connection.password` being empty was guarded against, there
are other possible cases (such as an extra field) that wasn't guarded
against, which ended up with this in the logs:
WARNING - ***-***-***-*** ***L***o***g***g***i***n***g*** ***e***r***r***o***r*** ***-***-***-***
Oops!
(cherry picked from commit 8814a59)
This commit adds an optimization where the recently modified files (detected by mtime) will be parsed even though it has not reached `min_file_process_interval`. This way you can increase `[scheduler] min_file_process_interval` to a higher value like `600` or so when you have large number of files to avoid unnecessary reparsing if files haven't changed, while still making sure that modified files are taken care of. (cherry picked from commit add7490)
(cherry picked from commit 9d06ee8)
…che#16088) There was a problem that when we initialized configuration, we've run validate() which - among others - checkd if the connection is an `sqlite` but when the SQLAlchemy connection was not configured via variable but via secret manager, it has fallen back to secret_backend, which should be configured via conf and initialized. The problem is that the "conf" object is not yet created, because the "validate()" method has not finished yet and "initialize_configuration" has not yet returned. This led to snake eating its own tail. This PR defers the validate() method to after secret backends have been initialized. The effect of it is that secret backends might be initialized with configuration that is not valid, but there are no real negative consequences of this. Fixes: apache#16079 Fixes: apache#15685 starting (cherry picked from commit 65519ab)
…aw` for KubeExecutor (apache#16108) (cherry picked from commit cdc9f1a)
Although `Connection.password` being empty was guarded against, there
are other possible cases (such as an extra field) that wasn't guarded
against, which ended up with this in the logs:
WARNING - ***-***-***-*** ***L***o***g***g***i***n***g*** ***e***r***r***o***r*** ***-***-***-***
Oops!
(cherry picked from commit 8814a59)
the totalwidth of the tree view will depend on the window size like before, but max out at 1200px (cherry picked from commit f2aa9b5)
Rather than dying with an exception, catch it and warn about that, asking users to report it to us. Additionally handle the specific case where a file handle/IO object is logged -- we definitely don't want to iterate over that! (cherry picked from commit 57bd6fb)
(cherry picked from commit 90f0088)
Infinite pools: Make their `total_slots` be `inf` instead of `-1` (cherry picked from commit 96f7643)
…pache#16392) (cherry picked from commit 4300731)
We should show the actual default/example value in the configuration
reference docs, not the templated values.
e.g. `{dag_id}` like you get in a generated airflow.cfg, not `{{dag_id}}
like is stored in the airflow.cfg template.
(cherry picked from commit cc3c13c)
- add `num_runs` as a meta field to add to the tree refresh request (cherry picked from commit 6087a09)
(cherry picked from commit 15ff238)
(cherry picked from commit 6f9c0ce)
We don't need a way to specify _in_ setup.py that this should be installed from a GitHub release -- it's never needed by users, and if you are developing the theme you can install the custom version yourself. (The variable name is confusing too -- it wasn't pulling from git, but from a published release on GitHub.) Removing this just means one less thing to update. (cherry picked from commit 129fc61)
…pache#16601) (cherry picked from commit 18cb0bb)
Istio service mesh is not compatible by default with Kubernetes Jobs. The normal behavior is that a Job will be started, get an istio-proxy sidecar attached to it via the istio mutating webhook, run until completion, then the 'main' container in the pod stops, but istio-proxy hangs around indefinitely. This change handles cleanly exiting the Istio sidecar 'istio-proxy' when a Kubernetes Executor task completes. (cherry picked from commit 84fa48f) (cherry picked from commit 6ed59bf) (cherry picked from commit ba60ede) (cherry picked from commit 80ac218)
…running K8sPodOperator (cherry picked from commit 569d32b)
(cherry picked from commit 280bcae)
fixes issue apache#15234. As of now, TI success & failure endpoints are POST only and behave differently as per the "confirmed" flag. They either render a confirmation page or updates the TI states on the basis of that flag, something which is not a great design. Also, as these endpoints are POST only, they throw a 404 error when someone clicks on the link received via email. To fix the issue, extracting the rendering functionalities into a diff endpoint "/confirm" & keeping these endpoints as pure POST endpoints.
Mistakenly checking for the wrong args in TI success API. Introduced in PR apache#16233.
When a task fails in executor while still queued in scheduler, the executor reports this failure but scheduler doesn't change the task state resulting in the task being queued until the scheduler is restarted. This commit fixes it by ensuring that when a task is reported to have failed in the executor, the task is failed in scheduler
jhtimmins
requested review from
XD-DENG,
ashb,
dimberman,
ephraimbuddy,
jedcunningham,
kaxil,
mik-laj,
potiuk,
ryanahamilton,
turbaszek and
vikramkoka
as code owners
boring-cyborg
bot
added
area:dev-tools
area:production-image
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Update version to 2.1.1+astro.1