Apache Airflow 2.3.0

New Features

• Add dynamic task mapping (https://github.com/apache/airflow/pulls?q=is%3Apr+is%3Amerged+label%3AAIP-42+milestone%3A%22Airflow+2.3.0%22)
• New Grid View replaces Tree View (#18675)
• Templated requirements.txt in Python Operators (#17349)
• Allow reuse of decorated tasks (#22941)
• Move the database configuration to a new section (#22284)
• Add SmoothOperator (#22813)
• Make operator's execution_timeout configurable (#22389)
• Events Timetable (#22332)
• Support dag serialization with custom ti_deps rules (#22698)
• Support log download in task log view (#22804)
• support for continue backfill on failures (#22697)
• Add dag-processor cli command (#22305)
• Add possibility to create users in LDAP mode (#22619)
• Add ignore_first_depends_on_past for scheduled jobs (#22491)
• Update base sensor operator to support XCOM return value (#20656)
• Add an option for run id in the ui trigger screen (#21851)
• Enable JSON serialization for connections (#19857)
• Add REST API endpoint for bulk update of DAGs (#19758)
• Add queue button to click-on-DagRun interface. (#21555)
• Add list-import-errors to airflow dags command (#22084)
• Store callbacks in database if standalone_dag_processor config is True. (#21731)
• Add LocalKubernetesExecutor (#19729)
• Add celery.task_timeout_error metric (#21602)
• Airflow db downgrade cli command (#21596)
• Add ALL_SKIPPED trigger rule (#21662)
• Add db clean CLI command for purging old data (#20838)
• Add celery_logging_level (#21506)
• Support different timeout value for dag file parsing (#21501)
• Support generating SQL script for upgrades (#20962)
• Add option to compress Serialized dag data (#21332)
• Branch python operator decorator (#20860)
• Add Audit Log View to Dag View (#20733)
• Add missing StatsD metric for failing SLA Callback notification (#20924)
• Add ShortCircuitOperator configurability for respecting downstream trigger rules (#20044)
• Allow using Markup in page title in Webserver (#20888)
• Add Listener Plugin API that tracks TaskInstance state changes (#20443)
• Add context var hook to inject more env vars (#20361)
• Add a button to set all tasks to skipped (#20455)
• Cleanup pending pods (#20438)
• Add config to warn public deployment exposure in UI (#18557)
• Log filename template records (#20165)
• Added windows extensions (#16110)
• Showing approximate time until next dag_run in Airflow (#20273)
• Extend config window on UI (#20052)
• Add show dag dependencies feature to CLI (#19985)
• Add cli command for 'airflow dags reserialize` (#19471)
• Add missing description field to Pool schema(REST API) (#19841)
• Introduce DagRun action to change state to queued. (#19353)
• Add DAG run details page (#19705)
• Add role export/import to cli tools (#18916)
• Adding dag_id_pattern parameter to the /dags endpoint (#18924)

Improvements

• Show schedule_interval/timetable description in UI (#16931)
• Added column duration to DAG runs view (#19482)
• Enable use of custom conn extra fields without prefix (#22607)
• Initialize finished counter at zero (#23080)
• Improve logging of optional provider features messages (#23037)
• Meaningful error message in resolve_template_files (#23027)
• Update ImportError items instead of deleting and recreating them (#22928)
• Add option --skip-init to db reset command (#22989)
• Support importing connections from files with ".yml" extension (#22872)
• Support glob syntax in .airflowignore files (#21392) (#22051)
• Hide pagination when data is a single page (#22963)
• Support for sorting DAGs in the web UI (#22671)
• Speed up has_access decorator by ~200ms (#22858)
• Add XComArg to lazy-imported list of Airflow module (#22862)
• Add more fields to REST API dags/dag_id/details endpoint (#22756)
• Don't show irrelevant/duplicated/"internal" Task attrs in UI (#22812)
• No need to load whole ti in current_state (#22764)
• Pickle dag exception string fix (#22760)
• Better verification of Localexecutor's parallelism option (#22711)
• log backfill exceptions to sentry (#22704)
• retry commit on MySQL deadlocks during backfill (#22696)
• Add more fields to REST API get DAG(dags/dag_id) endpoint (#22637)
• Use timetable to generate planned days for current year (#22055)
• Disable connection pool for celery worker (#22493)
• Make date picker label visible in trigger dag view (#22379)
• Expose try_number in airflow vars (#22297)
• Add generic connection type (#22310)
• Add a few more fields to the taskinstance finished log message (#22262)
• Pause auto-refresh if scheduler isn't running (#22151)
• Show DagModel details. (#21868)
• Add pip_install_options to PythonVirtualenvOperator (#22158)
• Show import error for airflow dags list CLI command (#21991)
• Pause auto-refresh when page is hidden (#21904)
• Default args type check (#21809)
• Enhance magic methods on XComArg for UX (#21882)
• py files don't have to be checked is_zipfiles in refresh_dag (#21926)
• Fix TaskDecorator type hints (#21881)
• Add 'Show record' option for variables (#21342)
• Use DB where possible for quicker airflow dag subcommands (#21793)
• REST API: add rendered fields in task instance. (#21741)
• Change the default auth backend to session (#21640)
• Don't check if py DAG files are zipped during parsing (#21538)
• Switch XCom implementation to use run_id (#20975)
• Action log on Browse Views (#21569)
• Implement multiple API auth backends (#21472)
• Change logging level details of connection info in get_connection() (#21162)
• Support mssql in airflow db shell (#21511)
• Support config worker_enable_remote_control for celery (#21507)
• Log memory usage in CgroupTaskRunner (#21481)
• Modernize DAG-related URL routes and rename "tree" to "grid" (#20730)
• Move Zombie detection to SchedulerJob (#21181)
• Improve speed to run airflow by 6x (#21438)
• Add more SQL template fields renderers (#21237)
• Simplify fab has access lookup (#19294)
• Log context only for default method (#21244)
• Log trigger status only if at least one is running (#21191)
• Add optional features in providers. (#21074)
• Better multiple_outputs inferral for @task.python (#20800)
• Improve handling of string type and non-attribute template_fields (#21054)
• Remove un-needed deps/version requirements (#20979)
• Correctly specify overloads for TaskFlow API for type-hinting (#20933)
• Introduce notification_sent to SlaMiss view (#20923)
• Rewrite the task decorator as a composition (#20868)
• Add "Greater/Smaller than or Equal" to filters in the browse views (#20602) (#20798)
• Rewrite DAG run retrieval in task command (#20737)
• Speed up creation of DagRun for large DAGs (5k+ tasks) by 25-130% (#20722)
• Make native environment Airflow-flavored like sandbox (#20704)
• Better error when param value has unexpected type (#20648)
• Add filter by state in DagRun REST API (List Dag Runs) (#20485)
• Prevent exponential memory growth in Tasks with custom logging handler (#20541)
• Set default logger in logging Mixin (#20355)
• Reduce deprecation warnings from www (#20378)
• Add hour and minute to time format on x-axis of all charts using nvd3.lineChart (#20002)
• Add specific warning when Task asks for more slots than pool defined with (#20178)
• UI: Update duration column for better human readability (#20112)
• Use Viewer role as example public role (#19215)
• Properly implement DAG param dict copying (#20216)
• ShortCircuitOperator push XCom by returning python_callable result (#20071)
• Add clear logging to tasks killed due to a Dagrun timeout (#19950)
• Change log level for Zombie detection messages (#20204)
• Better confirmation prompts (#20183)
• Only execute TIs of running DagRuns (#20182)
• Check and run migration in commands if necessary (#18439)
• Log only when Zombies exists (#20118)
• Increase length of the email and username (#19932)
• Add more filtering options for TI's in the UI (#19910)
• Dynamically enable "Test Connection" button by connection type (#19792)
• Avoid littering postgres server logs with "could not obtain lock" with HA schedulers (#19842)
• Renamed Connection.get_hook parameter to make it the same as in SqlSensor and SqlOperator. (#19849)
• Add hook_params in SqlSensor using the latest changes from PR #18718. (#18431)
• Speed up webserver boot time by delaying provider initialization (#19709)
• Configurable logging of XCOM value in PythonOperator (#19378)
• Minimize production js files (#19658)
• Add hook_params in BaseSqlOperator (#18718)
• Add missing "end_date" to hash components (#19281)
• More friendly output of the airflow plugins command + add timetables (#19298)
• Add sensor default timeout config (#19119)
• Update taskinstance REST API schema to include dag_run_id field (#19105)
• Adding feature in bash operator to append the user defined env variable to system env variable (#18944)
• Duplicate Connection: Added logic to query if a connection id exists before creating one (#18161)

Bug Fixes

• Use inherited 'trigger_tasks' method (#23016)
• In DAG dependency detector, use class type instead of class name (#21706)
• Fix tasks being wrongly skipped by schedule_after_task_execution (#23181)
• Fix X-Frame enabled behaviour (#23222)
• Allow extra to be nullable in connection payload as per schema(REST API). (#23183)
• Fix dag_id extraction for dag level access checks in web ui (#23015)
• Fix timezone display for logs on UI (#23075)
• Include message in graph errors (#23021)
• Change trigger dropdown left position (#23013)
• Don't add planned tasks for legacy DAG runs (#23007)
• Add dangling rows check for TaskInstance references (#22924)
• Validate the input params in connection CLI command (#22688)
• Fix trigger event payload ...

Apache Airflow 2.2.5

Bug Fixes

• Check and disallow a relative path for sqlite (#22530)
• Fixed dask executor and tests (#22027)
• Fix broken links to celery documentation (#22364)
• Fix incorrect data provided to tries & landing times charts (#21928)
• Fix assignment of unassigned triggers (#21770)
• Fix triggerer --capacity parameter (#21753)
• Fix graph autorefresh on page load (#21736)
• Fix filesystem sensor for directories (#21729)
• Fix stray order_by(TaskInstance.execution_date) (#21705)
• Correctly handle multiple '=' in LocalFileSystem secrets. (#21694)
• Log exception in local executor (#21667)
• Disable default_pool delete on web ui (#21658)
• Extends typing-extensions to be installed with python 3.8+ #21566 (#21567)
• Dispose unused connection pool (#21565)
• Fix logging JDBC SQL error when task fails (#21540)
• Filter out default configs when overrides exist. (#21539)
• Fix Resources __eq__ check (#21442)
• Fix max_active_runs=1 not scheduling runs when min_file_process_interval is high (#21413)
• Reduce DB load incurred by Stale DAG deactivation (#21399)
• Fix race condition between triggerer and scheduler (#21316)
• Fix trigger dag redirect from task instance log view (#21239)
• Log traceback in trigger excs (#21213)
• A trigger might use a connection; make sure we mask passwords (#21207)
• Update ExternalTaskSensorLink to handle templated external_dag_id (#21192)
• Ensure clear_task_instances sets valid run state (#21116)
• Fix: Update custom connection field processing (#20883)
• Truncate stack trace to DAG user code for exceptions raised during execution (#20731)
• Fix duplicate trigger creation race condition (#20699)
• Fix Tasks getting stuck in scheduled state (#19747)
• Fix: Do not render undefined graph edges (#19684)
• Set X-Frame-Options header to DENY only if X_FRAME_ENABLED is set to true. (#19491)

Doc only changes

• adding on_execute_callback to callbacks docs (#22362)
• Add documentation on specifying a DB schema. (#22347)
• Fix postgres part of pipeline example of tutorial (#21586)
• Extend documentation for states of DAGs & tasks and update trigger rules docs (#21382)
• DB upgrade is required when updating Airflow (#22061)
• Remove misleading MSSQL information from the docs (#21998)

Misc

• Add the new Airflow Trove Classifier to setup.cfg (#22241)
• Rename to_delete to to_cancel in TriggerRunner (#20658)
• Update Flask-AppBuilder to 3.4.5 (#22596)

Apache Airflow Helm Chart 1.5.0

New Features

• Add support for custom command and args in jobs (#20864)
• Support for priorityClassName (#20794)
• Add envFrom to the Flower deployment (#21401)
• Add annotations to cleanup pods (#21484)

Improvements

• Speedup liveness probe for scheduler and triggerer (#20833, #21108, #22041)
• Update git-sync to v3.4.0 (#21309)
• Remove default auth backend setting (#21640)

Bug Fixes

• Fix elasticsearch URL when username/password are empty (#21222)
• Mount airflow.cfg in wait-for-airflow-migrations containers (#20609)
• Grant pod log reader to triggerer serviceaccount (#21111)

Doc only changes

• Simplify chart docs for configuring Airflow (#21747)
• Add extra information about time synchronization needed (#21685)
• Fix extra containers docs (#20787)
• Fix example DAGs env var in quickstart (#21990)

Misc

• Use 2.2.4 as default Airflow version (#21745)
• Change Redis image to bullseye (#21875)

Apache Airflow 2.2.4

Bug Fixes

• Adding missing login provider related methods from Flask-Appbuilder (#21294)
• Fix slow DAG deletion due to missing dag_id index for job table (#20282)
• Add a session backend to store session data in the database (#21478)
• Show task status only for running dags or only for the last finished dag (#21352)
• Use compat data interval shim in log handlers (#21289)
• Fix mismatch in generated run_id and logical date of DAG run (#18707)
• Fix TriggerDagRunOperator extra link (#19410)
• Add possibility to create user in the Remote User mode (#19963)
• Avoid deadlock when rescheduling task (#21362)
• Fix the incorrect scheduling time for the first run of dag (#21011)
• Fix Scheduler crash when executing task instances of missing DAG (#20349)
• Deferred tasks does not cancel when DAG is marked fail (#20649)
• Removed duplicated dag_run join in Dag.get_task_instances() (#20591)
• Avoid unintentional data loss when deleting DAGs (#20758)
• Fix session usage in /rendered-k8s view (#21006)
• Fix airflow dags backfill --reset-dagruns errors when run twice (#21062)
• Do not set TaskInstance.max_tries in refresh_from_task (#21018)
• Don't require dag_id in body in dagrun REST API endpoint (#21024)
• Add Roles from Azure OAUTH Response in internal Security Manager (#20707)
• Allow Viewing DagRuns and TIs if a user has DAG "read" perms (#20663)
• Fix running airflow dags test <dag_id> <execution_dt> results in error when run twice (#21031)
• Switch to non-vendored latest connexion library (#20910)
• Bump flask-appbuilder to >=3.3.4 (#20628)
• upgrade celery to 5.2.3 (#19703)
• Bump croniter from <1.1 to <1.2 (#20489)
• Lift off upper bound for MarkupSafe (#20113)
• Avoid calling DAG.following_schedule() for TaskInstance.get_template_context() (#20486)
• Fix(standalone): Remove hardcoded Webserver port (#20429)
• Remove unnecssary logging in experimental API (#20356)
• Un-ignore DeprecationWarning (#20322)
• Deepcopying Kubernetes Secrets attributes causing issues (#20318)
• Fix(dag-dependencies): fix arrow styling (#20303)
• Adds retry on taskinstance retrieval lock (#20030)
• Correctly send timing metrics when using dogstatsd (fix schedule_delay metric) (#19973)
• Enhance multiple_outputs inference of dict typing (#19608)
• Fixing ses email backend (#18042)
• Pin Markupsafe until we are able to upgrade Flask/Jinja (#21664)

Doc only changes

• Added explaining concept of logical date in DAG run docs (#21433)
• Add note about Variable precedence with env vars (#21568)
• Update error docs to include before_send option (#21275)
• Augment xcom docs (#20755)
• Add documentation and release policy on "latest" constraints (#21093)
• Add a link to the DAG model in the Python API reference (#21060)
• Added an enum param example (#20841)
• Compare taskgroup and subdag (#20700)
• Add note about reserved params keyword (#20640)
• Improve documentation on Params (#20567)
• Fix typo in MySQL Database creation code (Set up DB docs) (#20102)
• Add requirements.txt description (#20048)
• Clean up default_args usage in docs (#19803)
• Add docker-compose explanation to conn localhost (#19076)
• Update CSV ingest code for tutorial (#18960)
• Adds Pendulum 1.x -> 2.x upgrade documentation (#18955)
• Updating explicit arg example in TaskFlow API tutorial doc (#18907)
• Adds back documentation about context usage in Python/@task (#18868)
• Clean up dynamic start_date values from docs (#19607)
• Docs for multiple pool slots (#20257)
• Update upgrading.rst with detailed code example of how to resolve post-upgrade warning (#19993)

Misc

• Deprecate some functions in the experimental API (#19931)
• Deprecate smart sensors (#20151)

Apache Airflow Helm Chart 1.4.0

New Features

• Add type to extra secrets param (#20599)
• Support elasticsearch connection scheme (#20564)
• Allows to disable built-in secret variables individually (#18974)
• Add support for securityContext (#18249)
• Add extra containers, volumes and volume mounts for jobs (#18808)
• Allow ingress multiple hostnames w/diff secrets (#18542)
• PgBouncer extra volumes, volume mounts, and sslmode (#19749)
• Allow specifying kerberos keytab (#19054)
• Allow disabling the Helm hooks (#18776, #20018)
• Add migration-wait-timeout (#20069)

Improvements

• Increase default livenessProbe timeout (#20698)
• Strict schema for k8s objects for values.yaml (#19181)
• Remove unnecessary pod_template_file defaults (#19690)
• Use built-in check-migrations command for Airflow>=2 (#19676)

Bug Fixes

• Fix precedence of affinity, nodeSelector, and tolerations (#20641)
• Fix chart elasticsearch default port 80 to 9200. (#20616)
• Fix network policy issue for webserver and flowerui (#20199)
• Use local definitions for k8s schema validation (#20544)
• Add custom labels for ingresses/PVCs (#20535)
• Fix extra secrets/configmaps labels (#20464)
• Fix flower restarts on update (#20316)
• Properly quote namespace names (#20266)

Doc only changes

• Add helm dependency update step to chart INSTALL (#20702)
• Reword section covering the envvar secrets (#20566)
• Add "Customizing Workers" page (#20331)
• Include Datadog example in production guide (#17996)
• Update production Helm guide database section to use k8s secret (#19892)
• Fix multiNamespaceMode docs to also cover KPO (#19879)
• Clarify Helm behaviour when it comes to loading default connections (#19708)

Misc

• Use 2.2.3 as default Airflow version (#20450)
• Add ArtifactHUB annotations for docs and screenshots (#20558)
• Add kubernetes 1.21 support (#19557)

Apache Airflow 2.2.3

Bug Fixes

• Lazy Jinja2 context (#20217)
• Exclude snowflake-sqlalchemy v1.2.5 (#20245)
• Move away from legacy importlib.resources API (#19091)
• Move setgid as the first command executed in forked task runner (#20040)
• Fix race condition when starting DagProcessorAgent (#19935)
• Limit httpx to <0.20.0 (#20218)
• Log provider import errors as debug warnings (#20172)
• Bump minimum required alembic version (#20153)
• Fix log link in gantt view (#20121)
• fixing #19028 by moving chown to use sudo (#20114)
• Lift off upper bound for MarkupSafe (#20113)
• Fix infinite recursion on redact log (#20039)
• Fix db downgrades (#19994)
• Context class handles deprecation (#19886)
• Fix possible reference to undeclared variable (#19933)
• Validate DagRun state is valid on assignment (#19898)
• Workaround occasional deadlocks with MSSQL (#19856)
• Enable task run setting to be able reinitialise (#19845)
• Fix log endpoint for same task (#19672)
• Cast macro datetime string inputs explicitly (#19592)
• Do not crash with stacktrace when task instance is missing (#19478)
• Fix log timezone in task log view (#19342) (#19401)
• Fix: Add taskgroup tooltip to graph view (#19083)
• Rename execution date in forms and tables (#19063)
• Simplify "invalid TI state" message (#19029)
• Handle case of nonexistent file when preparing file path queue (#18998)
• Do not create dagruns for DAGs with import errors (#19367)
• Fix field relabeling when switching between conn types (#19411)
• KubernetesExecutor should default to template image if used (#19484)
• Fix task instance api cannot list task instances with None state (#19487)
• Fix IntegrityError in DagFileProcessor.manage_slas (#19553)
• Declare data interval fields as serializable (#19616)
• Relax timetable class validation (#19878)
• Fix labels used to find queued KubernetesExecutor pods (#19904)
• Fix moved data migration check for MySQL when replcation is used (#19999)

Doc only changes

• Warn without tracebacks when example_dags are missing deps (#20295)
• Deferrable operators doc clarification (#20150)
• Ensure the example DAGs are all working (#19355)
• Updating core example DAGs to use TaskFlow API where applicable (#18562)
• Add xcom clearing behaviour on task retries (#19968)
• Add a short chapter focusing on adapting secret format for connections (#19859)
• Add information about supported OS-es for Apache Airflow (#19855)
• Update docs to reflect that changes to the base_log_folder require updating other configs (#19793)
• Disclaimer in KubernetesExecutor pod template docs (#19686)
• Add upgrade note on execution_date -> run_id (#19593)
• Expanding .output operator property information in TaskFlow tutorial doc (#19214)
• Add example SLA DAG (#19563)
• Add a proper example to patch DAG (#19465)
• Add DAG file processing description to Scheduler Concepts (#18954)
• Updating explicit arg example in TaskFlow API tutorial doc (#18907)
• Adds back documentation about context usage in Python/@task (#18868)
• Add release date for when an endpoint/field is added in the REST API (#19203)
• Better pod_template_file examples (#19691)
• Add decription on how you can customize image entrypoint (#18915)
• Dags-in-image pod template example should not have dag mounts (#19337)

Apache Airflow Helm Chart 1.3.0

New Features

• Chart: Add resources for cleanup and createuser jobs (#19263)
• Chart: Add labels to jobs created by cleanup pods (#19225)
• Add migration job resources (#19175)
• Allow custom pod annotations to all components (#18481)
• Chart: Make PgBouncer cmd/args configurable (#18910)
• Chart: Use python 3.7 by default; support disabling triggerer (#18920)

Improvements

• Chart: Increase default liveness probe timeout (#19003)
• Chart: Mount DAGs in triggerer (#18753)

Bug Fixes

• Allow Airflow UI to create worker pod via Clear > Run (#18272)
• Allow airflow standard images to run in openshift utilising the official helm chart #18136 (#18147)

Doc only changes

• Chart: Fix extraEnvFrom examples (#19144)
• Chart docs: Update webserver secret key reference configuration (#18595)
• Fix helm chart links in source install guide (#18588)

Misc

• Chart: Update default Airflow version to 2.2.1 (#19326)
• Modernize dockerfiles builds (#19327)
• Chart: Use strict k8s schemas for template validation (#19379)

Apache Airflow 2.2.2

Bug Fixes

• Fix bug when checking for existence of a Variable (#19395)
• Fix Serialization whenrelativedelta is passed as schedule_interval (#19418)
• Fix moving of dangling TaskInstance rows for SQL Server (#19425)
• Fix task instance modal in gantt view (#19258)
• Fix serialization of Params with set data type (#19267)
• Check if job object is None before calling .is_alive() (#19380)
• Task should fail immediately when pod is unprocessable (#19359)
• Fix downgrade for a DB Migration (#19390)
• Only mark SchedulerJobs as failed, not any jobs (#19375)
• Fix message on "Mark as" confirmation page (#19363)
• Bugfix: Check next run exists before reading data interval (#19307)
• Fix MySQL db migration with default encoding/collation (#19268)
• Fix hidden tooltip position (#19261)
• sqlite_default Connection has been hard-coded to /tmp, use gettempdir instead (#19255)
• Fix Toggle Wrap on DAG code page (#19211)
• Clarify "dag not found" error message in CLI (#19338)
• Add Note to SLA regarding schedule_interval (#19173)
• Use execution_date to check for existing DagRun for TriggerDagRunOperator (#18968)
• Add explicit session parameter in PoolSlotsAvailableDep (#18875)
• FAB still requires WTForms<3.0 (#19466)
• Fix missing dagruns when catchup=True (#19528)

Doc only changes

• Add missing parameter documentation for "timetable" (#19282)
• Improve Kubernetes Executor docs (#19339)
• Update image tag used in docker docs

Apache Airflow 2.2.1

Bug Fixes

• Fix Unexpected commit error in schedulerjob (#19213)
• Add DagRun.logical_date as a property (#19198)
• Clear ti.next_method and ti.next_kwargs on task finish (#19183)
• Faster PostgreSQL db migration to Airflow 2.2 (#19166)
• Remove incorrect type comment in Swagger2Specification._set_defaults classmethod (#19065)
• Add TriggererJob to jobs check command (#19179, #19185)
• Hide tooltip when next run is None (#19112)
• Crerate TI context with data interval compat layer (#19148)
• Fix queued dag runs changes catchup=False behaviour (#19130, #19145)
• add detailed information to logging when a dag or a task finishes. (#19097)
• Warn about unsupported Python 3.10 (#19060)
• Fix catchup by limiting queued dagrun creation using max_active_runs (#18897)
• Prevent scheduler crash when serialized dag is missing (#19113)
• Don't install SQLAlchemy/Pendulum adapters for other DBs (#18745)
• Workaround libstdcpp TLS error (#19010)
• Change ds, ts, etc. back to use logical date (#19088)
• Ensure task state doesn't change when marked as failed/success/skipped (#19095)
• Relax packaging requirement (#19087)
• Rename trigger page label to Logical Date (#19061)
• Allow Param to support a default value of None (#19034)
• Upgrade old DAG/task param format when deserializing from the DB (#18986)
• Don't bake ENV and _cmd into tmp config for non-sudo (#18772)
• CLI: Fail backfill command before loading DAGs if missing args (#18994)
• BugFix: Null execution date on insert to task_fail violating NOT NULL (#18979)
• Try to move "dangling" rows in upgradedb (#18953)
• Row lock TI query in SchedulerJob._process_executor_events (#18975)
• Sentry before send fallback (#18980)
• Fix XCom.delete error in Airflow 2.2.0 (#18956)
• Check python version before starting triggerer (#18926)

Doc only changes

• Update access control documentation for TaskInstances and DagRuns (#18644)
• Add information about keepalives for managed Postgres (#18850)
• Doc: Add Callbacks Section to Logging & Monitoring (#18842)
• Group PATCH DAGrun together with other DAGRun endpoints (#18885)

Apache Airflow 2.2.0

New Features

• AIP-39: Add (customizable) Timetable class to Airflow for richer scheduling behaviour (#15397, #16030,
  #16352, #17030, #17122, #17414, #17552, #17755, #17989, #18084, #18088, #18244, #18266, #18420, #18434,
  #18421, #18475, #18499, #18573, #18522, #18729, #18706, #18742, #18786, #18804)
• AIP-40: Add Deferrable "Async" Tasks (#15389, #17564, #17565, #17601, #17745, #17747, #17748, #17875,
  #17876, #18129, #18210, #18214, #18552, #18728, #18414)
• Add a Docker Taskflow decorator (#15330, #18739)
• Add Airflow Standalone command (#15826)
• Display alert messages on dashboard from local settings (#18284)
• Advanced Params using json-schema (#17100)
• Ability to test connections from UI or API (#15795, #18750)
• Add Next Run to UI (#17732)
• Add default weight rule configuration option (#18627)
• Add a calendar field to choose the execution date of the DAG when triggering it (#16141)
• Allow setting specific cwd for BashOperator (#17751)
• Show import errors in DAG views (#17818)
• Add pre/post execution hooks [Experimental] (#17576)
• Added table to view providers in Airflow ui under admin tab (#15385)
• Adds secrets backend/logging/auth information to provider yaml (#17625)
• Add date format filters to Jinja environment (#17451)
• Introduce RESTARTING state (#16681)
• Webserver: Unpause DAG on manual trigger (#16569)
• API endpoint to create new user (#16609)
• Add insert_args for support transfer replace (#15825)
• Add recursive flag to glob in filesystem sensor (#16894)
• Add conn to jinja template context (#16686)
• Add default_args for TaskGroup (#16557)
• Allow adding duplicate connections from UI (#15574)
• Allow specifying multiple URLs via the CORS config option (#17941)
• Implement API endpoint for DAG deletion (#17980)
• Add DAG run endpoint for marking a dagrun success or failed(#17839)
• Add support for kinit options [-f|-F] and [-a|-A] (#17816)
• Queue support for DaskExecutor using Dask Worker Resources (#16829, #18720)
• Make auto refresh interval configurable (#18107)

Improvements

• Small improvements for Airflow UI (#18715, #18795)
• Rename processor_poll_interval to scheduler_idle_sleep_time (#18704)
• Check the allowed values for the logging level (#18651)
• Fix error on triggering a dag that doesn't exist using dagrun_conf (#18655)
• Add muldelete action to TaskInstanceModelView (#18438)
• Avoid importing DAGs during clean DB installation (#18450)
• Require can_edit on DAG privileges to modify TaskInstances and DagRuns (#16634)
• Make Kubernetes job description fit on one log line (#18377)
• Always draw borders if task instance state is null or undefined (#18033)
• Inclusive Language (#18349)
• Improved log handling for zombie tasks (#18277)
• Adding Variable.update method and improving detection of variable key collisions (#18159)
• Add note about params on trigger DAG page (#18166)
• Change TaskInstance and TaskReschedule PK from execution_date to run_id (#17719)
• Adding TaskGroup support in BaseOperator.chain() (#17456)
• Advises the kernel to not cache log files generated by Airflow (#18054)
• Allow filtering DAGS by tags in the REST API (#18090)
• Optimize imports of Providers Manager (#18052)
• Adds capability of Warnings for incompatible community providers (#18020)
• Serialize the template_ext attribute to show it in UI (#17985)
• Add robots.txt and X-Robots-Tag header (#17946)
• Refactor BranchDayOfWeekOperator, DayOfWeekSensor (#17940)
• Update error message to guide the user into self-help mostly (#17929)
• Update to Celery 5 (#17397)
• Avoid redirect loop for users with no permissions (#17838)
• Improve graph view load time for dags with open groups (#17821)
• Add links to provider's documentation (#17736)
• Remove Marshmallow schema warnings (#17753)
• Rename none_failed_or_skipped by none_failed_min_one_success trigger rule (#17683)
• Remove [core] store_dag_code & use DB to get Dag Code (#16342)
• Rename task_concurrency to max_active_tis_per_dag (#17708)
• Import Hooks lazily individually in providers manager (#17682)
• Adding support for multiple task-ids in the external task sensor (#17339)
• Replace execution_date with run_id in airflow tasks run command (#16666)
• Avoid endless redirect loop when user has no roles (#17613)
• Make output from users cli command more consistent (#17642)
• Open relative extra links in place (#17477)
• Move worker_log_server_port option to the logging section (#17621)
• Use gunicorn to serve logs generated by worker (#17591)
• Improve validation of Group id (#17578)
• Simplify 404 page (#17501)
• Add XCom.clear so it's hookable in custom XCom backend (#17405)
• Add deprecation notice for SubDagOperator (#17488)
• Support DAGS folder being in different location on scheduler and runners (#16860)
• Remove /dagrun/create and disable edit form generated by F.A.B (#17376)
• Enable specifying dictionary paths in template_fields_renderers (#17321)
• error early if virtualenv is missing (#15788)
• Handle connection parameters added to Extra and custom fields (#17269)
• Fix airflow celery stop to accept the pid file. (#17278)
• Remove DAG refresh buttons (#17263)
• Deprecate dummy trigger rule in favor of always (#17144)
• Be verbose about failure to import airflow_local_settings (#17195)
• Include exit code in AirflowException str when BashOperator fails. (#17151)
• Adding EdgeModifier support for chain() (#17099)
• Only allows supported field types to be used in custom connections (#17194)
• Secrets backend failover (#16404)
• Warn on Webserver when using SQLite or SequentialExecutor (#17133)
• Extend init_containers defined in pod_override (#17537)
• Client-side filter dag dependencies (#16253)
• Improve executor validation in CLI (#17071)
• Prevent running airflow db init/upgrade migrations and setup in parallel. (#17078)
• Update chain() and cross_downstream() to support XComArgs (#16732)
• Improve graph view refresh (#16696)
• When a task instance fails with exception, log it (#16805)
• Set process title for serve-logs and LocalExecutor (#16644)
• Rename test_cycle to check_cycle (#16617)
• Set process title for Worker when using LocalExecutor (#16623)
• Add schema as DbApiHook instance attribute (#16521, #17423)
• Improve compatibility with MSSQL (#9973)
• Add transparency for unsupported connection type (#16220)
• Call resource based fab methods (#16190)
• Format more dates with timezone (#16129)
• Replace deprecated dag.sub_dag with dag.partial_subset (#16179)
• Treat AirflowSensorTimeout as immediate failure without retrying (#12058)
• Marking success/failed automatically clears failed downstream tasks (#13037)
• Add close/open indicator for import dag errors (#16073)
• Add collapsible import errors (#16072)
• Always return a response in TI's action_clear view (#15980)
• Add cli command to delete user by email (#15873)
• Use resource and action names for FAB permissions (#16410)
• Rename DAG concurrency ([core] dag_concurrency) settings for easier understanding (#16267, #18730)
• Calendar UI improvements (#16226)
• Refactor: SKIPPED should not be logged again as SUCCESS (#14822)
• Remove version limits for dnspython (#18046, #18162)
• Accept custom run ID in TriggerDagRunOperator (#18788)

Bug Fixes

• Make REST API patch user endpoint work the same way as the UI (#18757)
• Properly set start_date for cleared tasks (#18708)
• Ensure task_instance exists before running update on its state(REST API) (#18642)
• Make AirflowDateTimePickerWidget a required field (#18602)
• Retry deadlocked transactions on deleting old rendered task fields (#18616)
• Fix retry_exponential_backoff divide by zero error when retry delay is zero (#17003)
• Improve how UI handles datetimes (#18611, #18700)
• Bugfix: dag_bag.get_dag should return None, not raise exception (#18554)
• Only show the task modal if it is a valid instance (#18570)
• Fix accessing rendered {{ task.x }} attributes from within templates (#18516)
• Add missing email type of connection (#18502)
• Don't use flash for "same-page" UI messages. (#18462)
• Fix task group tooltip (#18406)
• Properly fix dagrun update state endpoint (#18370)
• Properly handle ti state difference between executor and scheduler (#17819)
• Fix stuck "queued" tasks in KubernetesExecutor (#18152)
• Don't permanently add zip DAGs to sys.path (#18384)
• Fix random deadlocks in MSSQL database (#18362)
• Deactivating DAGs which have been removed from files (#17121)
• When syncing dags to db remove dag_tag rows that are now unused (#8231)
• Graceful scheduler shutdown on error (#18092)
• Fix mini scheduler not respecting wait_for_downstream dep (#18338)
• Pass exception to run_finished_callback for Debug Executor (#17983)
• Make XCom.get_one return full, not abbreviated values (#18274)
• Use try/except when closing temporary file in task_runner (#18269)
• show next run if not none (#18273)
• Fix DB session handling in XCom.set (#18240)
• Sort adopted tasks in _check_for_stalled_adopted_tasks method (#18208)
• Fix external_executor_id not being set for manually run jobs (#17207)
• Fix deleting of zipped Dags in Serialized Dag Table (#18243)
• Return explicit error on user-add for duplicated email (#18224)
• Remove loading dots even when last run data is empty (#18230)
• Swap dag import error dropdown icons (#18207)
• Automatically create section when migrating config (#16814)
• Set encoding to utf-8 by default while reading task logs (#17965)
• Set task state to failed when pod is DELETED while running (#18095)
• Apply parent dag permissions to subdags (#1816...