Airflow 1.10.10, 2020-04-09
New Features
- [AIRFLOW-7048] Allow user to chose timezone to use in UI (#8046)
- Add Production Docker image support (#7832)
- Get Airflow Variables from Environment Variables (#7923)
- Get Airflow Variables from Hashicorp Vault (#7944)
- Get Airflow Variables from AWS Systems Manager Parameter Store (#7945)
- Get Airflow Variables from GCP Secrets Manager (#7946)
- [AIRFLOW-5705] Add secrets backend and support for AWS SSM / Get Airflow Connections from AWS Parameter Store(#6376)
- [AIRFLOW-7104] Add Secret backend for GCP Secrets Manager / Get Airflow Connections from GCP Secrets Manager (#7795)
- [AIRFLOW-7076] Add support for HashiCorp Vault as Secrets Backend / Get Airflow Connections from Hashicorp Vault (#7741)
- [AIRFLOW-6685] Add ThresholdCheckOperator (#7353)
- [AIRFLOW-7080] Add API endpoint to return a DAG's paused state (#7737)
Improvements
- Use same tooltip for Graph and Tree views for TaskInstances (#8043)
- Allow DateTimePicker in Webserver to actually pick times too (#8034)
- [AIRFLOW-5590] Add run_id to trigger DAG run API response (#6256)
- [AIRFLOW-6695] Can now pass dagrun conf when triggering dags via UI (#7312)
- [AIRFLOW-5336] Add ability to make updating FAB perms on webserver in it optional (#5940)
- [AIRFLOW-1467] Allow tasks to use more than one pool slot (#7160)
- [AIRFLOW-6987] Avoid creating default connections (#7629)
- [AIRFLOW-4175] S3Hook load_file should support ACL policy parameter (#7733)
- [AIRFLOW-4438] Add Gzip compression to S3_hook (#7680)
- Allow setting Airflow Variable values to empty string (#8021)
- Dont schedule dummy tasks (#7880)
- Prevent sequential scan of task instance table (#8014)
- [AIRFLOW-7017] Respect default dag view in trigger dag origin (#7667)
- [AIRFLOW-6837] Limit description length of a Dag on HomePage (#7457)
- [AIRFLOW-6989] Display Rendered template_fields without accessing Dag files (#7633)
- [AIRFLOW-5944] Rendering templated_fields without accessing DAG files (#6788)
- [AIRFLOW-5946] DAG Serialization: Store source code in db (#7217)
- [AIRFLOW-7079] Remove redundant code for storing template_fields (#7750)
- [AIRFLOW-7024] Add the verbose parameter support to SparkSqlOperator (#7676)
- [AIRFLOW-6733] Extend not replace template (#7366)
- [AIRFLOW-7001] Further fix for the MySQL 5.7 UtcDateTime (#7655)
- [AIRFLOW-6014] Handle pods which are preempted & deleted by kubernetes but not restarted (#6606)
- [AIRFLOW-6950] Remove refresh_executor_config from ti.refresh_from_db (#7577)
- [AIRFLOW-7016] Sort dag tags in the UI (#7661)
- [AIRFLOW-6762] Fix link to "Suggest changes on this page" (#7387)
- [AIRFLOW-6948] Remove ASCII Airflow from version command (#7572)
- [AIRFLOW-6767] Correct name for default Athena workgroup (#7394)
- [AIRFLOW-6905] Update pin.svg with new pinwheel (#7524)
- [AIRFLOW-6801] Make use of ImportError.timestamp (#7425)
- [AIRFLOW-6830] Add Subject/MessageAttributes to SNS hook and operator (#7451)
- [AIRFLOW-6630] Resolve handlebars advisory (#7284)
- [AIRFLOW-6945] MySQL 5.7 is used in v1-10-test as an option
- [AIRFLOW-6871] Optimize tree view for large DAGs (#7492)
- [AIRFLOW-7063] Fix dag.clear() slowness caused by count (#7723)
- [AIRFLOW-7023] Remove duplicated package definitions in setup.py (#7675)
- [AIRFLOW-7001] Time zone removed from MySQL TIMSTAMP field inserts
- [AIRFLOW-7105] Unify Secrets Backend method interfaces (#7830)
- Make BaseSecretsBackend.build_path generic (#7948)
- Allow hvac pakage installation using 'hashicorp' extra (#7915)
- Standardize SecretBackend class names (#7846)
- [AIRFLOW-5705] Make AwsSsmSecretsBackend consistent with VaultBackend (#7753)
- [AIRFLOW-7045] Update SQL query to delete RenderedTaskInstanceFields (#8051)
- Handle DST better in Task Instance tool tips (#8104)
Bug Fixes
- BugFix: Show task_id in the Graph View tooltip (#7859)
- [AIRFLOW-6730] Use total_seconds instead of seconds (#7363)
- [AIRFLOW-6167] Escape column name in create table in hive (#6741)
- [AIRFLOW-6628] DAG auto-complete now suggests from all acessible DAGs (#7251)
- [AIRFLOW-7113] Fix gantt render error (#7913)
- [AIRFLOW-6399] Add _access control to validate deserialized DAGs (#7896)
- [AIRFLOW-6399] Serialization: DAG access_control field should be decorated field in DAG serialization (#7879)
- [AIRFLOW-4453] Make behavior of
none_failed
consistent with documentation (#7464) - [AIRFLOW-4363] Fix JSON encoding error (#7628)
- [AIRFLOW-6683] Run REST API tests when DAGs are serialized (#7352)
- [AIRFLOW-6704] Copy common TaskInstance attributes from Task (#7324)
- [AIRFLOW-6734] Use configured base_template instead of hard-coding (#7367)
- [AIRFLOW-7098] Simple salesforce release 1.0.0 breaks the build (#7775)
- [AIRFLOW-6062] Executor would only delete workers in its own namespace (#7123)
- [AIRFLOW-7074] Add Permissions to view SubDAGs (#7752)
- [AIRFLOW-7025] Fix SparkSqlHook.run_query to handle its parameter properly (#7677)
- [AIRFLOW-6855] Escape project_dataset_table in SQL query in gcs to bq operator (#7475)
- [AIRFLOW-6949] Respect explicit conf to SparkSubmitOperator (#7575)
- [AIRFLOW-6588] write_stdout and json_format are boolean (#7199)
- [AIRFLOW-3439] Decode logs with 'utf-8' (#4474)
- [AIRFLOW-6878] Fix misconfigured default value for kube_client_request_args
- [AIRFLOW-5167] Update dependencies for GCP packages (#7116)
- [AIRFLOW-6821] Success callback not called when task marked as success from UI (#7447)
- [AIRFLOW-6740] Remove Undocumented, deprecated, dysfunctional PROXY_FIX_NUM_PROXIES (#7359)
- [AIRFLOW-6728] Change various DAG info methods to POST (#7364)
- [AIRFLOW-6997] Make sure worker pods initcontainers obtain env vars from config (#7663)
- [AIRFLOW-7062] Fix pydruid release breaking the build (#7720)
- [AIRFLOW-6040] ReadTimoutError in KubernetesExecutor should not raise exception (#7616)
- [AIRFLOW-6943] Fix utf-8 encodeed description in DAG in Python 2 (#7567)
- [AIRFLOW-6892] Fix broken non-wheel releases (#7514)
- [AIRFLOW-6789] BugFix: Fix Default Worker concurrency (#7494)
- [AIRFLOW-6840] Bump up version of future (#7471)
- [AIRFLOW-5705] Fix bugs in AWS SSM Secrets Backend (#7745)
- [AIRFLOW-5705] Fix bug in Secrets Backend (#7742)
- Fix CloudSecretsManagerBackend invalid connections_prefix (#7861)
- [AIRFLOW-7045] BugFix: DebugExecutor fails to change task state. (#8073)
- BugFix: Datetimepicker is stuck on the UI (#8092)
- [AIRFLOW-5277] Gantt chart respects per-user the Timezone UI setting (#8096)
- Fix timezones displayed in Task Instance tooltip (#8103)
- BugFix: Fix writing & deleting Dag Code for Serialized DAGs (#8151)
- Make the default TI pool slots '1' (#8153)
- Fix 500 error in Security screens (#8165)
- Fix Viewing Dag Code for Stateless Webserver (#8159)
- Fix issue with SQLAlchemy 1.3.16 (#8230)
Misc/Internal
- Fix Flaky TriggerDAG UI test (#8022)
- Remove unnecessary messages in CI (#7951)
- Fixes too high parallelism in CI (#7947)
- Install version is not persistent in breeze (#7914)
- Fixed automated check for image rebuild (#7912)
- Move Dockerfile to Dockerfile.ci (#7829)
- Generate requirements are now sorted (#8040)
- Change name of the common environment initialization function (#7805)
- Requirements now depend on python version (#7841)
- Bring back reset db explicitly called at CI entry (#7798)
- Fixes unclean installation of Airflow 1.10 (#7796)
- Change name of the common environment initialization function (#7805)
- [AIRFLOW-7029] Use separate docker image for running license check (#7678)
- [AIRFLOW-5842] Swtch to Debian buster image as a base (#7647)
- [AIRFLOW-5828] Move build logic out from hooks/build (#7618)
- [AIRFLOW-6839] Even more mypy speed improvements (#7460)
- [AIRFLOW-6820] split breeze into functions (#7433)
- [AIRFLOW-7097] Install gcloud beta componensts in CI image (#7772)
- [AIRFLOW-7018] fixing travis's job name escaping problem (#7668)
- [AIRFLOW-7054] Breeze has an option now to reset db at entry (#7710)
- [AIRFLOW-7005] Added exec command to Breeze (#7649)
- [AIRFLOW-7015] Detect Dockerhub repo/user when building on Dockerhub (#7673)
- [AIRFLOW-6727] Fix minor bugs in Release Management scripts (#7355)
- [AIRFLOW-7013] Automated check if Breeze image needs to be pulled (#7656)
- [AIRFLOW-7010] Skip in-container checks for Dockerhub builds (#7652)
- [AIRFLOW-7011] Pin JPype release to allow to build 1.10 images
- [AIRFLOW-7006] Fix missing +e in Breeze script (#7648)
- [AIRFLOW-6979] Fix breeze test-target specific test param issue (#7614)
- [AIRFLOW-6932] Add restart-environment command to Breeze
- [AIRFLOW-6919] Make Breeze DAG-test friedly (#7539)
- [AIRFLOW-6838] Introduce real subcommands for Breeze (#7515)
- [AIRFLOW-6763] Make systems tests ready for backport tests (#7389)
- [AIRFLOW-6866] Fix wrong export for Mac on Breeze (#7485)
- [AIRFLOW-6842] Skip fixing ownership on Mac (#7469)
- [AIRFLOW-6841] Fixed unbounded variable on Mac (#7465)
- [AIRFLOW-7067] Pinned version of Apache Airflow (#7730)
- [AIRFLOW-7058] Add support for different DB versions (#7717)
- [AIRFLOW-7002] Get rid of yaml "parser" in bash (#7646)
- [AIRFLOW-6972] Shorter frequently used commands in Breeze (#7608)
Doc only changes
- Fix typo for store_serialized_dags config (#7952)
- Fix broken link in README.md (#7893)
- Separate supported Postgres versions with comma (#7892)
- Fix grammar in setup.py (#7877)
- Add Jiajie Zhong to committers list (#8047)
- Update Security doc for 1.10.* for auth backends (#8072)
- Fix Example in config_templates for Secrets Backend (#8074)
- Add backticks in IMAGES.rst command description (#8075)
- Change version_added for store_dag_code config (#8076)
- [AIRFLOW-XXXX] Remove duplicate docs
- [AIRFLOW-XXXX] Remove the defunct limitation of Dag Serialization (#7716)
- [AIRFLOW-XXXX] Add prerequisite tasks for all GCP operators guide (#6049)
- [AIRFLOW-XXXX] Simplify AWS/Azure/Databricks operators listing (#6047)
- [AIRFLOW-XXXX] Add external reference to all GCP operator guide (#6048)
- [AIRFLOW-XXXX] Simplify GCP operators listing
- [AIRFLOW-XXXX] Simplify Qubole operators listing
- [AIRFLOW-XXXX] Add autogenerated TOC (#6038)
- [AIRFLOW-XXXX] Create "Using the CLI" page (#5823)
- [AIRFLOW-XXXX] Group references in one section (#5776)
- [AIRFLOW-XXXX] Fix analytics doc (#5885)
- [AIRFLOW-XXXX] Add S3 Logging section (#6039)
- [AIRFLOW-XXXX] Move Azure Logging section above operators (#6040)
- [AIRFLOW-XXXX] Update temp link to a fixed link (#7715)
- [AIRFLOW-XXXX] Add Updating.md section for 1.10.9 (#7385)
- [AIRFLOW-XXXX] Remove duplication in BaseOperator docstring (#7321)
- [AIRFLOW-XXXX] Update tests info in CONTRIBUTING.rst (#7466)
- [AIRFLOW-XXXX] Small BREEZE.rst update (#7487)
- [AIRFLOW-XXXX] Add instructions for logging to localstack S3 (#7461)
- [AIRFLOW-XXXX] Remove travis config warnings (#7467)
- [AIRFLOW-XXXX] Add communication chapter to contributing (#7204)
- [AIRFLOW-XXXX] Add known issue - example_dags/init.py (#7444)
- [AIRFLOW-XXXX] Fix breeze build-docs (#7445)
- [AIRFLOW-XXXX] Less verbose docker builds
- [AIRFLOW-XXXX] Speed up mypy runs (#7421)
- [AIRFLOW-XXXX] Fix location of kubernetes tests (#7373)
- [AIRFLOW-XXXX] Remove quotes from domains in Google Oauth (#4226)
- [AIRFLOW-XXXX] Add explicit info about JIRAs for code-related PRs (#7318)
- [AIRFLOW-XXXX] Fix typo commiter => committer (#7392)
- [AIRFLOW-XXXX] Remove duplicated paragraph in docs (#7662)
- Fix reference to KubernetesPodOperator (#8100)