forked from apache/airflow
-
Notifications
You must be signed in to change notification settings - Fork 0
/
CHANGELOG.txt
2041 lines (2018 loc) · 109 KB
/
CHANGELOG.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
AIRFLOW 1.10.1, 2018-11-13
New features:
[AIRFLOW-2524] Airflow integration with AWS Sagemaker
[AIRFLOW-2657] Add ability to delete DAG from web ui
[AIRFLOW-2780] Adds IMAP Hook to interact with a mail server
[AIRFLOW-2794] Add delete support for Azure blob
[AIRFLOW-2912] Add operators for Google Cloud Functions
[AIRFLOW-2974] Add Start/Restart/Terminate methods Databricks Hook
[AIRFLOW-2989] No Parameter to change bootDiskType for DataprocClusterCreateOperator
[AIRFLOW-3078] Basic operators for Google Compute Engine
[AIRFLOW-3147] Update Flask-AppBuilder version
[AIRFLOW-3231] Basic operators for Google Cloud SQL (deploy / patch / delete)
[AIRFLOW-3276] Google Cloud SQL database create / patch / delete operators
Improvements:
[AIRFLOW-393] Add progress callbacks for FTP downloads
[AIRFLOW-520] Show Airflow version on web page
[AIRFLOW-843] Exceptions now available in context during on_failure_callback
[AIRFLOW-2476] Update tabulate dependency to v0.8.2
[AIRFLOW-2592] Bump Bleach dependency
[AIRFLOW-2622] Add "confirm=False" option to SFTPOperator
[AIRFLOW-2662] support affinity & nodeSelector policies for kubernetes executor/operator
[AIRFLOW-2709] Improve error handling in Databricks hook
[AIRFLOW-2723] Update lxml dependency to >= 4.0.
[AIRFLOW-2763] No precheck mechanism in place during worker initialisation for the connection to metadata database
[AIRFLOW-2789] Add ability to create single node cluster to DataprocClusterCreateOperator
[AIRFLOW-2797] Add ability to create Google Dataproc cluster with custom image
[AIRFLOW-2854] kubernetes_pod_operator add more configuration items
[AIRFLOW-2855] Need to Check Validity of Cron Expression When Process DAG File/Zip File
[AIRFLOW-2904] Clean an unnecessary line in airflow/executors/celery_executor.py
[AIRFLOW-2921] A trivial incorrectness in CeleryExecutor()
[AIRFLOW-2922] Potential deal-lock bug in CeleryExecutor()
[AIRFLOW-2932] GoogleCloudStorageHook - allow compression of file
[AIRFLOW-2949] Syntax Highlight for Single Quote
[AIRFLOW-2951] dag_run end_date Null after a dag is finished
[AIRFLOW-2956] Kubernetes tolerations for pod operator
[AIRFLOW-2997] Support for clustered tables in Bigquery hooks/operators
[AIRFLOW-3006] Fix error when schedule_interval="None"
[AIRFLOW-3008] Move Kubernetes related example DAGs to contrib/example_dags
[AIRFLOW-3025] Allow to specify dns and dns-search parameters for DockerOperator
[AIRFLOW-3067] (www_rbac) Flask flash messages are not displayed properly (no background color)
[AIRFLOW-3069] Decode output of S3 file transform operator
[AIRFLOW-3072] Assign permission get_logs_with_metadata to viewer role
[AIRFLOW-3090] INFO logs are too verbose
[AIRFLOW-3103] Update Flask-Login
[AIRFLOW-3112] Align SFTP hook with SSH hook
[AIRFLOW-3119] Enable loglevel on celery worker and inherit from airflow.cfg
[AIRFLOW-3137] Make ProxyFix middleware optional
[AIRFLOW-3173] Add _cmd options for more password config options
[AIRFLOW-3177] Change scheduler_heartbeat metric from gauge to counter
[AIRFLOW-3193] Pin docker requirement version to v3
[AIRFLOW-3195] Druid Hook: Log ingestion spec and task id
[AIRFLOW-3197] EMR Hook is missing some parameters to valid on the AWS API
[AIRFLOW-3232] Make documentation for GCF Functions operator more readable
[AIRFLOW-3262] Can't get log containing Response when using SimpleHttpOperator
[AIRFLOW-3265] Add support for "unix_socket" in connection extra for Mysql Hook
Doc-only changes:
[AIRFLOW-1441] Tutorial Inconsistencies Between Example Pipeline Definition and Recap
[AIRFLOW-2682] Add how-to guide(s) for how to use basic operators like BashOperator and PythonOperator
[AIRFLOW-3104] .airflowignore feature is not mentioned at all in documentation
[AIRFLOW-3237] Refactor example DAGs
[AIRFLOW-3187] Update airflow.gif file with a slower version
[AIRFLOW-3159] Update Airflow documentation on GCP Logging
[AIRFLOW-3030] Command Line docs incorrect subdir
[AIRFLOW-2990] Docstrings for Hooks/Operators are in incorrect format
[AIRFLOW-3127] Celery SSL Documentation is out-dated
[AIRFLOW-2779] Add license headers to doc files
[AIRFLOW-2779] Add project version to license
Bug fixes:
[AIRFLOW-839] docker_operator.py attempts to log status key without first checking existence
[AIRFLOW-1104] Concurrency check in scheduler should count queued tasks as well as running
[AIRFLOW-1163] Add support for x-forwarded-* headers to support access behind AWS ELB
[AIRFLOW-1195] Cleared tasks in SubDagOperator do not trigger Parent dag_runs
[AIRFLOW-1508] Skipped state not part of State.task_states
[AIRFLOW-1762] Use key_file in SSHHook.create_tunnel()
[AIRFLOW-1837] Differing start_dates on tasks not respected by scheduler.
[AIRFLOW-1874] Support standard SQL in Check, ValueCheck and IntervalCheck BigQuery operators
[AIRFLOW-1917] print() from python operators end up with extra new line
[AIRFLOW-1970] Database cannot be initialized if an invalid fernet key is provided
[AIRFLOW-2145] Deadlock after clearing a running task
[AIRFLOW-2216] Cannot specify a profile for AWS Hook to load with s3 config file
[AIRFLOW-2574] initdb fails when mysql password contains percent sign
[AIRFLOW-2707] Error accessing log files from web UI
[AIRFLOW-2716] Replace new Python 3.7 keywords
[AIRFLOW-2744] RBAC app doesn't integrate plugins (blueprints etc)
[AIRFLOW-2772] BigQuery hook does not allow specifying both the partition field name and table name at the same time
[AIRFLOW-2778] Bad Import in collect_dag in DagBag
[AIRFLOW-2786] Variables view fails to render if a variable has an empty key
[AIRFLOW-2799] Filtering UI objects by datetime is broken
[AIRFLOW-2800] Remove airflow/ low-hanging linting errors
[AIRFLOW-2825] S3ToHiveTransfer operator may not may able to handle GZIP file with uppercase ext in S3
[AIRFLOW-2848] dag_id is missing in metadata table "job" for LocalTaskJob
[AIRFLOW-2860] DruidHook: time variable is not updated correctly when checking for timeout
[AIRFLOW-2865] Race condition between on_success_callback and LocalTaskJob's cleanup
[AIRFLOW-2893] Stuck dataflow job due to jobName mismatch.
[AIRFLOW-2895] Prevent scheduler from spamming heartbeats/logs
[AIRFLOW-2900] Code not visible for Packaged DAGs
[AIRFLOW-2905] Switch to regional dataflow job service.
[AIRFLOW-2907] Sendgrid - Attachments - ERROR - Object of type 'bytes' is not JSON serializable
[AIRFLOW-2938] Invalid 'extra' field in connection can raise an AttributeError when attempting to edit
[AIRFLOW-2979] Deprecated Celery Option not in Options list
[AIRFLOW-2981] TypeError in dataflow operators when using GCS jar or py_file
[AIRFLOW-2984] Cannot convert naive_datetime when task has a naive start_date/end_date
[AIRFLOW-2994] flatten_results in BigQueryOperator/BigQueryHook should default to None
[AIRFLOW-3002] ValueError in dataflow operators when using GCS jar or py_file
[AIRFLOW-3012] Email on sla miss is send only to first address on the list
[AIRFLOW-3046] ECS Operator mistakenly reports success when task is killed due to EC2 host termination
[AIRFLOW-3064] No output from `airflow test` due to default logging config
[AIRFLOW-3072] Only admin can view logs in RBAC UI
[AIRFLOW-3079] Improve initdb to support MSSQL Server
[AIRFLOW-3089] Google auth doesn't work under http
[AIRFLOW-3099] Errors raised when some blocs are missing in airflow.cfg
[AIRFLOW-3109] Default user permission should contain 'can_clear'
[AIRFLOW-3111] Confusing comments and instructions for log templates in UPDATING.md and default_airflow.cfg
[AIRFLOW-3124] Broken webserver debug mode (RBAC)
[AIRFLOW-3136] Scheduler Failing the Task retries run while processing Executor Events
[AIRFLOW-3138] Migration cc1e65623dc7 creates issues with postgres
[AIRFLOW-3161] Log Url link does not link to task instance logs in RBAC UI
[AIRFLOW-3162] HttpHook fails to parse URL when port is specified
[AIRFLOW-3183] Potential Bug in utils/dag_processing/DagFileProcessorManager.max_runs_reached()
[AIRFLOW-3203] Bugs in DockerOperator & Some operator test scripts were named incorrectly
[AIRFLOW-3238] Dags, removed from the filesystem, are not deactivated on initdb
[AIRFLOW-3268] Cannot pass SSL dictionary to mysql connection via URL
[AIRFLOW-3277] Invalid timezone transition handling for cron schedules
[AIRFLOW-3295] Require encryption in DaskExecutor when certificates are configured.
[AIRFLOW-3297] EmrStepSensor marks cancelled step as successful
AIRFLOW 1.10.0, 2018-08-03
--------------------------
[AIRFLOW-2870] Use abstract TaskInstance for migration
[AIRFLOW-2859] Implement own UtcDateTime (#3708)
[AIRFLOW-2140] Don't require kubernetes for the SparkSubmit hook
[AIRFLOW-2869] Remove smart quote from default config
[AIRFLOW-2857] Fix Read the Docs env
[AIRFLOW-2817] Force explicit choice on GPL dependency
[AIRFLOW-2716] Replace async and await py3.7 keywords
[AIRFLOW-2810] Fix typo in Xcom model timestamp
[AIRFLOW-2710] Clarify fernet key value in documentation
[AIRFLOW-2606] Fix DB schema and SQLAlchemy model
[AIRFLOW-2646] Fix setup.py not to install snakebite on Python3
[AIRFLOW-2604] Add index to task_fail
[AIRFLOW-2650] Mark SchedulerJob as succeed when hitting Ctrl-c
[AIRFLOW-2678] Fix db schema unit test to remove checking fab models
[AIRFLOW-2624] Fix webserver login as anonymous
[AIRFLOW-2654] Fix incorret URL on refresh in Graph View of FAB UI
[AIRFLOW-2668] Handle missing optional cryptography dependency
[AIRFLOW-2681] Include last dag run of externally triggered DAGs in UI.
[AIRFLOW-1840] Support back-compat on old celery config
[AIRFLOW-2612][AIRFLOW-2534] Clean up Hive-related tests
[AIRFLOW-2608] Implements/Standardize custom exceptions for experimental APIs
[AIRFLOW-2607] Fix failing TestLocalClient
[AIRFLOW-2638] dbapi_hook: support REPLACE INTO
[AIRFLOW-2542][AIRFLOW-1790] Rename AWS Batch Operator queue to job_queue
[AIRFLOW-2567] Extract result from the kubernetes pod as Xcom
[AIRFLOW-XXX] Adding REA Group to readme
[AIRFLOW-2601] Allow user to specify k8s config
[AIRFLOW-2559] Azure Fileshare hook
[AIRFLOW-1786] Enforce correct behavior for soft-fail sensors
[AIRFLOW-2355] Airflow trigger tag parameters in subdag
[AIRFLOW-2613] Fix Airflow searching .zip bug
[AIRFLOW-2627] Add a sensor for Cassandra
[AIRFLOW-2634][AIRFLOW-2534] Remove dependency for impyla
[AIRFLOW-2611] Fix wrong dag volume mount path for kubernetes executor
[AIRFLOW-2562] Add Google Kubernetes Engine Operators
[AIRFLOW-2630] Fix classname in test_sql_sensor.py
[AIRFLOW-2534] Fix bug in HiveServer2Hook
[AIRFLOW-2586] Stop getting AIRFLOW_HOME value from config file in bash operator
[AIRFLOW-2605] Fix autocommit for MySqlHook
[AIRFLOW-2539][AIRFLOW-2359] Move remaing log config to configuration file
[AIRFLOW-1656] Tree view dags query changed
[AIRFLOW-2617] add imagePullPolicy config for kubernetes executor
[AIRFLOW-2429] Fix security/task/sensors/ti_deps folders flake8 error
[AIRFLOW-2550] Implements API endpoint to list DAG runs
[AIRFLOW-2512][AIRFLOW-2522] Use google-auth instead of oauth2client
[AIRFLOW-2429] Fix operators folder flake8 error
[AIRFLOW-2585] Fix several bugs in CassandraHook and CassandraToGCSOperator
[AIRFLOW-2597] Restore original dbapi.run() behavior
[AIRFLOW-2590] Fix commit in DbApiHook.run() for no-autocommit DB
[AIRFLOW-1115] fix github oauth api URL
[AIRFLOW-2587] Add TIMESTAMP type mapping to MySqlToHiveTransfer
[AIRFLOW-2591][AIRFLOW-2581] Set default value of autocommit to False in DbApiHook.run()
[AIRFLOW-59] Implement bulk_dump and bulk_load for the Postgres hook
[AIRFLOW-2533] Fix path to DAG's on kubernetes executor workers
[AIRFLOW-2581] RFLOW-2581] Fix DbApiHook autocommit
[AIRFLOW-2578] Add option to use proxies in JiraHook
[AIRFLOW-2575] Make gcs to gcs operator work with large files
[AIRFLOW-437] Send TI context in kill zombies
[AIRFLOW-2566] Change backfill to rerun failed tasks
[AIRFLOW-1021] Fix double login for new users with LDAP
[AIRFLOW-XXX] Typo fix
[AIRFLOW-2561] Fix typo in EmailOperator
[AIRFLOW-2573] Cast BigQuery TIMESTAMP field to float
[AIRFLOW-2560] Adding support for internalIpOnly to DataprocClusterCreateOperator
[AIRFLOW-2565] templatize cluster_label
[AIRFLOW-83] add mongo hook and operator
[AIRFLOW-2558] Clear task/dag is clearing all executions
[AIRFLOW-XXX] Fix doc typos
[AIRFLOW-2513] Change `bql` to `sql` for BigQuery Hooks & Ops
[AIRFLOW-2557] Fix pagination for s3
[AIRFLOW-2545] Eliminate DeprecationWarning
[AIRFLOW-2500] Fix MySqlToHiveTransfer to transfer unsigned type properly
[AIRFLOW-2462] Change PasswordUser setter to correct syntax
[AIRFLOW-2525] Fix a bug introduced by commit dabf1b9
[AIRFLOW-2553] Add webserver.pid to .gitignore
[AIRFLOW-1863][AIRFLOW-2529] Add dag run selection widgets to gantt view
[AIRFLOW-2504] Log username correctly and add extra to search columns
[AIRFLOW-2551] Encode binary data with base64 standard rather than base64 url
[AIRFLOW-2537] Add reset-dagrun option to backfill command
[AIRFLOW-2526] dag_run.conf can override params
[AIRFLOW-2544][AIRFLOW-1967] Guard against next major release of Celery, Flower
[AIRFLOW-XXX] Add Yieldr to who is using airflow
[AIRFLOW-2547] Describe how to run tests using Docker
[AIRFLOW-2538] Update faq doc on how to reduce airflow scheduler latency
[AIRFLOW-2529] Improve graph view performance and usability
[AIRFLOW-2517] backfill support passing key values through CLI
[AIRFLOW-2532] Support logs_volume_subpath for KubernetesExecutor
[AIRFLOW-2466] consider task_id in _change_state_for_tis_without_dagrun
[AIRFLOW-2519] Fix CeleryExecutor with SQLAlchemy
[AIRFLOW-2402] Fix RBAC task log
[AIRFLOW-XXX] Add M4U to user list
[AIRFLOW-2536] docs about how to deal with airflow initdb failure
[AIRFLOW-2530] KubernetesOperator supports multiple clusters
[AIRFLOW-1499] Eliminate duplicate and unneeded code
[AIRFLOW-2521] backfill - make variable name and logging messages more acurate
[AIRFLOW-2429] Fix hook, macros folder flake8 error
[Airflow-XXX] add Prime to company list
[AIRFLOW-2525] Fix PostgresHook.copy_expert to work with "COPY FROM"
[AIRFLOW-2515] Add dependency on thrift_sasl to hive extra
[AIRFLOW-2523] Add how-to for managing GCP connections
[AIRFLOW-2510] Introduce new macros: prev_ds and next_ds
[AIRFLOW-1730] Unpickle value of XCom queried from DB
[AIRFLOW-2518] Fix broken ToC links in integration.rst
[AIRFLOW-1472] Fix SLA misses triggering on skipped tasks.
[AIRFLOW-2520] CLI - make backfill less verbose
[AIRFLOW-2107] add time_partitioning to run_query on BigQueryBaseCursor
[AIRFLOW-1057][AIRFLOW-1380][AIRFLOW-2362][2362] AIRFLOW Update DockerOperator to new API
[AIRFLOW-2415] Make airflow DAG templating render numbers
[AIRFLOW-2473] Fix wrong skip condition for TransferTests
[AIRFLOW-2472] Implement MySqlHook.bulk_dump
[AIRFLOW-2419] Use default view for subdag operator
[AIRFLOW-2498] Fix Unexpected argument in SFTP Sensor
[AIRFLOW-2509] Separate config docs into how-to guides
[AIRFLOW-2429] Add BaseExecutor back
[AIRFLOW-2429] Fix dag, example_dags, executors flake8 error
[AIRFLOW-2502] Change Single triple quotes to double for docstrings
[AIRFLOW-2503] Fix broken links in CONTRIBUTING.md
[AIRFLOW-2501] Refer to devel instructions in docs contrib guide
[AIRFLOW-2429] Fix contrib folder's flake8 errors
[AIRFLOW-2471] Fix HiveCliHook.load_df to use unused parameters
[AIRFLOW-2495] Update celery to 4.1.1
[AIRFLOW-2429] Fix api, bin, config_templates folders flake8 error
[AIRFLOW-2493] Mark template_fields of all Operators in the API document as "templated"
[AIRFLOW-2489] Update FlaskAppBuilder to 1.11.1
[AIRFLOW-2448] Enhance HiveCliHook.load_df to work with datetime
[AIRFLOW-2487] Enhance druid ingestion hook
[AIRFLOW-2397] Support affinity policies for Kubernetes executor/operator
[AIRFLOW-2482] Add test for rewrite method in GCS Hook
[AIRFLOW-2481] Fix flaky Kubernetes test
[AIRFLOW-2479] Improve doc FAQ section
[AIRFLOW-2485] Fix Incorrect logging for Qubole Sensor
[AIRFLOW-2486] Remove unnecessary slash after port
[AIRFLOW-2429] Make Airflow flake8 compliant
[AIRFLOW-2491] Resolve flask version conflict
[AIRFLOW-2484] Remove duplicate key in MySQL to GCS Op
[AIRFLOW-2458] Add cassandra-to-gcs operator
[AIRFLOW-2477] Improve time units for task duration and landing times charts for RBAC UI
[AIRFLOW-2474] Only import snakebite if using py2
[AIRFLOW-48] Parse connection uri querystring
[AIRFLOW-2467][AIRFLOW-2] Update import direct warn message to use the module name
[AIRFLOW-XXX] Fix order of companies
[AIRFLOW-2452] Document field_dict must be OrderedDict
[AIRFLOW-2420] Azure Data Lake Hook
[AIRFLOW-2213] Add Quoble check operator
[AIRFLOW-2465] Fix wrong module names in the doc
[AIRFLOW-1929] Modifying TriggerDagRunOperator to accept execution_date
[AIRFLOW-2460] Users can now use volume mounts and volumes
[AIRFLOW-2110][AIRFLOW-2122] Enhance Http Hook
[AIRFLOW-XXX] Updated contributors list
[AIRFLOW-2435] Add launch_type to ECSOperator to allow FARGATE
[AIRFLOW-2451] Remove extra slash ('/') char when using wildcard in gcs_to_gcs operator
[AIRFLOW-2461] Add support for cluster scaling on dataproc operator
[AIRFLOW-2376] Fix no hive section error
[AIRFLOW-2425] Add lineage support
[AIRFLOW-2430] Extend query batching to additional slow queries
[AIRFLOW-2453] Add default nil value for kubernetes/git_subpath
[AIRFLOW-2396] Add support for resources in kubernetes operator
[AIRFLOW-2169] Encode binary data with base64 before importing to BigQuery
[AIRFLOW-XXX] Add spotahome in user list
[AIRFLOW-2457] Update FAB version requirement
[AIRFLOW-2454][Airflow 2454] Support imagePullPolicy for k8s
[AIRFLOW-2450] update supported k8s versions to 1.9 and 1.10
[AIRFLOW-2333] Add Segment Hook and TrackEventOperator
[AIRFLOW-2442][AIRFLOW-2] Airflow run command leaves database connections open
[AIRFLOW-2016] assign template_fields for Dataproc Workflow Template sub-classes, not base class
[AIRFLOW-2446] Add S3ToRedshiftTransfer into the "Integration" doc
[AIRFLOW-2449] Fix operators.py to run all test cases
[AIRFLOW-2424] Add dagrun status endpoint and increased k8s test coverage
[AIRFLOW-2441] Fix bugs in HiveCliHook.load_df
[AIRFLOW-2358][AIRFLOW-201804] Make the Kubernetes example optional
[AIRFLOW-2436] Remove cli_logger in initdb
[AIRFLOW-2444] Remove unused option(include_adhoc) in cli backfill command
[AIRFLOW-2447] Fix TestHiveMetastoreHook to run all cases
[AIRFLOW-2445] Allow templating in kubernetes operator
[AIRFLOW-2086][AIRFLOW-2393] Customize default dagrun number in tree view
[AIRFLOW-2437] Add PubNub to list of current airflow users
[AIRFLOW-XXX] Add Quantopian to list of Airflow users
[AIRFLOW-1978] Add WinRM windows operator and hook
[AIRFLOW-2427] Add tests to named hive sensor
[AIRFLOW-2412] Fix HiveCliHook.load_file to address HIVE-10541
[AIRFLOW-2431] Add the navigation bar color parameter for RBAC UI
[AIRFLOW-2407] Resolve Python undefined names
[AIRFLOW-1952] Add the navigation bar color parameter
[AIRFLOW-2222] Implement GoogleCloudStorageHook.rewrite
[AIRFLOW-2426] Add Google Cloud Storage Hook tests
[AIRFLOW-2418] Bump Flask-WTF
[AIRFLOW-2417] Wait for pod is not running to end task
[AIRFLOW-1914] Add other charset support to email utils
[AIRFLOW-XXX] Update README.md with Craig@Work
[AIRFLOW-1899] Fix Kubernetes tests
[AIRFLOW-1812] Update logging example
[AIRFLOW-2313] Add TTL parameters for Dataproc
[AIRFLOW-2411] add dataproc_jars to templated_fields
[AIRFLOW-XXX] Add Reddit to Airflow users
[AIRFLOW-XXX] Fix wrong table header in scheduler.rst
[AIRFLOW-2409] Supply password as a parameter
[AIRFLOW-2410][AIRFLOW-75] Set the timezone in the RBAC Web UI
[AIRFLOW-2394] default cmds and arguments in kubernetes operator
[AIRFLOW-2406] Add Apache2 License Shield to Readme
[AIRFLOW-2404] Add additional documentation for unqueued task
[AIRFLOW-2400] Add Ability to set Environment Variables for K8s
[AIRFLOW-XXX] Add Twine Labs as an Airflow user
[AIRFLOW-1853] Show only the desired number of runs in tree view
[AIRFLOW-2401] Document the use of variables in Jinja template
[AIRFLOW-2403] Fix License Headers
[AIRFLOW-1313] Fix license header
[AIRFLOW-2398] Add BounceX to list of current airflow users
[AIRFLOW-2363] Fix return type bug in TaskHandler
[AIRFLOW-2389] Create a pinot db api hook
[AIRFLOW-2390] Resolve FlaskWTFDeprecationWarning
[AIRFLOW-1933] Fix some typos
[AIRFLOW-1960] Add support for secrets in kubernetes operator
[AIRFLOW-1313] Add vertica_to_mysql operator
[AIRFLOW-1575] Add AWS Kinesis Firehose Hook for inserting batch records
[AIRFLOW-2266][AIRFLOW-2343] Remove google-cloud-dataflow dependency
[AIRFLOW-2370] Implement --use_random_password in create_user
[AIRFLOW-2348] Strip path prefix from the destination_object when source_object contains a wildcard[]
[AIRFLOW-2391] Fix to Flask 0.12.2
[AIRFLOW-2381] Fix the flaky ApiPasswordTests test
[AIRFLOW-2378] Add Groupon to list of current users
[AIRFLOW-2382] Fix wrong description for delimiter
[AIRFLOW-2380] Add support for environment variables in Spark submit operator.
[AIRFLOW-2377] Improve Sendgrid sender support
[AIRFLOW-2331] Support init action timeout on dataproc cluster create
[AIRFLOW-1835] Update docs: Variable file is json
[AIRFLOW-1781] Make search case-insensitive in LDAP group
[AIRFLOW-2042] Fix browser menu appearing over the autocomplete menu
[AIRFLOW-XXX] Remove wheelhouse files from travis not owned by travis
[AIRFLOW-2336] Use hmsclient in hive_hook
[AIRFLOW-2041] Correct Syntax in python examples
[AIRFLOW-74] SubdagOperators can consume all celeryd worker processes
[AIRFLOW-2369] Fix gcs tests
[AIRFLOW-2365] Fix autocommit attribute check
[AIRFLOW-2068] MesosExecutor allows optional Docker image
[AIRFLOW-1652] Push DatabricksRunSubmitOperator metadata into XCOM
[AIRFLOW-2234] Enable insert_rows for PrestoHook
[AIRFLOW-2208][Airflow-22208] Link to same DagRun graph from TaskInstance view
[AIRFLOW-1153] Allow HiveOperators to take hiveconfs
[AIRFLOW-775] Fix autocommit settings with Jdbc hook
[AIRFLOW-2364] Warn when setting autocommit on a connection which does not support it
[AIRFLOW-2357] Add persistent volume for the logs
[AIRFLOW-766] Skip conn.commit() when in Auto-commit
[AIRFLOW-2351] Check for valid default_args start_date
[AIRFLOW-1433] Set default rbac to initdb
[AIRFLOW-2270] Handle removed tasks in backfill
[AIRFLOW-2344] Fix `connections -l` to work with pipe/redirect
[AIRFLOW-2300] Add S3 Select functionarity to S3ToHiveTransfer
[AIRFLOW-1314] Cleanup the config
[AIRFLOW-1314] Polish some of the Kubernetes docs/config
[AIRFLOW-1314] Improve error handling
[AIRFLOW-1999] Add per-task GCP service account support
[AIRFLOW-1314] Rebasing against master
[AIRFLOW-1314] Small cleanup to address PR comments (#24)
[AIRFLOW-1314] Add executor_config and tests
[AIRFLOW-1314] Improve k8s support
[AIRFLOW-1314] Use VolumeClaim for transporting DAGs
[AIRFLOW-1314] Create integration testing environment
[AIRFLOW-1314] Git Mode to pull in DAGs for Kubernetes Executor
[AIRFLOW-1314] Add support for volume mounts & Secrets in Kubernetes Executor
[AIRFLOW=1314] Basic Kubernetes Mode
[AIRFLOW-2326][AIRFLOW-2222] remove contrib.gcs_copy_operator
[AIRFLOW-2328] Fix empty GCS blob in S3ToGoogleCloudStorageOperator
[AIRFLOW-2350] Fix grammar in UPDATING.md
[AIRFLOW-2302] Fix documentation
[AIRFLOW-2345] pip is not used in this setup.py
[AIRFLOW-2347] Add Banco de Formaturas to Readme
[AIRFLOW-2346] Add Investorise as official user of Airflow
[AIRFLOW-2330] Do not append destination prefix if not given
[AIRFLOW-2240][DASK] Added TLS/SSL support for the dask-distributed scheduler.
[AIRFLOW-2309] Fix duration calculation on TaskFail
[AIRFLOW-2335] fix issue with jdk8 download for ci
[AIRFLOW-2184] Add druid_checker_operator
[AIRFLOW-2299] Add S3 Select functionarity to S3FileTransformOperator
[AIRFLOW-2254] Put header as first row in unload
[AIRFLOW-610] Respect _cmd option in config before defaults
[AIRFLOW-2287] Fix incorrect ASF headers
[AIRFLOW-XXX] Add Zego as an Apache Airflow user
[AIRFLOW-952] fix save empty extra field in UI
[AIRFLOW-1325] Add ElasticSearch log handler and reader
[AIRFLOW-2301] Sync files of an S3 key with a GCS path
[AIRFLOW-2293] Fix S3FileTransformOperator to work with boto3
[AIRFLOW-3212][AIRFLOW-2314] Remove only leading slash in GCS path
[AIRFLOW-1509][AIRFLOW-442] SFTP Sensor
[AIRFLOW-2291] Add optional params to ML Engine
[AIRFLOW-1774] Allow consistent templating of arguments in MLEngineBatchPredictionOperator
[AIRFLOW-2302] Add missing operators and hooks
[AIRFLOW-2312] Docs Typo Correction: Corresponding
[AIRFLOW-1623] Trigger on_kill method in operators
[AIRFLOW-2162] When impersonating another user, pass env variables to sudo
[AIRFLOW-2304] Update quickstart doc to mention scheduler part
[AIRFLOW-1633] docker_operator needs a way to set shm_size
[AIRFLOW-1340] Add S3 to Redshift transfer operator
[AIRFLOW-2303] Lists the keys inside an S3 bucket
[AIRFLOW-2209] restore flask_login imports
[AIRFLOW-2306] Add Bonnier Broadcasting to list of current users
[AIRFLOW-2305][AIRFLOW-2027] Fix CI failure caused by []
[AIRFLOW-2281] Add support for Sendgrid categories
[AIRFLOW-2027] Only trigger sleep in scheduler after all files have parsed
[AIRFLOW-2256] SparkOperator: Add Client Standalone mode and retry mechanism
[AIRFLOW-2284] GCS to S3 operator
[AIRFLOW-2287] Update license notices
[AIRFLOW-2296] Add Cinimex DataLab to Readme
[AIRFLOW-2298] Add Kalibrr to who uses airflow
[AIRFLOW-2292] Fix docstring for S3Hook.get_wildcard_key
[AIRFLOW-XXX] Update PR template
[AIRFLOW-XXX] Remove outdated migrations.sql
[AIRFLOW-2287] Add license header to docs/Makefile
[AIRFLOW-2286] Add tokopedia to the readme
[AIRFLOW-2273] Add Discord webhook operator/hook
[AIRFLOW-2282] Fix grammar in UPDATING.md
[AIRFLOW-2200] Add snowflake operator with tests
[AIRFLOW-2178] Add handling on SLA miss errors
[AIRFLOW-2169] Fix type 'bytes' is not JSON serializable in python3
[AIRFLOW-2215] Pass environment to subproces.Popen in base_task_runner
[AIRFLOW-2253] Add Airflow CLI instrumentation
[AIRFLOW-2274] Fix Dataflow tests
[AIRFLOW-2269] Add Custom Ink as an Airflow user
[AIRFLOW-2259] Dataflow Hook Index out of range
[AIRFLOW-2233] Update updating.md to include the info of hdfs_sensors renaming
[AIRFLOW-2217] Add Slack webhook operator
[AIRFLOW-1729] improve dagBag time
[AIRFLOW-2264] Improve create_user cli help message
[AIRFLOW-2260] [AIRFLOW-2260] SSHOperator add command template .sh files
[AIRFLOW-2261] Check config/env for remote base log folder
[AIRFLOW-2258] Allow import of Parquet-format files into BigQuery
[AIRFLOW-1430] Include INSTALL instructions to avoid GPL
[AIRFLOW-1430] Solve GPL dependency
[AIRFLOW-2251] Add Thinknear as an Airflow user
[AIRFLOW-2244] bugfix: remove legacy LongText code from models.py
[AIRFLOW-2247] Fix RedshiftToS3Transfer not to fail with ValueError
[AIRFLOW-2249] Add side-loading support for Zendesk Hook
[AIRFLOW-XXX] Add Qplum to Airflow users
[AIRFLOW-2228] Enhancements in ValueCheckOperator
[AIRFLOW-1206] Typos
[AIRFLOW-2060] Update pendulum version to 1.4.4
[AIRFLOW-2248] Fix wrong param name in RedshiftToS3Transfer doc
[AIRFLOW-1433][AIRFLOW-85] New Airflow Webserver UI with RBAC support
[AIRFLOW-1235] Fix webserver's odd behaviour
[AIRFLOW-1460] Allow restoration of REMOVED TI's
[airflow-2235] Fix wrong docstrings in two operators
[AIRFLOW-XXX] Fix chronological order for companies using Airflow
[AIRFLOW-2124] Upload Python file to a bucket for Dataproc
[AIRFLOW-2212] Fix ungenerated sensor API reference
[AIRFLOW-2226] Rename google_cloud_storage_default to google_cloud_default
[AIRFLOW-2211] Rename hdfs_sensors.py to hdfs_sensor.py for consistency
[AIRFLOW-2225] Update document to include DruidDbApiHook
[Airflow-2202] Add filter support in HiveMetastoreHook().max_partition()
[AIRFLOW-2220] Remove duplicate numeric list entry in security.rst
[AIRFLOW-XXX] Update tutorial documentation
[AIRFLOW-2215] Update celery task to preserve environment variables and improve logging on exception
[AIRFLOW-2185] Use state instead of query param
[AIRFLOW-2183] Refactor DruidHook to enable sql
[AIRFLOW-2203] Defer cycle detection
[AIRFLOW-2203] Remove Useless Commands.
[AIRFLOW-2203] Cache signature in apply_defaults
[AIRFLOW-2203] Speed up Operator Resources
[AIRFLOW-2203] Cache static rules (trigger/weight)
[AIRFLOW-2203] Store task ids as sets not lists
[AIRFLOW-2205] Remove unsupported args from JdbcHook doc
[AIRFLOW-2207] Fix flaky test that uses app.cached_app()
[AIRFLOW-2206] Remove unsupported args from JdbcOperator doc
[AIRFLOW-2140] Add Kubernetes scheduler to SparkSubmitOperator
[AIRFLOW-XXX] Add Xero to list of users
[AIRFLOW-2204] Fix webserver debug mode
[AIRFLOW-102] Fix test_complex_template always succeeds
[AIRFLOW-442] Add SFTPHook
[AIRFLOW-2169] Add schema to MySqlToGoogleCloudStorageOperator
[AIRFLOW-2184][AIRFLOW-2138] Google Cloud Storage allow wildcards
[AIRFLOW-1588] Cast Variable value to string
[AIRFLOW-2199] Fix invalid reference to logger
[AIRFLOW-2191] Change scheduler heartbeat logs from info to debug
[AIRFLOW-2106] SalesForce hook sandbox option
[AIRFLOW-2197] Silence hostname_callable config error message
[AIRFLOW-2150] Use lighter call in HiveMetastoreHook().max_partition()
[AIRFLOW-2186] Change the way logging is carried out in few ops
[AIRFLOW-2181] Convert password_auth and test_password_endpoints from DOS to UNIX
[AIRFLOW-2187] Fix Broken Travis CI due to AIRFLOW-2123
[AIRFLOW-2175] Check that filepath is not None
[AIRFLOW-2173] Don't check task IDs for concurrency reached check
[AIRFLOW-2168] Remote logging for Azure Blob Storage
[AIRFLOW-XXX] Add DocuTAP to list of users
[AIRFLOW-2176] Change the way logging is carried out in BQ Get Data Operator
[AIRFLOW-2177] Add mock test for GCS Download op
[AIRFLOW-2123] Install CI dependencies from setup.py
[AIRFLOW-2129] Presto hook calls _parse_exception_message but defines _get_pretty_exception_message
[AIRFLOW-2174] Fix typos and wrongly rendered documents
[AIRFLOW-2171] Store delegated credentials
[AIRFLOW-2166] Restore BQ run_query dialect param
[AIRFLOW-2163] Add HBC Digital to users of airflow
[AIRFLOW-2065] Fix race-conditions when creating loggers
[AIRFLOW-2147] Plugin manager: added 'sensors' attribute
[AIRFLOW-2059] taskinstance query is awful, un-indexed, and does not scale
[AIRFLOW-2159] Fix a few typos in salesforce_hook
[AIRFLOW-2132] Add step to initialize database
[AIRFLOW-2160] Fix bad rowid deserialization
[AIRFLOW-2161] Add Vevo to list of companies using Airflow
[AIRFLOW-2149] Add link to apache Beam documentation to create self executing Jar
[AIRFLOW-2151] Allow getting the session from AwsHook
[AIRFLOW-2097] tz referenced before assignment
[AIRFLOW-2152] Add Multiply to list of companies using Airflow
[AIRFLOW-1551] Add operator to trigger Jenkins job
[AIRFLOW-2034] Fix mixup between %s and {} when using str.format Convention is to use .format for string formating oustide logging, else use lazy format See comment in related issue https://github.com/apache/airflow/pull/2823/files Identified problematic case using following command line .git/COMMIT_EDITMSG:`grep -r '%s'./* | grep '\.format('`
[AIRFLOW-2102] Add custom_args to Sendgrid personalizations
[AIRFLOW-1035][AIRFLOW-1053] import unicode_literals to parse Unicode in HQL
[AIRFLOW-2127] Keep loggers during DB migrations
[AIRFLOW-2146] Resolve issues with BQ using DbApiHook methods
[AIRFLOW-2087] Scheduler Report shows incorrect Total task number
[AIRFLOW-2139] Remove unncecessary boilerplate to get DataFrame using pandas_gbq
[AIRFLOW-2125] Using binary package psycopg2-binary
[AIRFLOW-2142] Include message on mkdir failure
[AIRFLOW-1615] SSHHook: use port specified by Connection
[AIRFLOW-2122] Handle boolean values in sshHook
[AIRFLOW-XXX] Add Tile to the list of users
[AIRFLOW-2130] Add missing Operators to API Reference docs
[AIRFLOW-XXX] Add timeout units (seconds)
[AIRFLOW-2134] Add Alan to the list of companies that use Airflow
[AIRFLOW-2133] Remove references to GitHub issues in CONTRIBUTING
[AIRFLOW-2131] Remove confusing AirflowImport docs
[AIRFLOW-1852] Allow hostname to be overridable.
[AIRFLOW-2126] Add Bluecore to active users
[AIRFLOW-1618] Add feature to create GCS bucket
[AIRFLOW-2108] Fix log indentation in BashOperator
[AIRFLOW-2115] Fix doc links to PythonHosted
[AIRFLOW-XXX] Add contributor from Easy company
[AIRFLOW-1882] Add ignoreUnknownValues option to gcs_to_bq operator
[AIRFLOW-2089] Add on kill for SparkSubmit in Standalone Cluster
[AIRFLOW-2113] Address missing DagRun callbacks Given that the handle_callback method belongs to the DAG object, we are able to get the list of task directly with get_task and reduce the communication with the database, making airflow more lightweight.
[AIRFLOW-2112] Fix svg width for Recent Tasks on UI.
[AIRFLOW-2116] Set CI Cloudant version to <2.0
[AIRFLOW-XXX] Add PMC to list of companies using Airflow
[AIRFLOW-2100] Fix Broken Documentation Links
[AIRFLOW-1404] Add 'flatten_results' & 'maximum_bytes_billed' to BQ Operator
[AIRFLOW-800] Initialize valid Google BigQuery Connection
[AIRFLOW-1319] Fix misleading SparkSubmitOperator and SparkSubmitHook docstring
[AIRFLOW-1983] Parse environment parameter as template
[AIRFLOW-2095] Add operator to create External BigQuery Table
[AIRFLOW-2085] Add SparkJdbc operator
[AIRFLOW-1002] Add ability to clean all dependencies of removed DAG
[AIRFLOW-2094] Jinjafied project_id, region & zone in DataProc{*} Operators
[AIRFLOW-2092] Fixed incorrect parameter in docstring for FTPHook
[AIRFLOW-XXX] Add SocialCops to Airflow users
[AIRFLOW-2088] Fix duplicate keys in MySQL to GCS Helper function
[AIRFLOW-2091] Fix incorrect docstring parameter in BigQuery Hook
[AIRFLOW-2090] Fix typo in DataStore Hook
[AIRFLOW-1157] Fix missing pools crashing the scheduler
[AIRFLOW-713] Jinjafy {EmrCreateJobFlow,EmrAddSteps}Operator attributes
[AIRFLOW-2083] Docs: Use "its" instead of "it's" where appropriate
[AIRFLOW-2066] Add operator to create empty BQ table
[AIRFLOW-XXX] add Karmic to list of companies
[AIRFLOW-2073] Make FileSensor fail when the file doesn't exist
[AIRFLOW-2078] Improve task_stats and dag_stats performance
[AIRFLOW-2080] Use a log-out icon instead of a power button
[AIRFLOW-2077] Fetch all pages of list_objects_v2 response
[AIRFLOW-XXX] Add TM to list of companies
[AIRFLOW-1985] Impersonation fixes for using `run_as_user`
[AIRFLOW-2018][AIRFLOW-2] Make Sensors backward compatible
[AIRFLOW-XXX] Fix typo in concepts doc (dag_md)
[AIRFLOW-2069] Allow Bytes to be uploaded to S3
[AIRFLOW-2074] Fix log var name in GHE auth
[AIRFLOW-1927] Convert naive datetimes for TaskInstances
[AIRFLOW-1760] Password auth for experimental API
[AIRFLOW-2038] Add missing kubernetes dependency for dev
[AIRFLOW-2040] Escape special chars in task instance logs URL
[AIRFLOW-1968][AIRFLOW-1520] Add role_arn and aws_account_id/aws_iam_role support back to aws hook
[AIRFLOW-2048] Fix task instance failure string formatting
[AIRFLOW-2046] Fix kerberos error to work with python 3.x
[AIRFLOW-2063] Add missing docs for GCP
[AIRFLOW-XXX] Fix typo in docs
[AIRFLOW-1793] Use docker_url instead of invalid base_url
[AIRFLOW-2055] Elaborate on slightly ambiguous documentation
[AIRFLOW-2039] BigQueryOperator supports priority property
[AIRFLOW-2053] Fix quote character bug in BQ hook
[AIRFLOW-2057] Add Overstock to list of companies
[AIRFLOW-XXX] Add Plaid to Airflow users
[AIRFLOW-2044] Add SparkSubmitOperator to documentation
[AIRFLOW-2037] Add methods to get Hash values of a GCS object
[AIRFLOW-2050] Fix Travis permission problem
[AIRFLOW-2043] Add Intercom to list of companies
[AIRFLOW-2023] Add debug logging around number of queued files
[AIRFLOW-XXX] Add Pernod-ricard as a airflow user
[AIRFLOW-1453] Add 'steps' into template_fields in EmrAddSteps
[AIRFLOW-2015] Add flag for interactive runs
[AIRFLOW-1895] Fix primary key integrity for mysql
[AIRFLOW-2030] Fix KeyError:`i` in DbApiHook for insert
[AIRFLOW-1943] Add External BigQuery Table feature
[AIRFLOW-2033] Add Google Cloud Storage List Operator
[AIRFLOW-2006] Add local log catching to kubernetes operator
[AIRFLOW-2031] Add missing gcp_conn_id in the example in DataFlow docstrings
[AIRFLOW-2029] Fix AttributeError in BigQueryPandasConnector
[AIRFLOW-2028] Add JobTeaser to official users list
[AIRFLOW-2016] Add support for Dataproc Workflow Templates
[AIRFLOW-2025] Reduced Logging verbosity
[AIRFLOW-1267][AIRFLOW-1874] Add dialect parameter to BigQueryHook
[AIRFLOW-XXX] Fixed a typo
[AIRFLOW-XXX] Typo node to nodes
[AIRFLOW-2019] Update DataflowHook for updating Streaming type job
[AIRFLOW-2017][Airflow 2017] adding query output to PostgresOperator
[AIRFLOW-1889] Split sensors into separate files
[AIRFLOW-1950] Optionally pass xcom_pull task_ids
[AIRFLOW-1755] Allow mount below root
[AIRFLOW-511][Airflow 511] add success/failure callbacks on dag level
[AIRFLOW-192] Add weight_rule param to BaseOperator
[AIRFLOW-2008] Use callable for python column defaults
[AIRFLOW-1984] Fix to AWS Batch operator
[AIRFLOW-2000] Support non-main dataflow job class
[AIRFLOW-2003] Use flask-caching instead of flask-cache
[AIRFLOW-2002] Do not swallow exception on logging import
[AIRFLOW-2004] Import flash from flask not flask.login
[AIRFLOW-1997] Fix GCP operator doc strings
[AIRFLOW-1996] Update DataflowHook waitfordone for Streaming type job[]
[AIRFLOW-1995][Airflow 1995] add on_kill method to SqoopOperator
[AIRFLOW-1770] Allow HiveOperator to take in a file
[AIRFLOW-1994] Change background color of Scheduled state Task Instances
[AIRFLOW-1436][AIRFLOW-1475] EmrJobFlowSensor considers Cancelled step as Successful
[AIRFLOW-1517] Kubernetes operator PR fixes
[AIRFLOW-1517] addressed PR comments
[AIRFLOW-1517] started documentation of k8s operator
[AIRFLOW-1517] Restore authorship of resources
[AIRFLOW-1517] Remove authorship of resources
[AIRFLOW-1517] Add minikube for kubernetes integration tests
[AIRFLOW-1517] Restore authorship of resources
[AIRFLOW-1517] fixed license issues
[AIRFLOW-1517] Created more accurate failures for kube cluster issues
[AIRFLOW-1517] Remove authorship of resources
[AIRFLOW-1517] Add minikube for kubernetes integration tests
[AIRFLOW-1988] Change BG color of None state TIs
[AIRFLOW-790] Clean up TaskInstances without DagRuns
[AIRFLOW-1949] Fix var upload, str() produces "b'...'" which is not json
[AIRFLOW-1930] Convert func.now() to timezone.utcnow()
[AIRFLOW-1688] Support load.time_partitioning in bigquery_hook
[AIRFLOW-1975] Make TriggerDagRunOperator callback optional
[AIRFLOW-1480] Render template attributes for ExternalTaskSensor fields
[AIRFLOW-1958] Add **kwargs to send_email
[AIRFLOW-1976] Fix for missing log/logger attribute FileProcessHandler
[AIRFLOW-1982] Fix Executor event log formatting
[AIRFLOW-1971] Propagate hive config on impersonation
[AIRFLOW-1969] Always use HTTPS URIs for Google OAuth2
[AIRFLOW-1954] Add DataFlowTemplateOperator
[AIRFLOW-1963] Add config for HiveOperator mapred_queue
[AIRFLOW-1946][AIRFLOW-1855] Create a BigQuery Get Data Operator
[AIRFLOW-1953] Add labels to dataflow operators
[AIRFLOW-1967] Update Celery to 4.0.2
[AIRFLOW-1964] Add Upsight to list of Airflow users
[AIRFLOW-XXX] Changelog for 1.9.0
[AIRFLOW-1470] Implement BashSensor operator
[AIRFLOW-XXX] Pin sqlalchemy dependency
[AIRFLOW-1955] Do not reference unassigned variable
[AIRFLOW-1957] Add contributor to BalanceHero in Readme
[AIRFLOW-1517] Restore authorship of secrets and init container
[AIRFLOW-1517] Remove authorship of secrets and init container
[AIRFLOW-1935] Add BalanceHero to readme
[AIRFLOW-1939] add astronomer contributors
[AIRFLOW-1517] Kubernetes Operator
[AIRFLOW-1928] Fix @once with catchup=False
[AIRFLOW-1937] Speed up scheduling by committing in batch
[AIRFLOW-1821] Enhance default logging config by removing extra loggers
[AIRFLOW-1904] Correct DAG fileloc to the right filepath
[AIRFLOW-1909] Update docs with supported versions of MySQL server
[AIRFLOW-1915] Relax flask-wtf dependency specification
[AIRFLOW-1920] Update CONTRIBUTING.md to reflect enforced linting rules
[AIRFLOW-1942] Update Sphinx docs to remove deprecated import structure
[AIRFLOW-1846][AIRFLOW-1697] Hide Ad Hoc Query behind secure_mode config
[AIRFLOW-1948] Include details for on_kill failure
[AIRFLOW-1938] Clean up unused exception
[AIRFLOW-1932] Add GCP Pub/Sub Pull and Ack
[AIRFLOW-XXX] Purge coveralls
[AIRFLOW-XXX] Remove unused coveralls token
[AIRFLOW-1938] Remove tag version check in setup.py
[AIRFLOW-1916] Don't upload logs to remote from `run --raw`
[AIRFLOW-XXX] Fix failing PubSub tests on Python3
[AIRFLOW-XXX] Upgrade to python 3.5 and disable dask tests
[AIRFLOW-1913] Add new GCP PubSub operators
[AIRFLOW-1525] Fix minor LICENSE and NOTICE issues
[AIRFLOW-1687] fix fernet error without encryption
[AIRFLOW-1912] airflow.processor should not propagate logging
[AIRFLOW-1911] Rename celeryd_concurrency
[AIRFLOW-1885] Fix IndexError in ready_prefix_on_cmdline
[AIRFLOW-1854] Improve Spark Submit operator for standalone cluster mode
[AIRFLOW-1908] Fix celery broker options config load
[AIRFLOW-1907] Pass max_ingestion_time to Druid hook
[AIRFLOW-1909] Add away to list of users
[AIRFLOW-1893][AIRFLOW-1901] Propagate PYTHONPATH when using impersonation
[AIRFLOW-1892] Modify BQ hook to extract data filtered by column
[AIRFLOW-1829] Support for schema updates in query jobs
[AIRFLOW-1840] Make celery configuration congruent with Celery 4
[AIRFLOW-1878] Fix stderr/stdout redirection for tasks
[AIRFLOW-1897][AIRFLOW-1873] Task Logs for running instance not visible in WebUI
[AIRFLOW-1896] FIX bleach <> html5lib incompatibility
[AIRFLOW-1884][AIRFLOW-1059] Reset orphaned task state for external dagruns
[AIRFLOW-XXX] Fix typo in comment
[AIRFLOW-1869] Do not emit spurious warning on missing logs
[AIRFLOW-1888] Add AWS Redshift Cluster Sensor
[AIRFLOW-1887] Renamed endpoint url variable
[AIRFLOW-1873] Set TI.try_number to right value depending TI state
[AIRFLOW-1891] Fix non-ascii typo in default configuration template
[AIRFLOW-1879] Handle ti log entirely within ti
[AIRFLOW-1869] Write more error messages into gcs and file logs
[AIRFLOW-1876] Write subtask id to task log header
[AIRFLOW-1554] Fix wrong DagFileProcessor termination method call
[AIRFLOW-342] Do not use amqp, rpc as result backend
[AIRFLOW-966] Make celery broker_transport_options configurable
[AIRFLOW-1881] Make operator log in task log
[AIRFLOW-XXX] Added DataReply to the list of Airflow Users
[AIRFLOW-1883] Get File Size for objects in Google Cloud Storage
[AIRFLOW-1872] Set context for all handlers including parents
[AIRFLOW-1855][AIRFLOW-1866] Add GCS Copy Operator to copy multiple files
[AIRFLOW-1870] Enable flake8 tests
[AIRFLOW-1785] Enable Python 3 tests
[AIRFLOW-1850] Copy cmd before masking
[AIRFLOW-1665] Reconnect on database errors
[AIRFLOW-1559] Dispose SQLAlchemy engines on exit
[AIRFLOW-1559] Close file handles in subprocesses
[AIRFLOW-1559] Make database pooling optional
[AIRFLOW-1848][Airflow-1848] Fix DataFlowPythonOperator py_file extension doc comment
[AIRFLOW-1843] Add Google Cloud Storage Sensor with prefix
[AIRFLOW-1803] Time zone documentation
[AIRFLOW-1826] Update views to use timezone aware objects
[AIRFLOW-1827] Fix api endpoint date parsing
[AIRFLOW-1806] Use naive datetime when using cron
[AIRFLOW-1809] Update tests to use timezone aware objects
[AIRFLOW-1806] Use naive datetime for cron scheduling
[AIRFLOW-1807] Force use of time zone aware db fields
[AIRFLOW-1808] Convert all utcnow() to time zone aware
[AIRFLOW-1804] Add time zone configuration options
[AIRFLOW-1802] Convert database fields to timezone aware
[AIRFLOW-XXX] Add dask lock files to excludes
[AIRFLOW-1790] Add support for AWS Batch operator
[AIRFLOW-XXX] Update README.md
[AIRFLOW-1820] Remove timestamp from metric name
[AIRFLOW-1810] Remove unused mysql import in migrations.
[AIRFLOW-1838] Properly log collect_dags exception
[AIRFLOW-1842] Fixed Super class name for the gcs to gcs copy operator
[AIRFLOW-1845] Modal background now covers long or tall pages
[AIRFLOW-1229] Add link to Run Id, incl execution_date
[AIRFLOW-1842] Add gcs to gcs copy operator with renaming if required
[AIRFLOW-1841] change False to None in operator and hook
[AIRFLOW-1839] Fix more bugs in S3Hook boto -> boto3 migration
[AIRFLOW-1830] Support multiple domains in Google authentication backend
[AIRFLOW-1831] Add driver-classpath spark submit
[AIRFLOW-1795] Correctly call S3Hook after migration to boto3
[AIRFLOW-1811] Fix render Druid operator
[AIRFLOW-1819] Fix slack operator unittest bug
[AIRFLOW-1805] Allow Slack token to be passed through connection
[AIRFLOW-1816] Add region param to Dataproc operators
[AIRFLOW-868] Add postgres_to_gcs operator and unittests
[AIRFLOW-1613] make mysql_to_gcs_operator py3 compatible
[AIRFLOW-1817] use boto3 for s3 dependency
[AIRFLOW-1813] Bug SSH Operator empty buffer
[AIRFLOW-1801][AIRFLOW-288] Url encode execution dates
[AIRFLOW-1563] Catch OSError while symlinking the latest log directory
[AIRFLOW-1794] Remove uses of Exception.message for Python 3
[AIRFLOW-1799] Fix logging line which raises errors
[AIRFLOW-1102] Upgrade Gunicorn >=19.4.0
[AIRFLOW-1756] Fix S3TaskHandler to work with Boto3-based S3Hook
[AIRFLOW-1797] S3Hook.load_string didn't work on Python3
[AIRFLOW-646] Add docutils to setup_requires
[AIRFLOW-1792] Missing intervals DruidOperator
[AIRFLOW-1789][AIRFLOW-1712] Log SSHOperator stderr to log.warning
[AIRFLOW-1787] Fix task instance batch clear and set state bugs
[AIRFLOW-1780] Fix long output lines with unicode from hanging parent
[AIRFLOW-387] Close SQLAlchemy sessions properly
[AIRFLOW-1779] Add keepalive packets to ssh hook
[AIRFLOW-1669] Fix Docker and pin Moto to 1.1.19
[AIRFLOW-71] Add support for private Docker images
[AIRFLOW-XXX] Give a clue what the 'ds' variable is
[AIRFLOW-XXX] Correct typos in the faq docs page
[AIRFLOW-1571] Add AWS Lambda Hook
[AIRFLOW-1675] Fix docstrings for API docs
[AIRFLOW-1712][AIRFLOW-756][AIRFLOW-751] Log SSHOperator output
[AIRFLOW-1776] Capture stdout and stderr for logging
[AIRFLOW-1765] Make experimental API securable without needing Kerberos.
[AIRFLOW-1764] The web interface should not use the experimental API
[AIRFLOW-1771] Rename heartbeat to avoid confusion
[AIRFLOW-1769] Add support for templates in VirtualenvOperator
[AIRFLOW-1763] Fix S3TaskHandler unit tests
[AIRFLOW-1315] Add Qubole File & Partition Sensors
[AIRFLOW-1018] Make processor use logging framework
[AIRFLOW-1695] Add RedshiftHook using boto3
[AIRFLOW-1706] Fix query error for MSSQL backend
[AIRFLOW-1711] Use ldap3 dict for group membership
[AIRFLOW-1723] Make sendgrid a plugin
[AIRFLOW-1757] Add missing options to SparkSubmitOperator
[AIRFLOW-1734][Airflow 1734] Sqoop hook/operator enhancements
[AIRFLOW-1761] Fix type in scheduler.rst
[AIRFLOW-1731] Set pythonpath for logging
[AIRFLOW-1641] Handle executor events in the scheduler
[AIRFLOW-1744] Make sure max_tries can be set
[AIRFLOW-1732] Improve dataflow hook logging
[AIRFLOW-1736] Add HotelQuickly to Who Uses Airflow
[AIRFLOW-1657] Handle failing qubole operator
[AIRFLOW-1677] Fix typo in example_qubole_operator
[AIRFLOW-926] Fix JDBC Hook
[AIRFLOW-1520] Boto3 S3Hook, S3Log
[AIRFLOW-1716] Fix multiple __init__ def in SimpleDag
[AIRFLOW-XXX] Fix DateTime in Tree View
[AIRFLOW-1719] Fix small typo
[AIRFLOW-1432] Charts label for Y axis not visible
[AIRFLOW-1743] Verify ldap filters correctly
[AIRFLOW-1745] Restore default signal disposition
[AIRFLOW-1741] Correctly hide second chart on task duration page
[AIRFLOW-1728] Add networkUri, subnet, tags to Dataproc operator
[AIRFLOW-1726] Add copy_expert psycopg2 method to PostgresHook
[AIRFLOW-1330] Add conn_type argument to CLI when adding connection
[AIRFLOW-1698] Remove SCHEDULER_RUNS env var in systemd
[AIRFLOW-1694] Stop using itertools.izip
[AIRFLOW-1692] Change test_views filename to support Windows
[AIRFLOW-1722] Fix typo in scheduler autorestart output filename
[AIRFLOW-1723] Support sendgrid in email backend
[AIRFLOW-1718] Set num_retries on Dataproc job request execution
[AIRFLOW-1727] Add unit tests for DataProcHook
[AIRFLOW-1631] Fix timing issue in unit test
[AIRFLOW-1631] Fix local executor unbound parallelism
[AIRFLOW-1724] Add Fundera to Who uses Airflow?
[AIRFLOW-1683] Cancel BigQuery job on timeout.
[AIRFLOW-1714] Fix misspelling: s/seperate/separate/
[AIRFLOW-1681] Add batch clear in task instance view
[AIRFLOW-1696] Fix dataproc version label error
[AIRFLOW-1613] Handle binary field in MySqlToGoogleCloudStorageOperator
[AIRFLOW-1697] Mode to disable charts endpoint
[AIRFLOW-1691] Add better Google cloud logging documentation
[AIRFLOW-1690] Add detail to gcs error messages
[AIRFLOW-1682] Make S3TaskHandler write to S3 on close
[AIRFLOW-1634] Adds task_concurrency feature
[AIRFLOW-1676] Make GCSTaskHandler write to GCS on close
[AIRFLOW-1678] Fix erroneously repeated word in function docstrings
[AIRFLOW-1323] Made Dataproc operator parameter names consistent
[AIRFLOW-1590] fix unused module and variable
[AIRFLOW-1671] Add @apply_defaults back to gcs download operator
[AIRFLOW-988] Fix repeating SLA miss callbacks
[AIRFLOW-1611] Customize logging
[AIRFLOW-1668] Expose keepalives_idle for Postgres connections
[AIRFLOW-1658] Kill Druid task on timeout
[AIRFLOW-1669][AIRFLOW-1368] Fix Docker import
[AIRFLOW-891] Make webserver clock include date
[AIRFLOW-1560] Add AWS DynamoDB hook and operator for inserting batch items
[AIRFLOW-1654] Show tooltips for link icons in DAGs view
[AIRFLOW-1660] Change webpage width to full-width
[AIRFLOW-1664] write file as binary instead of str
[AIRFLOW-1659] Fix invalid obj attribute bug in file_task_handler.py
[AIRFLOW-1635] Allow creating GCP connection without requiring a JSON file
[AIRFLOW-1650] Fix custom celery config loading
[AIRFLOW-1647] Fix Spark-sql hook
[AIRFLOW-1587] Fix CeleryExecutor import error
[Airflow-1640][AIRFLOW-1640] Add qubole default connection
[AIRFLOW-1576] Added region param to Dataproc{*}Operators
[AIRFLOW-1643] Add healthjump to officially using list
[AIRFLOW-1626] Add Azri Solutions to Airflow users
[AIRFLOW-1636] Add AWS and EMR connection type
[AIRFLOW-1527] Refactor celery config
[AIRFLOW-1639] Fix Fernet error handling
[AIRFLOW-1637] Fix Travis CI build status link
[AIRFLOW-1628] Fix docstring of sqlsensor
[AIRFLOW-1331] add SparkSubmitOperator option
[AIRFLOW-1627] Only query pool in SubDAG init when necessary
[AIRFLOW-1629] Make extra a textarea in edit connections form
[AIRFLOW-1368] Automatically remove Docker container on exit
[AIRFLOW-289] Make airflow timezone independent
[AIRFLOW-1356] Add `--celery_hostname` to `airflow worker`
[AIRFLOW-1247] Fix ignore_all_dependencies argument ignored
[AIRFLOW-1621] Add tests for server side paging
[AIRFLOW-1591] Avoid attribute error when rendering logging filename
[AIRFLOW-1031] Replace hard-code to DagRun.ID_PREFIX
[AIRFLOW-1604] Rename logger to log
[AIRFLOW-1512] Add PythonVirtualenvOperator
[AIRFLOW-1617] Fix XSS vulnerability in Variable endpoint
[AIRFLOW-1497] Reset hidden fields when changing connection type
[AIRFLOW-1619] Add poll_sleep parameter to GCP dataflow operator
[AIRFLOW-XXX] Remove landscape.io config
[AIRFLOW-XXX] Remove non working service badges
[AIRFLOW-1177] Fix Variable.setdefault w/existing JSON
[AIRFLOW-1600] Fix exception handling in get_fernet
[AIRFLOW-1614] Replace inspect.stack() with sys._getframe()
[AIRFLOW-1519] Add server side paging in DAGs list
[AIRFLOW-1309] Allow hive_to_druid to take tblproperties
[AIRFLOW-1613] Make MySqlToGoogleCloudStorageOperator compaitible with python3
[AIRFLOW-1603] add PAYMILL to companies list
[AIRFLOW-1609] Fix gitignore to ignore all venvs
[AIRFLOW-1601] Add configurable task cleanup time
AIRFLOW 1.9.0, 2018-01-02
-------------------------
[AIRFLOW-1525] Fix minor LICENSE and NOTICE issues
[AIRFLOW-XXX] Bump version to 1.9.0
[AIRFLOW-1897][AIRFLOW-1873] Task Logs for running instance not visible in WebUI
[AIRFLOW-XXX] Make sure session is committed
[AIRFLOW-1896] FIX bleach <> html5lib incompatibility
[AIRFLOW-XXX] Fix log handler test
[AIRFLOW-1873] Set TI.try_number to right value depending TI state
[AIRFLOW-1554] Fix wrong DagFileProcessor termination method call
[AIRFLOW-1872] Set context for all handlers including parents
[AIRFLOW-XXX] Add dask lock files to excludes
[AIRFLOW-1839] Fix more bugs in S3Hook boto -> boto3 migration
[AIRFLOW-1795] Correctly call S3Hook after migration to boto3
[AIRFLOW-1813] Bug SSH Operator empty buffer
[AIRFLOW-1794] Remove uses of Exception.message for Python 3
[AIRFLOW-1799] Fix logging line which raises errors
[AIRFLOW-1102] Upgrade Gunicorn >=19.4.0
[AIRFLOW-1756] Fix S3TaskHandler to work with Boto3-based S3Hook
[AIRFLOW-1797] S3Hook.load_string didn't work on Python3
[AIRFLOW-1792] Missing intervals DruidOperator
[AIRFLOW-1789][AIRFLOW-1712] Log SSHOperator stderr to log.warning
[AIRFLOW-1669] Fix Docker and pin Moto to 1.1.19
[AIRFLOW-71] Add support for private Docker images
[AIRFLOW-1779] Add keepalive packets to ssh hook
[AIRFLOW-XXX] Give a clue what the 'ds' variable is
[AIRFLOW-XXX] Correct typos in the faq docs page
[AIRFLOW-1571] Add AWS Lambda Hook
[AIRFLOW-1675] Fix docstrings for API docs
[AIRFLOW-1712][AIRFLOW-756][AIRFLOW-751] Log SSHOperator output
[AIRFLOW-1776] Capture stdout and stderr for logging
[AIRFLOW-1765] Make experimental API securable without needing Kerberos.
[AIRFLOW-1764] The web interface should not use the experimental API
[AIRFLOW-1634] Adds task_concurrency feature
[AIRFLOW-1018] Make processor use logging framework
[AIRFLOW-1695] Add RedshiftHook using boto3
[AIRFLOW-1706] Fix query error for MSSQL backend
[AIRFLOW-1711] Use ldap3 dict for group membership
[AIRFLOW-1757] Add missing options to SparkSubmitOperator
[AIRFLOW-1734][Airflow 1734] Sqoop hook/operator enhancements
[AIRFLOW-1731] Set pythonpath for logging
[AIRFLOW-1641] Handle executor events in the scheduler
[AIRFLOW-1744] Make sure max_tries can be set
[AIRFLOW-1330] Add conn_type argument to CLI when adding connection
[AIRFLOW-926] Fix JDBC Hook
[AIRFLOW-1520] Boto3 S3Hook, S3Log
[AIRFLOW-XXX] Fix DateTime in Tree View
[AIRFLOW-1432] Charts label for Y axis not visible
[AIRFLOW-1743] Verify ldap filters correctly
[AIRFLOW-1745] Restore default signal disposition
[AIRFLOW-1741] Correctly hide second chart on task duration page
[AIRFLOW-1726] Add copy_expert psycopg2 method to PostgresHook
[AIRFLOW-1698] Remove SCHEDULER_RUNS env var in systemd
[AIRFLOW-1694] Stop using itertools.izip
[AIRFLOW-1692] Change test_views filename to support Windows
[AIRFLOW-1722] Fix typo in scheduler autorestart output filename
[AIRFLOW-1691] Add better Google cloud logging documentation
[AIRFLOW-1690] Add detail to gcs error messages
[AIRFLOW-1682] Make S3TaskHandler write to S3 on close
[AIRFLOW-1676] Make GCSTaskHandler write to GCS on close
[AIRFLOW-1635] Allow creating GCP connection without requiring a JSON file
[AIRFLOW-1323] Made Dataproc operator parameter names consistent
[AIRFLOW-1590] fix unused module and variable
[AIRFLOW-988] Fix repeating SLA miss callbacks
[AIRFLOW-1611] Customize logging
[AIRFLOW-1668] Expose keepalives_idle for Postgres connections
[AIRFLOW-1658] Kill Druid task on timeout
[AIRFLOW-1669][AIRFLOW-1368] Fix Docker import
[AIRFLOW-1560] Add AWS DynamoDB hook and operator for inserting batch items
[AIRFLOW-1654] Show tooltips for link icons in DAGs view
[AIRFLOW-1660] Change webpage width to full-width
[AIRFLOW-1664] write file as binary instead of str
[AIRFLOW-1659] Fix invalid obj attribute bug in file_task_handler.py
[AIRFLOW-1650] Fix custom celery config loading
[AIRFLOW-1647] Fix Spark-sql hook
[AIRFLOW-1587] Fix CeleryExecutor import error
[AIRFLOW-1636] Add AWS and EMR connection type
[AIRFLOW-1527] Refactor celery config
[AIRFLOW-1639] Fix Fernet error handling
[AIRFLOW-1628] Fix docstring of sqlsensor
[AIRFLOW-1331] add SparkSubmitOperator option
[AIRFLOW-1627] Only query pool in SubDAG init when necessary
[AIRFLOW-1629] Make extra a textarea in edit connections form
[AIRFLOW-1621] Add tests for server side paging