Static start_date and default arg cleanup for Microsoft providers example DAGs #19062
Conversation
boring-cyborg
bot
added
area:providers
kind:documentation
provider:microsoft-azure
| default_args={"container_name": AZURE_CONTAINER_NAME, "blob_name": BLOB_NAME}, | ||
| ) as dag: | ||
|
|
||
| # [START how_to_wait_for_blob] |
There was a problem hiding this comment.
Removed this tag as it was not being used in any documentation.
| }, | ||
| schedule_interval=timedelta(1), | ||
| default_args={'retries': 1}, | ||
| schedule_interval=timedelta(days=1), |
There was a problem hiding this comment.
Updated schedule_interval to use keyword args to be more explicit.
| catchup=False, | ||
| ) as dag: | ||
|
|
||
| cmd = 'ls -l' |
There was a problem hiding this comment.
Removed unused variable in DAG.
|
|
||
| To get information about jobs within a Azure Blob Storage use: | ||
| :class:`~airflow.providers.microsoft.azure.transfers.azure_blob_to_gcs.AzureBlobStorageToGCSOperator` | ||
|
|
There was a problem hiding this comment.
Space added for better formatting.
ephraimbuddy
left a comment
ephraimbuddy
left a comment
There was a problem hiding this comment.
LGTM
Some of these examples are used in system tests. Can you verify them by executing the system tests in https://github.com/apache/airflow/tree/main/tests/providers/microsoft/azure/transfers.
github-actions
bot
added
the
okay to merge
|
The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest main or amend the last commit of the PR, and push it with --force-with-lease. |
|
@ephraimbuddy FYI - I've confirmed Azure system tests pass locally.
|
josh-fell
deleted the
microsoft-providers-default-args-start-date-cleanup
branch
2 participants
There is an ongoing effort to enhance example DAGs by setting static values for
start_date(already complete for core example DAGs), cleaning up and/or implementing usefuldefault_args, and removing/limiting the use of redundant, default connection ID values.This PR mainly addresses these 3 cleanup areas in example DAGs across Microsoft providers. There are other small docs updates.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.