Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Move and convert all AWS example dags to system tests #30003

Merged
merged 6 commits into from
Mar 21, 2023
Merged

Move and convert all AWS example dags to system tests #30003

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
5cda1bd
Move and convert all example dags to system tests
vincbeck Mar 8, 2023
147a797
Fix example_appflow system test
vincbeck Mar 9, 2023
eb51f75
Fix example_eks_templated system test
vincbeck Mar 9, 2023
52a4746
Remove check_example_dags_in_provider_tocs test
vincbeck Mar 9, 2023
31a3094
Update tests/system/providers/amazon/aws/example_eks_templated.py
vincbeck Mar 20, 2023
4f3d291
Merge branch 'main' into vincbeck/example_dags
vincbeck Mar 20, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 0 additions & 16 deletions airflow/providers/amazon/aws/example_dags/__init__.py
View file
Open in desktop

This file was deleted.

41 changes: 0 additions & 41 deletions airflow/providers/amazon/aws/example_dags/example_ftp_to_s3.py
View file
Open in desktop

This file was deleted.

41 changes: 0 additions & 41 deletions airflow/providers/amazon/aws/example_dags/example_gcs_to_s3.py
View file
Open in desktop

This file was deleted.

50 changes: 0 additions & 50 deletions airflow/providers/amazon/aws/example_dags/example_imap_attachment_to_s3.py
View file
Open in desktop

This file was deleted.

47 changes: 0 additions & 47 deletions airflow/providers/amazon/aws/example_dags/example_mongo_to_s3.py
View file
Open in desktop

This file was deleted.

40 changes: 0 additions & 40 deletions airflow/providers/amazon/aws/example_dags/example_s3_to_ftp.py
View file
Open in desktop

This file was deleted.

40 changes: 0 additions & 40 deletions airflow/providers/amazon/aws/example_dags/example_s3_to_sftp.py
View file
Open in desktop

This file was deleted.

40 changes: 0 additions & 40 deletions airflow/providers/amazon/aws/example_dags/example_sftp_to_s3.py
View file
Open in desktop

This file was deleted.

1 change: 0 additions & 1 deletion docs/apache-airflow-providers-amazon/example-dags.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,3 @@ Example DAGs
You can learn how to use Amazon AWS integrations by analyzing the source code of the example DAGs:

* `Amazon AWS <https://github.com/apache/airflow/tree/providers-amazon/|version|/tests/system/providers/amazon/aws>`__
* `Amazon AWS (legacy) <https://github.com/apache/airflow/tree/providers-amazon/|version|/airflow/providers/amazon/aws/example_dags>`__
12 changes: 6 additions & 6 deletions docs/apache-airflow-providers-amazon/operators/appflow.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ Run Flow
To run an AppFlow flow keeping all filters as is, use:
:class:`~airflow.providers.amazon.aws.operators.appflow.AppflowRunOperator`.

.. exampleinclude:: /../../airflow/providers/amazon/aws/example_dags/example_appflow.py
.. exampleinclude:: /../../tests/system/providers/amazon/aws/example_appflow.py
:language: python
:dedent: 4
:start-after: [START howto_operator_appflow_run]
Expand All @@ -61,7 +61,7 @@ Run Flow Full
To run an AppFlow flow removing all filters, use:
:class:`~airflow.providers.amazon.aws.operators.appflow.AppflowRunFullOperator`.

.. exampleinclude:: /../../airflow/providers/amazon/aws/example_dags/example_appflow.py
.. exampleinclude:: /../../tests/system/providers/amazon/aws/example_appflow.py
:language: python
:dedent: 4
:start-after: [START howto_operator_appflow_run_full]
Expand All @@ -78,7 +78,7 @@ Run Flow Daily
To run an AppFlow flow filtering daily records, use:
:class:`~airflow.providers.amazon.aws.operators.appflow.AppflowRunDailyOperator`.

.. exampleinclude:: /../../airflow/providers/amazon/aws/example_dags/example_appflow.py
.. exampleinclude:: /../../tests/system/providers/amazon/aws/example_appflow.py
:language: python
:dedent: 4
:start-after: [START howto_operator_appflow_run_daily]
Expand All @@ -95,7 +95,7 @@ Run Flow Before
To run an AppFlow flow filtering future records and selecting the past ones, use:
:class:`~airflow.providers.amazon.aws.operators.appflow.AppflowRunBeforeOperator`.

.. exampleinclude:: /../../airflow/providers/amazon/aws/example_dags/example_appflow.py
.. exampleinclude:: /../../tests/system/providers/amazon/aws/example_appflow.py
:language: python
:dedent: 4
:start-after: [START howto_operator_appflow_run_before]
Expand All @@ -112,7 +112,7 @@ Run Flow After
To run an AppFlow flow filtering past records and selecting the future ones, use:
:class:`~airflow.providers.amazon.aws.operators.appflow.AppflowRunAfterOperator`.

.. exampleinclude:: /../../airflow/providers/amazon/aws/example_dags/example_appflow.py
.. exampleinclude:: /../../tests/system/providers/amazon/aws/example_appflow.py
:language: python
:dedent: 4
:start-after: [START howto_operator_appflow_run_after]
Expand All @@ -129,7 +129,7 @@ Skipping Tasks For Empty Runs
To skip tasks when some AppFlow run return zero records, use:
:class:`~airflow.providers.amazon.aws.operators.appflow.AppflowRecordsShortCircuitOperator`.

.. exampleinclude:: /../../airflow/providers/amazon/aws/example_dags/example_appflow.py
.. exampleinclude:: /../../tests/system/providers/amazon/aws/example_appflow.py
:language: python
:dedent: 4
:start-after: [START howto_operator_appflow_shortcircuit]
Expand Down
2 changes: 1 addition & 1 deletion docs/apache-airflow-providers-amazon/operators/athena.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ to run a query in Amazon Athena.

In the following example, we query an existing Athena table and send the results to
an existing Amazon S3 bucket. For more examples of how to use this operator, please
see the `Sample DAG <https://github.com/apache/airflow/blob/|version|/airflow/providers/amazon/aws/example_dags/example_athena.py>`__.
see the `Sample DAG <https://github.com/apache/airflow/blob/|version|/tests/system/providers/amazon/aws/example_athena.py>`__.

.. exampleinclude:: /../../tests/system/providers/amazon/aws/example_athena.py
:language: python
Expand Down
Loading