Skip to content

Commit

Permalink
Documentation fixes
Browse files Browse the repository at this point in the history
  • Loading branch information
bjankiewicz authored and bjankie1 committed Aug 16, 2022
1 parent f385f57 commit 9679ca1
Show file tree
Hide file tree
Showing 2 changed files with 13 additions and 13 deletions.
2 changes: 1 addition & 1 deletion airflow/providers/google/cloud/hooks/dataproc.py
Original file line number Diff line number Diff line change
Expand Up @@ -977,7 +977,7 @@ def list_batches(

class DataprocAsyncHook(GoogleBaseHook):
"""
Asynchronuous Hook for Google Cloud Dataproc APIs.
Asynchronous Hook for Google Cloud Dataproc APIs.
All the methods in the hook where project_id is used must be called with
keyword arguments rather than positional.
Expand Down
24 changes: 12 additions & 12 deletions docs/apache-airflow-providers-google/operators/cloud/dataproc.rst
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ A cluster configuration can look as followed:

.. exampleinclude:: /../../tests/system/providers/google/cloud/dataproc/example_dataproc_hive.py
:language: python
:dedent: 4
:dedent: 0
:start-after: [START how_to_cloud_dataproc_create_cluster]
:end-before: [END how_to_cloud_dataproc_create_cluster]

Expand All @@ -62,7 +62,7 @@ For create Dataproc cluster in Google Kubernetes Engine you should use this clus

.. exampleinclude:: /../../tests/system/providers/google/cloud/dataproc/example_dataproc_gke.py
:language: python
:dedent: 4
:dedent: 0
:start-after: [START how_to_cloud_dataproc_create_cluster_in_gke_config]
:end-before: [END how_to_cloud_dataproc_create_cluster_in_gke_config]

Expand All @@ -84,7 +84,7 @@ You can generate and use config as followed:

.. exampleinclude:: /../../tests/system/providers/google/cloud/dataproc/example_dataproc_cluster_generator.py
:language: python
:dedent: 4
:dedent: 0
:start-after: [START how_to_cloud_dataproc_create_cluster_generate_cluster_config]
:end-before: [END how_to_cloud_dataproc_create_cluster_generate_cluster_config]

Expand All @@ -98,7 +98,7 @@ An example of a new cluster config and the updateMask:

.. exampleinclude:: /../../tests/system/providers/google/cloud/dataproc/example_dataproc_update.py
:language: python
:dedent: 4
:dedent: 0
:start-after: [START how_to_cloud_dataproc_updatemask_cluster_operator]
:end-before: [END how_to_cloud_dataproc_updatemask_cluster_operator]

Expand Down Expand Up @@ -154,55 +154,55 @@ Example of the configuration for a PySpark Job:

.. exampleinclude:: /../../tests/system/providers/google/cloud/dataproc/example_dataproc_pyspark.py
:language: python
:dedent: 4
:dedent: 0
:start-after: [START how_to_cloud_dataproc_pyspark_config]
:end-before: [END how_to_cloud_dataproc_pyspark_config]

Example of the configuration for a SparkSQl Job:

.. exampleinclude:: /../../tests/system/providers/google/cloud/dataproc/example_dataproc_spark_sql.py
:language: python
:dedent: 4
:dedent: 0
:start-after: [START how_to_cloud_dataproc_sparksql_config]
:end-before: [END how_to_cloud_dataproc_sparksql_config]

Example of the configuration for a Spark Job:

.. exampleinclude:: /../../tests/system/providers/google/cloud/dataproc/example_dataproc_spark.py
:language: python
:dedent: 4
:dedent: 0
:start-after: [START how_to_cloud_dataproc_spark_config]
:end-before: [END how_to_cloud_dataproc_spark_config]

Example of the configuration for a Spark Job running in `deferrable mode <https://airflow.apache.org/docs/apache-airflow/stable/concepts/deferring.html>`__:

.. exampleinclude:: /../../tests/system/providers/google/cloud/dataproc/example_dataproc_spark_deferrable.py
:language: python
:dedent: 4
:dedent: 0
:start-after: [START how_to_cloud_dataproc_spark_deferrable_config]
:end-before: [END how_to_cloud_dataproc_spark_deferrable_config]

Example of the configuration for a Hive Job:

.. exampleinclude:: /../../tests/system/providers/google/cloud/dataproc/example_dataproc_hive.py
:language: python
:dedent: 4
:dedent: 0
:start-after: [START how_to_cloud_dataproc_hive_config]
:end-before: [END how_to_cloud_dataproc_hive_config]

Example of the configuration for a Hadoop Job:

.. exampleinclude:: /../../tests/system/providers/google/cloud/dataproc/example_dataproc_hadoop.py
:language: python
:dedent: 4
:dedent: 0
:start-after: [START how_to_cloud_dataproc_hadoop_config]
:end-before: [END how_to_cloud_dataproc_hadoop_config]

Example of the configuration for a Pig Job:

.. exampleinclude:: /../../tests/system/providers/google/cloud/dataproc/example_dataproc_pig.py
:language: python
:dedent: 4
:dedent: 0
:start-after: [START how_to_cloud_dataproc_pig_config]
:end-before: [END how_to_cloud_dataproc_pig_config]

Expand All @@ -211,7 +211,7 @@ Example of the configuration for a SparkR:

.. exampleinclude:: /../../tests/system/providers/google/cloud/dataproc/example_dataproc_sparkr.py
:language: python
:dedent: 4
:dedent: 0
:start-after: [START how_to_cloud_dataproc_sparkr_config]
:end-before: [END how_to_cloud_dataproc_sparkr_config]

Expand Down

0 comments on commit 9679ca1

Please sign in to comment.