Skip to content

Latest commit

 

History

History
1184 lines (840 loc) · 41.9 KB

vision.rst

File metadata and controls

1184 lines (840 loc) · 41.9 KB
Raw

Google Cloud Vision Operators

CloudVisionAddProductToProductSetOperator

Creates a new ReferenceImage resource.

For parameter definition, take a look at :class:`~airflow.contrib.operators.gcp_vision_operator.CloudVisionAddProductToProductSetOperator`

Arguments

Some arguments in the example DAG are taken from the OS environment variables:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :start-after: [START howto_operator_vision_args_common]
      :end-before: [END howto_operator_vision_args_common]


.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :start-after: [START howto_operator_vision_product_explicit_id]
      :end-before: [END howto_operator_vision_product_explicit_id]


.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :start-after: [START howto_operator_vision_product_set_explicit_id]
      :end-before: [END howto_operator_vision_product_set_explicit_id]

Using the operator

We are using the :class:`~google.cloud.vision_v1.types.Product`, :class:`~google.cloud.vision_v1.types.ProductSet` and Retry objects from Google libraries:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :start-after: [START howto_operator_vision_retry_import]
      :end-before: [END howto_operator_vision_retry_import]


.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :start-after: [START howto_operator_vision_product_set_import]
      :end-before: [END howto_operator_vision_product_set_import]


.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :start-after: [START howto_operator_vision_product_import]
      :end-before: [END howto_operator_vision_product_import]

If product_set_id and product_id was generated by the API it can be extracted from XCOM:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :dedent: 4
      :start-after: [START howto_operator_vision_add_product_to_product_set]
      :end-before: [END howto_operator_vision_add_product_to_product_set]

Otherwise it can be specified explicitly:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :dedent: 4
      :start-after: [START howto_operator_vision_add_product_to_product_set_2]
      :end-before: [END howto_operator_vision_add_product_to_product_set_2]

Templating

.. literalinclude:: ../../../../airflow/contrib/operators/gcp_vision_operator.py
    :language: python
    :dedent: 4
    :start-after: [START vision_add_product_to_product_set_template_fields]
    :end-before: [END vision_add_product_to_product_set_template_fields]

More information

See Google Cloud Vision Add Product To Product Set documentation.

CloudVisionAnnotateImageOperator

Run image detection and annotation for an image.

For parameter definition, take a look at :class:`~airflow.contrib.operators.gcp_vision_operator.CloudVisionAnnotateImageOperator`

Arguments

Some arguments in the example DAG are taken from the OS environment variables:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :start-after: [START howto_operator_vision_args_common]
      :end-before: [END howto_operator_vision_args_common]


.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :start-after: [START howto_operator_vision_annotate_image_url]
      :end-before: [END howto_operator_vision_annotate_image_url]

Using the operator

We are using the :class:`~google.cloud.enums` and Retry objects from Google libraries:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :start-after: [START howto_operator_vision_retry_import]
      :end-before: [END howto_operator_vision_retry_import]


.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :start-after: [START howto_operator_vision_enums_import]
      :end-before: [END howto_operator_vision_enums_import]



.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :dedent: 4
      :start-after: [START howto_operator_vision_annotate_image]
      :end-before: [END howto_operator_vision_annotate_image]

The result can be extracted from XCOM:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :dedent: 4
      :start-after: [START howto_operator_vision_annotate_image_result]
      :end-before: [END howto_operator_vision_annotate_image_result]

Templating

.. literalinclude:: ../../../../airflow/contrib/operators/gcp_vision_operator.py
    :language: python
    :dedent: 4
    :start-after: [START vision_annotate_image_template_fields]
    :end-before: [END vision_annotate_image_template_fields]

More information

See Google Cloud Vision Annotate Image documentation.

CloudVisionProductCreateOperator

Creates and returns a new product resource.

Possible errors regarding the Product object provided:

  • Returns INVALID_ARGUMENT if display_name is missing or longer than 4096 characters.
  • Returns INVALID_ARGUMENT if description is longer than 4096 characters.
  • Returns INVALID_ARGUMENT if product_category is missing or invalid.

For parameter definition, take a look at :class:`~airflow.contrib.operators.gcp_vision_operator.CloudVisionProductCreateOperator`

Arguments

Some arguments in the example DAG are taken from the OS environment variables:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :start-after: [START howto_operator_vision_args_common]
    :end-before: [END howto_operator_vision_args_common]


.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :start-after: [START howto_operator_vision_product_explicit_id]
    :end-before: [END howto_operator_vision_product_explicit_id]

Using the operator

We are using the Product and Retry objects from Google libraries:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :start-after: [START howto_operator_vision_product_import]
    :end-before: [END howto_operator_vision_product_import]


.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :start-after: [START howto_operator_vision_retry_import]
    :end-before: [END howto_operator_vision_retry_import]


.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :start-after: [START howto_operator_vision_product]
    :end-before: [END howto_operator_vision_product]

The product_id argument can be omitted (it will be generated by the API):

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :dedent: 4
    :start-after: [START howto_operator_vision_product_create]
    :end-before: [END howto_operator_vision_product_create]

Or it can be specified explicitly:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :dedent: 4
    :start-after: [START howto_operator_vision_product_create_2]
    :end-before: [END howto_operator_vision_product_create_2]

Templating

.. literalinclude:: ../../../../airflow/contrib/operators/gcp_vision_operator.py
    :language: python
    :dedent: 4
    :start-after: [START vision_product_create_template_fields]
    :end-before: [END vision_product_create_template_fields]

More information

See Google Cloud Vision Product create documentation.

CloudVisionProductDeleteOperator

Permanently deletes a product and its reference images.

Metadata of the product and all its images will be deleted right away, but search queries against ProductSets containing the product may still work until all related caches are refreshed.

Possible errors:

  • Returns NOT_FOUND if the product does not exist.

For parameter definition, take a look at :class:`~airflow.contrib.operators.gcp_vision_operator.CloudVisionProductDeleteOperator`

Arguments

Some arguments in the example DAG are taken from the OS environment variables:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :start-after: [START howto_operator_vision_args_common]
    :end-before: [END howto_operator_vision_args_common]


.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :start-after: [START howto_operator_vision_product_explicit_id]
    :end-before: [END howto_operator_vision_product_explicit_id]

Using the operator

If product_id was generated by the API it can be extracted from XCOM:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :dedent: 4
    :start-after: [START howto_operator_vision_product_delete]
    :end-before: [END howto_operator_vision_product_delete]

Otherwise it can be specified explicitly:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :dedent: 4
    :start-after: [START howto_operator_vision_product_delete_2]
    :end-before: [END howto_operator_vision_product_delete_2]

Templating

.. literalinclude:: ../../../../airflow/contrib/operators/gcp_vision_operator.py
    :language: python
    :dedent: 4
    :start-after: [START vision_product_delete_template_fields]
    :end-before: [END vision_product_delete_template_fields]

More information

See Google Cloud Vision Product delete documentation.

CloudVisionProductGetOperator

Gets information associated with a Product.

Possible errors:

  • Returns NOT_FOUND if the Product does not exist.

For parameter definition, take a look at :class:`~airflow.contrib.operators.gcp_vision_operator.CloudVisionProductGetOperator`

Arguments

Some arguments in the example DAG are taken from the OS environment variables:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :start-after: [START howto_operator_vision_args_common]
    :end-before: [END howto_operator_vision_args_common]


.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :start-after: [START howto_operator_vision_product_explicit_id]
    :end-before: [END howto_operator_vision_product_explicit_id]

Using the operator

If product_id was generated by the API it can be extracted from XCOM:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :dedent: 4
    :start-after: [START howto_operator_vision_product_get]
    :end-before: [END howto_operator_vision_product_get]

Otherwise it can be specified explicitly:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :dedent: 4
    :start-after: [START howto_operator_vision_product_get_2]
    :end-before: [END howto_operator_vision_product_get_2]

Templating

.. literalinclude:: ../../../../airflow/contrib/operators/gcp_vision_operator.py
    :language: python
    :dedent: 4
    :start-after: [START vision_product_get_template_fields]
    :end-before: [END vision_product_get_template_fields]

More information

See Google Cloud Vision Product get documentation.

CloudVisionProductSetCreateOperator

Creates a new ProductSet resource.

For parameter definition, take a look at :class:`~airflow.contrib.operators.gcp_vision_operator.CloudVisionProductSetCreateOperator`

Arguments

Some arguments in the example DAG are taken from the OS environment variables:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :start-after: [START howto_operator_vision_args_common]
    :end-before: [END howto_operator_vision_args_common]


.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :start-after: [START howto_operator_vision_product_set_explicit_id
    :end-before: [END howto_operator_vision_product_set_explicit_id

Using the operator

We are using the ProductSet and Retry objects from Google libraries:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :start-after: [START howto_operator_vision_product_set_import]
    :end-before: [END howto_operator_vision_product_set_import]


.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :start-after: [START howto_operator_vision_retry_import]
    :end-before: [END howto_operator_vision_retry_import]


.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :start-after: [START howto_operator_vision_product_set]
    :end-before: [END howto_operator_vision_product_set]

The product_set_id argument can be omitted (it will be generated by the API):

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :dedent: 4
    :start-after: [START howto_operator_vision_product_set_create]
    :end-before: [END howto_operator_vision_product_set_create]

Or it can be specified explicitly:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :dedent: 4
    :start-after: [START howto_operator_vision_product_set_create_2]
    :end-before: [END howto_operator_vision_product_set_create_2]

Templating

.. literalinclude:: ../../../../airflow/contrib/operators/gcp_vision_operator.py
    :language: python
    :dedent: 4
    :start-after: [START vision_productset_create_template_fields]
    :end-before: [END vision_productset_create_template_fields]

More information

See Google Cloud Vision ProductSet create documentation.

CloudVisionProductSetDeleteOperator

Permanently deletes a ProductSet. Products and ReferenceImages in the ProductSet are not deleted. The actual image files are not deleted from Google Cloud Storage.

For parameter definition, take a look at :class:`~airflow.contrib.operators.gcp_vision_operator.CloudVisionProductSetDeleteOperator`

Arguments

Some arguments in the example DAG are taken from the OS environment variables:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :start-after: [START howto_operator_vision_args_common]
    :end-before: [END howto_operator_vision_args_common]


.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :start-after: [START howto_operator_vision_product_set_explicit_id]
    :end-before: [END howto_operator_vision_product_set_explicit_id]

Using the operator

If product_set_id was generated by the API it can be extracted from XCOM:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :dedent: 4
    :start-after: [START howto_operator_vision_product_set_delete]
    :end-before: [END howto_operator_vision_product_set_delete]

Otherwise it can be specified explicitly:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :dedent: 4
    :start-after: [START howto_operator_vision_product_set_delete_2]
    :end-before: [END howto_operator_vision_product_set_delete_2]

Templating

.. literalinclude:: ../../../../airflow/contrib/operators/gcp_vision_operator.py
    :language: python
    :dedent: 4
    :start-after: [START vision_productset_delete_template_fields]
    :end-before: [END vision_productset_delete_template_fields]

More information

See Google Cloud Vision ProductSet delete documentation.

CloudVisionProductSetGetOperator

Gets information associated with a ProductSet.

For parameter definition, take a look at :class:`~airflow.contrib.operators.gcp_vision_operator.CloudVisionProductSetGetOperator`

Arguments

Some arguments in the example DAG are taken from the OS environment variables:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :start-after: [START howto_operator_vision_args_common]
    :end-before: [END howto_operator_vision_args_common]


.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :start-after: [START howto_operator_vision_product_set_explicit_id]
    :end-before: [END howto_operator_vision_product_set_explicit_id]

Using the operator

If product_set_id was generated by the API it can be extracted from XCOM:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :dedent: 4
    :start-after: [START howto_operator_vision_product_set_get]
    :end-before: [END howto_operator_vision_product_set_get]

Otherwise it can be specified explicitly:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :dedent: 4
    :start-after: [START howto_operator_vision_product_set_get_2]
    :end-before: [END howto_operator_vision_product_set_get_2]

Templating

.. literalinclude:: ../../../../airflow/contrib/operators/gcp_vision_operator.py
    :language: python
    :dedent: 4
    :start-after: [START vision_productset_get_template_fields]
    :end-before: [END vision_productset_get_template_fields]

More information

See Google Cloud Vision ProductSet get documentation.

CloudVisionProductSetUpdateOperator

Makes changes to a ProductSet resource. Only display_name can be updated currently.

Note

To locate the ProductSet resource, its name in the form projects/PROJECT_ID/locations/LOC_ID/productSets/PRODUCT_SET_ID is necessary.

You can provide the name directly as an attribute of the product_set object. However, you can leave it blank and provide location and product_set_id instead (and optionally project_id - if not present, the connection default will be used) and the name will be created by the operator itself.

This mechanism exists for your convenience, to allow leaving the project_id empty and having Airflow use the connection default project_id.

For parameter definition, take a look at :class:`~airflow.contrib.operators.gcp_vision_operator.CloudVisionProductSetUpdateOperator`

Arguments

Some arguments in the example DAG are taken from the OS environment variables:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :start-after: [START howto_operator_vision_args_common]
    :end-before: [END howto_operator_vision_args_common]


.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :start-after: [START howto_operator_vision_product_set_explicit_id]
    :end-before: [END howto_operator_vision_product_set_explicit_id]

Using the operator

We are using the ProductSet object from the Google Cloud Vision library:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :start-after: [START howto_operator_vision_product_set_import]
    :end-before: [END howto_operator_vision_product_set_import]


.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :start-after: [START howto_operator_vision_product_set]
    :end-before: [END howto_operator_vision_product_set]

Initialization of the task:

If product_set_id was generated by the API it can be extracted from XCOM:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :dedent: 4
    :start-after: [START howto_operator_vision_product_set_update]
    :end-before: [END howto_operator_vision_product_set_update]

Otherwise it can be specified explicitly:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :dedent: 4
    :start-after: [START howto_operator_vision_product_set_update_2]
    :end-before: [END howto_operator_vision_product_set_update_2]

Templating

.. literalinclude:: ../../../../airflow/contrib/operators/gcp_vision_operator.py
    :language: python
    :dedent: 4
    :start-after: [START vision_productset_update_template_fields]
    :end-before: [END vision_productset_update_template_fields]

More information

See Google Cloud Vision ProductSet update documentation.

CloudVisionProductUpdateOperator

Makes changes to a Product resource. Only the display_name, description, and labels fields can be updated right now. If labels are updated, the change will not be reflected in queries until the next index time.

Note

To locate the Product resource, its name in the form projects/PROJECT_ID/locations/LOC_ID/products/PRODUCT_ID is necessary.

You can provide the name directly as an attribute of the product object. However, you can leave it blank and provide location and product_id instead (and optionally project_id - if not present, the connection default will be used) and the name will be created by the operator itself.

This mechanism exists for your convenience, to allow leaving the project_id empty and having Airflow use the connection default project_id.

Possible errors:

  • Returns NOT_FOUND if the Product does not exist.
  • Returns INVALID_ARGUMENT if display_name is present in update_mask but is missing from the request or longer than 4096 characters.
  • Returns INVALID_ARGUMENT if description is present in update_mask but is longer than 4096 characters.
  • Returns INVALID_ARGUMENT if product_category is present in update_mask.

For parameter definition, take a look at :class:`~airflow.contrib.operators.gcp_vision_operator.CloudVisionProductUpdateOperator`

Arguments

Some arguments in the example DAG are taken from the OS environment variables:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :start-after: [START howto_operator_vision_args_common]
    :end-before: [END howto_operator_vision_args_common]


.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :start-after: [START howto_operator_vision_product_explicit_id]
    :end-before: [END howto_operator_vision_product_explicit_id]

Using the operator

We are using the Product object from the Google Cloud Vision library:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :start-after: [START howto_operator_vision_product_import]
    :end-before: [END howto_operator_vision_product_import]


.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :start-after: [START howto_operator_vision_product]
    :end-before: [END howto_operator_vision_product]

If product_id was generated by the API it can be extracted from XCOM:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :dedent: 4
    :start-after: [START howto_operator_vision_product_update]
    :end-before: [END howto_operator_vision_product_update]

Otherwise it can be specified explicitly:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
    :language: python
    :dedent: 4
    :start-after: [START howto_operator_vision_product_update_2]
    :end-before: [END howto_operator_vision_product_update_2]

Templating

.. literalinclude:: ../../../../airflow/contrib/operators/gcp_vision_operator.py
    :language: python
    :dedent: 4
    :start-after: [START vision_product_update_template_fields]
    :end-before: [END vision_product_update_template_fields]

More information

See Google Cloud Vision Product update documentation.

CloudVisionReferenceImageCreateOperator

Creates a new ReferenceImage resource.

For parameter definition, take a look at :class:`~airflow.contrib.operators.gcp_vision_operator.CloudVisionReferenceImageCreateOperator`

Arguments

Some arguments in the example DAG are taken from the OS environment variables:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :start-after: [START howto_operator_vision_args_common]
      :end-before: [END howto_operator_vision_args_common]


.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :start-after: [START howto_operator_vision_reference_image_args]
      :end-before: [END howto_operator_vision_reference_image_args]

Using the operator

We are using the :class:`~google.cloud.vision_v1.types.ReferenceImage` and Retry objects from Google libraries:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :start-after: [START howto_operator_vision_reference_image_import]
      :end-before: [END howto_operator_vision_reference_image_import]


.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :start-after: [START howto_operator_vision_retry_import]
      :end-before: [END howto_operator_vision_retry_import]


.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :start-after: [START howto_operator_vision_reference_image]
      :end-before: [END howto_operator_vision_reference_image]

The product_set_id argument can be omitted (it will be generated by the API):

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :dedent: 4
      :start-after: [START howto_operator_vision_reference_image_create]
      :end-before: [END howto_operator_vision_reference_image_create]

Or it can be specified explicitly:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :dedent: 4
      :start-after: [START howto_operator_vision_reference_image_create_2]
      :end-before: [END howto_operator_vision_reference_image_create_2]

Templating

.. literalinclude:: ../../../../airflow/contrib/operators/gcp_vision_operator.py
    :language: python
    :dedent: 4
    :start-after: [START vision_reference_image_create_template_fields]
    :end-before: [END vision_reference_image_create_template_fields]

More information

See Google Cloud Vision ReferenceImage create documentation.

CloudVisionRemoveProductFromProductSetOperator

Creates a new ReferenceImage resource.

For parameter definition, take a look at :class:`~airflow.contrib.operators.gcp_vision_operator.CloudVisionRemoveProductFromProductSetOperator`

Arguments

Some arguments in the example DAG are taken from the OS environment variables:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :start-after: [START howto_operator_vision_args_common]
      :end-before: [END howto_operator_vision_args_common]


.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :start-after: [START howto_operator_vision_product_explicit_id]
      :end-before: [END howto_operator_vision_product_explicit_id]


.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :start-after: [START howto_operator_vision_product_set_explicit_id]
      :end-before: [END howto_operator_vision_product_set_explicit_id]

Using the operator

We are using the :class:`~google.cloud.vision_v1.types.Product`, :class:`~google.cloud.vision_v1.types.ProductSet` and Retry objects from Google libraries:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :start-after: [START howto_operator_vision_retry_import]
      :end-before: [END howto_operator_vision_retry_import]


.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :start-after: [START howto_operator_vision_product_set_import]
      :end-before: [END howto_operator_vision_product_set_import]


.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :start-after: [START howto_operator_vision_product_import]
      :end-before: [END howto_operator_vision_product_import]

If product_set_id and product_id was generated by the API it can be extracted from XCOM:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :dedent: 4
      :start-after: [START howto_operator_vision_remove_product_from_product_set]
      :end-before: [END howto_operator_vision_remove_product_from_product_set]

Otherwise it can be specified explicitly:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :dedent: 4
      :start-after: [START howto_operator_vision_remove_product_from_product_set_2]
      :end-before: [END howto_operator_vision_remove_product_from_product_set_2]

Templating

.. literalinclude:: ../../../../airflow/contrib/operators/gcp_vision_operator.py
    :language: python
    :dedent: 4
    :start-after: [START vision_remove_product_from_product_set_template_fields]
    :end-before: [END vision_remove_product_from_product_set_template_fields]

More information

See Google Cloud Vision Remove Product From Product Set documentation.

CloudVisionDetectTextOperator

Run text detection for an image.

For parameter definition, take a look at :class:`~airflow.contrib.operators.gcp_vision_operator.CloudVisionDetectTextOperator`

Arguments

Some arguments in the example DAG are taken from the OS environment variables:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :start-after: [START howto_operator_vision_annotate_image_url]
      :end-before: [END howto_operator_vision_annotate_image_url]

Using the operator

We are using the :class:`Retry` objects from Google libraries:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :start-after: [START howto_operator_vision_retry_import]
      :end-before: [END howto_operator_vision_retry_import]


.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :dedent: 4
      :start-after: [START howto_operator_vision_detect_text]
      :end-before: [END howto_operator_vision_detect_text]

The result can be extracted from XCOM:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :dedent: 4
      :start-after: [START howto_operator_vision_detect_text_result]
      :end-before: [END howto_operator_vision_detect_text_result]

Templating

.. literalinclude:: ../../../../airflow/contrib/operators/gcp_vision_operator.py
    :language: python
    :dedent: 4
    :start-after: [START vision_detect_text_set_template_fields]
    :end-before: [END vision_detect_text_set_template_fields]

More information

See Google Cloud Vision Text Detection documentation.

CloudVisionDetectDocumentTextOperator

Run document text detection for an image.

For parameter definition, take a look at :class:`~airflow.contrib.operators.gcp_vision_operator.CloudVisionDetectDocumentTextOperator`

Arguments

Some arguments in the example DAG are taken from the OS environment variables:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :start-after: [START howto_operator_vision_annotate_image_url]
      :end-before: [END howto_operator_vision_annotate_image_url]

Using the operator

We are using the :class:`Retry` objects from Google libraries:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :start-after: [START howto_operator_vision_retry_import]
      :end-before: [END howto_operator_vision_retry_import]


.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :dedent: 4
      :start-after: [START howto_operator_vision_document_detect_text]
      :end-before: [END howto_operator_vision_document_detect_text]

The result can be extracted from XCOM:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :dedent: 4
      :start-after: [START howto_operator_vision_document_detect_text_result]
      :end-before: [END howto_operator_vision_document_detect_text_result]

Templating

.. literalinclude:: ../../../../airflow/contrib/operators/gcp_vision_operator.py
    :language: python
    :dedent: 4
    :start-after: [START vision_document_detect_text_set_template_fields]
    :end-before: [END vision_document_detect_text_set_template_fields]

More information

See Google Cloud Vision Document Text Detection documentation.

CloudVisionDetectImageLabelsOperator

Run image label detection for an image.

For parameter definition, take a look at :class:`~airflow.contrib.operators.gcp_vision_operator.CloudVisionDetectImageLabelsOperator`

Arguments

Some arguments in the example DAG are taken from the OS environment variables:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :start-after: [START howto_operator_vision_annotate_image_url]
      :end-before: [END howto_operator_vision_annotate_image_url]

Using the operator

We are using the :class:`Retry` objects from Google libraries:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :start-after: [START howto_operator_vision_retry_import]
      :end-before: [END howto_operator_vision_retry_import]


.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :dedent: 4
      :start-after: [START howto_operator_vision_detect_labels]
      :end-before: [END howto_operator_vision_detect_labels]

The result can be extracted from XCOM:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :dedent: 4
      :start-after: [START howto_operator_vision_detect_labels_result]
      :end-before: [END howto_operator_vision_detect_labels_result]

Templating

.. literalinclude:: ../../../../airflow/contrib/operators/gcp_vision_operator.py
    :language: python
    :dedent: 4
    :start-after: [START vision_detect_labels_template_fields]
    :end-before: [END vision_detect_labels_template_fields]

More information

See Google Cloud Vision Label Detection documentation.

CloudVisionDetectImageSafeSearchOperator

Run image label detection for an image.

For parameter definition, take a look at :class:`~airflow.contrib.operators.gcp_vision_operator.CloudVisionDetectImageSafeSearchOperator`

Arguments

Some arguments in the example DAG are taken from the OS environment variables:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :start-after: [START howto_operator_vision_annotate_image_url]
      :end-before: [END howto_operator_vision_annotate_image_url]

Using the operator

We are using the :class:`Retry` objects from Google libraries:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :start-after: [START howto_operator_vision_retry_import]
      :end-before: [END howto_operator_vision_retry_import]


.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :dedent: 4
      :start-after: [START howto_operator_vision_detect_safe_search]
      :end-before: [END howto_operator_vision_detect_safe_search]

The result can be extracted from XCOM:

.. exampleinclude:: ../../../../airflow/contrib/example_dags/example_gcp_vision.py
      :language: python
      :dedent: 4
      :start-after: [START howto_operator_vision_detect_safe_search_result]
      :end-before: [END howto_operator_vision_detect_safe_search_result]

Templating

.. literalinclude:: ../../../../airflow/contrib/operators/gcp_vision_operator.py
    :language: python
    :dedent: 4
    :start-after: [START vision_detect_safe_search_template_fields]
    :end-before: [END vision_detect_safe_search_template_fields]

More information

See Google Cloud Vision Safe Search Detection documentation.