Skip to content

Importing CreateCustomContainerTrainingJobOperator takes too much time #29613

@shahar1

Description

@shahar1

Apache Airflow version

Other Airflow 2 version (please specify below)

What happened

When importing CreateCustomContainerTrainingJobOperator it seems to take 3-4 seconds just to import the module.
I ran time python for two DAG files - one with the import line and one without it. The import line is simple as follows:
from airflow.providers.google.cloud.operators.vertex_ai.custom_job import CreateCustomContainerTrainingJobOperator

The results of the comparison:

airflow@airflow-scheduler-##########-#####:~/gcs/dags$ time python dag_w_import.py
real    0m4.884s
user    0m4.668s
sys     0m1.425s
airflow@airflow-scheduler-##########-#####:~/gcs/dags$ time python same_dag_wo_import.py
real    0m1.484s
user    0m1.310s
sys     0m0.163s

What you think should happen instead

The import of the module should be instant

How to reproduce

Described above

Operating System

Ubuntu 18.04.6 LTS

Versions of Apache Airflow Providers

apache-airflow-providers-google==8.6.0

Deployment

Google Cloud Composer

Deployment details

composer-2.1.6-airflow-2.4.3

Anything else

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions