Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Various example dag errors on db init on fresh install #20225

Closed
1 of 2 tasks
khalidmammadov opened this issue Dec 11, 2021 · 1 comment · Fixed by #20295
Closed
1 of 2 tasks

Various example dag errors on db init on fresh install #20225

khalidmammadov opened this issue Dec 11, 2021 · 1 comment · Fixed by #20295
Assignees
Labels
area:core kind:bug This is a clearly a bug

Comments

@khalidmammadov
Copy link
Contributor

Apache Airflow version

2.2.3rc1 (release candidate)

What happened

After fresh install of the latest release and running db init it getting below error messages among usual db init messages. Note that I deliberately didn't chose to use VENV but I think that should be handled gracefully rather than below errors. Also relates to #19355

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/airflow/models/dagbag.py", line 331, in _load_modules_from_file
    loader.exec_module(new_module)
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/local/lib/python3.8/site-packages/airflow/example_dags/tutorial_taskflow_api_etl_virtualenv.py", line 81, in <module>
    tutorial_etl_dag = tutorial_taskflow_api_etl_virtualenv()
  File "/usr/local/lib/python3.8/site-packages/airflow/models/dag.py", line 2984, in factory
    f(**f_kwargs)
  File "/usr/local/lib/python3.8/site-packages/airflow/example_dags/tutorial_taskflow_api_etl_virtualenv.py", line 76, in tutorial_taskflow_api_etl_virtualenv
    order_data = extract()
  File "/usr/local/lib/python3.8/site-packages/airflow/decorators/base.py", line 219, in factory
    op = decorated_operator_class(
  File "/usr/local/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 188, in apply_defaults
    result = func(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/airflow/decorators/python_virtualenv.py", line 61, in __init__
    super().__init__(kwargs_to_upstream=kwargs_to_upstream, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 188, in apply_defaults
    result = func(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/airflow/decorators/base.py", line 131, in __init__
    super().__init__(**kwargs_to_upstream, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 188, in apply_defaults
    result = func(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/airflow/operators/python.py", line 371, in __init__
    raise AirflowException('PythonVirtualenvOperator requires virtualenv, please install it.')
airflow.exceptions.AirflowException: PythonVirtualenvOperator requires virtualenv, please install it.
ERROR [airflow.models.dagbag.DagBag] Failed to import: /usr/local/lib/python3.8/site-packages/airflow/example_dags/example_python_operator.py
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/airflow/models/dagbag.py", line 331, in _load_modules_from_file
    loader.exec_module(new_module)
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/local/lib/python3.8/site-packages/airflow/example_dags/example_python_operator.py", line 86, in <module>
    virtualenv_task = callable_virtualenv()
  File "/usr/local/lib/python3.8/site-packages/airflow/decorators/base.py", line 219, in factory
    op = decorated_operator_class(
  File "/usr/local/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 188, in apply_defaults
    result = func(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/airflow/decorators/python_virtualenv.py", line 61, in __init__
    super().__init__(kwargs_to_upstream=kwargs_to_upstream, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 188, in apply_defaults
    result = func(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/airflow/decorators/base.py", line 131, in __init__
    super().__init__(**kwargs_to_upstream, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 188, in apply_defaults
    result = func(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/airflow/operators/python.py", line 371, in __init__
    raise AirflowException('PythonVirtualenvOperator requires virtualenv, please install it.')
airflow.exceptions.AirflowException: PythonVirtualenvOperator requires virtualenv, please install it.
WARNI [unusual_prefix_97712dd65736a8bf7fa3a692877470abf8eebade_example_kubernetes_executor] Could not import DAGs in example_kubernetes_executor.py
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/airflow/example_dags/example_kubernetes_executor.py", line 36, in <module>
    from kubernetes.client import models as k8s
ModuleNotFoundError: No module named 'kubernetes'
WARNI [unusual_prefix_97712dd65736a8bf7fa3a692877470abf8eebade_example_kubernetes_executor] Install Kubernetes dependencies with: pip install apache-airflow[cncf.kubernetes]
ERROR [airflow.models.dagbag.DagBag] Failed to import: /usr/local/lib/python3.8/site-packages/airflow/example_dags/example_kubernetes_executor.py
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/airflow/models/dagbag.py", line 331, in _load_modules_from_file
    loader.exec_module(new_module)
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/local/lib/python3.8/site-packages/airflow/example_dags/example_kubernetes_executor.py", line 51, in <module>
    "pod_override": k8s.V1Pod(metadata=k8s.V1ObjectMeta(annotations={"test": "annotation"}))
NameError: name 'k8s' is not defined

What you expected to happen

No errors, but warnings

How to reproduce

docker run --name test-airflow-2.2.3rc1 -it python:3.8.12-buster bash
airflow db init
pip install apache-airflow==2.2.3rc1

Operating System

debian

Versions of Apache Airflow Providers

No response

Deployment

Other Docker-based deployment

Deployment details

docker run --name test-airflow-2.2.3rc1 -it python:3.8.12-buster bash

Anything else

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@khalidmammadov khalidmammadov added area:core kind:bug This is a clearly a bug labels Dec 11, 2021
@boring-cyborg
Copy link

boring-cyborg bot commented Dec 11, 2021

Thanks for opening your first issue here! Be sure to follow the issue template!

@uranusjr uranusjr changed the title Various example dag errors on db init on fressh install Various example dag errors on db init on fresh install Dec 12, 2021
@jedcunningham jedcunningham self-assigned this Dec 13, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:core kind:bug This is a clearly a bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants