PythonVirtualEnvOperator with tasks test command installs package outside of virtualenv.

[starone99]

Apache Airflow version

2.6.3

What happened

If I use PythonVirtualEnvOperator with airflow tasks test command, it installs package outside of virtualenv.
It is quite strange that scheduled task does not behave like this but with tasks test command.
I checked this behavior in two OS, Debian GNU/Linux 11 (bullseye) and MacOS 13.0
The detail is in how to reproduce

What you think should happen instead

No response

How to reproduce

helmfile version: 1.10.0
airflow version: 2.6.3

0. make task using PythonVirtualEnvOperator with module that is preinstalled in system-site-packages
   in my case, it is openpyxl which is one of constraints of airflow.
   in 2.6.3 openpyxl==3.1.2
   following is my operator that I found the bug.

PythonVirtualenvOperator(
    task_id='test_venv',
    dag=dag,
    python_callable=dummy_callable,
    requirements='openpyxl==3.0.10',
)

1. deploy airflow with helmfile
2. go to worker with kubectl exec command
3. check version of openpyxl with pip freeze command. It shows 3.1.2
4. run command 'airflow tasks test dag_id task_id' with the task defined above.
5. recheck version of openpyxl with pip freeze command. It shows 3.0.10

Operating System

Debian GNU/Linux 11 (bullseye), MacOS 13.0

Versions of Apache Airflow Providers

No response

Deployment

Official Apache Airflow Helm Chart

Deployment details

No response

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[hussein-awala]

I just tested with dag:

import pendulum

from airflow.models import DAG
from airflow.operators.python import PythonVirtualenvOperator


with DAG(
    dag_id="venv_dag",
    start_date=pendulum.datetime(2021, 10, 1),
    schedule=None,
) as venv_dag:
    def print_hello():
        print("hello world")
    PythonVirtualenvOperator(
        task_id='test_venv',
        python_callable=print_hello,
        requirements='openpyxl==3.0.10',
    )

and I couldn't reproduce the issue.

I tried with tasks test command:

airflow tasks test venv_dag test_venv

And with:

if __name__ == '__main__':
    venv_dag.test(execution_date=pendulum.datetime(2021, 10, 1))

Could you provide more information about your issue? (for example provide the task log)

[potiuk]

I also could not reproduce it. I run the same DAG as @hussein-awala and it correctly installed openpyxl and did not uninstall it in the worker "main" environment.

[2023-07-31, 09:07:56 UTC] {process_utils.py:182} INFO - Executing cmd: /usr/local/bin/python -m virtualenv /tmp/venvz82thpo8 --system-site-packages
[2023-07-31, 09:07:56 UTC] {process_utils.py:186} INFO - Output:
[2023-07-31, 09:07:57 UTC] {process_utils.py:190} INFO - created virtual environment CPython3.8.17.final.0-64 in 286ms
[2023-07-31, 09:07:57 UTC] {process_utils.py:190} INFO -   creator CPython3Posix(dest=/tmp/venvz82thpo8, clear=False, no_vcs_ignore=False, global=True)
[2023-07-31, 09:07:57 UTC] {process_utils.py:190} INFO -   seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/airflow/.local/share/virtualenv)
[2023-07-31, 09:07:57 UTC] {process_utils.py:190} INFO -     added seed packages: pip==23.2, setuptools==68.0.0, wheel==0.40.0
[2023-07-31, 09:07:57 UTC] {process_utils.py:190} INFO -   activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
[2023-07-31, 09:07:57 UTC] {process_utils.py:182} INFO - Executing cmd: /tmp/venvz82thpo8/bin/pip install -r /tmp/venvz82thpo8/requirements.txt
[2023-07-31, 09:07:57 UTC] {process_utils.py:186} INFO - Output:
[2023-07-31, 09:07:58 UTC] {process_utils.py:190} INFO - Collecting openpyxl==3.0.10 (from -r /tmp/venvz82thpo8/requirements.txt (line 1))
[2023-07-31, 09:07:58 UTC] {process_utils.py:190} INFO -   Downloading openpyxl-3.0.10-py2.py3-none-any.whl (242 kB)
[2023-07-31, 09:07:58 UTC] {process_utils.py:190} INFO -      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 242.1/242.1 kB 5.7 MB/s eta 0:00:00
[2023-07-31, 09:07:58 UTC] {process_utils.py:190} INFO - Requirement already satisfied: et-xmlfile in /home/airflow/.local/lib/python3.8/site-packages (from openpyxl==3.0.10->-r /tmp/venvz82thpo8/requirements.txt (line 1)) (1.1.0)
[2023-07-31, 09:08:01 UTC] {process_utils.py:190} INFO - Installing collected packages: openpyxl
[2023-07-31, 09:08:01 UTC] {process_utils.py:190} INFO -   Attempting uninstall: openpyxl
[2023-07-31, 09:08:01 UTC] {process_utils.py:190} INFO -     Found existing installation: openpyxl 3.1.2
[2023-07-31, 09:08:01 UTC] {process_utils.py:190} INFO -     Not uninstalling openpyxl at /home/airflow/.local/lib/python3.8/site-packages, outside environment /tmp/venvz82thpo8
[2023-07-31, 09:08:01 UTC] {process_utils.py:190} INFO -     Can't uninstall 'openpyxl'. No files were found to uninstall.
[2023-07-31, 09:08:01 UTC] {process_utils.py:190} INFO - Successfully installed openpyxl-3.0.10
[2023-07-31, 09:08:01 UTC] {process_utils.py:190} INFO - 
[2023-07-31, 09:08:01 UTC] {process_utils.py:190} INFO - [notice] A new release of pip is available: 23.2 -> 23.2.1
[2023-07-31, 09:08:01 UTC] {process_utils.py:190} INFO - [notice] To update, run: python -m pip install --upgrade pip
[2023-07-31, 09:08:01 UTC] {process_utils.py:182} INFO - Executing cmd: /tmp/venvz82thpo8/bin/python /tmp/venvz82thpo8/script.py /tmp/venvz82thpo8/script.in /tmp/venvz82thpo8/script.out /tmp/venvz82thpo8/string_args.txt /tmp/venvz82thpo8/termination.log
[2023-07-31, 09:08:01 UTC] {process_utils.py:186} INFO - Output:
[2023-07-31, 09:08:01 UTC] {taskinstance.py:829} DEBUG - Refreshing TaskInstance <TaskInstance: venv_dag.test_venv manual__2023-07-31T09:07:55.967888+00:00 [running]> from DB

After it was run:

airflow@airflow-worker-0:/opt/airflow/dags$ pip freeze | grep openpyx
openpyxl==3.1.2

You need to explain in detail what is your configuration and show some logs of what's happening - my guess is that you likely manually updated openpyxl. You need to try again.

Converting it into discussion - this does not seem like an Airflow bug.