Add example of dockerfile with creating new virtualenv

[ayushthe1]

---

closes: #26881

Add a guide to the documentation about how to add your own python venv and install packages via requirements.txt ExternalPythonOperator

[potiuk]

I think the dockerfile does not build - you can try to build it locally with docker build command and see what the problem is.

[ayushthe1]

hey @stromal ,i am getting an error in dockerfile due to command COPY airflow/requirments.txt .Is the path specified here right or i need to do something else ?

[stromal]

@ayushthe1
I just heave base folder that contains everything called airflow and it has requirements.txt to install staff that is the whole point of the ExternalPythonOperator.
tha Dockerfile outside of (at /home/ubuntu) that so it can acces all folders around the ubuntu system.

requirements.txt ex,.:

pandas==1.2.0
pymongo==3.11

[potiuk]

Yeah - the requirements.txt files should also be added to the folder and referred to, (and also I think in the .rst doc you should include the content of the requirements.txt as separate exampleinclude.

[potiuk]

BTW, This is the main purpose of the automated tests for those examples - we could have missed explanation and we would not explain that requirements.txt file is there, but the failing test actually pointed out that we also need it there (and add explanation on it)

[ayushthe1]

hey @potiuk ,what should i add in the requirements.txt file ?

[potiuk]

Can be anything neutral. I think we have a few places with similar things:

For example this is what we have in example_python_operator:

        @task.virtualenv(
            task_id="virtualenv_python", requirements=["colorama==0.4.0"], system_site_packages=False
        )

[stromal]

@ayushthe1

numpy and pandas

[potiuk]

@ayushthe1

numpy and pandas

I tihnk numpy and pandas are definitely too much for such an example. We have this page - if someone adds an example how to install numpy and pandas there, this is the right place:

https://airflow.apache.org/docs/docker-stack/recipes.html

The example that should be part of "creating a venv" is there to show mechanism not the way how to solve particular problems with installing numpy and pandas and they will cloud the purpose of that example.

If you think @stromal that numpy and pandas are needed, you can add a chapter to recipes but this is not the scope of this PR. Scope of this PR is to show how to create a virtualenv in the current base Docker image.

[potiuk]

@ayushthe1 - shall we get it to completion :)?

[ayushthe1]

@ayushthe1 - shall we get it to completion :)?

Should I add coloroma==0.4.0 in requirement.txt ?

[potiuk]

@ayushthe1 - shall we get it to completion :)?

Should I add coloroma==0.4.0 in requirement.txt ?

Yes. Please.

[potiuk]

Hey @ayushthe1 - are you finishing that one? We are about to release 2.5 and it would be great to get that one in

[ayushthe1]

Hey @ayushthe1 - are you finishing that one? We are about to release 2.5 and it would be great to get that one in

Sorry, it's taking so much time to complete it. Currently my exams are going on and I will complete the work as soon as i get some free time.

[potiuk]

gentle ping :)

[ayushthe1]

gentle ping :)

sorry for the delay ,i will now start working back on this.

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.