Breeze: Enter Shell environment with Breeze

[potiuk]

Includes:

• port forwarding
• selecting different backends
• selecting different python version
• diagnostics information
• remember last used parameters

[Bowrna]

@potiuk I will pick this one next

[potiuk]

Great!

[Bowrna]

@potiuk can you guide me in old breeze to the part of code that gets executed when we do ./breeze shell command? I am not able to backtrack the code.

[potiuk]

Look for enter_breeze :)

[Bowrna]

initialization::create_directories()
Creates folders to mount inside the container.
initialization::check_docker_version()

This checks the docker version and instructs if failed to install docker. Check if the docker available is good to go

build_images::forget_last_answer()
To remove the forced answer given previously.

build_images::determine_docker_cache_strategy()
We use pulled docker image cache by default for CI images

build_images::rebuild_ci_image_if_needed()
Checks if rebuild CI image is needed or not. Checks if docker build is needed. If yes, confirms image rebuild.

breeze::print_badge()

This emits the airflow logo, backend(mysql, postgres etc), backend_version, checks the cache to see if we have to suppress the asciiart logo. If not, print the visual art for airflow

initialization::summarize_build_environment()

How do we get values for all env variables initialized here? Are all of them are constants or do we have to compute on-demand or pass as a param? I will have to verify this part.

breeze::print_setup_instructions()
Prints setup instruction in case we find that autocomplete is not set

resoure_check() in run_resource_check.py

breeze::prepare_command_file()
This generates script inside build_cache dir depending on the type of configuration chosen and env var for breeze and file that can be run to use the docker command.

@potiuk I tried to find the logical steps getting executed in entering the shell. Please correct me if any of my understanding is wrong.

[potiuk]

Looks good. Few comments:

build_images::forget_last_answer()

We can skip this one. It has been used in pre-commits and shoudl have been removed in the last buildx refactor. I will remove it shortly. It was used to remember which answer we gave when "build" pre-commit run, so that "mypy" and "flake" would not ask the same question again. But right now I opted for simply "warning" the user and continuing with the pre-commit rather than asking the question, so this is not needed any more.

initialization::summarize_build_environment()

We should use BuildParams for that (and possibly add stuff that is missing there). This one shoudl likely be different though for CI and PROD build. Right now the same "summary" is used for PROD/CI. but PROD build accept many more params and variables, so we could end up with two different summaries - one for CI and one for PROD image.

breeze::prepare_command_file()

This one is a bit too "complex" and counter-intuitive and we might want to simplify it. Initially, I've done it because I wanted to have an easy way to run the final "docker-compose" command which breeze runs in the same way breeze does by running this simple "dc" script generated. This is nice to debug and test but a bit anti-pattern (you generate code that you execute later) and complex to understand. Since we are in python, I would rather now do it differently. We have the --verbose flag that prints the command we execute, so rather than setting the variables in the "dc" scripts, I'd pass them to docker-compose command via -e command. I.e instaad of :

What currently generated dc_ci script does:

PYTHON_MAJOR_MINOR_VERSION=3.6
COMPOSE_FILE=file1;file2....
docker-compose "${@}"

I would simply run:

docker-compose run -e PYTHON_MAJOR_MINOR_VERSION=3.6 -f <file_1> -f <file2> .....

which is equivalent.

One more thing the dc_ci script does - it checks if the docker-compose is the right version, so we should also do the check in python.