Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding --no-cache-dir to dockerfile pip install #3455

Merged
merged 1 commit into from
Jul 6, 2023

Conversation

noyoshi
Copy link
Collaborator

@noyoshi noyoshi commented Jul 6, 2023

Code Pull Requests

Please provide the following:

  • This adds the --no-cache-dir to pip installs within the Dockerfile. Without doing this, the pip cache adds around 2.5 GB uncompressed to the resulting containers.

@arnavgarg1 arnavgarg1 self-requested a review July 6, 2023 20:11
Copy link
Contributor

@arnavgarg1 arnavgarg1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Can we just confirm that creating the docker container from the dockerfile takes ~ the same amount of time? If yes, then I think LGTM!

@noyoshi
Copy link
Collaborator Author

noyoshi commented Jul 6, 2023

Looks good to me when I ran it.

Do you know why the tests are failing?

@arnavgarg1
Copy link
Contributor

@noyoshi I think it might be because of torchmetrics. I created this PR to pin it, let's see: #3456

@github-actions
Copy link

github-actions bot commented Jul 6, 2023

Unit Test Results

       6 files  ±       0         6 suites  ±0   50m 53s ⏱️ - 25m 25s
2 743 tests +2 710  2 719 ✔️ +2 690  11 💤 +  7  13 +13 
8 229 runs  +8 130  8 157 ✔️ +8 070  33 💤 +21  39 +39 

For more details on these failures, see this check.

Results for commit bc3a0fc. ± Comparison against base commit 8a5147d.

This pull request removes 25 and adds 2735 tests. Note that renamed tests count towards both.
tests.integration_tests.test_cli ‑ test_reproducible_cli_runs[horovod-experiment-1919-0]
tests.integration_tests.test_cli ‑ test_reproducible_cli_runs[horovod-experiment-1919-1]
tests.integration_tests.test_cli ‑ test_reproducible_cli_runs[horovod-experiment-31-0]
tests.integration_tests.test_cli ‑ test_reproducible_cli_runs[horovod-experiment-31-1]
tests.integration_tests.test_cli ‑ test_reproducible_cli_runs[horovod-train-1919-0]
tests.integration_tests.test_cli ‑ test_reproducible_cli_runs[horovod-train-1919-1]
tests.integration_tests.test_cli ‑ test_reproducible_cli_runs[horovod-train-31-0]
tests.integration_tests.test_cli ‑ test_reproducible_cli_runs[horovod-train-31-1]
tests.integration_tests.test_cli ‑ test_train_cli_horovod
tests.integration_tests.test_experiment ‑ test_experiment_model_resume_distributed[horovod]
…
tests.ludwig.augmentation.test_augmentation_pipeline ‑ test_image_augmentation[augmentation_pipeline_ops0]
tests.ludwig.augmentation.test_augmentation_pipeline ‑ test_image_augmentation[augmentation_pipeline_ops1]
tests.ludwig.augmentation.test_augmentation_pipeline ‑ test_image_augmentation[augmentation_pipeline_ops2]
tests.ludwig.augmentation.test_augmentation_pipeline ‑ test_invalid_augmentation_parameters[None]
tests.ludwig.augmentation.test_augmentation_pipeline ‑ test_invalid_augmentation_parameters[augmentation_pipeline_ops1]
tests.ludwig.augmentation.test_augmentation_pipeline ‑ test_invalid_augmentation_parameters[augmentation_pipeline_ops2]
tests.ludwig.augmentation.test_augmentation_pipeline ‑ test_invalid_augmentation_parameters[augmentation_pipeline_ops4]
tests.ludwig.augmentation.test_augmentation_pipeline ‑ test_invalid_augmentation_parameters[random_horizontal_flip]
tests.ludwig.augmentation.test_augmentation_pipeline ‑ test_load_model_with_augmentation_pipeline
tests.ludwig.augmentation.test_augmentation_pipeline ‑ test_local_model_training_with_augmentation_pipeline[preprocessing0-encoder0-False]
…
This pull request removes 1 skipped test and adds 8 skipped tests. Note that renamed tests count towards both.
tests.integration_tests.test_horovod ‑ test_horovod_gpu_memory_limit
tests.ludwig.automl.test_base_config
tests.ludwig.automl.test_utils
tests.ludwig.backend.test_ray
tests.ludwig.benchmarking.test_profiler
tests.ludwig.data.test_ray_data
tests.ludwig.utils.test_fs_utils ‑ test_get_fs_and_path_invalid_windows
tests.ludwig.utils.test_hyperopt_ray_utils ‑ test_grid_strategy[test_1]
tests.ludwig.utils.test_hyperopt_ray_utils ‑ test_grid_strategy[test_2]

@arnavgarg1
Copy link
Contributor

@noyoshi yeah that was the culprit, feel free to either pull in master before merging or merge directly since your changes are unrelated!

@noyoshi noyoshi merged commit 231fd62 into master Jul 6, 2023
3 of 16 checks passed
@noyoshi noyoshi deleted the no-cache-dir-dockerfile branch July 6, 2023 22:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants