Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix Apex Build #298

Merged
merged 1 commit into from
Sep 19, 2023
Merged

Fix Apex Build #298

merged 1 commit into from
Sep 19, 2023

Conversation

MaxiBoether
Copy link
Contributor

As discussed in NVIDIA/apex#1594, it appears the only solution to make apex build on a recent release is to ignore the build dependencies during packaging... Otherwise, we get the error that the packaging package is missing, although we explicitly install it.

@github-actions
Copy link

✅ Result of Pytest Coverage

---------- coverage: platform linux, python 3.11.5-final-0 -----------

Name Stmts Miss Cover
modyn/common/benchmark/stopwatch.py 23 0 100%
modyn/common/ftp/ftp_server.py 31 18 42%
modyn/common/ftp/ftp_utils.py 43 31 28%
modyn/common/trigger_sample/trigger_sample_storage.py 72 4 94%
modyn/database/abstract_database_connection.py 35 0 100%
modyn/database/partition_by_meta.py 33 12 64%
modyn/evaluator/evaluator.py 25 1 96%
modyn/evaluator/evaluator_entrypoint.py 32 3 91%
modyn/evaluator/internal/dataset/evaluation_dataset.py 68 3 96%
modyn/evaluator/internal/grpc/evaluator_grpc_server.py 22 0 100%
modyn/evaluator/internal/grpc/evaluator_grpc_servicer.py 154 14 91%
modyn/evaluator/internal/metric_factory.py 18 1 94%
modyn/evaluator/internal/metrics/abstract_decomposable_metric.py 10 1 90%
modyn/evaluator/internal/metrics/abstract_evaluation_metric.py 29 2 93%
modyn/evaluator/internal/metrics/abstract_holistic_metric.py 10 1 90%
modyn/evaluator/internal/metrics/accuracy.py 20 2 90%
modyn/evaluator/internal/metrics/f1_score.py 65 0 100%
modyn/evaluator/internal/metrics/roc_auc.py 36 1 97%
modyn/evaluator/internal/pytorch_evaluator.py 107 21 80%
modyn/evaluator/internal/utils/evaluation_info.py 26 0 100%
modyn/evaluator/internal/utils/evaluation_process_info.py 8 0 100%
modyn/evaluator/internal/utils/evaluator_messages.py 3 0 100%
modyn/metadata_database/metadata_base.py 3 0 100%
modyn/metadata_database/metadata_database_connection.py 44 3 93%
modyn/metadata_database/models/pipelines.py 9 1 89%
modyn/metadata_database/models/sample_training_metadata.py 15 0 100%
modyn/metadata_database/models/selector_state_metadata.py 46 10 78%
modyn/metadata_database/models/trained_models.py 14 0 100%
modyn/metadata_database/models/trigger_partitions.py 10 0 100%
modyn/metadata_database/models/trigger_training_metadata.py 14 0 100%
modyn/metadata_database/models/triggers.py 10 0 100%
modyn/metadata_processor/internal/grpc/metadata_processor_grpc_servicer.py 18 0 100%
modyn/metadata_processor/internal/grpc/metadata_processor_server.py 24 0 100%
modyn/metadata_processor/internal/metadata_processor_manager.py 23 4 83%
modyn/metadata_processor/metadata_processor.py 11 0 100%
modyn/metadata_processor/metadata_processor_entrypoint.py 24 1 96%
modyn/metadata_processor/processor_strategies/abstract_processor_strategy.py 29 0 100%
modyn/metadata_processor/processor_strategies/basic_processor_strategy.py 17 2 88%
modyn/metadata_processor/processor_strategies/processor_strategy_type.py 6 1 83%
modyn/model_storage/internal/grpc/grpc_server.py 22 0 100%
modyn/model_storage/internal/grpc/model_storage_grpc_servicer.py 65 4 94%
modyn/model_storage/model_storage.py 24 5 79%
modyn/model_storage/model_storage_entrypoint.py 32 3 91%
modyn/models/articlenet/articlenet.py 30 16 47%
modyn/models/coreset_methods_support.py 29 1 97%
modyn/models/dlrm/cuda_ext/dot_based_interact.py 24 13 46%
modyn/models/dlrm/dlrm.py 67 9 87%
modyn/models/dlrm/nn/embeddings.py 123 64 48%
modyn/models/dlrm/nn/factories.py 24 9 62%
modyn/models/dlrm/nn/interactions.py 50 11 78%
modyn/models/dlrm/nn/mlps.py 77 23 70%
modyn/models/dlrm/nn/parts.py 60 4 93%
modyn/models/dlrm/setup.py 5 5 0%
modyn/models/dlrm/utils/install_lib.py 11 7 36%
modyn/models/dlrm/utils/utils.py 28 0 100%
modyn/models/fmownet/fmownet.py 25 0 100%
modyn/models/resnet18/resnet18.py 28 0 100%
modyn/models/tokenizers/distill_bert_tokenizer.py 11 0 100%
modyn/models/yearbooknet/yearbooknet.py 23 0 100%
modyn/selector/internal/grpc/selector_grpc_servicer.py 80 22 72%
modyn/selector/internal/grpc/selector_server.py 26 1 96%
modyn/selector/internal/selector_manager.py 104 33 68%
modyn/selector/internal/selector_strategies/abstract_selection_strategy.py 161 15 91%
modyn/selector/internal/selector_strategies/coreset_strategy.py 52 5 90%
modyn/selector/internal/selector_strategies/downsampling_strategies/abstract_downsampling_strategy.py 31 2 94%
modyn/selector/internal/selector_strategies/downsampling_strategies/downsampling_scheduler.py 55 2 96%
modyn/selector/internal/selector_strategies/downsampling_strategies/gradnorm_downsampling_strategy.py 5 0 100%
modyn/selector/internal/selector_strategies/downsampling_strategies/loss_downsampling_strategy.py 5 0 100%
modyn/selector/internal/selector_strategies/downsampling_strategies/no_downsampling_strategy.py 14 2 86%
modyn/selector/internal/selector_strategies/downsampling_strategies/utils.py 10 0 100%
modyn/selector/internal/selector_strategies/freshness_sampling_strategy.py 110 8 93%
modyn/selector/internal/selector_strategies/new_data_strategy.py 90 6 93%
modyn/selector/internal/selector_strategies/presampling_strategies/abstract_balanced_strategy.py 51 0 100%
modyn/selector/internal/selector_strategies/presampling_strategies/abstract_presampling_strategy.py 24 2 92%
modyn/selector/internal/selector_strategies/presampling_strategies/label_balanced_presampling_strategy.py 5 0 100%
modyn/selector/internal/selector_strategies/presampling_strategies/no_presampling_strategy.py 15 1 93%
modyn/selector/internal/selector_strategies/presampling_strategies/random_no_replacement_presampling_strategy.py 34 0 100%
modyn/selector/internal/selector_strategies/presampling_strategies/random_presampling_strategy.py 15 0 100%
modyn/selector/internal/selector_strategies/presampling_strategies/trigger_balanced_presampling_strategy.py 11 1 91%
modyn/selector/internal/selector_strategies/presampling_strategies/utils.py 13 0 100%
modyn/selector/selector.py 60 8 87%
modyn/selector/selector_entrypoint.py 24 1 96%
modyn/storage/internal/database/models/dataset.py 20 0 100%
modyn/storage/internal/database/models/file.py 17 0 100%
modyn/storage/internal/database/models/sample.py 44 7 84%
modyn/storage/internal/database/storage_base.py 3 0 100%
modyn/storage/internal/database/storage_database_connection.py 53 0 100%
modyn/storage/internal/database/storage_database_utils.py 21 0 100%
modyn/storage/internal/file_watcher/new_file_watcher.py 214 44 79%
modyn/storage/internal/file_watcher/new_file_watcher_watch_dog.py 59 9 85%
modyn/storage/internal/file_wrapper/abstract_file_wrapper.py 23 1 96%
modyn/storage/internal/file_wrapper/binary_file_wrapper.py 49 1 98%
modyn/storage/internal/file_wrapper/csv_file_wrapper.py 94 8 91%
modyn/storage/internal/file_wrapper/file_wrapper_type.py 8 0 100%
modyn/storage/internal/file_wrapper/single_sample_file_wrapper.py 48 2 96%
modyn/storage/internal/filesystem_wrapper/abstract_filesystem_wrapper.py 31 1 97%
modyn/storage/internal/filesystem_wrapper/filesystem_wrapper_type.py 6 0 100%
modyn/storage/internal/filesystem_wrapper/local_filesystem_wrapper.py 52 0 100%
modyn/storage/internal/grpc/grpc_server.py 20 0 100%
modyn/storage/internal/grpc/storage_grpc_servicer.py 146 13 91%
modyn/storage/storage.py 34 1 97%
modyn/storage/storage_entrypoint.py 24 1 96%
modyn/supervisor/entrypoint.py 41 5 88%
modyn/supervisor/internal/evaluation_result_writer/abstract_evaluation_result_writer.py 14 2 86%
modyn/supervisor/internal/evaluation_result_writer/json_result_writer.py 17 0 100%
modyn/supervisor/internal/evaluation_result_writer/tensorboard_result_writer.py 13 0 100%
modyn/supervisor/internal/grpc_handler.py 311 47 85%
modyn/supervisor/internal/triggers/amounttrigger.py 15 0 100%
modyn/supervisor/internal/triggers/timetrigger.py 27 1 96%
modyn/supervisor/internal/triggers/trigger.py 6 0 100%
modyn/supervisor/internal/utils/evaluation_status_tracker.py 23 0 100%
modyn/supervisor/internal/utils/training_status_tracker.py 121 12 90%
modyn/supervisor/supervisor.py 271 30 89%
modyn/tests/database/test_abstract_database_connection.py 19 0 100%
modyn/tests/evaluator/internal/dataset/test_evaluation_dataset.py 128 2 98%
modyn/tests/evaluator/internal/grpc/test_evaluator_grpc_server.py 20 0 100%
modyn/tests/evaluator/internal/grpc/test_evaluator_grpc_servicer.py 291 16 95%
modyn/tests/evaluator/internal/metrics/test_accuracy.py 45 0 100%
modyn/tests/evaluator/internal/metrics/test_f1_score.py 58 0 100%
modyn/tests/evaluator/internal/metrics/test_roc_auc.py 31 0 100%
modyn/tests/evaluator/internal/test_metric_factory.py 13 0 100%
modyn/tests/evaluator/internal/test_pytorch_evaluator.py 166 22 87%
modyn/tests/evaluator/test_evaluator.py 30 0 100%
modyn/tests/evaluator/test_evaluator_entrypoint.py 22 0 100%
modyn/tests/metadata_database/models/test_pipelines.py 33 0 100%
modyn/tests/metadata_database/models/test_sample_training_metadata.py 40 0 100%
modyn/tests/metadata_database/models/test_selector_state_metadata.py 46 0 100%
modyn/tests/metadata_database/models/test_trained_models.py 46 0 100%
modyn/tests/metadata_database/models/test_trigger_training_metadata.py 38 0 100%
modyn/tests/metadata_database/models/test_triggers.py 33 0 100%
modyn/tests/metadata_database/test_metadata_database_connection.py 29 0 100%
modyn/tests/metadata_processor/internal/grpc/test_metadata_processor_grpc_servicer.py 26 0 100%
modyn/tests/metadata_processor/internal/grpc/test_metadata_processor_server.py 27 0 100%
modyn/tests/metadata_processor/internal/test_metadata_processor_manager.py 42 3 93%
modyn/tests/metadata_processor/processor_strategies/test_abstract_processor_strategy.py 60 0 100%
modyn/tests/metadata_processor/processor_strategies/test_basic_processor_strategy.py 43 0 100%
modyn/tests/metadata_processor/test_metadata_processor.py 22 3 86%
modyn/tests/metadata_processor/test_metadata_processor_entrypoint.py 22 0 100%
modyn/tests/model_storage/internal/grpc/test_model_storage_grpc_server.py 13 0 100%
modyn/tests/model_storage/internal/grpc/test_model_storage_grpc_servicer.py 83 1 99%
modyn/tests/model_storage/test_model_storage.py 35 5 86%
modyn/tests/model_storage/test_model_storage_entrypoint.py 22 0 100%
modyn/tests/models/test_bert_tokenizer.py 24 0 100%
modyn/tests/models/test_dlrm.py 46 0 100%
modyn/tests/models/test_embedding_recorder.py 27 0 100%
modyn/tests/models/test_fmownet.py 25 0 100%
modyn/tests/models/test_resnet18.py 22 0 100%
modyn/tests/models/test_yearbook_net.py 47 0 100%
modyn/tests/selector/internal/grpc/test_selector_grpc_servicer.py 145 0 100%
modyn/tests/selector/internal/grpc/test_selector_server.py 42 0 100%
modyn/tests/selector/internal/selector_strategies/downsampling_strategies/test_abstract_downsampling_strategy.py 15 0 100%
modyn/tests/selector/internal/selector_strategies/downsampling_strategies/test_gradnorm_downsampling_strategy.py 13 0 100%
modyn/tests/selector/internal/selector_strategies/downsampling_strategies/test_loss_downsampling_strategy.py 17 0 100%
modyn/tests/selector/internal/selector_strategies/downsampling_strategies/test_no_downsampling_strategy.py 5 0 100%
modyn/tests/selector/internal/selector_strategies/downsampling_strategies/test_scheduler.py 114 0 100%
modyn/tests/selector/internal/selector_strategies/presampling_strategies/test_abstract_balanced_strategy.py 14 0 100%
modyn/tests/selector/internal/selector_strategies/presampling_strategies/test_empty_presampling_strategy.py 0 0 100%
modyn/tests/selector/internal/selector_strategies/presampling_strategies/test_label_balanced_presampling_strategy.py 164 0 100%
modyn/tests/selector/internal/selector_strategies/presampling_strategies/test_random_no_replacement_presampling_strategy.py 51 0 100%
modyn/tests/selector/internal/selector_strategies/presampling_strategies/test_random_presampling_strategy.py 94 0 100%
modyn/tests/selector/internal/selector_strategies/presampling_strategies/test_trigger_balanced_presampling.py 139 0 100%
modyn/tests/selector/internal/selector_strategies/test_abstract_selection_strategy.py 220 0 100%
modyn/tests/selector/internal/selector_strategies/test_coreset_strategy.py 232 0 100%
modyn/tests/selector/internal/selector_strategies/test_freshness_sampling_strategy.py 308 0 100%
modyn/tests/selector/internal/selector_strategies/test_new_data_strategy.py 519 0 100%
modyn/tests/selector/internal/test_selector_manager.py 155 3 98%
modyn/tests/selector/internal/trigger_sample/test_trigger_sample_storage.py 157 0 100%
modyn/tests/selector/test_selector.py 84 3 96%
modyn/tests/selector/test_selector_entrypoint.py 22 0 100%
modyn/tests/storage/internal/database/models/test_dataset.py 47 0 100%
modyn/tests/storage/internal/database/models/test_file.py 64 0 100%
modyn/tests/storage/internal/database/models/test_sample.py 73 0 100%
modyn/tests/storage/internal/database/test_database_storage_utils.py 21 2 90%
modyn/tests/storage/internal/database/test_storage_database_connection.py 54 3 94%
modyn/tests/storage/internal/file_watcher/test_new_file_watcher.py 377 13 97%
modyn/tests/storage/internal/file_watcher/test_new_file_watcher_watch_dog.py 95 1 99%
modyn/tests/storage/internal/file_wrapper/test_binary_file_wrapper.py 92 0 100%
modyn/tests/storage/internal/file_wrapper/test_csv_file_wrapper.py 165 1 99%
modyn/tests/storage/internal/file_wrapper/test_file_wrapper_type.py 6 1 83%
modyn/tests/storage/internal/file_wrapper/test_single_sample_file_wrapper.py 90 0 100%
modyn/tests/storage/internal/filesystem_wrapper/test_filesystem_wrapper_type.py 6 1 83%
modyn/tests/storage/internal/filesystem_wrapper/test_local_filesystem_wrapper.py 167 0 100%
modyn/tests/storage/internal/grpc/test_grpc_server.py 11 0 100%
modyn/tests/storage/internal/grpc/test_storage_grpc_servicer.py 273 3 99%
modyn/tests/storage/test_storage.py 42 1 98%
modyn/tests/storage/test_storage_entrypoint.py 21 0 100%
modyn/tests/supervisor/internal/evaluation_result_writer/test_abstract_evaluation_result_writer.py 7 0 100%
modyn/tests/supervisor/internal/evaluation_result_writer/test_json_result_writer.py 16 0 100%
modyn/tests/supervisor/internal/evaluation_result_writer/test_tensorboard_result_writer.py 21 0 100%
modyn/tests/supervisor/internal/test_grpc_handler.py 319 0 100%
modyn/tests/supervisor/internal/triggers/test_amounttrigger.py 30 0 100%
modyn/tests/supervisor/internal/triggers/test_timetrigger.py 26 0 100%
modyn/tests/supervisor/internal/triggers/test_trigger.py 5 0 100%
modyn/tests/supervisor/internal/utils/test_evaluation_status_tracker.py 41 0 100%
modyn/tests/supervisor/internal/utils/test_training_status_tracker.py 133 0 100%
modyn/tests/supervisor/test_entrypoint.py 39 0 100%
modyn/tests/supervisor/test_supervisor.py 412 2 99%
modyn/tests/trainer_server/internal/data/key_sources/test_local_key_source.py 91 0 100%
modyn/tests/trainer_server/internal/data/key_sources/test_selector_key_source.py 92 0 100%
modyn/tests/trainer_server/internal/data/test_data_utils.py 22 1 95%
modyn/tests/trainer_server/internal/data/test_local_dataset_writer.py 59 0 100%
modyn/tests/trainer_server/internal/data/test_online_dataset.py 294 3 99%
modyn/tests/trainer_server/internal/data/test_per_class_online_dataset.py 50 3 94%
modyn/tests/trainer_server/internal/grpc/test_trainer_server_grpc_server.py 17 0 100%
modyn/tests/trainer_server/internal/grpc/test_trainer_server_grpc_servicer.py 389 8 98%
modyn/tests/trainer_server/internal/metadata_collector/test_metadata_collector.py 41 0 100%
modyn/tests/trainer_server/internal/trainer/metadata_pytorch_callbacks/test_loss_callback.py 51 1 98%
modyn/tests/trainer_server/internal/trainer/remote_downsamplers/test_abstract_remote_downsampling_strategy.py 12 0 100%
modyn/tests/trainer_server/internal/trainer/remote_downsamplers/test_get_tensor_subset.py 56 0 100%
modyn/tests/trainer_server/internal/trainer/remote_downsamplers/test_remote_gradnorm_downsample.py 92 0 100%
modyn/tests/trainer_server/internal/trainer/remote_downsamplers/test_remote_loss_downsample.py 82 0 100%
modyn/tests/trainer_server/internal/trainer/test_pytorch_trainer.py 361 33 91%
modyn/tests/trainer_server/test_trainer_server.py 34 0 100%
modyn/tests/trainer_server/test_trainer_server_entrypoint.py 22 0 100%
modyn/tests/utils/test_utils.py 123 0 100%
modyn/trainer_server/custom_lr_schedulers/dlrm_lr_scheduler/dlrm_scheduler.py 33 33 0%
modyn/trainer_server/internal/dataset/data_utils.py 16 2 88%
modyn/trainer_server/internal/dataset/key_sources/abstract_key_source.py 21 5 76%
modyn/trainer_server/internal/dataset/key_sources/local_key_source.py 21 1 95%
modyn/trainer_server/internal/dataset/key_sources/selector_key_source.py 54 2 96%
modyn/trainer_server/internal/dataset/local_dataset_writer.py 68 4 94%
modyn/trainer_server/internal/dataset/online_dataset.py 131 5 96%
modyn/trainer_server/internal/dataset/per_class_online_dataset.py 14 0 100%
modyn/trainer_server/internal/grpc/trainer_server_grpc_server.py 22 0 100%
modyn/trainer_server/internal/grpc/trainer_server_grpc_servicer.py 224 31 86%
modyn/trainer_server/internal/metadata_collector/metadata_collector.py 33 0 100%
modyn/trainer_server/internal/mocks/mock_metadata_processor.py 22 2 91%
modyn/trainer_server/internal/trainer/metadata_pytorch_callbacks/base_callback.py 15 1 93%
modyn/trainer_server/internal/trainer/metadata_pytorch_callbacks/loss_callback.py 21 0 100%
modyn/trainer_server/internal/trainer/pytorch_trainer.py 337 110 67%
modyn/trainer_server/internal/trainer/remote_downsamplers/abstract_per_label_remote_downsample_strategy.py 9 3 67%
modyn/trainer_server/internal/trainer/remote_downsamplers/abstract_remote_downsampling_strategy.py 30 3 90%
modyn/trainer_server/internal/trainer/remote_downsamplers/remote_gradnorm_downsampling.py 37 3 92%
modyn/trainer_server/internal/trainer/remote_downsamplers/remote_loss_downsampling.py 29 3 90%
modyn/trainer_server/internal/utils/metric_type.py 3 0 100%
modyn/trainer_server/internal/utils/trainer_messages.py 4 0 100%
modyn/trainer_server/internal/utils/training_info.py 48 2 96%
modyn/trainer_server/internal/utils/training_process_info.py 10 0 100%
modyn/trainer_server/trainer_server.py 19 0 100%
modyn/trainer_server/trainer_server_entrypoint.py 32 3 91%
modyn/utils/utils.py 131 12 91%
TOTAL 14874 965 94%
Coverage HTML written to
================ 627 passed, 143

@MaxiBoether MaxiBoether merged commit 91b586e into main Sep 19, 2023
8 checks passed
@MaxiBoether MaxiBoether deleted the fix/MaxiBoether/apex-packaging branch September 19, 2023 12:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
1 participant
@MaxiBoether