MNT Solving issue with slow conda and splitting the doc and test .yml to speed the CI.

[bruAristimunha]

Closes #474

[codecov]

Codecov Report

Merging #479 (1c6725a) into master (3d8dca6) will not change coverage.
The diff coverage is n/a.

❗ Current head 1c6725a differs from pull request most recent head 83618c1. Consider uploading reports for the commit 83618c1 to get more accurate results

@@           Coverage Diff           @@
##           master     #479   +/-   ##
=======================================
  Coverage   83.52%   83.52%           
=======================================
  Files          60       60           
  Lines        4388     4388           
=======================================
  Hits         3665     3665           
  Misses        723      723           

[bruAristimunha]

I separated the testing and documentation generation in the GitHub workflow; with this, these jobs are executed in parallel. This has already improved documentation generation time from approximately 1 hour to half an hour.

Also, I removed the env creation with conda to use pure Python env. This increased the speed; only conda took between 3 and 8 minutes to resolve the env, plus 2 minutes for the dependencies. Now it takes us 1 minute.

The tests on Linux and macOS take 6 to 7 minutes, and on Windows, 9 to 11 minutes. The test on Windows has a limitation on the RAM used; I had to limit some tests on Windows because of this.

Now, about generating the documentation, the documentation was generated sequentially, which means the process took approximately 36 minutes (3 minutes with configs and 33 minutes with doc). Now it is taking 16 min, with 3 minutes of configuration and 13 minutes of Doc generation.

The time for each of our examples is described below:

computation time summary after:

• ../examples/applied_examples/plot_bcic_iv_4_ecog_trial.py: 473.50 sec 1986.4 MB
• ../examples/model_building/plot_how_train_test_and_tune.py: 301.11 sec 718.1 MB
• ../examples/advanced_training/plot_data_augmentation_search.py: 280.05 sec 718.1 MB
• ../examples/datasets_io/plot_benchmark_preprocessing.py: 163.00 sec 466.0 MB
• ../examples/model_building/plot_hyperparameter_tuning_with_scikit-learn.py: 161.44 sec 166.2 MB
• ../examples/model_building/plot_bcic_iv_2a_moabb_cropped.py: 128.40 sec 1243.4 MB
• ../examples/advanced_training/plot_relative_positioning.py: 62.57 sec 152.0 MB
• ../examples/advanced_training/plot_data_augmentation.py: 54.59 sec 749.8 MB
• ../examples/model_building/plot_bcic_iv_2a_moabb_trial.py: 53.66 sec 711.0 MB
• ../examples/applied_examples/plot_sleep_staging_chambon2018.py: 44.33 sec 9.0 MB
• ../examples/applied_examples/plot_bcic_iv_4_ecog_cropped.py: 33.78 sec 1162.6 MB
• ../examples/applied_examples/plot_sleep_staging_eldele2021.py: 20.49 sec 10.8 MB
• ../examples/applied_examples/plot_sleep_staging_usleep.py: 18.69 sec 13.5 MB
• ../examples/datasets_io/plot_dataset_example.py: 10.86 sec 293.3 MB
• ../examples/applied_examples/plot_tuh_eeg_corpus.py: 7.89 sec 8.7 MB
• ../examples/datasets_io/plot_load_save_datasets.py: 7.03 sec 142.8 MB
• ../examples/datasets_io/plot_split_dataset.py: 6.72 sec 118.0 MB
• ../examples/datasets_io/plot_tuh_discrete_multitarget.py: 2.62 sec 18.0 MB
• ../examples/datasets_io/plot_custom_dataset_example.py: 1.94 sec 94.1 MB
• ../examples/model_building/plot_regression.py: 1.59 sec 8.9 MB
• ../examples/datasets_io/plot_mne_dataset_example.py: 1.37 sec 8.6 MB
• ../examples/datasets_io/benchmark_lazy_eager_loading.py: 0.00 sec 0.0 MB

computation time summary before:

• ../examples/model_building/plot_how_train_test_and_tune.py: 123.54 sec 795.9 MB
• ../examples/advanced_training/plot_data_augmentation_search.py: 115.69 sec 979.1 MB
• ../examples/applied_examples/plot_bcic_iv_4_ecog_trial.py: 106.05 sec 2070.9 MB
• ../examples/datasets_io/plot_benchmark_preprocessing.py: 62.04 sec 406.4 MB
• ../examples/model_building/plot_hyperparameter_tuning_with_scikit-learn.py: 61.27 sec 180.2 MB
• ../examples/model_building/plot_bcic_iv_2a_moabb_cropped.py: 58.21 sec 1318.2 MB
• ../examples/advanced_training/plot_relative_positioning.py: 43.04 sec 293.2 MB
• ../examples/model_building/plot_bcic_iv_2a_moabb_trial.py: 41.53 sec 789.2 MB
• ../examples/advanced_training/plot_data_augmentation.py: 40.19 sec 811.2 MB
• ../examples/applied_examples/plot_sleep_staging_chambon2018.py: 28.14 sec 148.6 MB
• ../examples/applied_examples/plot_bcic_iv_4_ecog_cropped.py: 26.71 sec 905.7 MB
• ../examples/applied_examples/plot_sleep_staging_usleep.py: 12.81 sec 122.8 MB
• ../examples/applied_examples/plot_sleep_staging_eldele2021.py: 11.66 sec 25.6 MB
• ../examples/datasets_io/plot_dataset_example.py: 8.15 sec 293.3 MB
• ../examples/datasets_io/plot_load_save_datasets.py: 5.25 sec 142.8 MB
• ../examples/datasets_io/plot_split_dataset.py: 5.08 sec 123.4 MB
• ../examples/applied_examples/plot_tuh_eeg_corpus.py: 2.88 sec 8.7 MB
• ../examples/datasets_io/plot_tuh_discrete_multitarget.py: 1.98 sec 18.0 MB
• ../examples/datasets_io/plot_custom_dataset_example.py: 1.51 sec 93.2 MB
• ../examples/model_building/plot_regression.py: 1.48 sec 46.2 MB
• ../examples/datasets_io/plot_mne_dataset_example.py: 1.20 sec 8.5 MB
• ../examples/datasets_io/benchmark_lazy_eager_loading.py: 0.00 sec 0.0 MB

We can still improve this by changing the preprocessing function and defining a cache space in common for the examples. However, to do that, we need to change the CI, and I still need to figure out how to do that.

As I consider this PR a high priority for the code sprint tomorrow, I am applying a self-approval and integrating it into braindecode.