Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: fix unit test #678

Merged
merged 2 commits into from
May 17, 2023
Merged

chore: fix unit test #678

merged 2 commits into from
May 17, 2023

Conversation

wesk
Copy link
Collaborator

@wesk wesk commented May 16, 2023

Problem

Unit test started failing after #672 was merged, which adds an assert about the config space for the MOREA method. The unit test was returning an invalid config space, which failed the assert.

Solution

Have the mock blackbox return a specific config space.

Alternative considered: we can actually remove the mock_blackbox entirely, but then the test would run for >1sec.


By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@wesk wesk marked this pull request as ready for review May 16, 2023 18:14
@wesk wesk requested a review from mseeger May 16, 2023 18:14
@codecov
Copy link

codecov bot commented May 16, 2023

Codecov Report

Patch coverage: 62.50% and project coverage change: +0.09 🎉

Comparison is base (5440b05) 65.83% compared to head (be4b101) 65.92%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #678      +/-   ##
==========================================
+ Coverage   65.83%   65.92%   +0.09%     
==========================================
  Files         397      398       +1     
  Lines       27767    27782      +15     
==========================================
+ Hits        18280    18315      +35     
+ Misses       9487     9467      -20     
Impacted Files Coverage Δ
syne_tune/config_space.py 84.54% <ø> (ø)
...izer/schedulers/searchers/regularized_evolution.py 71.83% <25.00%> (-0.23%) ⬇️
...chmarking/benchmark_commons/test_hpo_main_local.py 100.00% <100.00%> (ø)
tst/schedulers/test_rea.py 100.00% <100.00%> (ø)

... and 3 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@@ -50,27 +51,37 @@ def test_tuner_is_run_the_expected_number_of_times(
"nas201-ImageNet16-120": nas201_mo_benchmark("ImageNet16-120"),
}

config_space = {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let us at least use the correct space for these benchmarks:

    hp_cols = ["hp_x0", "hp_x1", "hp_x2", "hp_x3", "hp_x4", "hp_x5"]

    config_space = {
        node: choice(
            ["avg_pool_3x3", "nor_conv_3x3", "skip_connect", "nor_conv_1x1", "none"]
        )
        for node in hp_cols
    }
    config_space["epochs"] = 200

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated

@wesk wesk requested a review from mseeger May 17, 2023 09:48
@mseeger mseeger merged commit 81ba774 into main May 17, 2023
30 checks passed
@mseeger mseeger deleted the 2023-05-16-fix-test branch May 17, 2023 09:49
@wesk wesk added the chore label May 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants