Add performance test for LexiFlow #812
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Anonymous-submission-repo
changed the title
skzhang1
requested review from
qingyun-wu and
sonichi
and removed request for
sonichi and
qingyun-wu
sonichi
reviewed
Nov 15, 2022
flaml/tune/searcher/flow2.py
Outdated
Comment on lines
367
to
368
| feasible_index_filter = np.where( | ||
| k_values.take(feasible_index) |
There was a problem hiding this comment.
k_values.take(feasible_index) is calculated twice. Use a var to calculate only once.
flaml/tune/searcher/flow2.py
Outdated
| feasible_index = [ | ||
| val for val in feasible_index if val in feasible_index_prior | ||
| ] | ||
| feasible_index = np.array(feasible_index).take(feasible_index_filter) |
There was a problem hiding this comment.
move np.array() to line 263.
flaml/tune/searcher/flow2.py
Outdated
| @@ -374,9 +374,7 @@ def update_fbest( | |||
| ] | |||
| ) | |||
| )[0].tolist() | |||
There was a problem hiding this comment.
remove tolist().
use
feasible_index_filter, _ =
flaml/tune/tune.py
Outdated
| feasible_index = [ | ||
| val for val in feasible_index if val in feasible_index_prior | ||
| ] | ||
| feasible_index = np.array(feasible_index).take(feasible_index_filter) |
Comment on lines
54
to
56
| assert ( | ||
| analysis.best_result["currin"] <= 2.2 | ||
| ), "lexicographic optimization not reproducible" |
There was a problem hiding this comment.
Why is the test named "reproducibility"? The assertion here is about the best currin score, not reproducibility.
Anonymous-submission-repo
requested review from
sonichi
and removed request for
qingyun-wu
qingyun-wu
reviewed
Nov 15, 2022
test/tune/test_lexiflow.py
Outdated
| @@ -138,5 +158,35 @@ def evaluate_function(configuration): | |||
| print(analysis.best_result) | |||
|
|
|||
|
|
|||
| def test_lexiflow_BraninCurrin(): | |||
There was a problem hiding this comment.
Suggested change
| def test_lexiflow_BraninCurrin(): | |
| def test_lexiflow_performance(): |
qingyun-wu
changed the title
qingyun-wu
approved these changes
Nov 15, 2022
sonichi
approved these changes
Nov 15, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why are these changes needed?
Related issue number
#767
Checks