Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix PBT issues with working dir and promotion of max fidelity trials #903

Merged
merged 99 commits into from
Aug 2, 2022
Merged
Changes from 3 commits
Commits
Show all changes
99 commits
Select commit Hold shift + click to select a range
b061b91
Use randint instead of choice in PBT.explore
bouthilx May 3, 2022
7729cef
Do not promote trials at last fidelity in PBT
bouthilx May 3, 2022
56ef0d4
Copy experiment when branching trial
bouthilx May 3, 2022
c999116
Fugly fix for trial working dir
bouthilx May 3, 2022
2c3ea09
isort
bouthilx May 3, 2022
de0fe44
Blackify
bouthilx May 3, 2022
8987e1d
Add prepare_trial arg to Runner and exp.workon
bouthilx Jun 22, 2022
f47850c
Adapt PBT to new way of trial working_dir copy
bouthilx Jun 22, 2022
1670090
Handle properly parent id in AlgoWrapper
bouthilx Jun 22, 2022
98b08a1
Compute remaining trials correctly in Runner
bouthilx Jun 22, 2022
04eb5e8
pylint
bouthilx Jun 22, 2022
0d06f22
Adjust functional tests for pbt and pb2
bouthilx Jun 22, 2022
c357931
Fix ExperimentClient.workon typing
bouthilx Jun 23, 2022
5c150f3
Add typing to Runner.__init__
bouthilx Jun 23, 2022
283b87b
Remove unnecessary `is not None`
bouthilx Jun 23, 2022
5113bb4
Add type hints to get_original_parent
bouthilx Jun 23, 2022
d6d3683
Add type hints to branching_rosenbrock
bouthilx Jun 23, 2022
5167330
Add type hints to test_branching_algos
bouthilx Jun 23, 2022
ca6a7cd
Remove unnecessary OrionState
bouthilx Jun 23, 2022
c34d3e8
Merge branch 'hotfix/pbt_cp_dir' of github.com:bouthilx/orion into ho…
bouthilx Jun 23, 2022
c05eb25
Add type hints to test_fidelity_upgrades
bouthilx Jun 23, 2022
b06ac03
Reenable PBT tests
bouthilx Jun 23, 2022
75dfb72
Merge branch 'hotfix/pbt_cp_dir' of github.com:bouthilx/orion into ho…
bouthilx Jun 23, 2022
8c29b3d
Remove useless shutil import
bouthilx Jun 23, 2022
a08e5ce
Add type hints to FakeClient.get_trial
bouthilx Jun 23, 2022
cd2ff72
Remove unnecessary tmp_path cleaning
bouthilx Jun 23, 2022
8b995b4
Merge branch 'hotfix/pbt_cp_dir' of github.com:bouthilx/orion into ho…
bouthilx Jun 23, 2022
0528bcf
Add type hints to new_runner
bouthilx Jun 23, 2022
f59808d
Add type hints to test_callback
bouthilx Jun 23, 2022
d913e82
Add type hints to test_prepare_trial_working_dir
bouthilx Jun 23, 2022
0d612bd
Rewrite test name
bouthilx Jun 23, 2022
82800ee
isort
bouthilx Jun 23, 2022
dbb9177
black
bouthilx Jun 23, 2022
15388bd
Merge branch 'hotfix/pbt_cp_dir' of github.com:bouthilx/orion into ho…
bouthilx Jun 23, 2022
f85d27e
Renable all tests for PBT
bouthilx Jun 23, 2022
90486b6
Remove Literal because only available in py3.8
bouthilx Jun 23, 2022
c63e3d3
isort
bouthilx Jun 23, 2022
dfb2b7b
Remove unrelevant tests
bouthilx Jun 27, 2022
2b45164
Add back Literal from another package
bouthilx Jun 27, 2022
5b7d217
isort
bouthilx Jun 27, 2022
42b549e
Reduce tests while debugging ConfigSpace version issue
bouthilx Jun 28, 2022
5b1c206
Merge branch 'develop' into hotfix/pbt_cp_dir
bouthilx Jul 4, 2022
a08e9e3
Add type-hints to build_params_hist
bouthilx Jul 4, 2022
697dd2a
Restore CI workflow
bouthilx Jul 4, 2022
780e25f
Merge branch 'hotfix/pbt_cp_dir' of github.com:bouthilx/orion into ho…
bouthilx Jul 4, 2022
cb4b35c
Restore all tests
bouthilx Jul 4, 2022
e76da18
Increase population size of PB2 to avoid early convergence
bouthilx Jul 4, 2022
f827cbb
Add ImportOptional and adjust algo imports
bouthilx Jul 5, 2022
5a18590
Fix wrong import_guard names
bouthilx Jul 5, 2022
fe14ac9
Remove useless else
bouthilx Jul 5, 2022
2dd5eeb
Make PB2 pass all the time
bouthilx Jul 5, 2022
b841b2a
Make test based on numpy indepedent of numpy version
bouthilx Jul 5, 2022
1ff694b
Skip PB2 in func tests if not installed
bouthilx Jul 5, 2022
aeacae0
Re-enable TPE fidelity functional test
bouthilx Jul 5, 2022
4432ac7
isort
bouthilx Jul 5, 2022
adae51e
Update doc about PBT
bouthilx Jul 5, 2022
1996834
Handle converging algos in func tests
bouthilx Jul 6, 2022
ece68bb
Fix Hyperband Typing and EVES state_dict
bouthilx Jul 13, 2022
26794fe
Fix PB2 repro
bouthilx Jul 13, 2022
aa29b86
Fix bug in test-suite
bouthilx Jul 13, 2022
198d65e
Use random objectives instead of incremental
bouthilx Jul 13, 2022
e54a1d9
Merge branch 'develop' into hotfix/pbt_cp_dir
bouthilx Jul 13, 2022
70de8be
Fix slips
bouthilx Jul 13, 2022
1edc6fc
Fix isort
bouthilx Jul 13, 2022
ea902b6
Please flake8
bouthilx Jul 14, 2022
70b6eab
Please flake8 once and for all :pray:
bouthilx Jul 14, 2022
f5959c5
Use conftest file for PBT fixtures
bouthilx Jul 14, 2022
c01b74a
Fix Ax tests now that objectives are random
bouthilx Jul 14, 2022
0681ab0
Fix EVES for cat data on random objectives
bouthilx Jul 14, 2022
1736ab0
Fix EVES sampling for close to convergence
bouthilx Jul 14, 2022
c87435f
Fix PB2 with random objectives
bouthilx Jul 14, 2022
224cd3f
Xfail properly EVES and PB2 on cat test
bouthilx Jul 15, 2022
55b2776
Adjust nevergrad tests based on random objectives
bouthilx Jul 15, 2022
99d59e1
Fix nevergrad config fixture
bouthilx Jul 15, 2022
f1fcd06
Fix handling of Fidelity in HEBO
bouthilx Jul 15, 2022
8dca072
Stop crashing PB2 when converging
bouthilx Jul 15, 2022
49ce21f
Relax RNG seed test
bouthilx Jul 15, 2022
1d970e8
Adjust tests of PBT because generate_offsprings does not raise anymore
bouthilx Jul 25, 2022
dab7dc1
Adjust HEBO fixture to new test-suite interface
bouthilx Jul 25, 2022
a593d7c
Adjust suggest_n for HEBO
bouthilx Jul 25, 2022
5b9b340
Swap parent classes of Hyperband
bouthilx Jul 26, 2022
9d5cf4e
Add type hint
bouthilx Jul 26, 2022
fbed0a3
Add type hints to prepare_trial_working_dir
bouthilx Jul 26, 2022
23bea94
Add type hints
bouthilx Jul 26, 2022
dbd4ac2
Remove deprecated note in docstring
bouthilx Jul 26, 2022
a4790a5
Add type hints
bouthilx Jul 26, 2022
0e702f0
Fix style
bouthilx Jul 26, 2022
a885b0d
Merge branch 'develop' into hotfix/pbt_cp_dir
bouthilx Jul 29, 2022
cf886dc
Remove test_trial_working_dir_is_created
bouthilx Jul 29, 2022
d3e4935
Fix control_randomness
bouthilx Jul 29, 2022
e7d5b92
Appease pylint for Protocol classes...
bouthilx Jul 29, 2022
a39c818
Rename Algo protocol to make it more generic
bouthilx Jul 29, 2022
4c94dd2
Using Protocol from typing_ext for py37
bouthilx Jul 29, 2022
7ef65b3
PB2 does not raise when converging, just like PBT
bouthilx Jul 30, 2022
0a67e66
Update src/orion/core/utils/random_state.py
bouthilx Aug 1, 2022
7230143
Remove useless asserts
bouthilx Aug 1, 2022
e3e27d6
Merge branch 'hotfix/pbt_cp_dir' of github.com:bouthilx/orion into ho…
bouthilx Aug 1, 2022
23bd733
Force pymoo dep to 0.5.0 for HEBO
bouthilx Aug 1, 2022
ec910bd
Make test robust to nb of trials
bouthilx Aug 1, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 8 additions & 7 deletions src/orion/core/utils/random_state.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,10 @@
import random
from contextlib import contextmanager
from dataclasses import dataclass, replace
from typing import Any, Protocol
from typing import Any

import numpy as np
from typing_extensions import Protocol

try:
import torch
Expand Down Expand Up @@ -80,26 +81,26 @@ def seed(cls, base_seed: int | None) -> RandomState:
return random_state


class Algo(Protocol):
"""Protocol for algorithms with a random state that can be saved and restored."""
class HasRandomState(Protocol): # pylint: disable=too-few-public-methods
"""Protocol for objects with a random state that can be saved and restored."""

random_state: RandomState | None


@contextmanager
def control_randomness(algo: Algo):
def control_randomness(has_random_state: HasRandomState):
"""Seeds the randomness inside the indented block of code using `self.random_state`."""
if algo.random_state is None:
if has_random_state.random_state is None:
lebrice marked this conversation as resolved.
Show resolved Hide resolved
yield
return

# Save the initial random state.
initial_rng_state = RandomState.current()
# Set the random state.
algo.random_state.set()
has_random_state.random_state.set()
yield
# Update the random state stored on `self`, so that the changes inside the block are
bouthilx marked this conversation as resolved.
Show resolved Hide resolved
# reflected in the RandomState object.
algo.random_state = RandomState.current()
has_random_state.random_state = RandomState.current()
# Reset the initial state.
initial_rng_state.set()