Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

examples/launch_asha_yahpo.py not working #714

Closed
jgolebiowski opened this issue Jun 14, 2023 · 2 comments · Fixed by #725
Closed

examples/launch_asha_yahpo.py not working #714

jgolebiowski opened this issue Jun 14, 2023 · 2 comments · Fixed by #725

Comments

@jgolebiowski
Copy link
Collaborator

Describe the bug
examples/launch_asha_yahpo.py fails with the following bug: https://github.com/awslabs/syne-tune/actions/runs/5260727444/jobs/9508020339?pr=711#logs

To reproduce
Steps to reproduce the behavior:

  1. Go to syne tune root
  2. Run the following command python examples/launch_asha_yahpo.py
  3. See error
Traceback (most recent call last):
  File "examples/launch_asha_yahpo.py", line 115, in <module>
    plot_yahpo_learning_curves(
  File "examples/launch_asha_yahpo.py", line 36, in plot_yahpo_learning_curves
    bb = trial_backend.blackbox
  File "/home/runner/work/syne-tune/syne-tune/syne_tune/blackbox_repository/simulated_tabular_backend.py", line 335, in blackbox
    self._blackbox = load_blackbox(
  File "/home/runner/work/syne-tune/syne-tune/syne_tune/blackbox_repository/repository.py", line 169, in load_blackbox
    return instantiate_yahpo(name, **yahpo_kwargs)
  File "/home/runner/work/syne-tune/syne-tune/syne_tune/blackbox_repository/conversion_scripts/scripts/yahpo_import.py", line 373, in instantiate_yahpo
    bench = benchmark_set.BenchmarkSet(scenario, active_session=False)
  File "/opt/hostedtoolcache/Python/3.8.16/x64/lib/python3.8/site-packages/yahpo_gym/benchmark_set.py", line 50, in __init__
    self.config_space = self._get_config_space()
  File "/opt/hostedtoolcache/Python/3.8.16/x64/lib/python3.8/site-packages/yahpo_gym/benchmark_set.py", line 371, in _get_config_space
    cs = CS_json.read(json_string)
  File "/opt/hostedtoolcache/Python/3.8.16/x64/lib/python3.8/site-packages/ConfigSpace/read_and_write/json.py", line 444, in read
    _construct_hyperparameter(
  File "/opt/hostedtoolcache/Python/3.8.16/x64/lib/python3.8/site-packages/ConfigSpace/read_and_write/json.py", line 5[24](https://github.com/awslabs/syne-tune/actions/runs/5260727444/jobs/9508020339?pr=711#step:9:25), in _construct_hyperparameter
    q=hyperparameter["q"],
KeyError: 'q'
Error: Process completed with exit code 1.

Expected behavior
Complete run

Paste the output of the pip freeze command below:
Done using SyneTune githb actions.

@jgolebiowski
Copy link
Collaborator Author

jgolebiowski commented Jun 15, 2023

Broken example removed from github actions in #718. Once this issue is fixed, we should revert the change in #718 and re-introduce this example to github actions

@sumny
Copy link

sumny commented Jun 22, 2023

Hi,

it appears that the new ConfigSpace 0.7.1 (released two weeks ago) conflicts with the serialized configspaces in YAHPO Gym (slds-lmu/yahpo_gym#79).
I would be surprised if any YAHPO benchmarks currently run within syne-tune if the newest ConfigSpace version is installed.

For now, should be fixable by requiring configspace<=0.6.1 in
syne_tune/blackbox_repository/conversion_scripts/scripts/requirements-yahpo.txt

Hope this helps, otherwise let me know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants