RuntimeError: Study.stop is supposed to be invoked inside an objective function or a callback

[caydenwei]

Expected behavior

optuna: 3.0.3 python: 3.8

when I try the example below, runtimeError is occured. But if I change the line while i < 9 to while i < 8 or a number less than 9 it works fine. It does not make sense, because there does have 9 permutations.

Environment

• Optuna version: 3.0.3
• Python version: 3.8
• OS: MacOS

Error messages, stack traces, or logs

Traceback (most recent call last):
  File "xxx", line 13, in <module>
    study.tell(trail, x + y)
  File "xxx/venv/lib/python3.8/site-packages/optuna/study/study.py", line 605, in tell
    return _tell_with_warning(
  File "xxx/venv/lib/python3.8/site-packages/optuna/study/_tell.py", line 195, in _tell_with_warning
    study.sampler.after_trial(study, frozen_trial, state, values)
  File "xxx/venv/lib/python3.8/site-packages/optuna/samplers/_grid.py", line 211, in after_trial
    study.stop()
  File "xxx/venv/lib/python3.8/site-packages/optuna/study/study.py", line 761, in stop
    raise RuntimeError(
RuntimeError: `Study.stop` is supposed to be invoked inside an objective function or a callback.

Steps to reproduce

# python code

import optuna

if __name__ == "__main__":
    search_space = {"x": [-50, 0, 50], "y": [-99, 0, 99]}
    study = optuna.create_study(sampler=optuna.samplers.GridSampler(search_space))

    i = 0
    while i < 9:
        trail = study.ask()
        x = trail.suggest_int('x', -1000, 1000)
        y = trail.suggest_int('y', -1000, 1000)
        study.tell(trail, x + y)
        
        i += 1

Additional context (optional)

No response

[toshihikoyanase]

Thank you for your report.

GridSampler tries to stop the study at the end of trial if it has no more grid to visit.

optuna/optuna/samplers/_grid.py

Lines 206 to 211 in 4bc3dfc

	if len(target_grids) == 0:
	study.stop()
	elif len(target_grids) == 1:
	grid_id = study._storage.get_trial_system_attrs(trial._trial_id)["grid_id"]
	if grid_id == target_grids[0]:
	study.stop()

However, study.stop() must be called in study.optimize and the error occurred.

I don't think the current GridSampler is designed to be used with Ask-and-Tell interface, but, as a workaround, we can simply ignore runtime error at study.tell, for example, as follows:

import optuna

if __name__ == "__main__":
    search_space = {"x": [-50, 0, 50], "y": [-99, 0, 99]}
    study = optuna.create_study(sampler=optuna.samplers.GridSampler(search_space))

    i = 0
    while i < 9:
        try:
            trail = study.ask()
            x = trail.suggest_int('x', -1000, 1000)
            y = trail.suggest_int('y', -1000, 1000)
            study.tell(trail, x + y)
        except RuntimeError:
            break

        i += 1
    print(len(study.trials))

If you can use study.optimize, we can write the code much simpler:

import optuna

def objective(trial):
    x = trial.suggest_int('x', -1000, 1000)
    y = trial.suggest_int('y', -1000, 1000)
    return x + y


if __name__ == "__main__":
    search_space = {"x": [-50, 0, 50], "y": [-99, 0, 99]}
    study = optuna.create_study(sampler=optuna.samplers.GridSampler(search_space))
    study.optimize(objective)

[caydenwei]

Okay, thanks. The reason I want to do this is that I don't want the 'optimize' stucking my program, and I need an asynchronous behavior, I would try another way.