TPESampler can't deal with Pruned Trials within Sample for Multiobjective study

[patrickbritz]

Expected behavior

It seems as though if there exists pruned trials within the independent sample... those trials don't have values obviously. This causes "zip(trial.values, study.directions)" to return the error "TypeError: 'NoneType' object is not iterable".

Environment

• Optuna version: 3.5.0
• Python version: 3.10
• OS: OS:Linux-4.14.256-197.484.amzn2.x86_64-x86_64-with-glibc2.26

Error messages, stack traces, or logs

[W 2024-02-19 13:30:42,152] Trial 51 failed with value None.
Traceback (most recent call last):
  File "/demos/optuna_bug.py", line 28, in <module>
    study.optimize(objective, n_trials=100)
  File "/demo/.venv/lib/python3.10/site-packages/optuna/study/study.py", line 451, in optimize
    _optimize(
  File "/demo/.venv/lib/python3.10/site-packages/optuna/study/_optimize.py", line 66, in _optimize
    _optimize_sequential(
  File "/demo/.venv/lib/python3.10/site-packages/optuna/study/_optimize.py", line 163, in _optimize_sequential
    frozen_trial = _run_trial(study, func, catch)
  File "/demo/.venv/lib/python3.10/site-packages/optuna/study/_optimize.py", line 251, in _run_trial
    raise func_err
  File "/demo/.venv/lib/python3.10/site-packages/optuna/study/_optimize.py", line 200, in _run_trial
    value_or_values = func(trial)
  File "/home/patrick/source/Research/strat_dev/demos/optuna_bug.py", line 6, in objective
    x = trial.suggest_int("x", -10, 1000)
  File "/demo/.venv/lib/python3.10/site-packages/optuna/_convert_positional_args.py", line 83, in converter_wrapper
    return func(**kwargs)
  File "/demo/.venv/lib/python3.10/site-packages/optuna/trial/_trial.py", line 326, in suggest_int
    suggested_value = int(self._suggest(name, distribution))
  File "/demo/.venv/lib/python3.10/site-packages/optuna/trial/_trial.py", line 635, in _suggest
    param_value = self.study.sampler.sample_independent(
  File "/demo/.venv/lib/python3.10/site-packages/optuna/samplers/_tpe/sampler.py", line 447, in sample_independent
    return self._sample(study, trial, {param_name: param_distribution})[param_name]
  File "/demo/.venv/lib/python3.10/site-packages/optuna/samplers/_tpe/sampler.py", line 490, in _sample
    weights_below = _calculate_weights_below_for_multi_objective(
  File "/demo/.venv/lib/python3.10/site-packages/optuna/samplers/_tpe/sampler.py", line 778, in _calculate_weights_below_for_multi_objective
    for value, direction in zip(trial.values, study.directions):
TypeError: 'NoneType' object is not iterable

Steps to reproduce

1. Setup a muli-objective study
2. Specify a TPESampler for the study
3. Prune trials within startup trials

import optuna
from optuna.samplers import TPESampler
import numpy as np

def objective(trial):
    x = trial.suggest_int("x", -10, 1000)
    y = trial.suggest_int("y", -10, 1000)
    # Simulated objective function
    # Here, we'll try to maximize two objectives, and we'll add some artificial pruning
    value1 = np.sin(x) * np.cos(y)
    value2 = np.cos(x) * np.sin(y)

    
    # Prune some trials to simulate a scenario with pruned trials
    if trial.number > 4:
        raise optuna.exceptions.TrialPruned()
    
    return value1, value2, 

# Create an in-memory storage
storage = optuna.storages.InMemoryStorage()

# Create a study with the multi-objective sampler
study = optuna.create_study(storage=storage, directions=['maximize', 'maximize'], sampler=TPESampler())

# Optimize the study
study.optimize(objective, n_trials=100)

Additional context (optional)

No response

[nzw0301]

Thank you for reporting this. I don't think this usage is expected because pruners do not support multi-objective study due to trial.report's restriction. So I'm not sure this is a bug, but at least this should be documented somewhere.

[bionicles]

Hi Kento / @nzw0301 I love optuna and am thrilled to pester you about this issue to ask how hard you think #5268 would be to implement,

tl;dr: (to enable multi-objective pruners and save tons of time and energy by pruning unpromising (or unsafe!) trajectories!)

[nzw0301]

Thank you for your comment and kind words! I think this will be hard work as attempted before as replied Inthe original issue unfortunately...