Integrate multi-objective optimization module

[HideakiImamura]

Motivation

In the current Optuna's codebase, the optuna.multi_objective module is separated from other modules. However, to improve maintenance and ensure code consistency, Optuna's basic modules (e.g. study, trial, etc) should be designed with multi-objective optimization in mind.

PR #1961 was submitted, but this work will be split into several small PRs. This issue tracks the integration of the multi-objective optimization module.

TODOs

• Multi-dimensional study directions and trial values: fix RDB schema Change the RDB schema for multi-objective integration #2030.
• Multi-dimensional study directions and trial values: fix study, frozen trial, and storages Integrate multi-objective optimization module for the storages, study, and frozen trial #1994.
• Pruning system: fix trial.report and trial.should_prune Raise NotImplementedError for trial.report and trial.should_prune during multi-objective optimization #2135.
• Multi-objective samplers: migrate NSGAIISampler Migrate NSGA2 sampler #2150.
• Multi-objective samplers: migrate MOTPESampler Migrate MOTPE sampler #2167.
• Visualization module: migrate plot_pareto_front Migrate optuna.multi_objective.visualization.plot_pareto_front #2110, Move examples/multi_objective/plot_pareto_front.py to examples/visualization/plot_pareto_front.py #2122.
• Visualization module: fix single-objective plot functions Raise ValueError if target is None and study is for multi-objective optimization for plot_contour #2112, Raise ValueError if target is None and study is for multi-objective optimization for plot_edf #2117, Raise ValueError if target is None and study is for multi-objective optimization for plot_optimization_history #2118, Raise ValueError if target is None and study is for multi-objective optimization for plot_parallel_coordinate #2120, Raise ValueError if target is None and study is for multi-objective optimization for plot_slice #2121, Raise ValueError if target is None and study is for multi-objective optimization for get_param_importances, BaseImportanceEvaluator.evaluate, and plot_param_importances #2137.
• Misc: fix multi-objective examples Use latest multi-objective functionality in multi-objective example #2123.
• Misc: fix other modules which assume the single-objective optimization Raise ValueError in TPE and CMA-ES if study is being used for multi-objective optimization #2136.
• Misc: deprecate the previous multi-objective module Deprecate previous multi-objective module #2124.

[github-actions]

This issue has not seen any recent activity.

[hvy]

Pinned this issue since I find myself coming back to it quite often and there was one slot left for the pins.

[hvy]

@not522 is working on Study.trials_dataframe to support MO studies. I think we can close this issue as resolved after that.

[hvy]

Let me close this issue as resolved now that #2181 addressing the above has been merged.