Add timeline plot with matplotlib as backend

[eukaryo]

Motivation

cf: #4470

In addition to plotly, I want to implement timeline plot using matplotlib as backend.

Description of the changes

• Add timeline plot using matplotlib in optuna/visualization/matplotlib/_timeline.py.
• Add unit tests in tests/visualization_tests/matplotlib_tests/test_timeline.py.

[codecov-commenter]

Codecov Report

Merging #4538 (5d16671) into master (2d0093a) will increase coverage by 0.08%.
The diff coverage is 100.00%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@            Coverage Diff             @@
##           master    #4538      +/-   ##
==========================================
+ Coverage   90.81%   90.89%   +0.08%     
==========================================
  Files         187      184       -3     
  Lines       13972    13954      -18     
==========================================
- Hits        12688    12683       -5     
+ Misses       1284     1271      -13     

Impacted Files	Coverage Δ
optuna/visualization/matplotlib/__init__.py	100.00% <100.00%> (ø)
optuna/visualization/matplotlib/_timeline.py	100.00% <100.00%> (ø)

... and 40 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[c-bata]

@amylase @HideakiImamura Could you review this PR if you have time?

[HideakiImamura]

Thanks for the PR. I have several comments. PTAL.

By the way, can we check the plot appearance using the visual regression test? The CI job does not generate the appropriate link.

[eukaryo]

Here is an example of plot with the current modified code.

[toshihikoyanase]

Could you update the list of plot functions in the reference, please?

optuna/docs/source/reference/visualization/matplotlib.rst

Lines 13 to 21 in 2d0093a

	optuna.visualization.matplotlib.plot_contour
	optuna.visualization.matplotlib.plot_edf
	optuna.visualization.matplotlib.plot_intermediate_values
	optuna.visualization.matplotlib.plot_optimization_history
	optuna.visualization.matplotlib.plot_parallel_coordinate
	optuna.visualization.matplotlib.plot_param_importances
	optuna.visualization.matplotlib.plot_pareto_front
	optuna.visualization.matplotlib.plot_slice
	optuna.visualization.matplotlib.is_available

[eukaryo]

I have updated optuna/docs/source/reference/visualization/matplotlib.rst and fixed docstring in plot_timeline function,

[github-actions]

🚀 Deployed on https://642648d03da4640e33ad5d1b--effervescent-dieffenbachia-ab819a.netlify.app

[eukaryo]

Oh, timeline_plot is not rendered... Maybe the reason is that optuna/optuna-visualization-regression-tests#8 is not merged yet. I have resolved comments in that PR. I think we can merge that PR now. (although there is still some room for follow-up)

[toshihikoyanase]

optuna/optuna-visualization-regression-tests#8 was merged! Let me re-render the plot.

[toshihikoyanase]

The timeline plots of Plotly backend were rendered as expected, but those of matplotlib backend weren't. This is because the CI checked out the current master instead of the PR head.

/usr/bin/git -c protocol.version=2 fetch --no-tags --prune --progress --no-recurse-submodules --depth=1 origin +01aa6d9733269aa58e0ae8931a1bd28b662d7942:refs/remotes/origin/master

I think we need #4555 to render plots using the code in PRs.

https://642648d03da4640e33ad5d1b--effervescent-dieffenbachia-ab819a.netlify.app/plot_timeline.html

[HideakiImamura]

Thanks for the update. Almost LGTM. I have a minor comment. PTAL.

[HideakiImamura]

I confirmed that the matplotlib version by using the following code.

import time
import datetime

import optuna
import matplotlib.pyplot as plt


def objective(trial):
    if trial.number % 3 == 0:
        time.sleep(0.003 * trial.number)
        return 1
    elif trial.number % 3 == 1:
        time.sleep(0.002 * trial.number)
        raise optuna.TrialPruned()
    else:
        time.sleep(0.001 * trial.number)
        raise ValueError("FAILED!")


study = optuna.create_study()
study.optimize(objective, n_trials=100, catch=(ValueError,))

start = datetime.datetime.now()
complete = start - datetime.timedelta(seconds=1.0)
study.add_trial(
    optuna.trial.FrozenTrial(
        number=100,
        trial_id=-1,
        state=optuna.trial.TrialState.COMPLETE,
        value=0.0,
        values=None,
        datetime_start=start,
        datetime_complete=complete,
        params={},
        distributions={},
        user_attrs={},
        system_attrs={},
        intermediate_values={},
    )
)

plotly_fig = optuna.visualization.plot_timeline(study)
plotly_fig.write_image("hoge_plotly.png")

plt_fig = optuna.visualization.matplotlib.plot_timeline(study)
plt.savefig("hoge_plot.png")

The generated plots as follows.

Plotly version	Matplotlib version

[eukaryo]

The current code generates the plots below.

Plotly version	Matplotlib version

[HideakiImamura]

Looks very good!

[toshihikoyanase]

Thank you for your update. Let me check again.

[toshihikoyanase]

Thank you for your update. Looks solid to me.

[toshihikoyanase]

[Notes] The reference page was rendered as expected:

[toshihikoyanase]

@amylase Let me merge this PR since it got two approvals.