Add rank plot matplotlib version #4541

cross32768 · 2023-03-24T09:54:18Z

Motivation

This is the follow-up PR of #4427 .

Description of the changes

Add plot_rank function in optuna.visualization.matplotlib.
It is matplotlib version of plot_rank function in optuna.visualization which was added by #4427 .

review-notebook-app · 2023-03-24T09:54:22Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

github-actions · 2023-04-02T23:08:19Z

This pull request has not seen any recent activity.

github-actions · 2023-04-11T23:07:56Z

This pull request has not seen any recent activity.

cross32768 · 2023-04-14T08:41:27Z

Below is the comparison of plotly version rank_plot and matplotlib version by this PR.

Simple Example

def objective(trial):
    x = trial.suggest_float("x", -1, 1)
    y = trial.suggest_float("y", -1, 1)
    return x + y

study = optuna.create_study()
study.optimize(objective, n_trials=100)

Plotly

matplotlib

cross32768 · 2023-04-14T08:45:04Z

When parameter is logscale or categorical

def objective(trial):
    x = trial.suggest_float("x", 1, 1000, log=True)
    y = trial.suggest_categorical("y", ["a", "b", "c"])
    
    if y == "a":
        return 2 * x
    else:
        return x

Plotly

matplotlib

cross32768 · 2023-04-14T08:53:00Z

When plotting a study with more than 2 parameters

def objective(trial):
    x = trial.suggest_float("x", -1, 1)
    y = trial.suggest_float("y", -1, 1)
    z = trial.suggest_float("z", -1, 1)
    return x * y * z

Plotly

matplotlib

cross32768 · 2023-04-14T08:57:31Z

MEMO: change of plotly version in recent PR #4602 should be included if PR #4602 is merged

toshihikoyanase · 2023-04-20T04:15:11Z

@Alnusjaponica Could you join the review, please? If you have any questions, please let me know.

Alnusjaponica

I made some suggestions, some of which can possibly be addressed in different PRs. Feel free to ignore those minor changes if you don't agree with them.

Alnusjaponica · 2023-04-20T04:34:00Z

optuna/visualization/matplotlib/_rank.py

+) -> "Axes":
+    """Plot parameter relations as scatter plots with colors indicating ranks of objective value.
+
+    Note that, if a parameter contains missing values, a trial with missing values is not plotted.


Suggested change

Note that, if a parameter contains missing values, a trial with missing values is not plotted.

Note that, trials with missing values will not be plotted.

Thank you for your suggestion. It looks good change, but I think to apply this change in the follow-up PR because this change should also apply original plotly-version.

"values" usually mean objective values, so maybe it's better to say "trials missing the specified parameters will not be plotted."

optuna/visualization/matplotlib/_rank.py

Alnusjaponica · 2023-04-20T07:10:36Z

optuna/visualization/matplotlib/_rank.py

+    target: Optional[Callable[[FrozenTrial], float]] = None,
+    target_name: str = "Objective Value",
+) -> "Axes":
+    """Plot parameter relations as scatter plots with colors indicating ranks of objective value.


Suggested change

"""Plot parameter relations as scatter plots with colors indicating ranks of objective value.

"""Plot parameter relations as scatter plots with colors indicating ranks of objective values.

I think that this change can also be resolved in the follow-up PR to re-write both of plotly version and matplotlib version.

More precisely, this should be "ranks of target value".

optuna/visualization/matplotlib/_rank.py

Co-authored-by: Shinichi Hemmi <50256998+Alnusjaponica@users.noreply.github.com>

cross32768 · 2023-04-20T20:02:01Z

I added edgecolor to scatter plot as in #4602 .

cross32768 · 2023-04-20T20:08:36Z

@Alnusjaponica Thank you for your review. I resolved most of comments and update my code to follow #4602. Could you take another look?

Alnusjaponica

Sorry for my delayed reply. LGTM!
@contramundum53 Could you take another look?

contramundum53 · 2023-05-01T05:29:09Z

optuna/visualization/matplotlib/_rank.py

+    if n_params == 0:
+        _, ax = plt.subplots()
+        ax.set_title(title)
+        return ax
+    if n_params == 1 or n_params == 2:
+        fig, axs = plt.subplots()
+        axs.set_title(title)
+        pc = _add_rank_subplot(axs, sub_plot_infos[0][0])
+    else:
+        fig, axs = plt.subplots(n_params, n_params)


Nit: Maybe we don't need these ifs and can directly have plt.subplots(len(sub_plot_infos), len(sub_plot_infos)).

I think these ifs are required by the following reason.

if n_params == 0: is required because in this case, sub_plot_infos is [] and sub_plot_infos[0][0] raises error if we remove this if.

if n_params == 1 or n_params == 2: is required because plt.subplots() or plt.subplots(1, 1) returns Axes object. On the other hand, plt.subplots(n, n) returns array of Axes object if n >= 2. Thus, ax = axs[x_i, y_i] raises error if we remove this if.

contramundum53 · 2023-05-01T05:30:32Z

Otherwise, LGTM!

contramundum53 · 2023-05-09T12:27:32Z

I see, LGTM!

Add matplotlib version of rank_plot

6ac3cbd

Delete notebook added by mistake

adbc9ae

github-actions bot added the optuna.visualization Related to the `optuna.visualization` submodule. This is automatically labeled by github-actions. label Mar 24, 2023

github-actions bot added stale Exempt from stale bot labeling. and removed stale Exempt from stale bot labeling. labels Apr 2, 2023

github-actions bot added the stale Exempt from stale bot labeling. label Apr 11, 2023

cross32768 added 2 commits April 14, 2023 17:09

Merge branch 'master' into rank_plot_matplotlib

fb179f4

Fix title & remove redundant code

e895156

cross32768 removed the stale Exempt from stale bot labeling. label Apr 14, 2023

cross32768 marked this pull request as ready for review April 14, 2023 08:54

cross32768 assigned contramundum53 Apr 14, 2023

toshihikoyanase mentioned this pull request Apr 17, 2023

plot_rank marker lines #4602

Merged

toshihikoyanase added the feature Change that does not break compatibility, but affects the public interfaces. label Apr 20, 2023

toshihikoyanase assigned Alnusjaponica Apr 20, 2023

Alnusjaponica requested changes Apr 20, 2023

View reviewed changes

cross32768 and others added 7 commits April 21, 2023 04:44

Update optuna/visualization/matplotlib/_rank.py

295ad0c

Co-authored-by: Shinichi Hemmi <50256998+Alnusjaponica@users.noreply.github.com>

Update optuna/visualization/matplotlib/_rank.py

9a9e470

Co-authored-by: Shinichi Hemmi <50256998+Alnusjaponica@users.noreply.github.com>

Update optuna/visualization/matplotlib/_rank.py

48a6181

Co-authored-by: Shinichi Hemmi <50256998+Alnusjaponica@users.noreply.github.com>

Update optuna/visualization/matplotlib/_rank.py

6a12771

Co-authored-by: Shinichi Hemmi <50256998+Alnusjaponica@users.noreply.github.com>

Update optuna/visualization/matplotlib/_rank.py

f65031c

Co-authored-by: Shinichi Hemmi <50256998+Alnusjaponica@users.noreply.github.com>

Merge branch 'master' into rank_plot_matplotlib

6e1bc4f

add edge color

fd9f39b

update the pattern of n_params==1

6ed287e

Alnusjaponica approved these changes Apr 27, 2023

View reviewed changes

Alnusjaponica removed their assignment Apr 27, 2023

cross32768 modified the milestone: v3.2.0 Apr 27, 2023

contramundum53 reviewed May 1, 2023

View reviewed changes

cross32768 mentioned this pull request May 8, 2023

Improve docstring of rank_plot function and its matplotlib version #4660

Merged

contramundum53 approved these changes May 9, 2023

View reviewed changes

contramundum53 added this to the v3.2.0 milestone May 9, 2023

contramundum53 merged commit ff12502 into optuna:master May 10, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add rank plot matplotlib version #4541

Add rank plot matplotlib version #4541

cross32768 commented Mar 24, 2023 •

edited

Loading

review-notebook-app bot commented Mar 24, 2023

github-actions bot commented Apr 2, 2023

github-actions bot commented Apr 11, 2023

cross32768 commented Apr 14, 2023

cross32768 commented Apr 14, 2023

cross32768 commented Apr 14, 2023 •

edited

Loading

cross32768 commented Apr 14, 2023

toshihikoyanase commented Apr 20, 2023

Alnusjaponica left a comment •

edited

Loading

Alnusjaponica Apr 20, 2023

cross32768 Apr 20, 2023

contramundum53 May 1, 2023

Alnusjaponica Apr 20, 2023

cross32768 Apr 20, 2023

contramundum53 May 1, 2023

cross32768 commented Apr 20, 2023

cross32768 commented Apr 20, 2023

Alnusjaponica left a comment

contramundum53 May 1, 2023

cross32768 May 8, 2023

contramundum53 commented May 1, 2023

contramundum53 commented May 9, 2023

	Note that, if a parameter contains missing values, a trial with missing values is not plotted.
	Note that, trials with missing values will not be plotted.

	"""Plot parameter relations as scatter plots with colors indicating ranks of objective value.
	"""Plot parameter relations as scatter plots with colors indicating ranks of objective values.

Add rank plot matplotlib version #4541

Add rank plot matplotlib version #4541

Conversation

cross32768 commented Mar 24, 2023 • edited Loading

Motivation

Description of the changes

review-notebook-app bot commented Mar 24, 2023

github-actions bot commented Apr 2, 2023

github-actions bot commented Apr 11, 2023

cross32768 commented Apr 14, 2023

Simple Example

cross32768 commented Apr 14, 2023

When parameter is logscale or categorical

cross32768 commented Apr 14, 2023 • edited Loading

When plotting a study with more than 2 parameters

cross32768 commented Apr 14, 2023

toshihikoyanase commented Apr 20, 2023

Alnusjaponica left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cross32768 commented Apr 20, 2023

cross32768 commented Apr 20, 2023

Alnusjaponica left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

contramundum53 commented May 1, 2023

contramundum53 commented May 9, 2023

cross32768 commented Mar 24, 2023 •

edited

Loading

cross32768 commented Apr 14, 2023 •

edited

Loading

Alnusjaponica left a comment •

edited

Loading