Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix parallel coordinate with missing value #3373

Conversation

nzw0301
Copy link
Collaborator

@nzw0301 nzw0301 commented Mar 10, 2022

Motivation

Resolve #3372

Description of the changes

If a trial has a missing parameter used in a parallel coordinate plot, remove the trail from the visualization data for Plotly.

Note that by this change, coordinates' max/min values might change because the master's implementation set max/min based on all complete trials' values. On the other hand, this PR set max/min value based on only used trials that have no missing params. Personally, this PR's change makes sense since the parallel coordinate plot creates empty space in the coordinate as in the matplotlib's plot on #3372: 1--2 on Objective Value axis or 0--1 on param_b axis.

Screenshot 2022-03-11 at 1 40 23

Implementing the same logic to determine the max/min range in matplotlib should be a follow-up.

@nzw0301 nzw0301 added bug Issue/PR about behavior that is broken. Not for typos/examples/CI/test but for Optuna itself. optuna.visualization Related to the `optuna.visualization` submodule. This is automatically labeled by github-actions. v3 Issue/PR for Optuna version 3. labels Mar 10, 2022
@@ -31,21 +31,21 @@ def test_plot_parallel_coordinate() -> None:
figure = plot_parallel_coordinate(study)
assert len(figure.data[0]["dimensions"]) == 3
assert figure.data[0]["dimensions"][0]["label"] == "Objective Value"
assert figure.data[0]["dimensions"][0]["range"] == (0.0, 2.0)
assert figure.data[0]["dimensions"][0]["values"] == (0.0, 2.0, 1.0)
assert figure.data[0]["dimensions"][0]["range"] == (0.0, 1.0)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

study.trials_dataframe()
   number  value             datetime_start          datetime_complete duration  params_param_a  params_param_b     state
0       0    0.0 2022-03-16 09:52:21.366738 2022-03-16 09:52:21.366738   0 days             1.0             2.0  COMPLETE
1       1    2.0 2022-03-16 09:52:21.366833 2022-03-16 09:52:21.366833   0 days             NaN             0.0  COMPLETE
2       2    1.0 2022-03-16 09:52:21.366892 2022-03-16 09:52:21.366892   0 days             2.5             1.0  COMPLETE

[note] study generated by testing utility has a trial with missing param, which arises a need to change existing tests.

Copy link
Member

@himkt himkt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the PR. I confirmed that this patch fixes a bug around missing parameters.

Before After
Screen Shot 2022-03-16 at 9 34 41 Screen Shot 2022-03-16 at 9 34 47

Copy link
Member

@HideakiImamura HideakiImamura left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change looks good to me. How about adding a document about this behavior?

@himkt
Copy link
Member

himkt commented Mar 16, 2022

Note that, if a parameter contains missing values, a trial with missing values is not plotted.

I think it is already documented (but didn't work as expected).

@nzw0301
Copy link
Collaborator Author

nzw0301 commented Mar 23, 2022

hi @HideakiImamura,
could you give me your approval?

I believe that your comment is answered by #3373 (comment).

Copy link
Member

@HideakiImamura HideakiImamura left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the delay. LGTM.

@HideakiImamura HideakiImamura merged commit b2684aa into optuna:master Mar 24, 2022
@HideakiImamura HideakiImamura added this to the v3.0.0-b0 milestone Mar 24, 2022
@nzw0301 nzw0301 deleted the fix-parallel-coordinate-with-missing-value branch March 24, 2022 09:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Issue/PR about behavior that is broken. Not for typos/examples/CI/test but for Optuna itself. optuna.visualization Related to the `optuna.visualization` submodule. This is automatically labeled by github-actions. v3 Issue/PR for Optuna version 3.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Plotly's parallel coordinate plot with missing value looks wrong
3 participants