Support DiscreteUniformDistribution in suggest_float.

[himkt]

Description

This PR introduces step parameter to Trial.suggest_float.

Close #1069.

[codecov-io]

Codecov Report

Merging #1081 into master will decrease coverage by 0.29%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #1081      +/-   ##
==========================================
- Coverage   90.66%   90.37%   -0.30%     
==========================================
  Files         126      126              
  Lines       10954    11070     +116     
==========================================
+ Hits         9931    10004      +73     
- Misses       1023     1066      +43     

Impacted Files	Coverage Δ
optuna/integration/chainermn.py	77.40% <100.00%> (ø)
optuna/trial.py	88.02% <100.00%> (+1.51%)	⬆️
tests/test_trial.py	98.19% <100.00%> (-1.13%)	⬇️
optuna/structs.py	51.28% <0.00%> (-30.84%)	⬇️
optuna/pruners/percentile.py	94.11% <0.00%> (-1.41%)	⬇️
optuna/storages/base.py	56.32% <0.00%> (-0.66%)	⬇️
tests/integration_tests/test_skopt.py	97.33% <0.00%> (ø)
optuna/storages/rdb/storage.py	96.01% <0.00%> (+<0.01%)	⬆️
optuna/storages/in_memory.py	97.86% <0.00%> (+0.01%)	⬆️
... and 5 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 55ece1b...bf3b9e5. Read the comment docs.

[himkt]

I made this PR ready for review since I want to discuss this PR with members.

In #1069, it seems that raising NotImplementedError is expected only when log=True and step!=1.
However, step would be negative when 0<step<1 and be undefined when step<=0.
When log=True and step<1, what behavior is expected?

When log=True and 1<step, I implemented suggest_float to call suggest_discrite_uniform with low=low, high=high and q=math.log(step). I'm not sure whether this implementation is correct.

Could anyone take a look at this PR, please?

[toshihikoyanase]

In #1069, it seems that raising NotImplementedError is expected only when log=True and step!=1.

I'm really sorry but I made a mistake in #1069. The condition should be log=True and step is not None.
I think we may need to add a new distribution like LogDiscreteFloatDistribution and its corresponding calculation to support the condition. This PR can focus on the API design if we separate the PRs.

[himkt]

@toshihikoyanase Thank you for clarification, I revised my PR to raise NotImplementedError when
log=True and step is not None in e7cd5e8.

[hvy]

In terms of API, the changes basically LGTM. I left some minor comments.

Just leaving a reference to another PR as they may conflict #1087.

[himkt]

@hvy I'm sorry and thank you for catching my violating conventions.
(In the thread of #1087, I couldn't see the reference to this PR...:thinking:)

[himkt]

Now I can see #1087 (reference).

[hvy]

Regarding how step may behave when log=True:

1. SMAC3 (ConfigSpace) treats the "step" as given in the log space when log=True. https://github.com/automl/ConfigSpace/blob/master/ConfigSpace/hyperparameters.pyx#L383 This seems a bit unintuitive to me at first glance as the bounds are not, but I might be missing something.
2. Hyperopt is clear by its API design, similar to SMAC3, "step" is given in the log space. https://github.com/hyperopt/hyperopt/blob/d87309463a1496cb66d83827afe204d0700e4aef/docs/templates/getting-started/search_spaces.md#parameter-expressions

So, it seems like treating the "step" in the log space is consistent with other libraries, but again, it's likely open for discussion.

[hvy]

Thanks for your PR and the quick fixed, LGTM!

[toshihikoyanase]

Thank you for your PR. I have two small comments.
FYI: the rendered document can be found https://35185-122299416-gh.circle-artifacts.com/0/docs/build/html/reference/trial.html#optuna.trial.Trial.suggest_float.

[toshihikoyanase]

Thank you for your update. I have two suggestions.

[himkt]

@toshihikoyanase Sorry for not replying, I revised my PR. PTAL.

[toshihikoyanase]

LGTM. Thank you for your contribution!

[himkt]

rel. #510