Same parameters for multiple trials under distributed parallel processing

[JoshKoh]

Running distributed parallel processing with postgresql.
Started 20 instances.
Then noticed multiple trials having same parameters:
Trial 13443 finished with value: 0.7391443119806878 and parameters: {'model_index': 7, 'band1_index': 25, 'band2_index': 17, 'band3_index': 16}. Best is trial 551 with value: 0.7406389004521836.
Trial 13444 finished with value: 0.7391443119806878 and parameters: {'model_index': 7, 'band1_index': 25, 'band2_index': 17, 'band3_index': 16}. Best is trial 810 with value: 0.7406389004521836.
Trial 13445 finished with value: 0.7391443119806878 and parameters: {'model_index': 7, 'band1_index': 25, 'band2_index': 17, 'band3_index': 16}. Best is trial 551 with value: 0.7406389004521836.
Trial 13446 finished with value: 0.7391443119806878 and parameters: {'model_index': 7, 'band1_index': 25, 'band2_index': 17, 'band3_index': 16}. Best is trial 551 with value: 0.7406389004521836.
Trial 13447 finished with value: 0.7391443119806878 and parameters: {'model_index': 7, 'band1_index': 25, 'band2_index': 17, 'band3_index': 16}. Best is trial 551 with value: 0.7406389004521836.

Sometimes up to 8 trials in a row will have same parameters and scores. It pretty consistent throughout the run.

Expected behavior

Each trial to test different hyperparameter combination?

Environment

• Optuna version: 1.5
• Python version: 3.7
• OS: Win10 Pro 64-bit
• (Optional) Other libraries and their versions:

Error messages, stack traces, or logs

None

[ytsmiling]

Thank you for reporting the bug. Recently, we resolved a distribution-related bug in #1490, and I suspect that this problem relates to the bug. Would you mind trying the latest master (please clone this repository and execute pip install -e .)?

[JoshKoh]

The Optuna version that I am using is built from the master branch.

[ytsmiling]

Thank you for the update. So the problem also occurs in the latest master (optuna v2.0.0). We'd like to investigate the problem more, but it might take time. Would you tell us which samplers/pruners you used or provide us reproducible codes?

[JoshKoh]

The issue seems to happen only when there's a small pool of parameters to choose from. In my case, I am trying to select 4 values sequentially from a possible 28 values (28 x 27 x 26 x 25). However, when I increase possible values to 1,200, the issue disappears.

[ytsmiling]

Thank you for the detailed information. I'm afraid to tell you that there are no methods to enforce different samples among different workers in such cases in the current optuna. We'd like to add the functionality as a future plan, but it won't be available in the near future.

[JoshKoh]

I am closing this issue since this will likely be a feature update in the future. Thanks.