Adding a new parameter to TPESampler

[jeromepatel]

Motivation

Currently, there is no argument to pass independent_sampler in TPESampler but in the future, it may be useful to users and may provide flexibility if we can add something as an argument to it like other samplers.

Description

In current implementation, There is a variable random_sampler there in TPESampler, whose after_trial method is called when a fallback occurs. Other samplers use independent_sampler by default initialized with RandomSampler, so in both the cases the performance is same if no argument is passed. However, if we include independent_sampler in TPESampler there will be flexibility when a new sampler can be used in place of RandomSampler (such as Sobol, issue #1797 ).

Additional context (optional)

This is discussion of independent_sampler in continuation of PR #2376. Let me know your views on this!

[jeromepatel]

I want to point out that currently, NSAGAIISampler and MOTPESampler don't have independent_sampler either, so we can consider them also (just for a reference of upgrade)

[HideakiImamura]

TPE supports independent sampling by default. So, it is inappropriate to add independent_sampler to TPESampler.

By the way, if the number of previous trials is smaller than n_startup_trials, the RandomSampler is used in TPESampler. I think it would be natural to allow this RandomSampler to be changed to any other sampler. For example, if SobolSampler or LatinHypercubeSampler are supported in the future, there is a good chance that we will want to replace it with something like that.

[HideakiImamura]

I'd like to discuss the name of the argument to be added. independent_sampler is inappropriate as I said. How about init_sampler? Do you have any opinions?

[jeromepatel]

@HideakiImamura Thank you for your suggestion. I agree with you on the point that TPESampler doesn't look appropriate with an argument named independent_sampler as it already supports it by default. As the proposal for a new sampler could improve flexibility (for eg. SobolSampler and others), the name init_sampler sounds perfect to me. I would like to know others' views on this.

[github-actions]

This issue has not seen any recent activity.

[github-actions]

This issue was closed automatically because it had not seen any recent activity. If you want to discuss it, you can reopen it freely.