Ability to schedule ablation study of optimized configs

[bpkroth]

Related to #807 it would also be useful to be able to schedule an ablation study of optimal configs vs. the defaults by toggling single parameters (or combinations) back to the default in order to determine what improved in the optimal config.

[bpkroth]

@eujing

[bpkroth]

@jsfreischuetz

[bpkroth]

See Also: #687

[eujing]

I think this can also be achieved with the manual optimizer in #855, if we list out the ablation configs explicitly as configs to execute during the experiment.
I guess the advantage is that we get repeats of each config in a cycle if we did it that way.

[bpkroth]

I think the whole point is to automate the enumeration of the configs.
I'd rather see us implement #687 and then extend it by adding a shedule_ablation_study(config) for a given config vs. the defaults.
Then any scheduler can determine which one to run first (or in parallel) - it doesn't really matter.

To use @motus terminology from a previous discussion, this is a method of enqueuing Trials into the queue (e.g., config suggestion), not of scheduling within that set.