docs: Adding links to API in tutorials (#787)

docs/source/tutorials/multifidelity/mf_asha.rst

@@ -38,6 +38,13 @@ decisions based on too little data.
 Our `launcher script <mf_setup.html#the-launcher-script>`__ runs the stopping
 variant of ASHA if ``method="ASHA-STOP"``.
 
+API docs:
+
+* Baseline: :class:`~syne_tune.optimizer.baselines.ASHA`
+* Additional arguments: :class:`~syne_tune.optimizer.schedulers.HyperbandScheduler`
+  (``type="stopping"`` selects the early stopping variant)
+
+
 Asynchronous Successive Halving: Promotion Variant
 --------------------------------------------------
 
@@ -53,6 +60,12 @@ trial is found, a new trial is started from scratch. Our
 `launcher script <mf_setup.html#the-launcher-script>`__ runs the stopping
 variant of ASHA if ``method="ASHA-PROM"``.
 
+API docs:
+
+* Baseline: :class:`~syne_tune.optimizer.baselines.ASHA`
+* Additional arguments: :class:`~syne_tune.optimizer.schedulers.HyperbandScheduler`
+  (``type="promotion"`` selects the promotion variant)
+
 If these two variants (stopping and promotion) are compared under ideal
 conditions, one sometimes does better than the other, and vice versa. However,
 they come with different requirements. The promotion variant pauses and resumes
@@ -98,6 +111,12 @@ this distribution is :math:`P(r_{min}) = [1:243/415, 3:98/415, 9:41/415,
 <mf_setup.html#the-launcher-script>`__ runs asynchronous Hyperband with 6
 brackets if ``method="ASHA6-STOP"``.
 
+API docs:
+
+* Baseline: :class:`~syne_tune.optimizer.baselines.ASHA`
+* Additional arguments: :class:`~syne_tune.optimizer.schedulers.HyperbandScheduler`
+  (``brackets`` selects the number of brackets, defaults to 1)
+
 As also noted in `ASHA <https://arxiv.org/abs/1810.05934>`__, the algorithm
 often works best with a single bracket, so that ``brackets=1`` is the default
 in Syne Tune. However, we will see further below that model-based variants of

docs/source/tutorials/multifidelity/mf_async_model.rst

@@ -121,7 +121,13 @@ above. The acquisition function is expected improvement (EI) at the rung level
 Our `launcher script <mf_setup.html#the-launcher-script>`__ runs (asynchronous)
 MOBSTER-JOINT if ``method="MOBSTER-JOINT"``. The searcher can be configured
 with ``search_options``, but MOBSTER-JOINT with the ``"exp-decay-sum"``
-covariance model is the default
+covariance model is the default.
+
+API docs:
+
+* Baseline: :class:`~syne_tune.optimizer.baselines.MOBSTER`
+* Additional arguments: :class:`~syne_tune.optimizer.schedulers.HyperbandScheduler`
+* ``search_options``: :class:`~syne_tune.optimizer.schedulers.searchers.GPMultiFidelitySearcher`
 
 As shown `below <mf_comparison.html>`__, MOBSTER can outperform ASHA
 significantly. This is achieved by starting many less trials that stop very
@@ -169,6 +175,13 @@ with an independent GPs model, justifying the Syne Tune default. In our
 experience so far, changing the covariance model in MOBSTER-JOINT has only
 marginal impact.
 
+API docs:
+
+* Baseline: :class:`~syne_tune.optimizer.baselines.MOBSTER`
+* Additional arguments: :class:`~syne_tune.optimizer.schedulers.HyperbandScheduler`
+* ``search_options``: :class:`~syne_tune.optimizer.schedulers.searchers.GPMultiFidelitySearcher`
+  (here, we use ``search_options["model"] = "gp_independent"``)
+
 MOBSTER and Hyperband
 ~~~~~~~~~~~~~~~~~~~~~
 
@@ -237,6 +250,14 @@ called HYPERTUNE-INDEP, while this latter variant is called HYPERTUNE-JOINT.
 Our `launcher script <mf_setup.html#the-launcher-script>`__ runs this variant
 if ``method="HYPERTUNE-JOINT"``.
 
+API docs:
+
+* Baseline: :class:`~syne_tune.optimizer.baselines.HyperTune`
+* Additional arguments: :class:`~syne_tune.optimizer.schedulers.HyperbandScheduler`
+* ``search_options``: :class:`~syne_tune.optimizer.schedulers.searchers.hypertune.HyperTuneSearcher`
+  (``search_options["model"] = "gp_independent"`` by default, but HYPERTUNE-JOINT
+  is using ``"gp_multitask"``)
+
 Finally, computations of Hyper-Tune can be
 `controlled in the same way as in MOBSTER <#controlling-mobster-computations>`__.
 
@@ -386,3 +407,10 @@ important points:
   `above <#controlling-mobster-computations>`__. Apart from the default for
   ``max_size_data_for_model``, we also use ``opt_skip_period = 3`` as default
   for DyHPO.
+
+API docs:
+
+* Baseline: :class:`~syne_tune.optimizer.baselines.DyHPO`
+* Additional arguments: :class:`~syne_tune.optimizer.schedulers.HyperbandScheduler`
+* ``search_options``: :class:`~syne_tune.optimizer.schedulers.searchers.GPMultiFidelitySearcher`
+  (note that ``search_options["model"]`` must not be equal to ``"gp_independent"``)

docs/source/tutorials/multifidelity/mf_sync_model.rst

@@ -33,6 +33,11 @@ data for the two densities to be properly fit. It then makes a TPE decision at
 this resource level. Our `launcher script <mf_setup.html#the-launcher-script>`__
 runs synchronous BOHB if ``method="BOHB"``.
 
+API docs:
+
+* Baseline: :class:`~syne_tune.optimizer.baselines.SyncBOHB`
+* Additional arguments: :class:`~syne_tune.optimizer.schedulers.synchronous.SynchronousGeometricHyperbandScheduler`
+
 While BOHB is often more efficient than SYNCHB, it is held back by synchronous
 decision-making. Note that BOHB does not model the random function
 :math:`f(\mathbf{x}, r)` directly, which makes it hard to properly react to
@@ -56,6 +61,11 @@ is represented by an independent Gaussian process (more details are given
 `It turns out <mf_comparison.html>`__ that ``SyncMOBSTER`` outperforms
 ``SyncBOHB`` substantially on the benchmark chosen here.
 
+API docs:
+
+* Baseline: :class:`~syne_tune.optimizer.baselines.SyncMOBSTER`
+* Additional arguments: :class:`~syne_tune.optimizer.schedulers.synchronous.SynchronousGeometricHyperbandScheduler`
+
 When running these experiments with the simulator backend, we note that
 suddenly it takes quite some time for an experiment to be finished. Still many
 times faster than real time, we now need many minutes instead of seconds. This
@@ -93,6 +103,11 @@ rung, the hope is to find well-performing configurations faster. Our
 `launcher script <mf_setup.html#the-launcher-script>`__ runs DEHB if
 ``method="DEHB"``.
 
+API docs:
+
+* Baseline: :class:`~syne_tune.optimizer.baselines.DEHB`
+* Additional arguments: :class:`~syne_tune.optimizer.schedulers.synchronous.GeometricDifferentialEvolutionHyperbandScheduler`
+
 The main feature of DEHB over synchronous Hyperband is that configurations can
 be modified at every rung. However, this feature also has a drawback. Namely,
 DEHB cannot make effective use of checkpointing. If a trial is resumed with a

docs/source/tutorials/multifidelity/mf_syncsh.rst

@@ -88,8 +88,12 @@ Our `launcher script <mf_setup.html#the-launcher-script>`__ runs synchronous
 successive halving if ``method="SYNCSH"``. The relevant parameters are
 ``grace_period`` ( :math:`r_{min}` ) and ``reduction_factor`` ( :math:`\eta` ).
 Moreover, for SH, we need to set ``brackets=1``, since otherwise an extension
-called *Hyperband* is run (to be discussed shortly). Our implementation is
-:class:`~syne_tune.optimizer.schedulers.synchronous.SynchronousHyperbandScheduler`.
+called *Hyperband* is run (to be discussed shortly).
+
+API docs:
+
+* Baseline: :class:`~syne_tune.optimizer.baselines.SyncHyperband`
+* Additional arguments: :class:`~syne_tune.optimizer.schedulers.synchronous.SynchronousGeometricHyperbandScheduler`
 
 Synchronous SH employs *pause-and-resume scheduling* (see
 `introduction <mf_introduction.html#multi-fidelity-scheduling>`__). Once a
@@ -158,8 +162,12 @@ NASBench-201 example:
 Our `launcher script <mf_setup.html#the-launcher-script>`__ runs synchronous
 Hyperband if ``method="SYNCHB"``. Since ``brackets`` is not used when creating
 ``SyncHyperband``, the maximum value 6 is chosen. We also use the default
-values for ``grace_period`` (1) and ``reduction_factor`` (3). Our implementation
-is :class:`~syne_tune.optimizer.schedulers.synchronous.SynchronousHyperbandScheduler`.
+values for ``grace_period`` (1) and ``reduction_factor`` (3).
+
+API docs:
+
+* Baseline: :class:`~syne_tune.optimizer.baselines.SyncHyperband`
+* Additional arguments: :class:`~syne_tune.optimizer.schedulers.synchronous.SynchronousGeometricHyperbandScheduler`
 
 The advantages of Hyperband over SH are mostly theoretical. In practice, while
 Hyperband can improve on SH if :math:`r_{min}` chosen for SH is clearly too

examples/launch_resume_tuning.py

@@ -1,3 +1,15 @@
+# Copyright 2021 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License").
+# You may not use this file except in compliance with the License.
+# A copy of the License is located at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# or in the "license" file accompanying this file. This file is distributed
+# on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
+# express or implied. See the License for the specific language governing
+# permissions and limitations under the License.
 from syne_tune.config_space import randint
 
 import shutil

syne_tune/experiments/util.py

@@ -0,0 +1,12 @@
+# Copyright 2021 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License").
+# You may not use this file except in compliance with the License.
+# A copy of the License is located at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# or in the "license" file accompanying this file. This file is distributed
+# on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
+# express or implied. See the License for the specific language governing
+# permissions and limitations under the License.

syne_tune/optimizer/baselines.py

@@ -146,8 +146,10 @@ class ASHA(HyperbandScheduler):
     """Asynchronous Sucessive Halving (ASHA).
 
     One of ``max_t``, ``max_resource_attr`` needs to be in ``kwargs``. For
-    ``type="promotion"``, the latter is more useful, see also
-    :class:`~syne_tune.optimizer.schedulers.HyperbandScheduler`.
+    ``type="promotion"``, the latter is more useful.
+
+    See also :class:`~syne_tune.optimizer.schedulers.HyperbandScheduler` for
+    ``kwargs`` parameters.
 
     :param config_space: Configuration space for evaluation function
     :param metric: Name of metric to optimize
@@ -183,8 +185,12 @@ class MOBSTER(HyperbandScheduler):
     ``"gp_independent"`` (independent GP models at each rung level, with shared
     ARD kernel).
 
-    See :class:`~syne_tune.optimizer.schedulers.searchers.GPMultifidelitySearcher`
-    for ``kwargs["search_options"]`` parameters.
+    See also:
+
+    * :class:`~syne_tune.optimizer.schedulers.HyperbandScheduler` for ``kwargs``
+      parameters
+    * :class:`~syne_tune.optimizer.schedulers.searchers.GPMultiFidelitySearcher`
+      for ``kwargs["search_options"]`` parameters
 
     :param config_space: Configuration space for evaluation function
     :param metric: Name of metric to optimize
@@ -225,11 +231,14 @@ class HyperTune(HyperbandScheduler):
         | VLDB 2022
         | https://arxiv.org/abs/2201.06834
 
-    See also
-    :class:`~syne_tune.optimizer.schedulers.searchers.bayesopt.gpautograd.hypertune.gp_model.HyperTuneIndependentGPModel`,
-    and see
-    :class:`~syne_tune.optimizer.schedulers.searchers.hypertune.HyperTuneSearcher`
-    for ``kwargs["search_options"]`` parameters.
+    See also:
+
+    * :class:`~syne_tune.optimizer.schedulers.HyperbandScheduler` for ``kwargs``
+      parameters
+    * :class:`~syne_tune.optimizer.schedulers.searchers.hypertune.HyperTuneSearcher`
+      for ``kwargs["search_options"]`` parameters
+    * :class:`~syne_tune.optimizer.schedulers.searchers.bayesopt.gpautograd.hypertune.gp_model.HyperTuneIndependentGPModel`
+      for implementation
 
     :param config_space: Configuration space for evaluation function
     :param metric: Name of metric to optimize
@@ -292,14 +301,16 @@ class DyHPO(HyperbandScheduler):
       data at higher rungs, the score values for promoting a trial are much worse
       than those for starting a new one.
 
-    See :class:`~syne_tune.optimizer.schedulers.searchers.GPMultifidelitySearcher`
+    See :class:`~syne_tune.optimizer.schedulers.HyperbandScheduler` for ``kwargs``
+    parameters, and
+    :class:`~syne_tune.optimizer.schedulers.searchers.GPMultiFidelitySearcher`
     for ``kwargs["search_options"]`` parameters. The following parameters are
     most important for DyHPO:
 
     * ``rung_increment`` (and ``grace_period``): These parameters determine the
       rung level spacing. DyHPO is run with linearly spaced rung levels
       :math:`r_{min} + k \nu`, where :math:`r_{min}` is ``grace_period`` and
-      :math:`\nu` is `rung_increment``. The default is 2.
+      :math:`\nu` is ``rung_increment``. The default is 2.
     * ``probability_sh``: See comment. The smaller this probability, the closer
       the method is to the published original, which tends to start many more
       trials than promote paused ones. On the other hand, if this probability is
@@ -394,7 +405,9 @@ class BOHB(HyperbandScheduler):
 
     See
     :class:`~syne_tune.optimizer.schedulers.searchers.kde.MultiFidelityKernelDensityEstimator`
-    for ``kwargs["search_options"]`` parameters.
+    for ``kwargs["search_options"]`` parameters, and
+    :class:`~syne_tune.optimizer.schedulers.HyperbandScheduler` for ``kwargs``
+    parameters.
 
     :param config_space: Configuration space for evaluation function
     :param metric: Name of metric to optimize
@@ -423,6 +436,9 @@ class SyncHyperband(SynchronousGeometricHyperbandScheduler):
     One of ``max_resource_level``, ``max_resource_attr`` needs to be in ``kwargs``.
     The latter is more useful, see also :class:`~syne_tune.optimizer.schedulers.HyperbandScheduler`.
 
+    If ``kwargs["brackets"]`` is not given, the maximum number of brackets is
+    used. Choose ``kwargs["brackets"] = 1`` for synchronous successive halving.
+
     :param config_space: Configuration space for evaluation function
     :param metric: Name of metric to optimize
     :param resource_attr: Name of resource attribute
@@ -457,6 +473,9 @@ class SyncBOHB(SynchronousGeometricHyperbandScheduler):
     The latter is more useful, see also
     :class:`~syne_tune.optimizer.schedulers.HyperbandScheduler`.
 
+    If ``kwargs["brackets"]`` is not given, the maximum number of brackets is
+    used. Choose ``kwargs["brackets"] = 1`` for synchronous successive halving.
+
     :param config_space: Configuration space for evaluation function
     :param metric: Name of metric to optimize
     :param resource_attr: Name of resource attribute
@@ -526,6 +545,9 @@ class SyncMOBSTER(SynchronousGeometricHyperbandScheduler):
     The latter is more useful, see also
     :class:`~syne_tune.optimizer.schedulers.HyperbandScheduler`.
 
+    If ``kwargs["brackets"]`` is not given, the maximum number of brackets is
+    used. Choose ``kwargs["brackets"] = 1`` for synchronous successive halving.
+
     The default surrogate model (``search_options["model"]`` in ``kwargs``) is
     ``"gp_independent"``, different to :class:`MOBSTER`.
 

syne_tune/optimizer/schedulers/smac_scheduler.py

@@ -1,3 +1,15 @@
+# Copyright 2021 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License").
+# You may not use this file except in compliance with the License.
+# A copy of the License is located at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# or in the "license" file accompanying this file. This file is distributed
+# on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
+# express or implied. See the License for the specific language governing
+# permissions and limitations under the License.
 import logging
 from typing import Optional, List, Dict, Any
 

tst/experiments/test_metric_name_mode.py

@@ -1,3 +1,15 @@
+# Copyright 2021 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License").
+# You may not use this file except in compliance with the License.
+# A copy of the License is located at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# or in the "license" file accompanying this file. This file is distributed
+# on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
+# express or implied. See the License for the specific language governing
+# permissions and limitations under the License.
 import pytest
 
 from syne_tune.util import metric_name_mode