Pushing the docs to 0.9/ for branch: 0.9.X, commit a2369ddbc332d16d8ff173b12404b03fea472492

Author: skoptci

Diff for: 0.9/.buildinfo

@@ -0,0 +1,4 @@
+# Sphinx build info version 1
+# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
+config: 8775eac5e78fff9d3a8c75b300e84394
+tags: 645f666f9bcd5a90fca523b33c5a78b7

Diff for: 0.9/.nojekyll

@@ -0,0 +1,79 @@
+{
+  "cells": [
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "collapsed": false
+      },
+      "outputs": [],
+      "source": [
+        "%matplotlib inline"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n# Parallel optimization\n\nIaroslav Shcherbatyi, May 2017.\nReviewed by Manoj Kumar and Tim Head.\nReformatted by Holger Nahrstaedt 2020\n\n.. currentmodule:: skopt\n\n## Introduction\n\nFor many practical black box optimization problems expensive objective can be\nevaluated in parallel at multiple points. This allows to get more objective\nevaluations per unit of time, which reduces the time necessary to reach good\nobjective values when appropriate optimization algorithms are used, see for\nexample results in [1]_ and the references therein.\n\n\nOne such example task is a selection of number and activation function of a\nneural network which results in highest accuracy for some machine learning\nproblem. For such task, multiple neural networks with different combinations\nof number of neurons and activation function type can be evaluated at the same\ntime in parallel on different cpu cores / computational nodes.\n\nThe \u201cask and tell\u201d API of scikit-optimize exposes functionality that allows to\nobtain multiple points for evaluation in parallel. Intended usage of this\ninterface is as follows:\n\n1. Initialize instance of the `Optimizer` class from skopt\n2. Obtain n points for evaluation in parallel by calling the `ask` method of an optimizer instance with the `n_points` argument set to n > 0\n3. Evaluate points\n4. Provide points and corresponding objectives using the `tell` method of an optimizer instance\n5. Continue from step 2 until eg maximum number of evaluations reached\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "collapsed": false
+      },
+      "outputs": [],
+      "source": [
+        "print(__doc__)\nimport numpy as np"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Example\n\nA minimalistic example that uses joblib to parallelize evaluation of the\nobjective function is given below.\n\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "collapsed": false
+      },
+      "outputs": [],
+      "source": [
+        "from skopt import Optimizer\nfrom skopt.space import Real\nfrom joblib import Parallel, delayed\n# example objective taken from skopt\nfrom skopt.benchmarks import branin\n\noptimizer = Optimizer(\n    dimensions=[Real(-5.0, 10.0), Real(0.0, 15.0)],\n    random_state=1,\n    base_estimator='gp'\n)\n\nfor i in range(10):\n    x = optimizer.ask(n_points=4)  # x is a list of n_points points\n    y = Parallel(n_jobs=4)(delayed(branin)(v) for v in x)  # evaluate points in parallel\n    optimizer.tell(x, y)\n\n# takes ~ 20 sec to get here\nprint(min(optimizer.yi))  # print the best objective found"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Note that if `n_points` is set to some integer > 0 for the `ask` method, the\nresult will be a list of points, even for `n_points` = 1. If the argument is\nset to `None` (default value) then a single point (but not a list of points)\nwill be returned.\n\nThe default \"minimum constant liar\" [1]_ parallelization strategy is used in\nthe example, which allows to obtain multiple points for evaluation with a\nsingle call to the `ask` method with any surrogate or acquisition function.\nParallelization strategy can be set using the \"strategy\" argument of `ask`.\nFor supported parallelization strategies see the documentation of\nscikit-optimize.\n\n.. [1] `<https://hal.archives-ouvertes.fr/hal-00732512/document>`_\n"
+      ]
+    }
+  ],
+  "metadata": {
+    "kernelspec": {
+      "display_name": "Python 3",
+      "language": "python",
+      "name": "python3"
+    },
+    "language_info": {
+      "codemirror_mode": {
+        "name": "ipython",
+        "version": 3
+      },
+      "file_extension": ".py",
+      "mimetype": "text/x-python",
+      "name": "python",
+      "nbconvert_exporter": "python",
+      "pygments_lexer": "ipython3",
+      "version": "3.9.7"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
+}

Diff for: 0.9/_downloads/006e45de69c37f7b58e1da3a6fd18e4e/parallel-optimization.ipynb

@@ -0,0 +1,147 @@
+"""
+==========================
+Comparing surrogate models
+==========================
+
+Tim Head, July 2016.
+Reformatted by Holger Nahrstaedt 2020
+
+.. currentmodule:: skopt
+
+Bayesian optimization or sequential model-based optimization uses a surrogate
+model to model the expensive to evaluate function `func`. There are several
+choices for what kind of surrogate model to use. This notebook compares the
+performance of:
+
+* gaussian processes,
+* extra trees, and
+* random forests
+
+as surrogate models. A purely random optimization strategy is also used as
+a baseline.
+"""
+
+print(__doc__)
+import numpy as np
+np.random.seed(123)
+import matplotlib.pyplot as plt
+
+#############################################################################
+# Toy model
+# =========
+#
+# We will use the :class:`benchmarks.branin` function as toy model for the expensive function.
+# In a real world application this function would be unknown and expensive
+# to evaluate.
+
+from skopt.benchmarks import branin as _branin
+
+def branin(x, noise_level=0.):
+    return _branin(x) + noise_level * np.random.randn()
+
+#############################################################################
+
+from matplotlib.colors import LogNorm
+
+
+def plot_branin():
+    fig, ax = plt.subplots()
+
+    x1_values = np.linspace(-5, 10, 100)
+    x2_values = np.linspace(0, 15, 100)
+    x_ax, y_ax = np.meshgrid(x1_values, x2_values)
+    vals = np.c_[x_ax.ravel(), y_ax.ravel()]
+    fx = np.reshape([branin(val) for val in vals], (100, 100))
+
+    cm = ax.pcolormesh(x_ax, y_ax, fx,
+                       norm=LogNorm(vmin=fx.min(),
+                                    vmax=fx.max()),
+                       cmap='viridis_r')
+
+    minima = np.array([[-np.pi, 12.275], [+np.pi, 2.275], [9.42478, 2.475]])
+    ax.plot(minima[:, 0], minima[:, 1], "r.", markersize=14,
+            lw=0, label="Minima")
+
+    cb = fig.colorbar(cm)
+    cb.set_label("f(x)")
+
+    ax.legend(loc="best", numpoints=1)
+
+    ax.set_xlabel("X1")
+    ax.set_xlim([-5, 10])
+    ax.set_ylabel("X2")
+    ax.set_ylim([0, 15])
+
+
+plot_branin()
+
+#############################################################################
+# This shows the value of the two-dimensional branin function and
+# the three minima.
+#
+#
+# Objective
+# =========
+#
+# The objective of this example is to find one of these minima in as
+# few iterations as possible. One iteration is defined as one call
+# to the :class:`benchmarks.branin` function.
+#
+# We will evaluate each model several times using a different seed for the
+# random number generator. Then compare the average performance of these
+# models. This makes the comparison more robust against models that get
+# "lucky".
+
+from functools import partial
+from skopt import gp_minimize, forest_minimize, dummy_minimize
+
+func = partial(branin, noise_level=2.0)
+bounds = [(-5.0, 10.0), (0.0, 15.0)]
+n_calls = 60
+
+#############################################################################
+
+
+def run(minimizer, n_iter=5):
+    return [minimizer(func, bounds, n_calls=n_calls, random_state=n)
+            for n in range(n_iter)]
+
+# Random search
+dummy_res = run(dummy_minimize)
+
+# Gaussian processes
+gp_res = run(gp_minimize)
+
+# Random forest
+rf_res = run(partial(forest_minimize, base_estimator="RF"))
+
+# Extra trees
+et_res = run(partial(forest_minimize, base_estimator="ET"))
+
+#############################################################################
+# Note that this can take a few minutes.
+
+from skopt.plots import plot_convergence
+
+plot = plot_convergence(("dummy_minimize", dummy_res),
+                        ("gp_minimize", gp_res),
+                        ("forest_minimize('rf')", rf_res),
+                        ("forest_minimize('et)", et_res),
+                        true_minimum=0.397887, yscale="log")
+
+plot.legend(loc="best", prop={'size': 6}, numpoints=1)
+
+#############################################################################
+# This plot shows the value of the minimum found (y axis) as a function
+# of the number of iterations performed so far (x axis). The dashed red line
+# indicates the true value of the minimum of the :class:`benchmarks.branin` function.
+#
+# For the first ten iterations all methods perform equally well as they all
+# start by creating ten random samples before fitting their respective model
+# for the first time. After iteration ten the next point at which
+# to evaluate :class:`benchmarks.branin` is guided by the model, which is where differences
+# start to appear.
+#
+# Each minimizer only has access to noisy observations of the objective
+# function, so as time passes (more iterations) it will start observing
+# values that are below the true value simply because they are fluctuations.