Merge pull request #54 from rodrigo-arenas/0.6.X

[PR] Extended Callbacks Methods

docs/api/callbacks.rst

@@ -5,6 +5,7 @@ Callbacks
 
 .. autosummary::
    base.BaseCallback
+   ProgressBar
    ConsecutiveStopping
    DeltaThreshold
    TimerStopping
@@ -17,6 +18,10 @@ Callbacks
    :members:
    :undoc-members: False
 
+.. autoclass:: ProgressBar
+   :members:
+   :undoc-members: False
+
 .. autoclass:: ConsecutiveStopping
    :members:
    :undoc-members: False

docs/release_notes.rst

@@ -12,6 +12,8 @@ This is the current development version, these features are not yet available th
 Features:
 ^^^^^^^^^
 
+* Added the :class:`~sklearn_genetic.callbacks.ProgressBar` callback, it uses tqdm progress bar to shows
+  how many generations are left in the training progress.
 * Added the :class:`~sklearn_genetic.callbacks.TensorBoard` callback to log the
   generation metrics, watch in real time while the models are trained
   and compare different runs in your TensorBoard instance.
@@ -22,8 +24,14 @@ Features:
   its class name to know which callbacks were responsible of the stopping.
 * Added support for extra methods coming from scikit-learn's BaseSearchCV, it is
   still partial support, missing properties like `cv_results_`, `best_index_` and `multimetric_`.
-* tqdm progress bar is now displayed when the .fit method of `GASearchCV` is called, it shows
-  how many generations are left in the training progress.
+* Added methods `on_start` and `on_end` to :class:`~sklearn_genetic.callbacks.base.BaseCallback`.
+  Now the algorithms check for the callbacks like this:
+
+  - **on_start**: When the evolutionary algorithm is called from the GASearchCV.fit method
+
+  - **on_step:** When the evolutionary algorithm finish a generation (no change here).
+
+  - **on_end:** At the end of the last generation.
 
 ^^^^^^^^^^
 Bug Fixes:
@@ -47,7 +55,8 @@ Docs:
 * Edited all demos to be in the jupyter notebook format.
 * Added embedded jupyter notebooks examples.
 * The modules of the package now have a summary of their classes/functions in the docs.
-* Updated the callbacks tutorials to add new TensorBoard callback.
+* Updated the callbacks and custom callbacks tutorials to add new TensorBoard callback and
+  the new methods on the base callback.
 
 What's new in 0.5.0
 -------------------

docs/tutorials/callbacks.rst

@@ -10,7 +10,9 @@ Common callbacks includes different rules to stop the algorithm or log artifacts
 The callbacks are passed to the ``.fit`` method
 of the :class:`~sklearn_genetic.GASearchCV` class.
 
-The callbacks are evaluated at the end of each generation fit, so it looks like this:
+The callbacks are evaluated at start of the training using the `on_start` method,
+at the end of each generation fit using `on_step` method and at the
+end of the training using `on_end`, so it looks like this:
 
 .. image:: ../images/callbacks_evaluation_0.png
 
@@ -23,6 +25,8 @@ until that training point.
 Now lets see how to use them, we'll take
 the data set and model used in :ref:`basic-usage`. The available callbacks are:
 
+* ProgressBar
+
 * ConsecutiveStopping
 
 * DeltaThreshold
@@ -35,6 +39,32 @@ the data set and model used in :ref:`basic-usage`. The available callbacks are:
 
 * LogbookSaver
 
+ProgressBar
+-----------
+
+This callback display a tqdm bar with your training process, the length of the bar
+is the max number of generations (population_size + 1) that the algorithm would run,
+each step is a generation.
+
+You can pass any ``tqdm.auto.tqdm`` valid arguments as kwargs or leave it as default.
+To use this bar set:
+
+.. code:: python3
+
+    from sklearn_genetic.callbacks import ProgressBar
+    callback = ProgressBar()
+
+Now we just have to pass it to the estimator during the fitting
+
+.. code:: python3
+
+    # Already defined GASearchCV instance
+    evolved_estimator.fit(X, y, callbacks=callback)
+
+During the training it will be displayed like this:
+
+.. image:: ../images/progress_bar.gif
+
 ConsecutiveStopping
 -------------------
 

docs/tutorials/custom_callback.rst

@@ -5,10 +5,28 @@ sklearn-genetic-opt comes with some pre-defined callbacks,
 but you can make one of your own by defining a callable with
 certain methods.
 
+Parameters
+----------
+
 The callback must be a class with inheritance from the class
 :class:`~sklearn_genetic.callbacks.base.BaseCallback` that implements the
-``__call__`` and ``on_step`` methods, the result of them must be a bool,
+following methods:
+
+**on_start:** This is evaluated before starting the generation 0, it should return ``None``
+or ``False``. It expects the parameter `estimator`.
+
+**on_step:** This is called at the end of each generation, the result of them must be a bool,
 ``True`` means that the optimization must stop, ``False``, means it can continue.
+It expects the parameters `record`, `logbook` and `estimator`.
+
+**on_end:** This method is called at the end of the las generation or after an stopping
+callback meets its criteria. It expects the parameters `logbook` and `estimator`,
+it should return ``None`` or ``False``.
+
+All of those methods are optional, but at least one should be defined.
+
+Example
+-------
 
 In this example, we are going to define a dummy callback that
 stops the process if there have been more that `N` fitness values
@@ -19,8 +37,6 @@ Those are a dictionary, a deap's Logbook object and the
 current :class:`~sklearn_genetic.GASearchCV` respectively
 with the current iteration metrics, all the past iterations metrics
 and all the properties saved in the estimator.
-You can choice which to use, but all of them must be parameters
-on the ``on_step`` and ``__call__`` methods.
 
 So to check inside the logbook, we could define a function like this:
 
@@ -72,14 +88,37 @@ that will have all this parameters, so we can rewrite it like this:
 
            return False
 
-       def __call__(self, record, logbook, estimator=None):
-           return self.on_step(record, logbook, estimator)
+Now, let's expend it to add the others method, just to print a message:
+
+.. code-block:: python
+
+   from sklearn_genetic.callbacks.base import BaseCallback
 
+   class DummyThreshold(BaseCallback):
+       def __init__(self, threshold, N, metric='fitness'):
+           self.threshold = threshold
+           self.N = N
+           self.metric = metric
 
-**Note:** The implementation of the ``__call__`` method is optional, by default
-its behavior is inherited from the :class:`~sklearn_genetic.callbacks.base.BaseCallback`.
-It's in this example for deeper understanding of how the callbacks are coded and
-to avoid unexpected overwrites.
+       def on_start(self, estimator=None):
+           print("This training is starting!")
+
+       def on_step(self, record, logbook, estimator=None):
+           # Not enough data points
+           if len(logbook) <= self.N:
+               return False
+           # Get the last N metrics
+           stats = logbook.select(self.metric)[(-self.N - 1):]
+
+           n_met_condition = [x for x in stats if x < self.threshold]
+
+           if len(n_met_condition) > self.N:
+               return True
+
+           return False
+
+       def on_end(self, logbook=None, estimator=None):
+           print("I'm done with training!")
 
 So that is it, now you can initialize the DummyThreshold
 and pass it to a in the ``fit`` method of a
@@ -92,4 +131,7 @@ and pass it to a in the ``fit`` method of a
 
 Here there is an output example of this callback:
 
-.. image:: ../images/custom_callback_dummy_0.png
+.. image:: ../images/custom_callback_dummy_0.png
+
+Notice that there is an extra INFO message, that is general for all the callbacks
+that stops the training.