Skip to content

Commit

Permalink
Merge f47adb8 into 2c33155
Browse files Browse the repository at this point in the history
  • Loading branch information
jlnav committed Oct 9, 2023
2 parents 2c33155 + f47adb8 commit ba7fcc4
Show file tree
Hide file tree
Showing 7 changed files with 273 additions and 76 deletions.
2 changes: 1 addition & 1 deletion docs/data_structures/persis_info.rst
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ Examples:
.. literalinclude:: ../../libensemble/alloc_funcs/start_only_persistent.py
:linenos:
:start-at: if gen_count < persis_info.get("num_gens_started", 0):
:end-before: # Give evaluated results back to the persistent gen
:end-before: # Give evaluated results back to a running persistent gen
:emphasize-lines: 1
:caption: libensemble/alloc_funcs/start_only_persistent.py

Expand Down
107 changes: 74 additions & 33 deletions docs/function_guides/generator.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,42 +5,84 @@ Generator Functions

Generator and :ref:`Simulator functions<funcguides-sim>` have relatively similar interfaces.

.. code-block:: python
Writing a Generator
-------------------

def my_generator(Input, persis_info, gen_specs, libE_info):
batch_size = gen_specs["user"]["batch_size"]
.. tab-set::

.. tab-item:: Non-decorated
:sync: nodecorate

.. code-block:: python
def my_generator(Input, persis_info, gen_specs, libE_info):
batch_size = gen_specs["user"]["batch_size"]
Output = np.zeros(batch_size, gen_specs["out"])
# ...
Output["x"], persis_info = generate_next_simulation_inputs(Input["f"], persis_info)
return Output, persis_info
.. tab-item:: Decorated
:sync: decorate

.. code-block:: python
Output = np.zeros(batch_size, gen_specs["out"])
...
Output["x"], persis_info = generate_next_simulation_inputs(Input["f"], persis_info)
from libensemble.specs import input_fields, output_data
return Output, persis_info
@input_fields(["f"])
@output_data([("x", float)])
def my_generator(Input, persis_info, gen_specs, libE_info):
batch_size = gen_specs["user"]["batch_size"]
Output = np.zeros(batch_size, gen_specs["out"])
# ...
Output["x"], persis_info = generate_next_simulation_inputs(Input["f"], persis_info)
return Output, persis_info
Most ``gen_f`` function definitions written by users resemble::

def my_generator(Input, persis_info, gen_specs, libE_info):

where:

* ``Input`` is a selection of the :ref:`History array<funcguides-history>`
* :ref:`persis_info<datastruct-persis-info>` is a dictionary containing state information
* :ref:`gen_specs<datastruct-gen-specs>` is a dictionary of generator parameters, including which fields from the History array got sent
* ``libE_info`` is a dictionary containing libEnsemble-specific entries
* ``Input`` is a selection of the :ref:`History array<funcguides-history>`, a NumPy array.
* :ref:`persis_info<datastruct-persis-info>` is a dictionary containing state information.
* :ref:`gen_specs<datastruct-gen-specs>` is a dictionary of generator parameters.
* ``libE_info`` is a dictionary containing miscellaneous entries.

Valid generator functions can accept a subset of the above parameters. So a very simple generator can start::

def my_generator(Input):

If gen_specs was initially defined::
If ``gen_specs`` was initially defined:

.. tab-set::

.. tab-item:: Non-decorated function
:sync: nodecorate

.. code-block:: python
gen_specs = GenSpecs(
gen_f=my_generator,
inputs=["f"],
outputs=["x", float, (1,)],
user={"batch_size": 128},
)
.. tab-item:: Decorated function
:sync: decorate

.. code-block:: python
gen_specs = {
"gen_f": some_function,
"in": ["f"],
"out:" ["x", float, (1,)],
"user": {
"batch_size": 128
}
}
gen_specs = GenSpecs(
gen_f=my_generator,
user={"batch_size": 128},
)
Then user parameters and a *local* array of outputs may be obtained/initialized like::

Expand All @@ -56,10 +98,9 @@ Then return the array and ``persis_info`` to libEnsemble::

return Output, persis_info

Between the ``Output`` definition and the ``return``, any level and complexity
of computation can be performed. Users are encouraged to use the :doc:`executor<../executor/overview>`
to submit applications to parallel resources if necessary, or plug in components from
other libraries to serve their needs.
Between the ``Output`` definition and the ``return``, any computation can be performed.
Users can try an :doc:`executor<../executor/overview>` to submit applications to parallel
resources, or plug in components from other libraries to serve their needs.

.. note::

Expand All @@ -74,17 +115,17 @@ Persistent Generators
While non-persistent generators return after completing their calculation, persistent
generators do the following in a loop:

1. Receive simulation results and metadata; exit if metadata instructs
2. Perform analysis
3. Send subsequent simulation parameters
1. Receive simulation results and metadata; exit if metadata instructs.
2. Perform analysis.
3. Send subsequent simulation parameters.

Persistent generators don't need to be re-initialized on each call, but are typically
more complicated. The :doc:`APOSMM<../examples/aposmm>`
optimization generator function included with libEnsemble is persistent so it can
maintain multiple local optimization subprocesses based on results from complete simulations.
more complicated. The persistent :doc:`APOSMM<../examples/aposmm>`
optimization generator function included with libEnsemble maintains
local optimization subprocesses based on results from complete simulations.

Use ``gen_specs["persis_in"]`` to specify fields to send back to the generator throughout the run.
``gen_specs["in"]`` only describes the input fields when the function is **first called**.
Use ``GenSpecs.persis_in`` to specify fields to send back to the generator throughout the run.
``GenSpecs.inputs`` only describes the input fields when the function is **first called**.

Functions for a persistent generator to communicate directly with the manager
are available in the :ref:`libensemble.tools.persistent_support<p_gen_routines>` class.
Expand Down Expand Up @@ -159,7 +200,7 @@ a worker can be initiated in *active receive* mode by the allocation
function (see :ref:`start_only_persistent<start_only_persistent_label>`).
The persistent worker can then send and receive from the manager at any time.

Ensure there are no communication deadlocks in this mode. In manager--worker message exchanges, only the worker-side
Ensure there are no communication deadlocks in this mode. In manager-worker message exchanges, only the worker-side
receive is blocking by default (a non-blocking option is available).

Cancelling Simulations
Expand Down
93 changes: 68 additions & 25 deletions docs/function_guides/simulator.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,42 +5,86 @@ Simulator Functions

Simulator and :ref:`Generator functions<funcguides-gen>` have relatively similar interfaces.

.. code-block:: python
Writing a Simulator
-------------------

.. tab-set::

.. tab-item:: Non-decorated
:sync: nodecorate

.. code-block:: python
def my_simulation(Input, persis_info, sim_specs, libE_info):
batch_size = sim_specs["user"]["batch_size"]
Output = np.zeros(batch_size, sim_specs["out"])
# ...
Output["f"], persis_info = do_a_simulation(Input["x"], persis_info)
return Output, persis_info
.. tab-item:: Decorated
:sync: decorate

.. code-block:: python
from libensemble.specs import input_fields, output_data
def my_simulation(Input, persis_info, sim_specs, libE_info):
batch_size = sim_specs["user"]["batch_size"]
Output = np.zeros(batch_size, sim_specs["out"])
...
Output["f"], persis_info = do_a_simulation(Input["x"], persis_info)
@input_fields(["x"])
@output_data([("f", float)])
def my_simulation(Input, persis_info, sim_specs, libE_info):
batch_size = sim_specs["user"]["batch_size"]
Output = np.zeros(batch_size, sim_specs["out"])
# ...
Output["f"], persis_info = do_a_simulation(Input["x"], persis_info)
return Output, persis_info
return Output, persis_info
Most ``sim_f`` function definitions written by users resemble::

def my_simulation(Input, persis_info, sim_specs, libE_info):

where:

* ``Input`` is a selection of the :ref:`History array<funcguides-history>`
* :ref:`persis_info<datastruct-persis-info>` is a dictionary containing state information
* :ref:`sim_specs<datastruct-sim-specs>` is a dictionary of simulation parameters, including which fields from the History array got sent
* ``libE_info`` is a dictionary containing libEnsemble-specific entries
* ``Input`` is a selection of the :ref:`History array<funcguides-history>`, a NumPy array.
* :ref:`persis_info<datastruct-persis-info>` is a dictionary containing state information.
* :ref:`sim_specs<datastruct-sim-specs>` is a dictionary of simulation parameters.
* ``libE_info`` is a dictionary containing libEnsemble-specific entries.

Valid simulator functions can accept a subset of the above parameters. So a very simple simulator function can start::

def my_simulation(Input):

If sim_specs was initially defined::
If ``sim_specs`` was initially defined:

.. tab-set::

.. tab-item:: Non-decorated function
:sync: nodecorate

.. code-block:: python
sim_specs = SimSpecs(
sim_f=my_simulation,
inputs=["x"],
outputs=["f", float, (1,)],
user={"batch_size": 128},
)
.. tab-item:: Decorated function
:sync: decorate

.. code-block:: python
sim_specs = SimSpecs(
sim_f=my_simulation,
user={"batch_size": 128},
)
sim_specs = {
"sim_f": some_function,
"in": ["x"],
"out:" ["f", float, (1,)],
"user": {
"batch_size": 128
}
}
Then user parameters and a *local* array of outputs may be obtained/initialized like::

Expand All @@ -55,10 +99,9 @@ Then return the array and ``persis_info`` to libEnsemble::

return Output, persis_info

Between the ``Output`` definition and the ``return``, any level and complexity
of computation can be performed. Users are encouraged to use the :doc:`executor<../executor/overview>`
to submit applications to parallel resources if necessary, or plug in components from
other libraries to serve their needs.
Between the ``Output`` definition and the ``return``, any computation can be performed.
Users can try an :doc:`executor<../executor/overview>` to submit applications to parallel
resources, or plug in components from other libraries to serve their needs.

Executor
--------
Expand All @@ -73,7 +116,7 @@ for an additional example to try out.
Persistent Simulators
---------------------

Although comparatively uncommon, simulator functions can also be written
Simulator functions can also be written
in a persistent fashion. See the :ref:`here<persistent-gens>` for a general API overview
of writing persistent generators, since the interface is largely identical. The only
differences are to pass ``EVAL_SIM_TAG`` when instantiating a ``PersistentSupport``
Expand Down
4 changes: 4 additions & 0 deletions libensemble/sim_funcs/one_d_func.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,11 @@

import numpy as np

from libensemble.specs import input_fields, output_data


@input_fields(["x"])
@output_data([("f", float)])
def one_d_example(x, persis_info, sim_specs, _):
"""
Evaluates the six hump camel function for a single point ``x``.
Expand Down
3 changes: 3 additions & 0 deletions libensemble/sim_funcs/six_hump_camel.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,12 @@
import numpy as np

from libensemble.message_numbers import EVAL_SIM_TAG, FINISHED_PERSISTENT_SIM_TAG, PERSIS_STOP, STOP_TAG
from libensemble.specs import input_fields, output_data
from libensemble.tools.persistent_support import PersistentSupport


@input_fields(["x"])
@output_data([("f", float)])
def six_hump_camel(H, persis_info, sim_specs, libE_info):
"""
Evaluates the six hump camel function for a collection of points given in ``H["x"]``.
Expand Down

0 comments on commit ba7fcc4

Please sign in to comment.