Merge pull request #82 from Libensemble/develop

Develop

README.rst

@@ -99,7 +99,7 @@ libensemble/tests/cov_merge/index.html. The Travis CI coverage results are
 given online at
 `Coveralls <https://coveralls.io/github/Libensemble/libensemble?branch=master>`_. 
 
-Note for v0.2.0: The job_controller tests can be run using the direct-launch or
+Note: The job_controller tests can be run using the direct-launch or
 Balsam job controllers. However, currently only the direct-launch versions can
 be run on Travis CI, which reduces the test coverage results.
 
@@ -109,7 +109,7 @@ Basic Usage
 
 The examples directory contains example libEnsemble calling scripts, sim functions, gen functions, alloc functions and job submission scripts.
 
-See the `user-guide <http://libensemble.readthedocs.org>`_ for more information.
+See the `user-guide <https://libensemble.readthedocs.io/en/latest/user_guide.html>`_ for more information.
 
 
 Documentation

docs/conf.py

@@ -89,9 +89,9 @@ def __getattr__(cls, name):
 # built documents.
 #
 # The short X.Y version.
-version = '0.2.0'
+version = '0.3.0'
 # The full version, including alpha/beta/rc tags.
-release = '0.2.0'
+release = '0.3.0'
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.

docs/data_structures/alloc_specs.rst

@@ -4,14 +4,22 @@ alloc_specs
 Allocation function specifications to be set in user calling script and passed to libE.libE()::
 
     alloc_specs: [dict, optional] :
+
+        Required keys :  
+        
         'alloc_f' [func] :
             Default: give_sim_work_first
+            
+        Optional keys :
+        
         'out' [list of tuples] :
             Default: [('allocated',bool)]
-        'batch_mode' [bool] :
-            Default: []
-        'num_inst' [int] :
-            Default: []
-            
-        The 'batch_mode' and 'num_inst' are specific arguments for the allocation function give_sim_work_first
 
+
+:Notes:
+
+* The alloc_specs has the default keys as given above, but may be overidden by the user.
+* The tuples defined in the 'out' list are entered into the master :ref:`history array<datastruct-history-array>`
+
+
+:Examples:

docs/data_structures/exit_criteria.rst

@@ -8,9 +8,9 @@ Exit criteria for libEnsemble::
         Optional keys (At least one must be given) :
         
         'sim_max' [int] : 
-            Stop after this many sim_f evaluations have been completed
+            Stop after this many points have been evaluated (by sim_f) in current run
         'gen_max' [int] : 
-            Stop after this many points have been generated by gen_f
+            Stop after this many points have been generated by gen_f in current run
         'elapsed_wallclock_time' [float] : 
             Stop after this amount of seconds have elapsed (since the libEnsemble manager has been initialized)
         'stop_val' [(str,float)] : 

docs/data_structures/gen_specs.rst

@@ -19,20 +19,35 @@ Generation function specifications to be set in user calling script and passed t
         Optional keys :
     
         'save_every_k' [int] :
-            Save history array every k steps
+            Save history array to file after every k generated points.
+
+:Notes:
+
+* The user may define other fields to be passed to the generator function.
+* The tuples defined in the 'out' list are entered into the master :ref:`history array<datastruct-history-array>`
+* The generator 'out' field will generally include a variable(s) which is used for the simulator 'in' field,
+  in which case only the variable name is required for the simulator 'in' field.  E.g. The
+  **test_6-hump_camel_uniform_sampling.py** example below, matches the corresponding
+  :ref:`sim_specs example<sim-specs-exmple1>`, where 'x' is defined in the gen_specs 'out' field to give
+  two positional floats.
+
 
 :Examples:
 
+.. _gen-specs-exmple1:
+
 From: libensemble/tests/regression_tests/test_6-hump_camel_uniform_sampling.py::
 
     gen_specs = {'gen_f': uniform_random_sample,
                  'in': ['sim_id'],
-                 'out': [('x',float,2),
-                        ],
+                 'out': [('x',float,2)],
                  'lb': np.array([-3,-2]),
                  'ub': np.array([ 3, 2]),
                  'gen_batch_size': 500,
-                 'batch_mode': True,
-                 'num_inst':1,
                  'save_every_k': 300
                  }
+
+In this example, the generation function *uniform_random_sample* will generate 500 random points
+uniformly over the 2D domain defined by ``gen_specs['ub']`` and ``gen_specs['lb']``.
+The libEnsemble manager is set to dump the history array to file after every 300 generated points,
+though in this case it will only happen after 500 points due to the batch size.

docs/data_structures/history_array.rst

@@ -8,9 +8,9 @@ Stores the history of the output from gen_f, sim_f, and alloc_f::
     H: numpy structured array
         History array storing rows for each point. 
 
-Fields in ``H`` are those in specified ``sim_specs['out']``,
-``gen_specs['out']``, and ``alloc_specs['out']``. All values are intitialed to
-0 for integers, 0.0 for floats, and False for booleans. 
+Fields in ``H`` include those specified in ``sim_specs['out']``,
+``gen_specs['out']``, and ``alloc_specs['out']``. All values are initiated to
+0 for integers, 0.0 for floats, and False for booleans.
 
 
 

docs/data_structures/libE_specs.rst

@@ -10,10 +10,5 @@ Specifications for libEnsemble::
             libEnsemble communicator. Default: MPI.COMM_WORLD
         'color' [int] :
             Communicator color. Default: 0
-        'manager_ranks' [set] :
-            Default: [0]
-        'worker_ranks' [set] :
-            Default: [1 to comm.Get_size()-1]
         'queue_update_function' [func] :
             Default: []
-            

docs/data_structures/persis_info.rst

@@ -8,7 +8,12 @@ Supply persistent information to libEnsemble::
     persis_info: [dict] :
         Dictionary containing persistent info
 
-If worker ``i`` is sends back ``persis_info``, it is stored in ``persis_info[i]``. This functionality can be used to, for example, pass a random stream back to the manager to be included in future work from the allocation function. 
+Holds data that is passed to and from workers updating some state information. A typical example
+is a randon number generator to be used in consecutive calls to a generator.
+
+If worker ``i`` sends back ``persis_info``, it is stored in ``persis_info[i]``. This functionality
+can be used to, for example, pass a random stream back to the manager to be included in future work
+from the allocation function. 
 
 :Examples:
 

docs/data_structures/sim_specs.rst

@@ -21,17 +21,23 @@ Simulation function specifications to be set in user calling script and passed t
         Optional keys :
         
         'save_every_k' [int] :
-            Save history array every k steps
+            Save history array to file after every k simulated points.
         'sim_dir' [str] :
             Name of simulation directory which will be copied for each worker
         'sim_dir_prefix' [str] :
             A prefix path specifying where to create sim directories
         
         Additional entires in sim_specs will be given to sim_f
         
-        
+:Notes:
+
+* The user may define other fields to be passed to the simulator function.
+* The tuples defined in the 'out' list are entered into the master :ref:`history array<datastruct-history-array>`
+
 :Examples:
 
+.. _sim-specs-exmple1:
+
 From: libensemble/tests/regression_tests/test_6-hump_camel_uniform_sampling.py::
 
     sim_specs = {'sim_f': six_hump_camel, # This is the function whose output is being minimized
@@ -41,3 +47,6 @@ From: libensemble/tests/regression_tests/test_6-hump_camel_uniform_sampling.py::
                  'save_every_k': 400  
                  }
 
+Note that the dimensions and type of the 'in' field variable 'x' is specified by the corresponding
+generator 'out' field 'x' (see :ref:`gen_specs example<gen-specs-exmple1>`).
+Only the variable name is then required in the sim_specs.

docs/job_controller/job_controller.rst

@@ -31,12 +31,12 @@ Job Class
 ---------
 
 Jobs are created and returned though the job_controller launch function. Jobs can be passed as arguments
-to the job_controller poll and kill functions. Job information can be queired through the job attributes below and the query funcitons. Note that the job attributes are only updated when they are polled (or though other
+to the job_controller poll and kill functions. Job information can be queired through the job attributes below and the query functions. Note that the job attributes are only updated when they are polled (or though other
 job controller functions).
 
 .. autoclass:: Job
   :member-order: bysource
-  :members: workdir_exists, file_exists_in_workdir, read_file_in_workdir, stdout_exists, read_stdout
+  :members: workdir_exists, file_exists_in_workdir, read_file_in_workdir, stdout_exists, read_stdout, stderr_exists, read_stderr
 
 .. autoclass:: BalsamJob
   :show-inheritance:
@@ -50,7 +50,7 @@ Job Attributes
 
 Following is a list of job status and configuration attributes that can be retrieved from a job.
 
-:NOTE: These should not be set directly. Jobs are launched by the job controller and job information can be queired through the job attributes below and the query funcitons. 
+:NOTE: These should not be set directly. Jobs are launched by the job controller and job information can be queired through the job attributes below and the query functions. 
 
 Job Status attributes include:
 
@@ -60,7 +60,6 @@ Job Status attributes include:
 :job.errcode: (int) The errorcode/return code used by the underlying process manager
 :job.finished: (Boolean) True means job has finished running - not whether was successful
 :job.success: (Boolean) Did job complete succesfully (e.g. returncode is zero)
-:job.manager_signal: (String) Contains any signals received by manager. ('none'|'finish'|'kill')
 
 Run configuration attributes - Some will be auto-generated:
 

docs/job_controller/overview.rst

@@ -5,7 +5,7 @@ The Job Controller module can be used by the worker or user-side code to issue a
 
 At the top-level calling script, a registry and job_controller are created and the executable gen or sim applications are registered to these (these are applications that will be runnable parallel jobs). If an alternative job_controller, such as Balsam, is to be used, then these can be created as in the example. Once in the user-side worker code (sim/gen func), the job_controller can be retrieved without any need to specify the type.
 
-**Example usage (code runnable with or without Balsam backend):**
+**Example usage (code runnable with or without a Balsam backend):**
 
 In calling function::
 
@@ -35,8 +35,8 @@ In user sim func::
         time.sleep(delay)
         
         # Has manager sent a finish signal
-        jobctl.manager_poll(job)
-        if job.manager_signal == 'finish':
+        jobctl.manager_poll()
+        if jobctl.manager_signal == 'finish':
             jobctl.kill(job)        
         
         # Poll job to see if completed

docs/release_notes.rst

@@ -3,6 +3,28 @@ Release Notes
 =============
 
 
+Release 0.3.0
+-------------
+
+:Date: September 7, 2018
+
+* Issues with killing jobs have been fixed (#21)
+* Fix to job_controller manager_poll to work with multiple jobs (#62)
+* API change: persis_info now included as an argument to libE and is returned from libE instead of gen_info
+* Gen funcs: aposmm_logic module renamed to aposmm.
+* New example gen and allocation functions.
+* Updated Balsam launch script (with new Balsam workflow)
+* History is dumped to file on manager or worker exception and MPI aborted (with exit code 1) (#46)
+* Default logging level changed to DEBUG and redirected to file ensemble.log
+* Added directory of standalone tests (comms, job kills, and nested MPI launches)
+* Improved and speeded up unit tests (#68)
+* Considerable documentation enhancements
+
+:Known issues:
+
+* OpenMPI is not supported with direct MPI launches as nested MPI launches are not supported.
+
+
 Release 0.2.0
 -------------
 

docs/user_guide.rst

@@ -14,10 +14,10 @@ three routines:
 * alloc_f: Decides whether sim_f or gen_f should be called (and with what input/resources) as workers become available.
 
 Example sim_f, gen_f, and alloc_f routines can be found in the
-examples/sim_funcs, examples/gen_funcs, and examples/alloc_funcs directories,
-respectively. Examples of scripts used for calling libEnsemble can be found at
-examples/calling_scripts/. To enable portability, a job_controller interface is
-supplied for users to launch and monitor jobs in their user-provided sim_f and
+``examples/sim_funcs/``, ``examples/gen_funcs/``, and ``examples/alloc_funcs/`` directories,
+respectively. Examples of scripts used for calling libEnsemble can be found in
+``examples/calling_scripts/``. To enable portability, a :doc:`job_controller<job_controller/overview>` 
+interface is supplied for users to launch and monitor jobs in their user-provided sim_f and
 gen_f routines.
 
 The default alloc_f tells each available worker to call sim_f with the highest
@@ -53,7 +53,7 @@ to support) and plan to have examples of:
   intermediate quantities in order to stop related calculations and preempt
   future calculations associated with poor parameter values.
 
-* A user has a simulation sim_f with multiple fidelities, with the
+* A user has a sim_f with multiple fidelities, with the
   higher-fidelity evaluations requiring more computational resources, and a
   gen_f/alloc_f that decides which parameters should be evaluated and at what
   fidelity level. libEnsemble can coordinate these evaluations without
@@ -64,7 +64,7 @@ to support) and plan to have examples of:
   use the points from the APOSMM gen_f to identify optima; and after a point is
   ruled to be an optimum, a different gen_f can produce a collection of
   parameters necessary for sensitivity analysis of sim_f.
-  
+
 
 Naturally, combinations of these use cases are supported as well. An example of
 such a combination is using libEnsemble to solve an optimization problem that
@@ -79,13 +79,10 @@ corresponding sim_f output. Similarly, gen_f and sim_f are expected to return
 output in numpy structured arrays. The names of the fields to be given as input
 to gen_f and sim_f must be an output from gen_f or sim_f. In addition to the
 fields output from sim_f and gen_f, the final history returned from libEnsemble
-will include the fields. (Note that the libEnsemble history can contain
-pointers to data instead of the data itself. In some applications, this can
-greatly the size of the history and reduce the amount of data communicated
-to/from the manager.):
+will include the following fields:
 
 * sim_id' [int]: Each unit of work output from gen_f must have an associated
-  sim_id. The generator can assign this, but users must careful to ensure
+  sim_id. The generator can assign this, but users must be careful to ensure
   points are added in order. For example, if alloc_f allows for two gen_f
   instances to be running simultaneously, alloc_f should ensure that both don’t
   generate points with the same sim_id.
@@ -103,4 +100,19 @@ to/from the manager.):
 * returned' [bool]: Has this worker completed the evaluation of this unit of
   work?
 
-* paused' [bool]: Has this evaluation been paused?
+
+LibEnsemble Output
+------------------
+
+The history array is returned to the user by libEnsemble. In the case that libEnsemble
+aborts on an exception, the existing history array is dumped to a file libE_history_at_abort_<sim_count>.npy, where sim_count is the number of points evaluated.
+
+Other libEnsemble files produced by default are:
+
+**libe_summary.txt**: This contains a one-line summary of all user calculations. During run-time
+a summary file is produced for each worker in directory named *libe_stat_files*. At then end of 
+the run these are concatenated into *libe_summary.txt*. If a run aborts, the *libe_stat_files* directory
+will remain.
+
+**ensemble.log**: This is the logging output from libEnsemble. The default logging is DEBUG level, as
+this can provide useful diagnostics. If this file is not removed, multiple runs will append output.