Feature/improved libE field naming

docs/data_structures/history_array.rst

@@ -33,8 +33,8 @@ run with the generated ``x`` and ``theta`` values.
 For ease, the ``gen_f`` and ``sim_f`` are expected to return output as NumPy
 structured arrays. The output from the ``gen_f`` or ``sim_f`` functions can be
 the exact elements declared in the gen/sim_specs['out'] lists, or can be a
-subset of these lists. (The manager's history array will update any fields given
-back to it.)
+subset of these lists. (The manager's history array will update any fields 
+returned to it.)
 
 The names of the input fields for ``gen_f`` and ``sim_f`` must exist in the manager's
 history array (i.e.,~ they must be output fields from ``gen_f`` or ``sim_f`` or
@@ -57,12 +57,13 @@ The reserved boolean field ``cancel_requested`` can also be set in a user
 function to request that libEnsemble cancels evaluation of the point.
 
 The remaining reserved fields are protected (populated by libEnsemble), and
-store information about each entry. These include boolean fields for the current
-scheduling status of the point (``given`` out for evaluation, ``returned`` from
-evaluation, and ``given_back`` to the generator). Timing fields give the time
-(since the epoch) corresponding to each state, and when the point was generated.
-Other protected fields include the worker IDs on which points were generated or
-evaluated.
+store information about each entry. These include boolean fields for the
+current scheduling status of the point (``sim_started`` when the sim evaluation
+has started out, ``sim_ended`` when sim evaluation has completed, and
+``gen_informed`` when the sim output has been passed back to the generator).
+Timing fields give the time (since the epoch) corresponding to each state, and
+when the point was generated. Other protected fields include the worker IDs on
+which points were generated or evaluated.
 
 The user fields and the reserved fields together make up the final history array
 returned by libEnsemble.
@@ -82,37 +83,27 @@ The full list of these reserved fields is given below.
 
 :*Protected fields*:
 
-* ``given`` [bool]: True if this ``gen_f`` output been given to a libEnsemble
-  worker to be evaluated by a ``sim_f``.
+``gen_worker`` [int]: Worker that generated this entry
 
-* ``given_time`` [float]: Time this entry
-  was *first* given to a worker to be evaluated by a ``sim_f``.
+``gen_started_time`` [ float]: Time gen_worker was initiated that produced this entry
 
-* ``last_given_time`` [float]: Time this entry
-  was *last* given to a worker to be evaluated by a ``sim_f``.
+``gen_ended_time`` [ float]: Time gen_worker requested this entry
 
-* ``returned`` [bool]: True if this entry has been evaluated by a ``sim_f``.
+``sim_worker`` [int]: Worker that did (or is doing) the sim evaluation for this entry
 
-* ``returned_time`` [float]: Time this entry was *last* returned from a ``sim_f``.
+``sim_started`` [bool]: True if entry was given to sim_worker for sim evaluation
 
-* ``given_back`` [bool]: True if this ``gen_f`` output been given back to a ``gen_f``
-  worker after being returned from evaluation.
+``sim_started_time`` [float]: Time entry was given to sim_worker for a sim evaluation
 
-.. * ``given_back_time`` [float]: Time this entry
-..   was *first* given back to a ``gen_f`` worker.
+``sim_ended`` [bool]: True if entry's sim evaluation completed
 
-* ``last_given_back_time`` [float]: Time this entry
-  was *last* given back to a ``gen_f`` worker.
+``sim_ended_time`` [float]: Time entry's sim evaluation completed
 
-* ``sim_worker`` [int]: libEnsemble worker that performed the ``sim_f`` evaluation.
+``gen_informed`` [bool]: True if gen_worker was informed about the sim evaluation of this entry
 
-* ``gen_worker`` [int]: libEnsemble worker that generated this entry.
+``gen_informed_time`` [float]: Time gen_worker was informed about the sim evaluation of this entry
 
-* ``gen_time`` [float]: Time this entry was put into ``H`` by the manager.
-
-* ``last_gen_time`` [float]: Time this entry was last requested by a ``gen_f``.
-
-* ``kill_sent`` [bool]: True if a kill signal has been sent to the worker evaluating this entry.
+``kill_sent`` [bool]: True if a kill signal was sent to worker for this entry
 
 Other than ``'sim_id'`` and ``cancel_requested``, reserved fields cannot be
 overwritten by user functions unless ``libE_specs['safe_mode']`` is set to ``False``.
@@ -128,7 +119,7 @@ Example workflow updating history array
 The history array is initialized using the libEnsemble reserved field and the
 user-provided ``gen_specs['out']`` and ``sim_specs['out']`` entries.
 In the figure below, only the
-reserved fields: ``sim_id``, ``given``, and ``returned`` are shown for brevity.
+reserved fields: ``sim_id``, ``sim_started``, and ``sim_ended`` are shown for brevity.
 
     .. figure:: ../images/history_init.png
        :scale: 40

docs/function_guides/allocator.rst

@@ -96,9 +96,9 @@ routine can be found in ``libE_info``, passed into the allocation function::
     libE_info =  {'exit_criteria': dict,               # Criteria for ending routine
                   'elapsed_time': float,               # Time elapsed since start of routine
                   'manager_kill_canceled_sims': bool,  # True if manager is to send kills to cancelled simulations
-                  'given_count': int,                  # Total number of points given for simulation function evaluation
-                  'returned_count': int,               # Total number of points returned from simulation function evaluations
-                  'given_back_count': int,             # Total number of evaluated points given back to a generator function
+                  'sim_started_count': int,              # Total number of points given for simulation function evaluation
+                  'sim_ended_count': int,                # Total number of points returned from simulation function evaluations
+                  'gen_informed_count': int,           # Total number of evaluated points given back to a generator function
                   'sim_max_given': bool,               # True if `sim_max` simulations have been given out to workers
                   'use_resource_sets': bool}           # True if num_resource_sets has been explicitly set.
 
@@ -113,7 +113,7 @@ allocation function and detect impending timeouts, then pack up cleanup work req
 or mark points for cancellation.
 
 The remaining values above are useful for efficient filtering of H values
-(e.g. ``returned_count``), saves a filtering an entire column of H.
+(e.g. ``sim_ended_count``), saves a filtering an entire column of H.
 
 .. note:: An error occurs when the ``alloc_f`` returns nothing while
           all workers are idle

docs/tutorials/calib_cancel_tutorial.rst

@@ -186,9 +186,9 @@ prepared for irregular sending /receiving of data.
 .. Between routines to call the allocation function and distribute allocated work
 .. to each Worker, the Manager selects points from the History array that are:
 ..
-..     1) Marked as ``'given'`` by the allocation function
+..     1) Marked as ``'sim_started'`` by the allocation function
 ..     2) Marked with ``'cancel_requested'`` by the generator
-..     3) *Not* been marked as ``'returned'`` by the Manager
+..     3) *Not* been marked as ``'sim_ended'`` by the Manager
 ..     4) *Not* been marked with ``'kill_sent'`` by the Manager
 ..
 .. If any points match these characteristics, the Workers that are processing these
@@ -198,7 +198,7 @@ prepared for irregular sending /receiving of data.
 .. function calls, any points in the Manager's History array that have ``'cancel_requested'``
 .. as ``True`` are not allocated::
 ..
-..     task_avail = ~H['given'] & ~H['cancel_requested']
+..     task_avail = ~H['sim_started'] & ~H['cancel_requested']
 ..
 .. This ``alloc_f`` also can prioritize allocating points that have
 .. higher ``'priority'`` values from the ``gen_f`` values in the local History array::

docs/tutorials/local_sine_tutorial.rst

@@ -292,7 +292,7 @@ columns might be rearranged).
 
 .. code-block::
 
-  ['y', 'given_time', 'gen_worker', 'sim_worker', 'given', 'returned', 'x', 'allocated', 'sim_id', 'gen_time']
+  ['y', 'sim_started_time', 'gen_worker', 'sim_worker', 'sim_started', 'sim_ended', 'x', 'allocated', 'sim_id', 'gen_ended_time']
   [(-0.37466051, 1.559+09, 2, 2,  True,  True, [-0.38403059],  True,  0, 1.559+09)
   (-0.29279634, 1.559+09, 2, 3,  True,  True, [-2.84444261],  True,  1, 1.559+09)
   ( 0.29358492, 1.559+09, 2, 4,  True,  True, [ 0.29797487],  True,  2, 1.559+09)

libensemble/alloc_funcs/fast_alloc_and_pausing.py

@@ -78,7 +78,7 @@ def give_sim_work_first(W, H, sim_specs, gen_specs, alloc_specs, persis_info, li
                     if 'local_pt' in H.dtype.names and H['local_pt'][a1][0]:
                         persis_info['local_pt_ids'].add(pt_id)
 
-                    if np.all(H['returned'][a1]):
+                    if np.all(H['sim_ended'][a1]):
                         persis_info['complete'].add(pt_id)
                         values = gen_specs['user']['combine_component_func'](H['f_i'][a1])
                         persis_info['best_complete_val'] = min(persis_info['best_complete_val'], values)
@@ -124,13 +124,13 @@ def give_sim_work_first(W, H, sim_specs, gen_specs, alloc_specs, persis_info, li
             if len(H):
                 # Don't give gen instances in batch mode if points are unfinished
                 if (alloc_specs['user'].get('batch_mode')
-                        and not all(np.logical_or(H['returned'][last_size:], H['paused'][last_size:]))):
+                        and not all(np.logical_or(H['sim_ended'][last_size:], H['paused'][last_size:]))):
                     break
                 # Don't call APOSMM if there are runs going but none need advancing
                 if len(persis_info[lw]['run_order']):
                     runs_needing_to_advance = np.zeros(len(persis_info[lw]['run_order']), dtype=bool)
                     for run, inds in enumerate(persis_info[lw]['run_order'].values()):
-                        runs_needing_to_advance[run] = np.all(H['returned'][inds])
+                        runs_needing_to_advance[run] = np.all(H['sim_ended'][inds])
 
                     if not np.any(runs_needing_to_advance):
                         break

libensemble/alloc_funcs/fast_alloc_to_aposmm.py

@@ -49,13 +49,13 @@ def give_sim_work_first(W, H, sim_specs, gen_specs, alloc_specs, persis_info, li
 
             if len(H):
                 # Don't give gen instances in batch mode if points are unfinished
-                if user.get('batch_mode') and not support.all_returned(pt_filter=~H['paused'], low_bound=last_size):
+                if user.get('batch_mode') and not support.all_sim_ended(pt_filter=~H['paused'], low_bound=last_size):
                     break
                 # Don't call APOSMM if there are runs going but none need advancing
                 if len(persis_info[lw]['run_order']):
                     runs_needing_to_advance = np.zeros(len(persis_info[lw]['run_order']), dtype=bool)
                     for run, inds in enumerate(persis_info[lw]['run_order'].values()):
-                        runs_needing_to_advance[run] = H['returned'][inds[-1]]
+                        runs_needing_to_advance[run] = H['sim_ended'][inds[-1]]
 
                     if not np.any(runs_needing_to_advance):
                         break

libensemble/alloc_funcs/give_sim_work_first.py

@@ -33,8 +33,8 @@ def give_sim_work_first(W, H, sim_specs, gen_specs, alloc_specs, persis_info, li
 
     if 'cancel_sims_time' in user:
         # Cancel simulations that are taking too long
-        rows = np.where(np.logical_and.reduce((H['given'], ~H['returned'], ~H['cancel_requested'])))[0]
-        inds = time.time() - H['last_given_time'][rows] > user['cancel_sims_time']
+        rows = np.where(np.logical_and.reduce((H['sim_started'], ~H['sim_ended'], ~H['cancel_requested'])))[0]
+        inds = time.time() - H['sim_started_time'][rows] > user['cancel_sims_time']
         to_request_cancel = rows[inds]
         for row in to_request_cancel:
             H[row]['cancel_requested'] = True
@@ -52,7 +52,7 @@ def give_sim_work_first(W, H, sim_specs, gen_specs, alloc_specs, persis_info, li
     gen_count = support.count_gens()
     Work = {}
 
-    points_to_evaluate = ~H['given'] & ~H['cancel_requested']
+    points_to_evaluate = ~H['sim_started'] & ~H['cancel_requested']
     for wid in support.avail_worker_ids():
 
         if np.any(points_to_evaluate):
@@ -69,7 +69,7 @@ def give_sim_work_first(W, H, sim_specs, gen_specs, alloc_specs, persis_info, li
                 break
 
             # Do not start gen instances in batch mode if workers still working
-            if user.get('batch_mode') and not support.all_returned(H):
+            if user.get('batch_mode') and not support.all_sim_ended(H):
                 break
 
             # Give gen work

libensemble/alloc_funcs/inverse_bayes_allocf.py

@@ -31,7 +31,7 @@ def only_persistent_gens_for_inverse_bayes(W, H, sim_specs, gen_specs, alloc_spe
 
         # if > 1 persistent generator, assign the correct work to it
         inds_generated_by_wid = (H['gen_worker'] == wid)
-        if support.all_returned(H, inds_generated_by_wid):
+        if support.all_sim_ended(H, inds_generated_by_wid):
 
             # Has sim_f completed everything from this persistent worker?
             # Then give back everything in the last batch
@@ -45,7 +45,7 @@ def only_persistent_gens_for_inverse_bayes(W, H, sim_specs, gen_specs, alloc_spe
 
             Work[wid] = support.gen_work(wid, ['like'], inds_to_send_back, persis_info.get(wid), persistent=True)
 
-    points_to_evaluate = ~H['given'] & ~H['cancel_requested']
+    points_to_evaluate = ~H['sim_started'] & ~H['cancel_requested']
     for wid in support.avail_worker_ids(persistent=False):
         if np.any(points_to_evaluate):
 

libensemble/alloc_funcs/only_one_gen_alloc.py

@@ -38,7 +38,7 @@ def ensure_one_active_gen(W, H, sim_specs, gen_specs, alloc_specs, persis_info,
 
         elif not support.test_any_gen() and gen_flag:
 
-            if not support.all_returned(H):
+            if not support.all_sim_ended(H):
                 break
 
             # Give gen work

libensemble/alloc_funcs/persistent_aposmm_alloc.py

@@ -31,9 +31,9 @@ def persistent_aposmm_alloc(W, H, sim_specs, gen_specs, alloc_specs, persis_info
     Work = {}
 
     if persis_info.get('first_call', True):
-        assert support.all_given(H), "Initial points in H have never been given."
-        assert support.all_returned(H), "Initial points in H have never been returned."
-        assert support.all_given_back(H), "Initial points in H have never been given back to gen."
+        assert support.all_sim_started(H), "Initial points in H have never been given."
+        assert support.all_sim_ended(H), "Initial points in H have never been returned."
+        assert support.all_gen_informed(H), "Initial points in H have never been given back to gen."
 
         persis_info['samples_in_H0'] = sum(H['local_pt'] == 0)
         persis_info['next_to_give'] = len(H)  #
@@ -44,11 +44,11 @@ def persistent_aposmm_alloc(W, H, sim_specs, gen_specs, alloc_specs, persis_info
 
     # If any persistent worker's calculated values have returned, give them back.
     for wid in support.avail_worker_ids(persistent=EVAL_GEN_TAG):
-        if (persis_info.get('sample_done') or sum(H['returned']) >= init_sample_size + persis_info['samples_in_H0']):
+        if (persis_info.get('sample_done') or sum(H['sim_ended']) >= init_sample_size + persis_info['samples_in_H0']):
             # Don't return if the initial sample is not complete
             persis_info['sample_done'] = True
 
-            returned_but_not_given = np.logical_and(H['returned'], ~H['given_back'])
+            returned_but_not_given = np.logical_and(H['sim_ended'], ~H['gen_informed'])
             if np.any(returned_but_not_given):
                 point_ids = np.where(returned_but_not_given)[0]
                 Work[wid] = support.gen_work(wid, gen_specs['persis_in'], point_ids, persis_info.get(wid),

libensemble/alloc_funcs/start_fd_persistent.py

@@ -36,21 +36,21 @@ def finite_diff_alloc(W, H, sim_specs, gen_specs, alloc_specs, persis_info, libE
 
         # What (x_ind, f_ind) pairs have all of the evaluation of all n_ind
         # values complete.
-        inds_not_sent_back = ~H['given_back']
+        inds_not_sent_back = ~H['gen_informed']
         H_tmp = H[inds_not_sent_back]
 
         inds_to_send = np.array([], dtype=int)
         for x_ind in range(gen_specs['user']['n']):
             for f_ind in range(gen_specs['user']['p']):
-                inds = np.logical_and.reduce((H_tmp['x_ind'] == x_ind, H_tmp['f_ind'] == f_ind, H_tmp['returned']))
+                inds = np.logical_and.reduce((H_tmp['x_ind'] == x_ind, H_tmp['f_ind'] == f_ind, H_tmp['sim_ended']))
                 if sum(inds) == gen_specs['user']['nf']:
                     inds_to_send = np.append(inds_to_send, H_tmp['sim_id'][inds])
 
         if len(inds_to_send):
             Work[wid] = support.gen_work(wid, gen_specs['persis_in'], inds_to_send, persis_info.get(wid),
                                          persistent=True)
 
-    points_to_evaluate = ~H['given'] & ~H['cancel_requested']
+    points_to_evaluate = ~H['sim_started'] & ~H['cancel_requested']
     for wid in support.avail_worker_ids(persistent=False):
         if np.any(points_to_evaluate):
             # perform sim evaluations (if they exist in History).