Libensemble · shuds13 · Apr 11, 2025 · Apr 7, 2025 · Apr 7, 2025 · Apr 7, 2025
diff --git a/.codecov.yml b/.codecov.yml
@@ -3,5 +3,4 @@ ignore:
   - "libensemble/tools/forkable_pdb.py"
   - "libensemble/tools/live_data/*"
   - "libensemble/sim_funcs/executor_hworld.py"
-  - "libensemble/gen_funcs/persistent_ax_multitask.py"
-  - "libensemble/gen_funcs/persistent_gpCAM.py"
+  - "libensemble/gen_funcs/persistent_tasmanian.py"
diff --git a/.wci.yml b/.wci.yml
@@ -16,8 +16,8 @@ description: |
 language: Python
 
 release:
-  version: 1.4.3
-  date: 2024-12-16
+  version: 1.5.0
+  date: 2025-04-10
 
 documentation:
   general: https://libensemble.readthedocs.io

diff --git a/CHANGELOG.rst b/CHANGELOG.rst
@@ -8,6 +8,46 @@ GitHub issues are referenced, and can be viewed with hyperlinks on the `github r
 
 .. _`github releases page`: https://github.com/Libensemble/libensemble/releases
 
+Release 1.5.0
+--------------
+
+:Date: Apr 10, 2025
+
+General Updates:
+
+* Migrate package build system to `pyproject.toml` (with `pixi` support). #1459
+* Improve handling when no MPI found. #1514
+* `ensemble.save_output()` can save without appending attributes `append_attrs=False`.  #1531
+* Improve handling of worker-specific `persis_info` fields when they are not initially provided. #1531
+  * Bugfix: Fix `final_gen_send` when there are no worker-specific `persis_info` fields.
+  * Handle worker-generated `persis_info` fields.
+  * Ensure `persis_info` is initialized to an empty dictionary in user functions instead of `None`.
+
+Examples:
+
+* Update Ax generator for `Ax v0.5.0`. #1508
+* Rename gpCAM generators. #1516
+  * `persistent_gpCAM_ask_tell` to `persistent_gpCAM`
+  * `persistent_gpCAM_simple` to `persistent_gpCAM_covar` (in fact less simple)
+* Persistent generators return `None` as first return value unless `H_o` is updated. #1515
+* Add LUMI to known platforms. #1546
+
+Documentation:
+
+* Revamp Examples and HPC section of documentation. #1501, #1536, #1539
+* Add tutorial and notebook demonstrating surrogate model creation with gpCAM. #1531
+* Update Aurora guide. #1510
+* Update and documented APOSMM/WarpX example. #1543
+
+:Note:
+
+* Tests were run on Linux and MacOS with Python versions 3.10, 3.11, 3.12, 3.13
+* Heterogeneous workflows tested on Aurora (ALCF), Polaris (ALCF), LUMI (EuroHPC JU), and Perlmutter (NERSC).
+
+:Known Issues:
+
+* See known issues section in the documentation.
+
 Release 1.4.3
 --------------
 

diff --git a/LICENSE b/LICENSE
@@ -1,6 +1,6 @@
 BSD 3-Clause License
 
-Copyright (c) 2018-2024, UChicago Argonne, LLC and the libEnsemble Development Team
+Copyright (c) 2018-2025, UChicago Argonne, LLC and the libEnsemble Development Team
 All Rights Reserved.
 
 Redistribution and use in source and binary forms, with or without

diff --git a/docs/advanced_installation.rst b/docs/advanced_installation.rst
@@ -9,7 +9,7 @@ automatically installed alongside libEnsemble:
 * Python_       ``>= 3.10``
 * NumPy_        ``>= 1.21``
 * psutil_       ``>= 5.9.4``
-* `pydantic`_   ``<= 1.10.12``
+* `pydantic`_   ``>= 1.10.12``
 * pyyaml_       ``>= v6.0``
 * tomli_        ``>= 1.2.1``
 

diff --git a/docs/examples/sim_funcs.rst b/docs/examples/sim_funcs.rst
@@ -24,7 +24,7 @@ Ideal for simple debugging of generator processes or system testing.
    Borehole function with kills <sim_funcs/borehole_kills>
    Chwirut1 vector-valued function <sim_funcs/chwirut1>
    Inverse Bayesian likelihood <sim_funcs/inverse_bayes>
-   Norm  <sim_funcs/simple_sim>
+   Norm <sim_funcs/simple_sim>
    Rosenbrock test optimization function <sim_funcs/rosenbrock>
    Six Hump Camel <sim_funcs/six_hump_camel>
    Test noisy function <sim_funcs/noisy_vector_mapping>
@@ -36,8 +36,8 @@ Functions that run user applications
 These use the executor to launch applications and in some cases
 handle dynamic CPU/GPU allocation.
 
-The ``Variable resources`` module contains basic examples, while the ``Template`` 
-examples use a simple  MPI/OpenMP (with GPU offload option) application (``forces``) 
+The ``Variable resources`` module contains basic examples, while the ``Template``
+examples use a simple MPI/OpenMP (with GPU offload option) application (``forces``)
 to demonstrate libEnsemble’s capabilities on various HPC systems. The
 build_forces.sh_ file gives compile lines for building the simple ``forces``
 application on various platforms (use -DGPU to build for GPU).

diff --git a/docs/examples/sim_funcs/forces_simf_gpu.rst b/docs/examples/sim_funcs/forces_simf_gpu.rst
@@ -1,4 +1,4 @@
-Template for GPU executables 
+Template for GPU executables
 ----------------------------
 
 .. role:: underline

diff --git a/docs/examples/sim_funcs/forces_simf_gpu_multi_app.rst b/docs/examples/sim_funcs/forces_simf_gpu_multi_app.rst
@@ -10,7 +10,7 @@ ranks and GPU resources as requested by the generator.
 This makes efficient use of each node as the expensive GPU simulations will use the GPUs on
 the node/s, while the rest of the CPU cores are assigned to the simple CPU-only simulations.
 
-For a realistic use-case see https://journals.aps.org/prab/abstract/10.1103/PhysRevAccelBeams.26.084601
+See this publication_ for a real-world demonstration of these capabilities.
 
 .. automodule:: forces_multi_app.forces_simf
    :members:
@@ -39,5 +39,6 @@ up by each worker and these will be used when the simulation is run, unless over
 More information is available in the :doc:`Forces GPU tutorial <../../tutorials/forces_gpu_tutorial>`
 and the video_ demonstration on Frontier_.
 
-.. _video: https://www.youtube.com/watch?v=H2fmbZ6DnVc
 .. _Frontier: https://docs.olcf.ornl.gov/systems/frontier_user_guide.html
+.. _publication: https://doi.org/10.1103/PhysRevAccelBeams.26.084601
+.. _video: https://www.youtube.com/watch?v=H2fmbZ6DnVc
diff --git a/docs/examples/submission_scripts.rst b/docs/examples/submission_scripts.rst
@@ -1 +1,5 @@
 .. include:: ../platforms/example_scripts.rst
+    :end-before: .. _slurm_mpi_distributed:
+
+.. include:: ../platforms/example_scripts.rst
+    :start-after: .. _slurm_mpi_distributed:
diff --git a/docs/platforms/example_scripts.rst b/docs/platforms/example_scripts.rst
@@ -7,18 +7,17 @@ for more information about the respective systems and configuration.
 
 .. note::
     It is **highly recommended** that the directive lines (e.g., #SBATCH) in batch
-    submission scripts do **NOT** specify  processor, task, or GPU configuration info
-    --- these lines should only specify the number of nodes required.
+    submission scripts do **NOT** specify processor, task, or GPU configuration
+    information---these lines should only specify the number of nodes required.
 
     For example, do not specify ``#SBATCH --gpus-per-node=4`` in order to use four
     GPUs on the node, when each worker may use less than this, as this may assign
-    all of the GPUs to a single MPI invocation. Instead,  the configuration should
+    all of the GPUs to a single MPI invocation. Instead, the configuration should
     be supplied either
     :doc:`in the simulation function<../examples/sim_funcs/forces_simf_gpu>`
     or, if using dynamic resources,
     :doc:`in the generator<../examples/sim_funcs/forces_simf_gpu_vary_resources>`.
 
-
 General examples
 ----------------
 
@@ -43,7 +42,6 @@ LSF - Basic
     :caption: /examples/libE_submission_scripts/submit_lsf_simple.sh
     :language: bash
 
-
 System Examples
 ---------------
 

diff --git a/docs/platforms/platforms_index.rst b/docs/platforms/platforms_index.rst
@@ -46,14 +46,12 @@ which runs the generator on the manager (using a thread) as below.
 
        A SLURM batch script may include:
 
-
        .. code-block:: bash
 
           #SBATCH --nodes 3
 
           python run_libe_forces.py --nworkers 3
 
-
 When using **gen_on_manager**, set ``nworkers`` to the number of workers desired for running simulations.
 
 Dedicated Mode
@@ -64,7 +62,6 @@ True, the MPI executor will not launch applications on nodes where libEnsemble P
 processes (manager and workers) are running. Workers launch applications onto the
 remaining nodes in the allocation.
 
-
 .. list-table::
    :widths: 60 40
 
@@ -84,29 +81,27 @@ remaining nodes in the allocation.
 
        A SLURM batch script may include:
 
-
        .. code-block:: bash
 
           #SBATCH --nodes 3
 
           python run_libe_forces.py --nworkers 3
 
-
 Note that **gen_on_manager** is not set in the above example.
 
-Distributed  Running
---------------------
+Distributed Running
+-------------------
 
 In the **distributed** approach, libEnsemble can be run using the **mpi4py**
 communicator, with workers distributed across nodes. This is most often used
 when workers run simulation code directly, via a Python interface. The user
-script is invoked with an MPI runner, for example (using an `mpich` based MPI)::
+script is invoked with an MPI runner, for example (using an `mpich`-based MPI)::
 
     mpirun -np 4 -ppn 1 python myscript.py
 
 The distributed approach, can also be used with the executor, to co-locate workers
-with the applications they submit. To ensure workers are placed as required in this
-case, requires :ref:`a careful MPI rank placement <slurm_mpi_distributed>`.
+with the applications they submit. Ensuring that workers are placed as required in this
+case requires :ref:`a careful MPI rank placement <slurm_mpi_distributed>`.
 
     .. image:: ../images/distributed_new_detailed.png
         :alt: distributed
@@ -116,7 +111,6 @@ case, requires :ref:`a careful MPI rank placement <slurm_mpi_distributed>`.
 This allows the libEnsemble worker to read files produced by the application on
 local node storage.
 
-
 Configuring the Run
 -------------------
 
@@ -140,7 +134,7 @@ and partitions these to workers. The :doc:`MPI Executor<../executor/mpi_executor
 accesses the resources available to the current worker when launching tasks.
 
 Zero-resource workers
-~~~~~~~~~~~~~~~~~~~~~
+---------------------
 
 Users with persistent ``gen_f`` functions may notice that the persistent workers
 are still automatically assigned system resources. This can be resolved by using
@@ -159,7 +153,6 @@ Varying resources
 libEnsemble also features :ref:`dynamic resource assignment<var-resources-gpu>`, whereby the
 number of processes and/or the number of GPUs can be a set for each simulation by the generator.
 
-
 Overriding Auto-Detection
 -------------------------
 
@@ -172,8 +165,6 @@ libE_specs option.
 When using the MPI Executor, it is possible to override the detected information using the
 `custom_info` argument. See the :doc:`MPI Executor<../executor/mpi_executor>` for more.
 
-
-
 Systems with Launch/MOM Nodes
 -----------------------------
 
@@ -212,7 +203,6 @@ or *to entirely different systems*.
 Submission scripts for running on launch/MOM nodes and for using Balsam can be found in
 the :doc:`examples<example_scripts>`.
 
-
 .. _globus_compute_ref:
 
 Globus Compute - Remote User Functions

diff --git a/docs/resource_manager/resource_detection.rst b/docs/resource_manager/resource_detection.rst
@@ -18,17 +18,17 @@ LSF             LSB_HOSTS/LSB_MCPU_HOSTS
 PBS             PBS_NODEFILE
 ===========  ===========================
 
-These environment variable names can be modified via the  :ref:`resource_info<resource_info>`
+These environment variable names can be modified via the :ref:`resource_info<resource_info>`
 :class:`libE_specs<libensemble.specs.LibeSpecs>` option.
 
-On other systems you may have to supply a node list in a file called **node_list**
-in your run directory. For example, on ALCF system Cooley_, the session node list
+On other systems, you may have to supply a node list in a file called **node_list**
+in your run directory. For example, on the ALCF system Cooley_, the session node list
 can be obtained as follows::
 
             cat $COBALT_NODEFILE > node_list
 
 Resource detection can be disabled by setting
-``libE_specs["disable_resource_manager"] = True``, and users can simply supply run
+``libE_specs["disable_resource_manager"] = True``, and users can supply run
 configuration options on the Executor submit line.
 
 This will usually work sufficiently on

diff --git a/docs/resource_manager/zero_resource_workers.rst b/docs/resource_manager/zero_resource_workers.rst
@@ -53,7 +53,7 @@ concurrency desired by the ensemble, taking into account generators and simulato
 
 Users can set generator resources using the *libE_specs* options
 ``gen_num_procs`` and/or ``gen_num_gpus``, which take integer values.
-If only  ``gen_num_gpus`` is set, then the number of processors is set to match.
+If only ``gen_num_gpus`` is set, then the number of processors is set to match.
 
 To vary generator resources, ``persis_info`` settings can be used in allocation
 functions before calling the ``gen_work`` support function. This takes the

diff --git a/docs/tutorials/forces_gpu_tutorial.rst b/docs/tutorials/forces_gpu_tutorial.rst
@@ -7,9 +7,9 @@ to the GPU. The libEnsemble scripts in this example are available under
 forces_gpu_ in the libEnsemble repository.
 
 This example is based on the
-:doc:`simple forces tutorial  <../tutorials/executor_forces_tutorial>` with
+:doc:`simple forces tutorial <../tutorials/executor_forces_tutorial>` with
 a slightly modified simulation function (to assign GPUs) and a greatly increased
-number of particles (allows live GPU usage to be viewed).
+number of particles (to allow real-time GPU usage to be viewed).
 
 In the first example, each worker will be using one GPU. The code will assign the
 GPUs available to each worker, using the appropriate method. This works on systems
@@ -35,7 +35,6 @@ from the simple forces example are highlighted:
     # Optional - to print GPU settings
     from libensemble.tools.test_support import check_gpu_setting
 
-
     def run_forces(H, persis_info, sim_specs, libE_info):
         """Launches the forces MPI app and auto-assigns ranks and GPU resources.
 
@@ -154,7 +153,6 @@ and use this information however you want.
             output = np.zeros(1, dtype=sim_specs["out"])
             output["energy"][0] = final_energy
 
-
         return output
 
     The above code will assign a GPU to each worker on CUDA-capable systems,

diff --git a/docs/tutorials/gpcam_tutorial.rst b/docs/tutorials/gpcam_tutorial.rst
@@ -10,7 +10,7 @@ In each iteration, a batch of points is produced for concurrent evaluation, maxi
 Ensure that libEnsemble, and gpCAM are installed via: ``pip install libensemble gpcam``
 
 Generator function
------------------
+------------------
 
 The gpCAM generator function is called ``persistent_gpCAM``.
 
@@ -179,7 +179,7 @@ For running applications using parallel resources in the simulator see the `forc
         return term1 + term2 + term3
 
 Calling Script
--------------
+--------------
 
 Our calling script configures libEnsemble, the generator function, and the simulator function. It then create the ensemble object and runs the ensemble.
 
@@ -275,7 +275,7 @@ At the end of our calling script we run the ensemble.
     pprint(H[["sim_id", "x", "f"]][:16]) # See first 16 results
 
 Rerun and test model at known points
------------------------------------
+------------------------------------
 
 To see how the accuracy of the surrogate model improves, we can use previously evaluated points as test points and run again with a different seed.
 
@@ -292,7 +292,7 @@ To see how the accuracy of the surrogate model improves, we can use previously e
     print(persis_info)
 
 Viewing model progression
-------------------------
+-------------------------
 
 Now we can check how our model's values compared against the values at known test points as the ensemble progresses.
 The comparison is based on the **mean squared error** between the gpCAM model and our known

diff --git a/examples/libE_submission_scripts/bebop_submit_pbs_central.sh b/examples/libE_submission_scripts/bebop_submit_pbs_central.sh
@@ -5,7 +5,6 @@
 #PBS -A [project]
 #PBS -N libE_example
 
-
 cd $PBS_O_WORKDIR
 # Choose MPI backend. Note that the built mpi4py in your environment should match.
 module load oneapi/mpi