Implement Parameters as Pydantic BaseModels and improve exploration diagnostics

[AngelFP]

This PR implements a solution to the lack of parameter metadata in the history file. As proposed by @ax3l a couple of months ago, this solution is based on taking advantage of Pydantic's BaseModel to be able to serialize and save the exploration parameters to a JSON file. In this way, we can later recover these parameters and all their attributes for analyzing the output of an exploration.

Main changes

• All Parameters (i.e., varying parameters, objectives, analyzed parameters, etc.) are now Pydantic BaseModels. This allows for a few nice things:

  • The inputs from the user are now automatically validated, and we can easily define custom validators for more complex attributes like the dtype of the parameters.
  • We can serialize the optimization parameters to JSON and save them to a file that can be read by post-processing tools/scripts. With this, it is no longer needed to add information to the libE history file about which parameters were inputs, objectives, etc. The complete information of these parameters is now contained in the JSON file.

• A new ExplorationDiagnostics class has been implemented that allows for easily analyzing the output of an optimas Exploration. This class takes care of reading and processing the history file as well as the JSON file so that the user does not need to interact with them directly. The ExplorationDiagnostics replaces the old PostProcOptimization class, which was by now outdated (it was the last remnant of libE_opt).

• Added new section to documentation with example usage of ExplorationDiagnostics. This requires installing mpi4py, for which the RTD environment had to be changed to mamba.

Notes

• The implementation was done using Pydantic V2, which is currently requires using the develop branch of libEnsemble until v1.2 is released (Refactor/ support both Pydantic 1 and 2 Libensemble/libensemble#1135).
• Version number updated to 0.4.0.

Example use of ExplorationDiagnostics

Since all information about the history and the parameters is included in the diagnostics, we can now use it for fully automated post-processing, for example:

from optimas.diagnostics import ExplorationDiagnostics

diags = ExplorationDiagnostics("/path/to/exploration")

fig, ax = plt.subplots()
vps = diags.varying_parameters
df = diags.history
f1 = diags.objectives[0]
ax.axvline(vps[0].lower_bound)
ax.axvline(vps[0].upper_bound)
ax.set_xlabel(vps[0].name)
ax.axhline(vps[1].lower_bound)
ax.axhline(vps[1].upper_bound)
ax.set_ylabel(vps[1].name)
ax.scatter(df[vps[0].name], df[vps[1].name], c=df[f1.name])

[AngelFP]

@delaossa this should fix #80 (which #81 was addressing) because now you no longer need to import the original optimas script for post-processing.

[ax3l]

This is very cool, thank you! 🎉

Do you want to add a bit of docs on the JSON file to the manual to finalize?