Utopia is a comprehensive modelling framework for complex and evolving systems. It aims to provide the tools to conveniently implement a model, configure and perform simulation runs, and evaluate the resulting data.
Utopia is free software and licensed under the GNU Lesser General Public License Version 3 or any later version.
For the copyright notice and the list of copyright holders, see COPYING.md.
Complex and evolving systems are investigated with a wide array of different models types.
Among the most popular are cellular automata (CA), agent-based models (ABMs), and network models.
Utopia provides a library to build these types of models in modern C++.
Its core template library contains functions for implementing model dynamics and scaffoldings for building new models.
The Utopia dataIO library is capable of storing hierarchical simulation data efficiently in the H5 file format.
Within Utopia, and when using Utopia as a dependency, models are integrated into a simulation infrastructure for conveniently executing and analyzing them.
The utopya Python package constitutes Utopia's frontend.
It configures and performs the simulation runs and embeds the model into a data processing pipeline, such that the simulation data can directly be analyzed and visualized.
All parts of the frontend make use of a hierarchic, recursively-updated YAML-based configuration structure.
Using the paramspace package, this allows to easily define parameter sweeps, which can then be carried out as simultaneous simulations on massively parallel, distributed machines.
The dantro-based data processing pipeline automates visualization, thereby coupling the model implementation and its analysis closer together.
Several models are readily included in the framework, among them Conway's Game of Life, one CA- and one agent-based contagious disease model, and a network-based model of social opinion dynamics. Investigating these models by performing simulation runs with a few varying parameters, or sensitivity analysis over a large parameter space, requires little to no programming skills.
For introductory guides, feature lists, FAQs, and API references refer to the online user manual at docs.utopia-project.org
Utopia development happens in the utopia-project group on GitLab.com.
Additionally, you can retrieve the code and get in contact via a mirrored repository on GitHub.
Note that we used to host our own GitLab instance at ts-gitlab.iup.uni-heidelberg.de; traces of that past development platform may still be around this project somewhere ... 👻
Utopia was reviewed and published in the Journal of Open Source Software (JOSS). Please cite at least the following publication if you use Utopia (or a modified version thereof) for your own work:
Lukas Riedel, Benjamin Herdeanu, Harald Mack, Yunus Sevinchan, and Julian Weninger. 2020. “Utopia: A Comprehensive and Collaborative Modeling Framework for Complex and Evolving Systems.” Journal of Open Source Software 5 (53): 2165. DOI: 10.21105/joss.02165.
The CITATION.cff file in this repository follows the citation file format and contains additional metadata to reference this software, its authors, and associated publications.
See the user manual for more information and BibTeX data.
- How to install
- Quickstart
- Documentation and Guides
- Information for Developers
- Dependencies
- Troubleshooting
The following instructions will install Utopia into a development environment on your machine. If you simply want to run Utopia, you can do so via a ready-to-use docker image; see below for more information.
These instructions are intended for 'clean' macOS (both Intel and Apple Silicon) or recent Ubuntu setups. Since Windows supports the installation of Ubuntu via Windows Subsystem for Linux, Utopia can also be used on Windows. Follow the WSL Installation Guide to install Ubuntu, then follow the instructions for Ubuntu in this README.
Note: Utopia is always tested against a recent Ubuntu release, currently Ubuntu 22.04. However, you can use Utopia with any earlier release, as long as the dependencies can be fulfilled.
First, create a Utopia directory at a place of your choice.
This is where the Utopia repository will be cloned to.
When working with or developing for Utopia, auxiliary data will have a place there as well.
In your terminal, enter the Utopia directory you just created and invoke the clone command:
git clone https://gitlab.com/utopia-project/utopia.gitAlternatively, you can clone via SSH, using the address from the "Clone" button on the project page.
After cloning, there will be a new utopia directory (mirroring this repository) inside your top-level Utopia directory.
Install the third-party dependencies using a package manager.
Note: If you have Anaconda installed, you already have a working Python installation on your system, and you can omit installing the python packages below.
However, notice that there might be issues during the configuration step.
Have a look at the troubleshooting section to see how to address them.
apt update
apt install cmake gcc g++ gfortran git libarmadillo-dev libboost-all-dev \
libhdf5-dev libspdlog-dev libyaml-cpp-dev pkg-config \
python3-dev python3-pip python3-venvFurther, we recommend installing the following optional packages:
apt update
apt install ffmpeg graphviz doxygenYou will probably need administrator rights.
Note: For simplicity, we suggest installing the meta-package libboost-all-dev which includes the whole Boost library.
If you want a minimal installation (only the strictly required components), use the following packages instead:
libboost-dev libboost-test-dev libboost-graph-dev libboost-regex-dev.
First, install the Apple Command Line Tools:
xcode-select --installThere are two popular package managers on macOS, Homebrew and MacPorts; we recommend to use Homebrew. Here are the installation instructions for both:
-
Homebrew:
Install the required packages:
brew update brew install armadillo boost cmake hdf5 pkg-config python3 spdlog yaml-cpp
Further, we recommend installing the following optional packages:
brew update brew install ffmpeg graphviz doxygen
-
MacPorts:
Please be aware that
portcommands typically require administrator rights (sudo).Install the required packages:
port install armadillo boost cmake hdf5 python37 py37-pip spdlog yaml-cpp
Select the installed versions of Python and Pip:
port select --set python python37 port select --set python3 python37 port select --set pip pip37 port select --set pip3 pip37
Further, we recommend installing the following optional packages:
port install ffmpeg graphviz doxygen
Enter the repository and create your desired build directory:
cd utopia
mkdir buildNow, enter the build directory and invoke CMake (and mind the caveats below):
cd build
cmake ..Note: If you are using MacPorts, append the location of your Python installation to the CMake command (this is only required when calling CMake on a clean build directory):
cmake -DPython_ROOT_DIR=/opt/local ..The terminal output will show the configuration steps, which includes the installation of further Python dependencies and the creation of a virtual environment.
After this, you can build a specific or all Utopia models using:
make dummy # builds only the dummy model
make -j4 all # builds all models, using 4 CPUsYou should now be able to run a Utopia model.
Being in the build directory, call:
source ./activateto enter the virtual environment, where the Utopia Command Line Interface (CLI) is available.
(If you later want to exit the virtual environment, call the deactivate command.)
Note: If you are using csh or fish shells, use the respective activate scripts located in build/utopia-env/bin/ (see below).
The utopia command is now available and gives you control over running and evaluating model simulations:
utopia run dummyThe model output will be written into ~/utopia_output/dummy/<timestamp>.
For more information on how to use the command line interface, see the information for users below and the documentation.
This step is optional, but recommended.
To make sure that Utopia works as expected on your machine, build and carry out all tests.
The following instructions will enable additional, optional features of Utopia.
-
Install the optional dependencies for multithreading.
-
On Ubuntu, we recommend using the GCC compiler with Intel TBB:
apt update && apt install libtbb-devAlternatively, one may install the Intel oneAPI base toolkit following these installation instructions. The only required package is
intel-basekit. It includes the oneDPL library we use for parallelization. -
On macOS, with Homebrew (please mind the note below!):
brew update && brew install onedpl -
On macOS with MacPorts, we are currently unsure whether multithreading is workable.
-
-
Enter the Utopia build directory, and call CMake again. This time, enable the use of multithreading with the
MULTITHREADINGoption:cd build cmake -DMULTITHREADING=On ..At the end of its output, CMake should now report that the "Multithreading" feature has been enabled. If the requirements for multithreading are not met, however, the build will fail.
-
Re-compile the models. Inside the build directory, call
make all
-
Depending on the algorithms used inside the respective model, it will automatically exploit the multithreading capabilities of your system when executed! 🎉
Docker is a free OS-level virtualization software. It allows running any application in a closed environment container.
The Utopia docker image is a way to run Utopia models and evaluate them without having to go through the installation procedure. It is also suitable for framework and model development.
The image and instructions on how to create a container from it can be found on the ccees/utopia docker hub page.
This section gives a glimpse into working with Utopia. It's not more than a glimpse; after playing around with this, consult the documentation to gain access to more information material, and especially: the Utopia Tutorial.
The Utopia command line interface (CLI) is, by default, only available in a Python virtual environment, in which utopya (the Utopia frontend) and its dependencies are installed.
To conveniently work with the frontend, you should thus enter the virtual environment.
Execute one of the commands below, depending on which type of shell you're using:
source ./build/activate # For bash, zsh, or similar
source ./build/utopia-env/activate.csh # For csh
source ./build/utopia-env/activate.fish # For fishNow, your shell should be prefixed with (utopia-env).
(To exit the virtual environment, simply call deactivate.)
All the following should take place inside this virtual environment.
As you have already done with the dummy model, the basic command to run a model named SomeModel is:
utopia run SomeModelwhere SomeModel needs to be replaced with a valid model name.
To get a list of available models, run the utopia models ls command.
Alternatively, have a look at the src/utopia/models directory, where they are implemented.
The utopia run command carries out a pre-configured simulation for that model, loads the data, and performs automated plots.
Note:
- The script will always run through, even if there were errors in the
individual parts. Thus, you should check the terminal output for errors
and warnings (red or yellow, respectively). If you want the CLI to fail on
errors, you can also set the
--debugflag. - By default, you can find the simulation output in
~/utopia_output/SomeModel/<timestamp>. It contains the written data, all the used configuration files, and the default plots.
For further information, e.g. on how you can pass a custom configuration, check the CLI help:
utopia --help
utopia run --helpThe best place to continue from here is the tutorial.
This Readme only covers the very basics. For all further documentation, tutorials, guides, model descriptions, frequently asked questions and more, you should consult the actual Utopia documentation.
The latest build (corresponding to the latest commit to master) is available
online.
See below on how to build the documentation locally.
To work comfortably with Utopia, access to the GitLab needs to be easy. The best way to achieve that is by registering a so-called SSH key with your GitLab account.
To do that, follow the linked instructions to generate a key pair and to add a key to your GitLab account.
To build the documentation locally, first make sure that all submodules are downloaded, because they are needed for building the documentation:
git submodule update --init --recursiveThen navigate to the build directory and run
make docIn case you also want to generate (some of) the figures that are embedded into the documentation, set the UTOPIA_DOC_GENERATE_FIGURES environment variable:
UTOPIA_DOC_GENERATE_FIGURES=True make docAfter building, carrying out a link check and running some documentation tests is advisable:
make check_docsThe Sphinx-built user documentation will then be located at build/doc/html/.
The C++ doxygen-documentation can be found at build/doc/doxygen/html/.
Open the respective index.html files to browse the documentation.
CMake will inspect your system paths and use the default compiler.
You can use the CC and CXX environment variables to select a specific C and C++ compiler, respectively.
As compiler paths are cached by CMake, changing the compiler requires you to delete the build directory and re-run CMake.
Ubuntu 22.04 LTS (Jammy Jellyfish), for example, provides GCC versions 10 to 12. To use GCC 12, for example, first install it via APT:
apt update && apt install gcc-12 g++-12Then create the build directory, enter it, and set the CC and CXX environment variables when calling CMake:
mkdir build && cd build
CC=/usr/bin/gcc-12 CXX=/usr/bin/g++-12 cmake ..Alternatively, you can export these variables in separate bash commands before executing CMake.
If you followed the instructions above, you have a Release build which is
optimized for maximum performance. If you need to debug, you should reconfigure
the entire project with CMake by navigating to the build folder and calling
cmake -DCMAKE_BUILD_TYPE=Debug ..Afterwards, call
make <something>to rebuild executables. CMake handles the required compiler flags automatically.
The build type (as most other CMake flags) persists until it is explicitly changed by the user. To build optimized executables again, reconfigure with
cmake -DCMAKE_BUILD_TYPE=Release ..Utopia contains unit tests to ensure consistency by checking whether class members and functions are working correctly. This is done both for the C++ and the Python code. The tests are integrated into the GitLab Continuous Integration pipeline, meaning that tests are run upon every push to the project and failing tests can be easily detected. Tests can also be executed locally, to test a (possibly altered) version of Utopia before committing and pushing changes to the GitLab.
See the user manual for more details on how to run tests.
Working inside a clone or a fork of this repository is generally not a good idea: it makes updating harder, prohibits efficient version control on the models, and makes it more difficult to include additional dependencies or code.
To make setting up such a project repository as easy as possible, we provide a template repository, which can be used to start a new Utopia project!
Follow the instructions in the models_template project for more information.
Make sure to call the following command to export package information to the CMake project registry, such that a separate repository can find Utopia:
cmake -DCMAKE_EXPORT_PACKAGE_REGISTRY=On ..| Software | Required Version | Tested Version | Comments |
|---|---|---|---|
| GCC | >= 9 | 11.2 | Full C++17 support required |
| or: clang | >= 9 | 14.0 | Full C++17 support required |
| or: Apple LLVM | >= 9 | Full C++17 support required | |
| CMake | >= 3.16 | 3.22 | |
| pkg-config | >= 0.29 | 0.29 | |
| HDF5 | >= 1.10.4 | 1.10.7 | |
| Boost | >= 1.67 | 1.74 | required components: graph, regex and unit_test_framework |
| Armadillo | >= 9.600 | 10.8.2 | |
| yaml-cpp | >= 0.6.2 | 0.7.0 | |
| spdlog | >= 1.3 | 1.9.2 | |
| Python3 | >= 3.6 | 3.10.4 |
Utopia aims to allow rapid development, and is thus being tested only against the more recent releases of its dependencies. Currently, Utopia is tested against the packages provided by Ubuntu 22.04. However the above version requirements (i.e., those enforced by the build system) can be fulfilled also with Ubuntu 19.10.
To get Utopia running on a system with an earlier Ubuntu version, the above dependencies still need to be fulfilled. You can use the Ubuntu Package Search to find the versions available on your system. If a required version is not available, private package repositories may help to install a more recent version of a dependency.
If you encounter difficulties with the installation procedure for any of these dependencies, please file an issue in the GitLab project.
Utopia also has some Python dependencies, which are automatically installed during the configuration step of the installation.
The following table includes the most important Python dependencies:
| Software | Version | Comments |
|---|---|---|
| utopya | >= 1.3 | The (outsourced) Utopia frontend package |
| dantro | >= 0.18 | Handle, transform, and visualize hierarchically organized data |
| paramspace | >= 2.5 | Makes parameter sweeps easy |
In addition, the following packages are optionally used for development of the framework or its models.
| Software | Version | Comments |
|---|---|---|
| pytest | For model tests | |
| pre-commit | >= 2.18 | For pre-commit hooks |
| black | >= 22.6 | For formatting python code |
| Sphinx | == 5.* | Builds the Utopia documentation |
These requirements are defined in the .utopia-env-requirements.txt file; in case installation fails, a warning will be emitted during configuration.
The following depencies are recommended to be installed, but are not strictly required by Utopia:
| Software | Version | Comments |
|---|---|---|
| doxygen | >= 1.8 | Builds the C++ code documentation |
| graphviz | >= 2.40 | Used by doxygen to create dependency diagrams |
| ffmpeg | >= 4.1 | Used for creating videos |
| Intel oneDPL | Intel parallelization library | |
| TBB | >= 2018.5 | Intel multithreading library |
-
If you have a previous installation and the build fails inexplicably, removing the
builddirectory completely and starting anew from the configuration step should help.
In cases where the installation used to work but at some point stopped working, this should be a general remedy. If the problem does not seem to be related to the Python environment, deleting onlybuild/CMakeCache.txtmay already suffice and save some configuration time. -
If you have trouble with more recent HDF5 versions on macOS, one workaround is to use an older version:
brew install hdf5@1.10 brew link hdf5@1.10 --overwrite
-
Anaconda ships its own version of the HDF5 library which is not compatible with Utopia. To tell CMake where to find the correct version of the library, add the following argument (without the comments!) to the
cmake ..command during configuration:-DHDF5_ROOT=/usr/ # on Ubuntu -DHDF5_ROOT=/usr/local/ # on macOS (Homebrew)
For Ubuntu, the full command would then be:
cd build cmake -DHDF5_ROOT=/usr/ ..