Skip to content

Utopia is a comprehensive modelling framework for complex and evolving systems. Docs @ https://docs.utopia-project.org — NOTE: This repository is a READ-ONLY-MIRROR of the actual development repository; please open issues and MRs there:

License

GPL-3.0 and 2 other licenses found

Licenses found

GPL-3.0
LICENSE
Unknown
COPYING.md
LGPL-3.0
LICENSE.LESSER
Notifications You must be signed in to change notification settings

utopia-foss/utopia

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Utopia Logo

Utopia

Utopia is a comprehensive modelling framework for complex and evolving systems. It aims to provide the tools to conveniently implement a model, configure and perform simulation runs, and evaluate the resulting data.

Utopia is free software and licensed under the GNU Lesser General Public License Version 3 or any later version. For the copyright notice and the list of copyright holders, see COPYING.md.

Overview

Complex and evolving systems are investigated with a wide array of different models types. Among the most popular are cellular automata (CA), agent-based models (ABMs), and network models. Utopia provides a library to build these types of models in modern C++. Its core template library contains functions for implementing model dynamics and scaffoldings for building new models. The Utopia dataIO library is capable of storing hierarchical simulation data efficiently in the H5 file format. Within Utopia, and when using Utopia as a dependency, models are integrated into a simulation infrastructure for conveniently executing and analyzing them.

The utopya Python package constitutes Utopia's frontend. It configures and performs the simulation runs and embeds the model into a data processing pipeline, such that the simulation data can directly be analyzed and visualized. All parts of the frontend make use of a hierarchic, recursively-updated YAML-based configuration structure. Using the paramspace package, this allows to easily define parameter sweeps, which can then be carried out as simultaneous simulations on massively parallel, distributed machines. The dantro-based data processing pipeline automates visualization, thereby coupling the model implementation and its analysis closer together.

Several models are readily included in the framework, among them Conway's Game of Life, one CA- and one agent-based contagious disease model, and a network-based model of social opinion dynamics. Investigating these models by performing simulation runs with a few varying parameters, or sensitivity analysis over a large parameter space, requires little to no programming skills.

For introductory guides, feature lists, FAQs, and API references refer to the online user manual at docs.utopia-project.org

Utopia development happens in the utopia-project group on GitLab.com. Additionally, you can retrieve the code and get in contact via a mirrored repository on GitHub. Note that we used to host our own GitLab instance at ts-gitlab.iup.uni-heidelberg.de; traces of that past development platform may still be around this project somewhere ... 👻

How to cite Utopia

Utopia was reviewed and published in the Journal of Open Source Software (JOSS). Please cite at least the following publication if you use Utopia (or a modified version thereof) for your own work:

Lukas Riedel, Benjamin Herdeanu, Harald Mack, Yunus Sevinchan, and Julian Weninger. 2020. “Utopia: A Comprehensive and Collaborative Modeling Framework for Complex and Evolving Systems.” Journal of Open Source Software 5 (53): 2165. DOI: 10.21105/joss.02165.

The CITATION.cff file in this repository follows the citation file format and contains additional metadata to reference this software, its authors, and associated publications. See the user manual for more information and BibTeX data.

Contents of this README


How to install

The following instructions will install Utopia into a development environment on your machine. If you simply want to run Utopia, you can do so via a ready-to-use docker image; see below for more information.

Step-by-step Instructions

These instructions are intended for 'clean' macOS (both Intel and Apple Silicon) or recent Ubuntu setups. Since Windows supports the installation of Ubuntu via Windows Subsystem for Linux, Utopia can also be used on Windows. Follow the WSL Installation Guide to install Ubuntu, then follow the instructions for Ubuntu in this README.

Note: Utopia is always tested against a recent Ubuntu release, currently Ubuntu 22.04. However, you can use Utopia with any earlier release, as long as the dependencies can be fulfilled.

⚠️ If you encounter difficulties, have a look at the troubleshooting section. If this does not resolve your installation problems, please file an issue in the GitLab project.

1 — Clone Utopia

First, create a Utopia directory at a place of your choice. This is where the Utopia repository will be cloned to. When working with or developing for Utopia, auxiliary data will have a place there as well.

In your terminal, enter the Utopia directory you just created and invoke the clone command:

git clone https://gitlab.com/utopia-project/utopia.git

Alternatively, you can clone via SSH, using the address from the "Clone" button on the project page.

After cloning, there will be a new utopia directory (mirroring this repository) inside your top-level Utopia directory.

2 — Install dependencies

Install the third-party dependencies using a package manager.

Note: If you have Anaconda installed, you already have a working Python installation on your system, and you can omit installing the python packages below. However, notice that there might be issues during the configuration step. Have a look at the troubleshooting section to see how to address them.

On Ubuntu (22.04)
apt update
apt install cmake gcc g++ gfortran git libarmadillo-dev libboost-all-dev \
            libhdf5-dev libspdlog-dev libyaml-cpp-dev pkg-config \
            python3-dev python3-pip python3-venv

Further, we recommend installing the following optional packages:

apt update
apt install ffmpeg graphviz doxygen

You will probably need administrator rights.

Note: For simplicity, we suggest installing the meta-package libboost-all-dev which includes the whole Boost library. If you want a minimal installation (only the strictly required components), use the following packages instead: libboost-dev libboost-test-dev libboost-graph-dev libboost-regex-dev.

On macOS

First, install the Apple Command Line Tools:

xcode-select --install

There are two popular package managers on macOS, Homebrew and MacPorts; we recommend to use Homebrew. Here are the installation instructions for both:

  • Homebrew:

    Install the required packages:

    brew update
    brew install armadillo boost cmake hdf5 pkg-config python3 spdlog yaml-cpp

    Further, we recommend installing the following optional packages:

    brew update
    brew install ffmpeg graphviz doxygen
  • MacPorts:

    Please be aware that port commands typically require administrator rights (sudo).

    Install the required packages:

    port install armadillo boost cmake hdf5 python37 py37-pip spdlog yaml-cpp

    Select the installed versions of Python and Pip:

    port select --set python python37
    port select --set python3 python37
    port select --set pip pip37
    port select --set pip3 pip37

    Further, we recommend installing the following optional packages:

    port install ffmpeg graphviz doxygen

3 — Configure and build

Enter the repository and create your desired build directory:

cd utopia
mkdir build

Now, enter the build directory and invoke CMake (and mind the caveats below):

cd build
cmake ..

Note: If you are using MacPorts, append the location of your Python installation to the CMake command (this is only required when calling CMake on a clean build directory):

cmake -DPython_ROOT_DIR=/opt/local ..

The terminal output will show the configuration steps, which includes the installation of further Python dependencies and the creation of a virtual environment.

After this, you can build a specific or all Utopia models using:

make dummy     # builds only the dummy model
make -j4 all   # builds all models, using 4 CPUs

4 — Run a model 🎉

You should now be able to run a Utopia model. Being in the build directory, call:

source ./activate

to enter the virtual environment, where the Utopia Command Line Interface (CLI) is available. (If you later want to exit the virtual environment, call the deactivate command.)

Note: If you are using csh or fish shells, use the respective activate scripts located in build/utopia-env/bin/ (see below).

The utopia command is now available and gives you control over running and evaluating model simulations:

utopia run dummy

The model output will be written into ~/utopia_output/dummy/<timestamp>. For more information on how to use the command line interface, see the information for users below and the documentation.

5 — Make sure everything works

This step is optional, but recommended.

To make sure that Utopia works as expected on your machine, build and carry out all tests.

Optional Installation Steps

The following instructions will enable additional, optional features of Utopia.

Enable Multithreading in Utopia Models

  1. Install the optional dependencies for multithreading.

    • On Ubuntu, we recommend using the GCC compiler with Intel TBB:

      apt update && apt install libtbb-dev

      Alternatively, one may install the Intel oneAPI base toolkit following these installation instructions. The only required package is intel-basekit. It includes the oneDPL library we use for parallelization.

    • On macOS, with Homebrew (please mind the note below!):

      brew update && brew install onedpl
    • On macOS with MacPorts, we are currently unsure whether multithreading is workable.

  2. Enter the Utopia build directory, and call CMake again. This time, enable the use of multithreading with the MULTITHREADING option:

    cd build
    cmake -DMULTITHREADING=On ..

    At the end of its output, CMake should now report that the "Multithreading" feature has been enabled. If the requirements for multithreading are not met, however, the build will fail.

  3. Re-compile the models. Inside the build directory, call

    make all
  4. Depending on the algorithms used inside the respective model, it will automatically exploit the multithreading capabilities of your system when executed! 🎉

Utopia via Docker

Docker is a free OS-level virtualization software. It allows running any application in a closed environment container.

The Utopia docker image is a way to run Utopia models and evaluate them without having to go through the installation procedure. It is also suitable for framework and model development.

The image and instructions on how to create a container from it can be found on the ccees/utopia docker hub page.

Quickstart

This section gives a glimpse into working with Utopia. It's not more than a glimpse; after playing around with this, consult the documentation to gain access to more information material, and especially: the Utopia Tutorial.

How to run a model?

The Utopia command line interface (CLI) is, by default, only available in a Python virtual environment, in which utopya (the Utopia frontend) and its dependencies are installed. To conveniently work with the frontend, you should thus enter the virtual environment. Execute one of the commands below, depending on which type of shell you're using:

source ./build/activate                  # For bash, zsh, or similar
source ./build/utopia-env/activate.csh   # For csh
source ./build/utopia-env/activate.fish  # For fish

Now, your shell should be prefixed with (utopia-env). (To exit the virtual environment, simply call deactivate.) All the following should take place inside this virtual environment.

As you have already done with the dummy model, the basic command to run a model named SomeModel is:

utopia run SomeModel

where SomeModel needs to be replaced with a valid model name. To get a list of available models, run the utopia models ls command. Alternatively, have a look at the src/utopia/models directory, where they are implemented.

The utopia run command carries out a pre-configured simulation for that model, loads the data, and performs automated plots.

Note:

  • The script will always run through, even if there were errors in the individual parts. Thus, you should check the terminal output for errors and warnings (red or yellow, respectively). If you want the CLI to fail on errors, you can also set the --debug flag.
  • By default, you can find the simulation output in ~/utopia_output/SomeModel/<timestamp>. It contains the written data, all the used configuration files, and the default plots.

For further information, e.g. on how you can pass a custom configuration, check the CLI help:

utopia --help
utopia run --help

The best place to continue from here is the tutorial.

Utopia Documentation

This Readme only covers the very basics. For all further documentation, tutorials, guides, model descriptions, frequently asked questions and more, you should consult the actual Utopia documentation.

The latest build (corresponding to the latest commit to master) is available online.

See below on how to build the documentation locally.

Information for Developers

SSH Repository Access

To work comfortably with Utopia, access to the GitLab needs to be easy. The best way to achieve that is by registering a so-called SSH key with your GitLab account.

To do that, follow the linked instructions to generate a key pair and to add a key to your GitLab account.

Building the documentation locally

To build the documentation locally, first make sure that all submodules are downloaded, because they are needed for building the documentation:

git submodule update --init --recursive

Then navigate to the build directory and run

make doc

In case you also want to generate (some of) the figures that are embedded into the documentation, set the UTOPIA_DOC_GENERATE_FIGURES environment variable:

UTOPIA_DOC_GENERATE_FIGURES=True make doc

After building, carrying out a link check and running some documentation tests is advisable:

make check_docs

The Sphinx-built user documentation will then be located at build/doc/html/. The C++ doxygen-documentation can be found at build/doc/doxygen/html/. Open the respective index.html files to browse the documentation.

Choosing a different compiler

CMake will inspect your system paths and use the default compiler. You can use the CC and CXX environment variables to select a specific C and C++ compiler, respectively. As compiler paths are cached by CMake, changing the compiler requires you to delete the build directory and re-run CMake.

Ubuntu 22.04 LTS (Jammy Jellyfish), for example, provides GCC versions 10 to 12. To use GCC 12, for example, first install it via APT:

apt update && apt install gcc-12 g++-12

Then create the build directory, enter it, and set the CC and CXX environment variables when calling CMake:

mkdir build && cd build
CC=/usr/bin/gcc-12 CXX=/usr/bin/g++-12 cmake ..

Alternatively, you can export these variables in separate bash commands before executing CMake.

Build Types

If you followed the instructions above, you have a Release build which is optimized for maximum performance. If you need to debug, you should reconfigure the entire project with CMake by navigating to the build folder and calling

cmake -DCMAKE_BUILD_TYPE=Debug ..

Afterwards, call

make <something>

to rebuild executables. CMake handles the required compiler flags automatically.

The build type (as most other CMake flags) persists until it is explicitly changed by the user. To build optimized executables again, reconfigure with

cmake -DCMAKE_BUILD_TYPE=Release ..

Testing

Utopia contains unit tests to ensure consistency by checking whether class members and functions are working correctly. This is done both for the C++ and the Python code. The tests are integrated into the GitLab Continuous Integration pipeline, meaning that tests are run upon every push to the project and failing tests can be easily detected. Tests can also be executed locally, to test a (possibly altered) version of Utopia before committing and pushing changes to the GitLab.

See the user manual for more details on how to run tests.

Setting up a separate repository for models

Working inside a clone or a fork of this repository is generally not a good idea: it makes updating harder, prohibits efficient version control on the models, and makes it more difficult to include additional dependencies or code.

To make setting up such a project repository as easy as possible, we provide a template repository, which can be used to start a new Utopia project! Follow the instructions in the models_template project for more information.

Make sure to call the following command to export package information to the CMake project registry, such that a separate repository can find Utopia:

cmake -DCMAKE_EXPORT_PACKAGE_REGISTRY=On ..

Dependencies

Software Required Version Tested Version Comments
GCC >= 9 11.2 Full C++17 support required
or: clang >= 9 14.0 Full C++17 support required
or: Apple LLVM >= 9 Full C++17 support required
CMake >= 3.16 3.22
pkg-config >= 0.29 0.29
HDF5 >= 1.10.4 1.10.7
Boost >= 1.67 1.74 required components: graph, regex and unit_test_framework
Armadillo >= 9.600 10.8.2
yaml-cpp >= 0.6.2 0.7.0
spdlog >= 1.3 1.9.2
Python3 >= 3.6 3.10.4

Utopia aims to allow rapid development, and is thus being tested only against the more recent releases of its dependencies. Currently, Utopia is tested against the packages provided by Ubuntu 22.04. However the above version requirements (i.e., those enforced by the build system) can be fulfilled also with Ubuntu 19.10.

To get Utopia running on a system with an earlier Ubuntu version, the above dependencies still need to be fulfilled. You can use the Ubuntu Package Search to find the versions available on your system. If a required version is not available, private package repositories may help to install a more recent version of a dependency.

If you encounter difficulties with the installation procedure for any of these dependencies, please file an issue in the GitLab project.

Python

Utopia also has some Python dependencies, which are automatically installed during the configuration step of the installation.

The following table includes the most important Python dependencies:

Software Version Comments
utopya >= 1.3 The (outsourced) Utopia frontend package
dantro >= 0.18 Handle, transform, and visualize hierarchically organized data
paramspace >= 2.5 Makes parameter sweeps easy

In addition, the following packages are optionally used for development of the framework or its models.

Software Version Comments
pytest For model tests
pre-commit >= 2.18 For pre-commit hooks
black >= 22.6 For formatting python code
Sphinx == 5.* Builds the Utopia documentation

These requirements are defined in the .utopia-env-requirements.txt file; in case installation fails, a warning will be emitted during configuration.

Recommended

The following depencies are recommended to be installed, but are not strictly required by Utopia:

Software Version Comments
doxygen >= 1.8 Builds the C++ code documentation
graphviz >= 2.40 Used by doxygen to create dependency diagrams
ffmpeg >= 4.1 Used for creating videos
Intel oneDPL Intel parallelization library
TBB >= 2018.5 Intel multithreading library

Troubleshooting

  • If you have a previous installation and the build fails inexplicably, removing the build directory completely and starting anew from the configuration step should help.
    In cases where the installation used to work but at some point stopped working, this should be a general remedy. If the problem does not seem to be related to the Python environment, deleting only build/CMakeCache.txt may already suffice and save some configuration time.

  • If you have trouble with more recent HDF5 versions on macOS, one workaround is to use an older version:

    brew install hdf5@1.10
    brew link hdf5@1.10 --overwrite
  • Anaconda ships its own version of the HDF5 library which is not compatible with Utopia. To tell CMake where to find the correct version of the library, add the following argument (without the comments!) to the cmake .. command during configuration:

    -DHDF5_ROOT=/usr/           # on Ubuntu
    -DHDF5_ROOT=/usr/local/     # on macOS (Homebrew)

    For Ubuntu, the full command would then be:

    cd build
    cmake -DHDF5_ROOT=/usr/ ..

About

Utopia is a comprehensive modelling framework for complex and evolving systems. Docs @ https://docs.utopia-project.org — NOTE: This repository is a READ-ONLY-MIRROR of the actual development repository; please open issues and MRs there:

Topics

Resources

License

GPL-3.0 and 2 other licenses found

Licenses found

GPL-3.0
LICENSE
Unknown
COPYING.md
LGPL-3.0
LICENSE.LESSER

Code of conduct

Stars

Watchers

Forks