chore: use Cbc in unit testing, adapt documentation (#796)

* chore: use Cbc in unit testing, adapt documentation

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

* add back Cbc installation for Github Actions

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

* docs: make more explicit that the commands from the other section need to be copied and run in docker compose tutorial

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

---------

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

.github/workflows/lint-and-test.yml

@@ -49,6 +49,7 @@ jobs:
             ${{ runner.os }}-pip-
       - run: |
           ci/setup-postgres.sh
+          sudo apt-get -y install coinor-cbc
       - name: Install FlexMeasures & exact dependencies for tests
         run: make install-for-test
         if: github.event_name == 'push' && steps.cache.outputs.cache-hit != 'true'

documentation/configuration.rst

@@ -55,9 +55,13 @@ Default: ``False``
 FLEXMEASURES_LP_SOLVER
 ^^^^^^^^^^^^^^^^^^^^^^
 
-The command to run the scheduling solver. This is the executable command which FlexMeasures calls via the `pyomo library <http://www.pyomo.org/>`_. Other values might be ``cplex``, ``glpk`` or ``appsi_highs`` for `HiGHS <https://highs.dev/>`_. Consult `their documentation <https://pyomo.readthedocs.io/en/stable/solving_pyomo_models.html#supported-solvers>`_ to learn more. 
+The command to run the scheduling solver. This is the executable command which FlexMeasures calls via the `pyomo library <http://www.pyomo.org/>`_. Potential values might be ``cbc``, ``cplex``, ``glpk`` or ``appsi_highs``. Consult `their documentation <https://pyomo.readthedocs.io/en/stable/solving_pyomo_models.html#supported-solvers>`_ to learn more. 
+We have tested FlexMeasures with `HiGHS <https://highs.dev/>`_ and `Cbc <https://coin-or.github.io/Cbc/intro>`_.
+Note that you need to install the solver, read more at :ref:`installing-a-solver`.
+
+Default: ``"appsi_highs"``  (in unit testing, we use ``cbc`` as it runs well for us in Python3.8)
+
 
-Default: ``"cbc"``
 
 FLEXMEASURES_HOSTS_AND_AUTH_START
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

documentation/dev/docker-compose.rst

@@ -90,7 +90,7 @@ Let's go into the `flexmeasures-worker` container:
 
     $ docker exec -it flexmeasures-worker-1 bash
 
-There, we add the price data, as described in :ref:`tut_toy_schedule_price_data`. Create the prices and add them to the FlexMeasures DB in the container's bash session.
+There, we'll now add the price data, as described in :ref:`tut_toy_schedule_price_data`. Copy the commands from that section and run them in the container's bash session, to create the prices and add them to the FlexMeasures DB.
 
 Next, we put a scheduling job in the worker's queue. This only works because we have the Redis container running ― the toy tutorial doesn't have it. The difference is that we're adding ``--as-job``:
 

documentation/host/deployment.rst

@@ -45,13 +45,28 @@ The web server is told about the WSGI script, but also about the object which re
 Keep in mind that FlexMeasures is based on `Flask <https://flask.palletsprojects.com/>`_, so almost all knowledge on the web on how to deploy a Flask app also helps with deploying FlexMeasures. 
 
 
+.. _installing-a-solver:
+
 Install the linear solver on the server
 ---------------------------------------
 
-To compute schedules, FlexMeasures uses the `CBC <https://github.com/coin-or/Cbc>`_ (FlexMeasures solver by default) or `HiGHS <https://highs.dev/>`_ mixed integer linear optimization solver.
+To compute schedules, FlexMeasures uses the `HiGHS <https://highs.dev/>`_ mixed integer linear optimization solver (FlexMeasures solver by default) or `Cbc <https://github.com/coin-or/Cbc>`_.
 Solvers are used through `Pyomo <http://www.pyomo.org>`_\ , so in principle supporting a `different solver <https://pyomo.readthedocs.io/en/stable/solving_pyomo_models.html#supported-solvers>`_ would be possible.
 
-CBC needs to be present on the server where FlexMeasures runs, under the ``cbc`` command.
+They need to be installed in addition to FlexMeasures. Here is advice on how to install the two solvers we test internally:
+
+
+.. note:: We default to HiGHS, as it seems more powerful, but during unit tests we currently run Cbc, as it works for us on Python3.8
+
+
+HiGHS can be installed using pip:
+
+.. code-block:: bash
+
+   $ pip install highspy
+
+
+Cbc needs to be present on the server where FlexMeasures runs, under the ``cbc`` command.
 
 You can install it on Debian like this:
 
@@ -66,10 +81,4 @@ pass a directory for the installation.
 
 In case you want to install a later version, adapt the version in the script. 
 
-HiGHS can be installed using pip:
-
-.. code-block:: bash
-
-   $ pip install highspy
-
 

documentation/index.rst

@@ -222,6 +222,7 @@ The platform operator of FlexMeasures can be an Aggregator.
     host/docker
     host/data
     host/deployment
+    configuration
     host/queues
     host/error-monitoring
     host/modes
@@ -241,7 +242,6 @@ The platform operator of FlexMeasures can be an Aggregator.
     :maxdepth: 1
 
     dev/introduction
-    configuration
     dev/api
     dev/ci
     dev/auth

flexmeasures/utils/config_defaults.py

@@ -196,6 +196,9 @@ class TestingConfig(Config):
     SECURITY_HASHING_SCHEMES: list[str] = ["hex_md5"]
     SECURITY_DEPRECATED_HASHING_SCHEMES: list[str] = []
     FLEXMEASURES_MODE: str = "test"
+    FLEXMEASURES_LP_SOLVER: str = (
+        "cbc"  # this solver is currently the one we know is working in Python3.8
+    )
     FLEXMEASURES_PLANNING_HORIZON: timedelta = timedelta(
         hours=2 * 24
     )  # if more than 2 days, consider setting up more days of price data for tests