Unclear how configure plugin in "docker compose" approach

[rhpijnacker]

Hello,

I'm trying to set-up an on-premise FlexMeasures deployment using docker-compose.yml as starting point.
After creating the account and administrator user, I want to add the ENTSOE plugin to fetch energy price data.

I added a copy of the flexmeasures-entsoe plugin code to the flexmeasures-instance folder and configured the entries in flexmeasures.cfg as described. However, it is not clear to me whether the plugin should be running in the server or in the worker docker container. Should I use the /app/instance/ path, so it is only loaded in the server docker, or should I use /usr/var/flexmeasures-instance? The first path is not mounted in the worker, which is throwing errors. But using the second path, the worker is complaining about loading the entsoe-py dependency, as the requirements.txt is not applied in the worker.

[nhoening]

Hi @rhpijnacker I believe this is more of a personal choice. (and maybe this is actually more fitting of an issue on the flexmeasures repo)

At Seita, we even run another kind of service (next to web server and queue worker) must for cron jobs - and the entso tasks are each modelled as a cron job.

If you stay within the current docker-compose architecture, the worker node might be conceptually the more fitting choice (it would do everything that isn't serving web requests). It's a full Ubuntu, so it has a crontab.

Installing requirements.txt then only really has to happen in the worker. I see what you mean though, in that pip install -r instance/requirements.txt only happens in the server - there is no reason I can think of that it shouldn't happen in the worker, as well. In our production infrastructure, we install plugins plus dependencies in both.

I would welcome a pull request which upgrades the worker to be able to run plugins (maybe installing extra requirements from pip install -r /usr/var/flexmeasures-instance/requirements.txt ?

[nhoening]

Conversation can go on in the FM PR.