Refactor docker image and create a helm chart

[ludomikula]

I took the initial helm chart from @rhythmicdevil as a starting point .
There are couple of main changes:

1. Docker image was split into two images - one for backend (server) and one for frontend (client)
2. Helm chart as well as docker-compose now pull openblocks-ce-backend and openblocks-ce-frontend images
3. Docker compose now starts mongo database and redis as a separate service
4. Mongo database and redis are no longer part of the image
5. Helm chart creates two services, one for frontend and one for backend

Reasons why I split the image:

• docker container should have only one responsibility
• smaller docker images
• better horizontal scaling

[CLAassistant]

All committers have signed the CLA.

[jerry-goodman]

Thank you so much for your contribution, it's really nice. We'd like to have some discussions with you about the changes to image splits.
We did consider splitting openblocks-ce image into multiple containers before, and the all-in-one image was created to make it easier to get started with running an Openblocks instance, when we launched some new functions and services, such as WebSocket, workflow, etc. Instead of updating your docker-compose.yml, you can simply update the openblocks container to get the latest changes.
Maybe we can provide a way to prevent container from running embedded MongoDB or Redis at all.
Looking forward to your reply and opinions 😄

[ludomikula]

Yes, I can see your reasons for having an all-in-one image, but updating the docker-compose should not be that difficult either, right? :)

For development or educational purpose it's ok, but for a production purpose it's not that good especially if you want to run it in cluster.
When you need to scale your deployment, the smaller the image and the smaller memory/cpu consumption the better.
If you have an all-in-one monolithic container, you are not able to scale it efficiently, because the bottleneck might be just one of the applications installed within the container.
If the bottleneck is the database, you want to scale just that one, if the bottleneck is the server application, just scale that one etc. Even if you don't start the applications like you mentioned, it will still be part of the container taking space - this might not look that important, but it adds up.

There are a lot of different use cases for which you will be deploying openblocks and in my opinion having the containers separated makes it possible to handle all of them efficiently.
Imagine you serve one or couple of openblocks applications to one customer. To make it completely separated, I would install a separate instance per customer to make it easier to do backups/upgrades/maintenance as well as scaling. If one customer has 10 parallel users and the other has 1000 parallel users I can set the resources appropriately without affecting each other.

If you want to install openblocks as SaaS service, you would definitely need to be able to scale server application and have a separate database and redis. With enough users joining, I would probably split the server application service into separate auth/api/login services (handling requests for one endpoint eg. /login) where one instance would take care just for authentication etc, to scale just the part of the application which might be a bottleneck. (By splitting the service I mean running a server application container which would handle just /login requests, another one to handle just /api requests etc).

It's also a best practice to decouple applications in containers (https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#decouple-applications).

[jerry-goodman]

Hi @ludomikula.
Our biggest cost is to maintain multiple docker images (all-in-one and backend/frontend), which will increase the cost of our team's testing and troubleshooting.
If you use the external MongoDB and Redis, the image is stateless. It is easy to scale horizontally. The refined distribution of traffic can also be solved by adding a layer of load balance.
The image size is really a problem. We can use the startup command to make unrelated services not occupy memory. The disk space is a little larger, but it should not be a big problem now.

[ludomikula]

Would it be feasible to build it this way (which could satisfy everybody):
One Dockerfile with 2 intermediary images and one final image.

Intermediary:

• openblocks-ce-backend (only the server app)
• openblocks-ce-frontend (only the client app + node server)
  Both these images would work also standalone.

Final image (all-in-one) would:

• copy the whole backend installation from openblocks-ce-backend image
• copy the whole frontend installation from openblocks-ce-frontend image
• add nginx, mongodb, redis and supervisor like it's done today
• supervisor would reuse startup scripts from copied backend and frontend to start them up

This way you can build all-in-one image same way like it is done now, but it gives you the ability to create also the standalone images like this:

docker build -f deploy/docker/Dockerfile -t openblocksdev/openblocks-ce-backend --target openblocks-ce-backend .
docker build -f deploy/docker/Dockerfile -t openblocksdev/openblocks-ce-frontend --target openblocks-ce-frontend .

[jerry-goodman]

@ludomikula. This is a great solution, thank you so much for your suggestion! Not sure if you'd be interested in contributing to it.
Node server is used to serve the data source plugin written in JavaScript like s3, evaluate js code, etc. So I think the node server as being a standalone service might be a better way. Or package with the backend server.
Looking forward to your reply and opinions 😄

[ludomikula]

Yeah I will update this pull request to do it that way. I would probably opt for doing separate nodejs image.
What do you suggest to name the node server image? Something like openblocks-ce-datasource ?
Also, is the nodejs server optional (if you can omit that specific datasource functionality) or is it mandatory?

[jerry-goodman]

1. The node server will have more functionality in the future, so openblocks-ce-node-service might be better.
2. I think it should be mandatory.

[ludomikula]

@jerry-goodman I have reworked the docker building process to create the intermediary images as well as the all-in-one image which is composed from the "standalone" ones. Will re-add the helm chart once it's updated.

See deploy/docker/README.md for some details. Any suggestions/fixes/ideas welcome :)

[jerry-goodman]

Thanks for your contribution, I'll take a closer look at this later.

[jerry-goodman]

Hi @ludomikula. I found some points that do not support the arm architecture after tests. We can merge your pr first, and I will modify it later. Thanks again for your contribution 💪💪.