[Official] Contentify Cloud

[chriskonnertz]

Contentify Cloud

Contentify Cloud is nothing less than a true paradigm shift in the development, support and operation of Contentify. For a looong time (years!) one of the biggest issue with Contentify was the most important thing: How to get it run?! Over the years we have implemented plenty of improvements to help people with this step. But, at the end of the day, it was not enough. Too many had serious trouble.

Contentify Cloud is the definitive solution: Instead of letting you alone with the challenge to configure the hosting environment and installing Contentify, we will take care. We will offer you managed hosting for your Contentify website and it will be 100% free. This is no joke and no trick. We will combine the advantages of an open source CMS with the advantages of software as a service (SaaS) to create a new experience for gamers all around the world.

Free Plan

There will be different plans, presumably "Free" and "Premium". We will start with the "Free" plan. It will have a lot of restrictions in order to minimize costs, since you get it for free while we have to pay for the server infrastructure. These are the restrcitions:

• No domain included. That's really nothing to put on our bills. Domains can be bought super cheap for just a few cents per months so it's not a big deal for you, but it would be for us, because small costs add up to big costs if a lot of users would use this. It also makes organisation more complicate.
• The size of the disk space will be limited to a small size, probably 300 MB. This is enough for small websites. You can use external storages to extend the space or you will be able to upgrade to a paid plan with more space. The good new is, it will be SSD disks.
• CPU + RAM will be shared. We will make sure there's enough power to run small websites. If your website starts to use to much of these resources we will ask you to switch to a paid plan. If you do not switch to a paid plan we will explicitly restrict the resources your website can use.
• No backup store included. If you want to have permanent, complete backups, then you will have to use external storage.
• No load balancing, no scalability, no DDOS protection, no caching other than file system based etc. -> the paid "Premium" plan might cover these requirements
• At least at the beginning we will not offer different server locations - servers will be located in Germany. However, this might change later on.
• Websites that are considered "no longer in use" will be archived or deleted. This will happen in a merciless way. Obviously we cannot carry any dead websites with us.

You will be able to move your entire website to an external hosting if you prefer. We do not jail your website.

These restrictions might seem harsh at the first glance, but keep in mind that this offering is completetly free and not supported by putting advertisement on your website. We have to keep costs at a minimal level. So we have to cut off whatever we can by ensuring the free plan makes the majority of you happy. And we believe this is the case. Even though there are a lot of restrictions we believe that most of you will be satisfied by the free plan.

Features

All plans include at least these features:

• Web environment prepared to run Contentify - you do not have to configure a server
• We monitor your website and ensure it is always online
• We automatically create daily backups of our server infrastructure - so your data is safe (note that this only covers failures o the servers, not mistakes that you make!)
• Contentify being ready to installed - just a few clicks and your website will be up and running
• Technical support for hosting-related issues
• Whenever a new version of Contenfiy is released, we do the update
• Cron job running to perform background tasks
• SSL certificate
• Emailing, so your website can send mails out-of-the-box without the need to use an external service

ToDo List -> Contentify 3.1

• Update the "Update" module to avoid confusion when the website is hosted in the cloud. Also add a "Read for update" option.
• Display system information (for example free web space) in the backend so website admins of websites hosted in the cloud know what's going on
• Add a few more settings so website admins can customize their website without having access to config files
• Add SQL query module or page where website administrators can run SQL queries
• Add unique instance ID to identify instances of Contentify websites (this does not include any form of authentication)
• Add console installer script so Contentify can be installed completely via console and without user input. We want to have this to deploy new instances of Contentify in the cloud.

More about v3.0: #451

[funtimeerror]

@chriskonnertz AWS/GCP? docker? scaling on the backend? :)

[chriskonnertz]

• AWS is too expensive, GCP: I don't know but I assume it is expensive as well. Free hosting obviously means costs have to be minimized in a drastical way.
• Docker might be they way to go but not right from the beginning. Bulding a complex infrastructure right from the start means too much overhead.
• There will be different plans. We will start with the free plan. The free plan will have a lot of resctrictions. More sophisticated plans will not be free but will probably allow upscaling.

[acamilleri]

Hi @chriskonnertz

You can try Scaleway Kapsule. It's managed Kubernetes cluster, the control plane it's free, you only pay for worker nodes used.

You can start with only one worker node and activating (for free) the autoscaler to auto add new workers (you can set a limit) when all ressources are consumed.

The cheapest worker is 7,99€/month for 3 vCPU and 4 GB RAM.

According to use a Kubernetes cluster, a load balancer is required to redirect request only to working node and auto-update a list of node (when node are replaced, deleted or added).

The cheapest load balancer is 8.99€/month, but it managed too.

Now, we need to think about the database, how you want store data ? Shared database cluster out of Kubernetes ? Shared database cluster in Kubernetes ? One dedicated database instance per Contentify instance in Kubernetes ?

I think, the important thing in this project is the service availability, free or premium users must have a good availability of their website.

Several days i create an infrastructure example on Bitbucket, i don't know if you have seen my message. I will copy paste the content here. Let me feedback about it and your opinion about the service you want.

Keep in your mind, i can help you about this subject if you want.

---

My message on Bitbucket:

My first idea, is to using Kubernetes to have some flexibility and resilience.

Kubernetes have some auto-healing and auto-scaling feature, autoscaling can help you to control the limit of the platform and the price too.
For exemple, in Scaleway (i dont know the price of GCP / AWS / DigitalOcean but it’s maybe higher):

• The control plane is free and should never be paid.
• The first compute node available it’s DEV1-M (3 vCPU / 4G RAM) 7,99€ available regions Paris only.
• The smallest load-balancer is 8,99€ (Unlimited traffic / Bandwidth limit: 200 Mbps/s) available regions Paris and Amsterdam.
• The block storage price: 0,08 €/Go/mois available regions Paris and Amsterdam.

Domain can be not managed in Scaleway (it’s an early access) but we can go to another cloud provider with API.

Infrastructure schema

How it works ?

• Load Balancer is the mandatory entrypoint to keep only one entrypoint define in domain zone (don’t manage endpoint with dns, dns protocol have some caching)
• Traefik manage request routing
• Each customer/user have a dedicated Contentify instance with dedicated ressources (https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#resource-requests-and-limits-of-pod-and-container)
• A shared mysql cluster (can be remove to have a dedicated instance without HA for each Contentify instance)

How works a request ?

• Request arrived in Load Balancer (attached to Traefik Ingress Controller)
• Traefik Ingress Controller redirect the request to the pod corresponding of the .cloud.contentify.com host in the request headers

How deploy this infrastructure and his changes ?

• Terraform in CI-CD ❤ https://www.terraform.io/

How to test code without using a paid cluster ?

• Docker-desktop / Minikube with skaffold to test on local ❤

How to monitor the infrastructure ?

• An (external ?) or internal Prometheus instance

• node_exporter as DaemonSet to watch cluster ressources used

• Prometheus scrape traefik metrics

• Implement /metrics on Contentify ? (not needed in V1 i think)

How to deploy and configure an Contentify instance ?

• a Go API with some basic endpoint ?
  • /create : create instance
  • /delete : delete instance

Beautiful way but not required.

To keep a basic API and manage instance easily, is to create a Contentify Operator, i don’t use it at the moment, but i’m reading the book (http://shop.oreilly.com/product/0636920234357.do).

The Operator it’s an extends function of Kubernetes, it can help you to maintain your ressources. It manage it with Code, you can listen on Kubernetes events related of your ressources to do anything (eg: You want to upgrade a Contentify instance, you can tell to make a backup before any upgrade, …)

[chriskonnertz]

Hey,

sorry for the delay...

I have a couple of questions, because I am not realy used to work with Kubernetes.

1. Does Scalewaly provide a Kapsule interface and is it available in English also? Or just French? (I think English because it's "Kubernetes Dashboard" right?)
2. So in one Worker Node there can be multiple Contentify websites, right?
3. So if there was one load balancer of type LP-GP-S, all incoming traffic would have to be less than 200 Mbit/s? Only the incoming traffic, right?
4. The load balancer directes the traffic between the worker nodes, or also between the websites?
5. How many worker nodes do we need at least? 1 or 2?
6. Is there something like a trial offer for the load balancer / compute node? I understand thet uebrnetes requires a laod balancer, but if for the start there are only a couple of websites, it's a bit over the top...
7. Block storage is only needed if the 40 GB of the compute node is not enough, right?
8. Is a worker node physically? So every thing in the worker node is running on the same computer?
9. How would Backups work?

Sorry for asking a lot of questions but I am not a Kubernetes specialist. :P

[acamilleri]

Hey,

No problem for the delay :), don't afraid to ask about infra, it can be challenge my proposal and help you to understand how it works.

Does Scalewaly provide a Kapsule interface and is it available in English also? Or just French? (I think English because it's "Kubernetes Dashboard" right?)

Scaleway provide a Kubernetes cluster as Kapsule name (kapsule is just the commercial name). Kubernetes needs to have an etcd database and some master node to work, theses resources are managed by Scaleway. Scaleway provide a little interface on his console to manage cluster (see scaleway resources used, create, delete or update). Kubernetes provide with a dashboard you can enabled when you will create your Kubernetes cluster (cf: https://www.scaleway.com/en/docs/get-started-with-scaleway-kubernetes-kapsule/#-Creating-a-Cluster).

The Kubernetes dashboard is in english and the Scaleway console is in french and english.

So in one Worker Node there can be multiple Contentify websites, right?

Yes an only one worker can host some Contentify website.
A worker node it's a virtual machine, on Scaleway a virtual machine is a compute instance like AWS, GCP and it's a droplet for DigitalOcean. You are limited by CPU and RAM.

There are some limitations on Scaleway Kapsule:

• The type of compute instance you can used (DEV1-M (3 vCPU / 4G RAM) 7,99€) and you can't go below. list of compute instance available for Kapsule
• Kapsule is only available in Paris.

So if there was one load balancer of type LP-GP-S, all incoming traffic would have to be less than 200 Mbit/s? Only the incoming traffic, right?

Yes, It's just a bandwidth limit on the incoming traffic. 200 Mbit/s for some small website it's ok.

The load balancer directes the traffic between the worker nodes, or also between the websites?

The load balancer directes the traffic between the workers nodes to arrive in Traefik and Traefik redirect the traffic to the correct website.

How many worker nodes do we need at least? 1 or 2?
You can start with one worker node, but it's not recommended.

A worker node are exposed to the virtual machine lifecycle (hypervisor crashed and vm are unexpectedly shutdown, the cloud provider can make a maintenance, ...)
I recommended to start a cluster with Kubernetes worker pool of two nodes and limited to 5. The pool must be created with auto-healing and auto-scaling feature (free).
Auto-healing replace a worker node in a critical status.
Auto-scaling remove or adding worker node (related to your pool limitation) based on resources used.

When a worker node are shutdown unexpectedly, container (Contentify website) are balanced to an another worker node to limit the disruption time.

Is there something like a trial offer for the load balancer / compute node? I understand thet uebrnetes requires a laod balancer, but if for the start there are only a couple of websites, it's a bit over the top...

Load Balancer is not required, but recommended.

Why a load balancer is recommended ? The load balancer have a list of worker node are available in the Kubernetes cluster, when any worker are destroyed or shutdown unexpectedly the load balancer we stop to forward any request to theses worker.

If you start the project without any load balancer, customer may have some error requests when worker shutdown or destroyed unexpectedly, because the dns cache will keep the IP of the destroyed worker node.

The Load Balancer provide a Virtual IP Address to ensure they are already available and you just need to put them in your domain, and the list of worker node are every time update.

Block storage is only needed if the 40 GB of the compute node is not enough, right?

Block storage is only needed for stateful application. Kubernetes it's just an orchestrator of container, a container can move between worker node during an update or an disruption of one or multiple worker node. A container (eg: database) must always have its files regardless of whether it is moved from one worker to another. If you store it's data on the worker node and the container is moved to another worker you will loose data. If the worker node are destroyed, you will loose data too.

Maybe, you can choose to host database outside of Kubernetes and you don't need any block storage.

Is a worker node physically? So every thing in the worker node is running on the same computer?

Yes, a worker node it's a VM (eg with DigitalOcean: droplet).

How would Backups work?

It's like you want. In Contentify, the must important thing it's the Database and file uploaded in the website.

Some open source project can running on Kubernetes to manage backup with different stockage backend.

[stale]

This issue has been automatically marked as stale, because it has not had recent activity. It will be closed in 30 days if no further activity occurs. Thank you for your contributions.