Discussion: Reasoning on the architecture and deployment of Chapter

[rkpasia]

First of all, hi everyone!
I've followed the discussion and the evolution of this project these last weeks.

In this issue I would like to take a step back and reason on the whole architecture of this project, from a deployment/management point of view.

So far, what I understood, is that Chapter should be single-container hosted, or at least "contained". If that's correct, let's move on and reason with this assumption in mind.

So, first observation:

• Next.js for the frontend (client and server-side rendering)
• Express for the backend

If Chapter is going to be a single Docker container, it should encapsulate both the front-end and the back-end of the application. Hence, Next.js could be served with server-side rendering.
Note: if Next.js is served statically, Chaper would not be able to use dynamic routes! No dynamic pages.
If next is server-side rendered, what's the pro of having an entire other app for rest APIs?
Would they still be used by the Next front-end to fetch data? Data that such front-end would be able to get by himself because it's just server-side rendered?

Maybe I'm just not getting the whole picture, but something could be redundant.

Second observation:

• Where should be located the database?

Is the database of the Chapter located in the same docker container? What about scalability?
If the Chapter needs more computation power, hence a replica of the container, will the database be duplicated?

As I said, these are consideration about the deployment/architecture of Chapter.
Maybe there is already an answer to all of those questions, but currently I think they are not that clear.
If that's not the case, I hope for a good and constructive discussion 😄

[timmyichen]

I think the plan is to have the web app be hosted on a single container, and other services hosted on other containers - so there will be a separate postgresql container. If they want to use a different hosted database it should be trivial to change the env variables to point to the external, non-docker DB.

[jacobbogers]

Any help needed to setup the database side of things, I can do the container, installation of schema scripts, DB rights, and creation of test/mock data?

[rkpasia]

To reply to both @timmyichen and @jacobbogers:

I think the problem needs to have a broader picture. If we look at what Chapter should be:

This will be a self-hosted Docker container that you can one-click deploy to the cloud, then configure through an admin panel. No coding required.

I'm worried about the section where we state "No coding required".

The deployment of what should be a single container, is becoming more and more a "microservice" architecture, or (more realistically) a distributed monolyth.

In this discussion I'm more worried about the overall user experience.
The users of chapter realistically shouldn't be coders. Maybe they are just first-time organizers of something that they don't even know what shape will take, hence are looking for a tool with less friction as possible (and even less budget as possible).

How are we going to deploy a multi-container Chapter? How are we going to let users manage a multi-container service, without coding experience?

Ok, as @jacobbogers said, let's do a dockercompose file, that's great. But who is going to deploy it and where?

I think this project will have to consider also a "deployment configurator" that will enable users to deploy a Chapter with selected providers and so on.

At least, this is my personal perspective considering the user-experience and deployment problems that could arise in the future.

[jacobbogers]

I see multi container is already being used with a dockercompose file))
I leave my comment as reference, a decision has been made, before I posted the suggestion ))

see here

You can distribute the docker compose file via npm as a cli command, aka npx @fcc/chapter start (this is just an example of the principle)

regardless if you use 1 or more containers, dockercompose would make it trivial for the installer since it takes care of all the command line arguments.

[rkpasia]

Again, @jacobbogers, you're absolutely right about dockercompose 😊 I'm all in with multi-container, it just wasn't explicit in the description of the project.

Anyway, the thing that concern me the most, is that we can't ask users without any experience of coding to launch the terminal and make a deployment with docker, no matter how easy it'll be.

Otherwise we're failing on the assumption that Chapter should be 1-click install.

[allella]

There have been no discussions of preferred or required hosts and obviously the intention is to allow anybody to own and host a Chapter wherever they want.

However, I'm assuming Quincy's "1-Click" target is based on there being cheaper hosts like DigitalOcean for which "1-Click" install profiles can be created.

Are there other similar hosting or micro-services platforms where install profiles / images could be used to by those who prefer the most simple approach.

The person doing the "1-click" would need other technical and administrative capabilities, like setting up billing and changing DNS records. So, we likely have the benefit of the primary administrator of the organization being more skilled than the promise requires.

I'd be more concerned about how these 1-Click installs are going to be upgraded than how to get the initial technical stack running.

[jacobbogers]

@allella, a "one click" would need something like an installable "exe" (windows) or a linux repo (yum , apt-get) etc etc, should be have this already for the 1st MVP release? It would be cool to have, I admit,

[allella]

@jacobbogers I think the point is that the base installation would be at an even higher infrastructure-level, not an operating system "click" or step.

The 1-Click would target the least technical and the more technical folks would almost assuredly opt for the other options with more control of the setup process.

As an example, I can go to Go Daddy and install a Wordpress or Drupal site in seconds, but I never would because I want to control and tweak the setup. Technical people are not the target of a 1-Click option.

There are hosting platforms (AWS, Digital Ocean, other IaaS types) which allow people to visit a URL and click a big button to "start" the process of automatically provisioning a server, or a collection of services, from a pre-defined image. After the "1-Click" you'd still need to provide account, billing, and optional install preferences. This administrative process require additional clicks, so I'd agree that nothing is truly a single click.

Again, I'll just say there are hosts like https://marketplace.digitalocean.com who are marketing this concept, so I believe that's where Quincy is thinking in the long-term. 1-Click, in whatever form it becomes, is not an MVP promise, nor is it likely to appeal to the people who want more control.

[jacobbogers]

@allella, I fully understand what you mean, In the past I deployed git repos to heroku in "one click" (maybe 2). Last time I used heroku it has a free tire (last time I checked),

[rkpasia]

@allella I agree with you. I still think this project could spin-off a Chapter configurator after the first releases of the MVP. The configurator, in my mind, should handle the deployment for non-tech people that would like to use Chapter. That could be achieved trough the setup of api interfaces with many IaaS providers.
But at this point, it should be considered afterwards the MVP.

What about the concern related to Next.js?

[allella]

@rkpasia NEXT.js was decided early on.

JS not my native tech-stack and I have no thoughts on the stack, but here are some earlier conversations.

#34 #56 #2

[allella]

@rkpasia @jacobbogers are there any action items or PRs to come out of this conversation? Seems like the concerns were more over Quincy's wording of "one-click" and "no coding required", which might be simple .md tweaks, if anything?

[allella]

@3piecechickendinnerTB This is an earlier conversation on the thoughts of using Docker containers for deployment.

One major point, regardless of the architecture, is that this group is not hosting one large instance for everyone in the world. Rather, each organization will self-host for whichever local chapters they see fit. So, even for large national or international organizations, the instances are likely to host dozens or lower hundreds of local chapters. So, scaling to thousands of chapters or millions of user is unlikely to be a concern.

[allella]

That said, the viability of Docker as the single best way to deploy as a "one-click" (or at least as simple as possible) is open for debate.

One big concern of mine is how organizations are going to update the instances with new updates with minimal risk of something breaking on a migration or related code or library updates.

[allella]

@MihirSomani please check out the recent and prior conversations here.

I'm sure there are other conversations in different issues, but this one is pretty relevant trying to match the hosting to the vision of Chapter.

[3piecechickendinnerTB]

@3piecechickendinnerTB This is an earlier conversation on the thoughts of using Docker containers for deployment.

One major point, regardless of the architecture, is that this group is not hosting one large instance for everyone in the world. Rather, each organization will self-host for whichever local chapters they see fit. So, even for large national or international organizations, the instances are likely to host dozens or lower hundreds of local chapters. So, scaling to thousands of chapters or millions of user is unlikely to be a concern.

Definitely agree on the use of Docker. My concern is that we have only one environment that acts as development and production and we are not using a registry as of yet. I'm very new to Docker and CI/CD but am learning quickly.

[MihirSomani]

I believe that the "1-click install" should be a post MVP feature. Although Docker containers make it possible to decouple several micro-services, they still require several environment variables to be set which will be specific to each organization. Even though some platforms like DigitalOcean offer services for creating 1-click-install profiles, ensuring a smooth configuration for organizations requires several considerations to be taken into account before creating the install profiles.

[Zeko369]

We will probably setup a heroku oneclick because it's pretty easy to get it working

[allella]

@3piecechickendinnerTB @MihirSomani

@Zeko369 is looking for guidance on Docker configuration. The conversation in is Discord #implementation-details right now. If you have time to assist, then please drop in. We may also create an issue on Github if Fran thinks it warrants an issue.