Feature: Add support for Kubernetes deployment through Kustomize

[saviogl]

Description

This PR adds initial support for Kubernetes deployments through Kustomize. This allows for a more K8s flexible deployment K8s model that can cater to users particular requirements.

It uses Airbyte's provided Kustomize distribution, and include the additional dependent resources (metabase, n8n, hasura).

There are still improvements that can be made for a true production deployment, but I think it's a good starting point to get feedback and have it integrated.

Currently it contains a base setup with no overlays, that mimics much like the local development setup provided through docker-compose. As we move forward we can start creating overlays to cater to general use cases.

Noticeable features:

• Auto Initialization using faros-init docker image (farosai/faros-ce-init:latest)
  • This is done by using a K8s Job that is deployed along with the stack
• Creation of ConfigMap based on basic .env for auto setup
  • For this I ended up creating a hard link from the root file so that we could run kustomize without the flag --load-restrictor LoadRestrictionsNone. This flag prevents files from being loaded outside of the kustomization.yaml folder.
  • This provides a ConfigMap that can be mounted onto the appropriate manifests
• Easy overrides with Kustomize which solves for secret management, security requirements, and other governance needs

Deployment

For deploying this stack provided that you have access to a cluster you can just run:

kubectl apply -k kube/base

Type of change

• New feature (non-breaking change which adds functionality)
• This change requires a documentation update

Checklist

(Delete what does not apply)

• Have you checked to there aren't other open Pull Requests for the same update/change?
• Have you lint your code locally before submission?
• Have you added an explanation of what your changes do and why you'd like us to include them?
• Have you written new tests for your core changes, as applicable?
• Have you successfully run tests with your changes locally?

[tovbinm]

Thanks for adding this! Can you please add kube/README.md with a some documentation on how to use it. (We can then port it to our official documentation website https://commuity.faros.ai)

[tovbinm]

I am not able to access any databases from Metabase UI. Is it configured correctly?

[saviogl]

Yeah it is, I got all the applications working - Do you have more information?

This might indicate that the initialization (faros-init) didn't run properly and didn't set up metabase's configuration.... 🤔

Do you have more logs context of the deployment?

[ypc-faros]

Did the initialization using faros-ce-init work for you?

I was able to port forward Hasura for example, but I can see it wasn't initialized.

[saviogl]

It works for me more often than not in Mac, but it will depend a bit on the performance of the cluster to make sure things boot up properly.

Running K8s in a Mac is often slower due to the virtualization compared to Linux. I've booted up on both, but the MacOS takes a bit longer and depending on how much retries are configured it can exhaust the retries. Linux is pretty snappy and haven't had an issue with it. I'm thinking that's likely tied to the local resources from the Mac perspective

Have you checked to see the logs on the faros-init job?

[ypc-faros]

It works for me more often than not in Mac, but it will depend a bit on the performance of the cluster to make sure things boot up properly.

Running K8s in a Mac is often slower due to the virtualization compared to Linux. I've booted up on both, but the MacOS takes a bit longer and depending on how much retries are configured it can exhaust the retries. Linux is pretty snappy and haven't had an issue with it. I'm thinking that's likely tied to the local resources from the Mac perspective

Have you checked to see the logs on the faros-init job?

Thanks! I gave my minikube cluster more resources and was able to bring it up. I'll do some local testing and get back to you soon! 👍

[saviogl]

@ypc-faros Sounds good, let me know how it goes.

There's more on what we can improve upon both on resource constraints as well as dependency management but this feel like a good starting point from what I've tested so far, and unblocks people to get going with this deployment

[ypc-faros]

allow => allows

[ypc-faros]

replace => replaced

[ypc-faros]

replace => replaced

[ypc-faros]

replace => replaced

[ypc-faros]

Thank you @saviogl 🥳 !!!