Skip to content

Commit

Permalink
Feat/update docs (#562)
Browse files Browse the repository at this point in the history
* Update input manifest docs

* Add simple AWS only example

* Add simple Azure only example

* Add `server_type` description

* Add simple GCP only example

* Add simple Hetzner only example

* Add simple OCI only example

* Rework the example manifest

* Clean up claudie workflow docs

* Address issues with md lint

* Fix link to dynamoDB

* Remove unnecessary code blocks

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
  • Loading branch information
MiroslavRepka and github-actions[bot] committed Feb 6, 2023
1 parent af904a3 commit 7ec5cdb
Show file tree
Hide file tree
Showing 23 changed files with 1,313 additions and 170 deletions.
25 changes: 14 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

Platform for managing multi-cloud Kubernetes clusters with each nodepool in a different cloud-provider.

# Features
## Features

### Manage multi-cloud Kubernetes clusters

Expand All @@ -15,7 +15,7 @@ Simply insert credentials to your cloud projects, define your cluster, and watch

![Infra Diagram](docs/infra-diagram.png)

### Management via IaC
### Management via IaC

Declaratively define your infrastructure with a simple, easy to understand YAML [syntax](./docs/input-manifest/input-manifest.md).
See example [manifest](./docs/input-manifest/example.yaml).
Expand All @@ -24,18 +24,19 @@ See example [manifest](./docs/input-manifest/example.yaml).

To scale-up or scale-down, simply change a few lines in the input manifest and Claudie will take care of the rest in the matter of minutes.

### Loadbalancing
### Loadbalancing

Claudie has its own managed load-balancing solution, which you can use for Ingresses, the Kubernetes API server, or generally anything. Check out our [LB docs](https://github.com/Berops/claudie/tree/master/docs/loadbalancing).

### Persistent storage volumes
### Persistent storage volumes

Claudie comes pre-configured with a storage solution, with ready-to-use Storage Classes. See [Storage docs](https://github.com/Berops/claudie/tree/master/docs/storage) to learn more.

# Get started using Claudie
## Get started using Claudie

1. Deploy Claudie Kubernetes [manifests/claudie](https://github.com/Berops/claudie/tree/master/manifests/claudie) into a Kubernetes cluster:
```

```sh
kubectl apply -k manifests/claudie
```

Expand All @@ -49,24 +50,26 @@ After the input manifest is successfully applied, the kubeconfig to your newly
built clusters is output as a secret in the `claudie` namespace with a name in
the form of `<cluster-name>-<cluster-hash>-kubeconfig`.
# Get involved
## Get involved
<!-- Contributor guidelines -->
Everyone is more than welcome to open an issue, a PR or to start a discussion.
Everyone is more than welcome to open an issue, a PR or to start a discussion.
For more information about contributing please read the [contribution guidelines](./docs/contributing/contributing.md).
If you want to have a chat with us, feel free to join our ["claudie-workspace" Slack workspace](https://join.slack.com/t/claudie-workspace/shared_invite/zt-1imfso8r4-xwrpZjL9kt61FT1LjvWD5w).
# Security
## Security
While we strive to create secure software, there is always a chance that we
miss something.
If you've discovered something that requires our attention, see [our security
policy](SECURITY.md) to learn how to proceed.

# Roadmap
## Roadmap
<!-- Add a roadmap for claudie so users know which features are being worked on and which will in future -->
To see the vision behind Claudie, please refer to the [roadmap](./docs/roadmap/roadmap.md) document.

### LICENSE
## LICENSE

Apache-2.0 (see [LICENSE](LICENSE) for details).
1 change: 1 addition & 0 deletions SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
Thanks for helping to make this project safe for everyone.

## Reporting security issues

If you discover a security issue in this repo, please report it to us at
**<a href="mailto:security&commat;berops&period;com">security&commat;<!--do not scrape me-->berops&period;<!--do not scrape me-->com</a>**.

Expand Down
Binary file modified docs/claudie-workflow/claudie-diagram.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
67 changes: 43 additions & 24 deletions docs/claudie-workflow/claudie-workflow.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,11 @@
# Claudie
### A single platform for multiple clouds

## A single platform for multiple clouds

![claudie schema](claudie-diagram.jpg)

### Microservices

- [Context-box](https://github.com/Berops/claudie/tree/master/services/context-box)
- [Scheduler](https://github.com/Berops/claudie/tree/master/services/scheduler)
- [Builder](https://github.com/Berops/claudie/tree/master/services/builder)
Expand All @@ -13,10 +16,13 @@
- [Frontend](https://github.com/Berops/claudie/tree/master/services/frontend)

### Data stores

- [MongoDB](https://github.com/Berops/claudie/tree/master/manifests/claudie/mongo)
- [Minio](https://github.com/Berops/claudie/tree/master/manifests/claudie/minio)
- [DynamoDB](https://github.com/Berops/claudie/tree/master/manifests/claudie/dynamo)

### Tools used

- [Terraform](https://github.com/hashicorp/terraform)
- [Ansible](https://github.com/ansible/ansible)
- [KubeOne](https://github.com/kubermatic/kubeone)
Expand All @@ -26,11 +32,12 @@
- [K8s-sidecar](https://github.com/kiwigrid/k8s-sidecar)
- [gRPC](https://grpc.io/)


## Context-box

Context box is Claudie's "control unit". It holds pending configs, which need to be processed, periodically checks for new/changed configs and receives new configs from `frontend`.

### API

```go
// Saves the config parsed by Frontend.
rpc SaveConfigFrontEnd(SaveConfigRequest) returns (SaveConfigResponse);
Expand All @@ -54,56 +61,60 @@ Context box is Claudie's "control unit". It holds pending configs, which need to
// Deletes the config from the database.
rpc DeleteConfigFromDB(DeleteConfigRequest) returns (DeleteConfigResponse);
```

### Flow

- Receives a `config` from Frontend, calculates its `msChecksum` and saves it to the database
- Periodically checks for `config` changes and pushes the `config` to the `schedulerQueue` if `msChecksum` != `dsChecksum`
- Periodically checks for `config` changes and pushes the `config` to the `builderQueue` if `dsChecksum` != `csChecksum`
- Receives a `config` with the `desiredState` from Scheduler and saves it to the database
- Receives a `config` with the `currentState` from Builder and saves it to the database

### Variables used

| variable | meaning |
| ------------ | ---------------------- |
| `msChecksum` | manifest checksum |
| `dsChecksum` | desired state checksum |
| `csChecksum` | current state checksum |


## Scheduler

Scheduler brings the infrastructure to a desired the state based on the manifest contained in the config that is received from Context-box.

Scheduler also monitors the health of current infrastructure and manages any operations based on actual health state (e.g. replacement of broken nodes, etc. *[work in progress]*).

### API
```
This service is a gRPC client, thus it does not provide any API
```

>This service is a gRPC client, thus it does not provide any API
### Flow

- Periodically pulls `config` from Context-Box's `schedulerQueue`
- Creates `desiredState` with `dsChecksum` based on the `config`
- Sends the `config` file back to Context-box


## Builder

Builder aligns the current state of the infrastructure with the desired state. It calls methods on `terraformer`, `ansibler`, `kube-eleven` and `kuber` in order to manage the infrastructure. It follows that Builder also takes care of deleting nodes from a kubernetes cluster by finding differences between `desiredState` and `currentState`.

### API
```
This service is a gRPC client, thus it does not provide any API
```

>This service is a gRPC client, thus it does not provide any API
### Flow

- Periodically polls Context-Box's `builderQueue` for changes in `config`, pulls it when changed
- Calls Terraformer, Ansibler, Kube-eleven and Kuber
- Creates `currentState`
- Sends updated `config` with the `currentState` to Context-box


## Terraformer

Terraformer creates or destroys infrastructure (specified in the desired state) via Terraform calls.

### API

```go
// Builds the infrastructure based on the provided desired state (includes addition/deletion of *stuff*).
rpc BuildInfrastructure(BuildInfrastructureRequest) returns (BuildInfrastructureResponse);
Expand All @@ -112,19 +123,22 @@ Terraformer creates or destroys infrastructure (specified in the desired state)
```

### Flow

- Receives a `config` from Builder
- Uses Terraform to create infrastructure based on the `desiredState`
- Updates the `currentState` in the `config`
- Upon receiving a deletion request, Terraformer destroys the infrastructure based on the current state


## Ansibler

Ansibler uses Ansible to:
- set up Wireguard VPN between the nodes
- set up nginx load balancer
- install dependencies for nodes in a kubernetes cluster

- set up Wireguard VPN between the nodes
- set up nginx load balancer
- install dependencies for nodes in a kubernetes cluster

### API

```go
// InstallNodeRequirements installs any requirements there are on all of the nodes.
rpc InstallNodeRequirements(InstallRequest) returns (InstallResponse);
Expand All @@ -138,45 +152,49 @@ Ansibler uses Ansible to:
```

### Flow

- Receives a `configToDelete` from Builder for `TeardownLoadBalancers()`
- Finds the new ApiEndpoint among the control nodes of the k8s-cluster.
- Sets up new certs for the endpoint to be reachable
- Receives a `config` from Builder for `InstallVPN()`
- Sets up ansible *inventory*, and installs the Wireguard full mesh VPN using a playbook
- Updates the `currentState` in a `config`
- Receives a `config` from Builder for `InstallNodeRequirements()`
- Sets up ansible *inventory*, and installs any prerequisities, as per individual nodes' requirements
- Sets up ansible *inventory*, and installs any prerequisites, as per individual nodes' requirements
- Updates the `currentState` in a `config`
- Receives a `config` from Builder for `SetUpLoadbalancers()`
- Sets up the ansible inventory, and installs nginx load balancers
- Creates and verifies the DNS configuration for the load balancers


## Kube-eleven

Kube-eleven uses [KubeOne](https://github.com/kubermatic/kubeone) to set up kubernetes clusters.
After cluster creation, it assures the cluster stays healthy and keeps running smoothly.

### API

```go
// BuildCluster builds the kubernetes clusters specified in the provided config.
rpc BuildCluster(BuildClusterRequest) returns (BuildClusterResponse);
```

### Flow

- Receives a `config` object from Builder
- Generates KubeOne manifest based on the `desiredState`
- Uses KubeOne to provision a kubernetes cluster
- Updates the `currentState` in the `config`


## Kuber

Kuber manipulates the cluster resources using `kubectl`.

### API

```go
// StoreClusterMetatada creates a secret, which holds the private key and a list of public IP addresses of the cluster supplied.
// StoreClusterMetadata creates a secret, which holds the private key and a list of public IP addresses of the cluster supplied.
rpc StoreClusterMetadata(StoreClusterMetadataRequest) returns (StoreClusterMetadataResponse);
// DeleteClusterMetatada deletes the secret holding the private key and public IP addresses of the cluster supplied.
// DeleteClusterMetadata deletes the secret holding the private key and public IP addresses of the cluster supplied.
rpc DeleteClusterMetadata(DeleteClusterMetadataRequest) returns (DeleteClusterMetadataResponse);
// SetUpStorage installs Longhorn into the cluster.
rpc SetUpStorage(SetUpStorageRequest) returns (SetUpStorageResponse);
Expand All @@ -189,24 +207,25 @@ Kuber manipulates the cluster resources using `kubectl`.
```

### Flow

- Receives a `config` from Builder for `SetUpStorage()`
- Applies the `longhorn` deployment
- Receives a `config` from Builder for `StoreKubeconfig()`
- Creates a kubernetes secret that holds the kubeconfig of the Claudie-created cluster
- Upon infrastructure deletion request, Kuber deletes the kubeconfig secret of the cluster being deleted


## Frontend

Frontend is a layer between the user and Claudie.
New manifests are added as secrets into the kubernetes cluster where `k8s-sidecar` saves them into Frontend's file system
and notifies the Frontend service via a HTTP request that the new manifests are now available.

### API
```
This service is a gRPC client, thus it does not provide any API
```

>This service is a gRPC client, thus it does not provide any API
### Flow

- User applies a new secret holding a manifest
- `k8s-sidecar` detects it and saves it to Frontend's file system
- `k8s-sidecar` notifies Frontend via a HTTP request that changes have been made
Expand Down
Loading

0 comments on commit 7ec5cdb

Please sign in to comment.