When an individual or a company deploys a kubernetes cluster for workload management. What is the next thing required. Security, Monitoring, Logging, Tracing, Alerting and CI/CD for workloads, a lot of great opensource tools exist for these processes and selecting a tools based on industry's standards and recommendations is quite a hard & duanting task as it requires a lot of research that fulfill the requirements.
We at Stakater understand the industry's problem and keeping that in mind we have developed a consolidated solution named StakaterPlatform
that will help the community to adopt Security, Monitoring, Logging, Tracing, Alerting and CI/CD processes for thier workloads.
StakaterPlatform
gives a head-start to individuals and companies with a set of seven stacks containing best of the breed opensource tools based on industry's recommendation and best practices for monitoring
, logging
, security
, tracing
and alerting
of Kubernetes cluster. End-users only need to focus on developing their application while everything else will be managed by StakaterPlatform
!
StakaterPlatform
uses community managed helm charts and docker images.
For detailed walk-through please visit StakaterPlatform on Stakater Playbook
StakaterPlatform consist of 7 stacks:
Table for Stacks tools:
Control | Delivery | Logging | Monitoring | Security | Alerting | Tracing |
---|---|---|---|---|---|---|
External DNS | Jenkins | Elasticsearch | Prometheus Operator | Keycloak | Uptime Robot | Istio |
Nginx Ingress | Sonatype-Nexus | Fluentd | Metrics Server | Proxy Injector | Slack | |
Descheduler | RDLM | Eventrouter | ||||
Forecastle | Kibana | |||||
Reloader | Cerebro | |||||
Sealed Secret Controller | Elasticsearch Curator | |||||
K8S Dashboard | Logrotate | |||||
Gitwebhookproxy | Konfigurator | |||||
Ingress Monitor Controller | ||||||
Xposer |
- Kubernetes cluster with at least 8 VCPUS & 32 GB of RAM
- A working domain (e.g.
stakaterplatform.com
) - SSL Certificate for that domain. Creating & using custom SSL certificates
- Flux: You define the entire desired state of your cluster in git and flux ensures that the current state matches the one declared in repo.
- Kubeseal: Required to encrypt base64 encoded secrets to commit-able(git) sealed-secrets
Installation tutorial for local machine can be seen here
- kubectl (between v1.11 & v1.15.3)
- helm (v2.15.0 or lower)
- kubeseal
-
Duplicate this repository.
-
Update configuration variables in
variables.config
file and provide the relevant values. -
[Recommended but optional] To take full advantage of the tool stack configure Additional Variables as well.
-
Ensure that correct context is set for kubectl & helm.
-
run
make configure
, this will make all required substitutions based on configuration variables in the repository. When prompted commit those changes, don't commit them if you want to run everything from your local machine i.e. No flux hence no GitOps. -
For deployment there are two options:
a. Using flux, Add the public SSH key of flux(configs/flux.pub) to your Git repository with write access. For flux run
make deploy-flux
and provide git credentials on prompt.b. Without flux, just don't add flux key and this will remove the process of GitOps(using flux) from your cluster. It removes the need of having your repository available remotely. Run
make deploy
-
Estimated time for everything to be up and running is 5-10. Use the printed dashboard token to access the Kubernetes dashboard at
dashboard-control.YOURDOMAINNAME
-
Visit
https://forecastle-control.YOURDOMAINNAME
and you'll be able to view all applications deployed by StakaterPlatform.
Note: Since variables.config
file and configs/
directory contains private information those files are not being
tracked in git and won't/shouldn't be committed. In case you want to commit those changes run make track-secrets
.
- Duplicate this repository in a GitLab account.
- Update configuration variables in
variables.config
file and provide the relevant values. - Create a Personal Access Token on GitLab and note down Personal Access Token (Available only when created)
- Configure CI/CD Environment variables in
<Your GitLab Project> -> Settings -> CI/CD
. - Add Following variables for the pipeline
Pipeline Variables | Description |
---|---|
KUBE_CONFIG | Base64 encoded KubeConfig of the kubernetes cluster you want to deploy on |
REPO_ACCESS_TOKEN | Personal access token generated in Step 3 |
TARGET | Make file target. Value: pipeline-deploy |
AWS_ACCESS_KEY_ID | (Define only if CLOUD_PROVIDER is aws ) AWS Access Key Id. |
AWS_SECRET_ACCESS_KEY | (Define only if CLOUD_PROVIDER is aws ) AWS Secret Access Key. |
- Add the public SSH key(Deploy Keys) to your GitLab account with write access printed at the end of pipeline logs.
- Once key is added StakaterPlatform will deploy on your cluster (Estimated time: 5-10 minutes). 🎊
- Use the printed dashboard token to access the Kubernetes dashboard at
dashboard-control.DOMAIN
Run make verify
to run tests to ensure that all the relevant endpoints are up and running.
Run pipeline with Pipeline variable: TARGET
= verify
StakaterPlatform offers a tool Forecastle to access tools from a single panel. Go to forecastle-control.DOMAIN
to access forecastle after complete deployment where you can access all the tools offered by StakaterPlatform and verify if working fine
See default username and passwords set for tools here
Variables | Description | Default |
---|---|---|
CLOUD_PROVIDER | Name of the k8s cloud provider | nil ( aws | azure ) |
DNS_PROVIDER | Cloud DNS Provider | aws (Route53) |
EXTERNAL_DNS_AWS_ACCESS_KEY_ID | AWS Access Key Id having access to create/delete/update Route53 HostedZone entries | nil |
EXTERNAL_DNS_AWS_SECRET_ACCESS_KEY | AWS Secret Access Key having access to create/delete/update Route53 HostedZone entries | nil |
DOMAIN | Domain to use for StakaterPlatform | nil |
BASE64_ENCODED_SSL_CERTIFICATE_CA_CRT | Base64 encoded Intermediate Certificate value | nil |
BASE64_ENCODED_SSL_CERTIFICATE_TLS_CRT | Base64 encoded Server Certificate value | nil |
BASE64_ENCODED_SSL_CERTIFICATE_TLS_KEY | Base64 encoded Certificate Key value | nil |
STAKATER_PLATFORM_SSH_GIT_URL | SSH URL for your Github repo. | nil (e.g git@github.com/stakater/StakaterPlatform.git . Notice : is replaced with / in the URL ) |
STAKATER_PLATFORM_BRANCH | Branch to use for STAKATER_PLATFORM_SSH_GIT_URL |
master |
For generating certificates view: Creating & using custom SSL certificates
See Detailed Configurations for configuring available tools in the stacks.
See Stakater Playbook for stack definitions and configuration options.
Run make destroy
to remove StakaterPlatform from your cluster.
- Replace all secrets with sealed-secrets. Stakater Workshop - Sealed Secrets In coming updates for StakaterPlatform this will be followed by default
- Change default usernames and passwords for all tools (
variables.config
) - Add your own SSH keys for flux
- Use Identity Providers (e.g. Google, Active Directory etc.) and configure keyCloak to use that. KeyCloak with Identity Providers
- Use keycloak for SSO over all exposed applications
- Enable mTLS between microservices to prevent unauthorized connections between pods
- Only allow restricted access to users other than the administrator for dashboard
- Don't share your kubeconfig file and other secrets, it's suggested to keep such things in vault
StakaterPlatform has been tested on following kubernetes flavors:
Cloud | Kubernetes Version | Stakater Platform Version |
---|---|---|
AWS ![]() |
1.14.6 | v0.0.1 |
Azure ![]() |
1.14.8 | v0.0.1 |
If you have questions, check the Documentation and talk to us on slack #community on Stakater Slack.