- Ortelius in a box
- Project Description
- Future Endeavours
- Pre-requisite checklist
- Developer Tools
- Pre Flight Checks & Troubleshooting Tips
- Ortelius | Supply Chain Catalog & DevOps Intelligence
- Ortelius
- Docker.com | Application Containerisation Platform
- Terraform | Provisioning & Managing IaC
- KinD | Kubernetes in Docker
- Container Runtime Interface (CRI) CLI | Container Troubleshooting Tool
- Kubernetes.io | The Conductor of Containerised Applications
- KubeShark Realtime visibility into K8's
- DevSpace | Dev Container inside K8s
- GitHub Codespaces & Dev Containers | Devcontainers for your GitHub repos
- Leapp | Cross-Platform Cloud Access Application
- Localstack | AWS Cloudstack on your local machine with Docker extensions
- Thunderclient | API Testing Inside VS Code
The goal is to open up the Microservices world and give visibility to developers so that it does not feel like you are developing with a blind fold on.
Ortelius and its supporting Microservices are deployed into a Kind cluster using Terraform.
DBeaver or any suitable database client for Postgresql can be used to access the Postgresql database. The Postgres database is persisted using volume mounts.
Devspace is the tool for deploying and connecting VS Code to the Dev Container inside the Ortelius namespace. The Dev Container will contain all the tools required for development of the Ortelius technology stack.
Kubeshark can be used by the developer for Microservice API troubleshooting.
The LocalStack extension in your Docker Desktop opens up the cloud world for you to develop against AWS services running locally on your machine.
Leapp is the one ring to rule them all when it comes to cloud provider authentication and role switching.
Thunderclient is a powerful API tester inside VS Code.
The great thing is that this is immutable and transportable to any operating system that supports Docker, Kubectl, Helm, Kind and Terraform.
Whilst I have tried to cover every scenario in the documentation I welcome feedback for any ambiguous or missing instructions.
- Migrate the Ortelius eco-system to AWS using LocalStack.
- Improve the Devcontainer experience.
- Implement our Event Driven Architecture POC.
- If you have Helm Charts installed run
helm repo update - Kube config is expected to be in the default location
$KUBECONFIG - If you get an
AJAXerror when trying to login then logout first withhttp://localhost:8080/dmadminweb/Logoutor you could use incognito mode (Its a cookie issue) - If you get incorrect username or password the database is probably borked, destroy and re-deploy
- If everything goes completely haywire manually remove everything such as
*.tfstateand the Ortelius Docker containers representing the Kind K8s nodes and executeterraform plan -auto-approveandterraform apply -auto-approve - Please make sure you have a folder in your root
/tmp/postgresand please make sure you add this mount point to your Docker Desktop underSettings --> Resources --> File Sharing(/tmp is sufficient) - If you are already using ports 8080 or 5432 in your environment you will need to change the ports in
main.tfhere
# ortelius http port | change port 8080 to something else if it clashes with your environment
extra_port_mappings {
container_port = 31000
host_port = 8080 <-- make the port change here
listen_address = "0.0.0.0"
}
# postgresql port | change port 5432 to something else if it clashes with your environment
extra_port_mappings {
container_port = 31316
host_port = 5432 <-- make the port change here
listen_address = "0.0.0.0"
}
}
Ortelius | Supply Chain Catalog & DevOps Intelligence
Ortelius is central catalog of supply chain and DevOps intelligence. It is designed to track and version composition details for every component of your software supply chain along with all consuming logical applications. With Ortelius, you can easily view your logical application's SBOMs, CVEs, service dependencies, and inventory based on versions, even in a decoupled microservices architecture. Ortelius aggregates DevOps, security and supply chain data for each independent component moving through the pipeline. It is particularly useful in cloud-native, microservices architectures where the logical application becomes ambiguous. Ortelius tracks who is consuming shared components, versions them when they are updated and then creates new release candidates for every logical application that is impacted by a component change. It then aggregates that data to the logical application level so you don't have to. The latest version of Ortelius is maintained by the Ortelius Community managed by the Continuous Delivery Foundation (Linux Foundation). It was originally created by DeployHub and OpenMake Software. Our mission is to simplify the adoption of modern architecture through a world-class microservice catalog driven by a supportive and diverse global open source community.
- Ortelius on ArtifactHUB is here
- Helm Charts are here
- Chart.yaml is here
- Ortelius architecture is here
- Ortelius architecture diagram is here
How to access PostgreSQL with DBeaver
- Free multi-platform database tool for developers, database administrators, analysts and all people who need to work with databases.
- Supports all popular databases:
MySQL, PostgreSQL, SQLite, Oracle, DB2, SQL Server, Sybase, MS Access, Teradata, Firebird, Apache Hive, Phoenix, Presto, etc. - You can use DBeaver Community Edition which is available here
- Supports ChatGPT integration for smart completion and code generation as an optional extension
- Apache License 2.0
- PostgreSQL localhost mount
- Password
postgres
- Ortelius Database Tables
- List all the contexts
kubectl config get context
- Switch to the kind-ortelius context
kubectl config use-context kind-ortelius
- Help with contexts
kubectl config -h
- List all namespaces
kubectl get namespace -A
- Control Plane
ortelius-control-plane - Worker
ortelius-worker
kubectl get nodes -A
Docker.com | Application Containerisation Platform
Docker is a platform for developing, shipping, and running applications. It uses containers, which are lightweight, standalone, executable packages of software that include everything needed to run the application, including the code, runtime, libraries, environment variables, and system tools. Containers provide a consistent, isolated environment for applications to run, making it easier to develop, test, and deploy applications. Docker allows developers to automate the deployment of applications into containers and manage containers as a single unit. It simplifies the process of deploying applications, making it easier to scale applications, and enabling organizations to adopt a microservices architecture.
- Account setup
- Install
- Get familiar with the basic commands
- Use Devdocs and the Docker documentation here
- Terms & Conditions
Terraform | Provisioning & Managing IaC
Terraform is an open-source tool for provisioning and managing infrastructure as code. It provides a simple, declarative syntax for defining infrastructure resources, such as virtual machines, DNS entries, and databases. Terraform can manage popular service providers as well as custom in-house solutions. By describing infrastructure as code, Terraform enables versioning, testing, and collaboration of infrastructure changes. Terraform can create, update, and delete resources in parallel, while minimizing the risk of conflicts and errors. Terraform also provides a state management system that tracks changes to infrastructure over time, making it easier to roll back changes if necessary. With Terraform, organizations can automate their infrastructure management processes, improve reliability, and increase efficiency.
- Install Terraform here
- Documentation is here
- Terms & Conditions
- Licence
- Clone the repo
- Navigate to
/ - Run the following
terraform init
terraform plan
- Open videos in a new tab
time terraform apply --auto-approve
terraform apply --auto-approve
- Open videos in a new tab
terraform destroy --auto-approve
- Open videos in a new tab
In total, there 5 log levels which can be used for debugging purposes:
TRACEone of the most descriptive log levels, if you set the log level to TRACE, Terraform will write every action and step into the log file.DEBUGa little bit more sophisticated logging which is used by developers at critical or more complex pieces of code to reduce debugging time.INFOthe info log level is useful when needing to log some informative instructions or readme type instructions.WARNused when something is not critical but would be nice to include in the form of a log so that the developer can make adjustments later.ERRORas the name suggests, this is used if something is terribly wrong and is a blocker.
export TF_LOG="DEBUG"
export TF_LOG_PATH="/abraham/terraform-debug.log"
KinD | Kubernetes in Docker
Kind (Kubernetes in Docker) is a tool for running local Kubernetes clusters using Docker containers as nodes. It provides an easy-to-use environment for testing and developing applications that run on a Kubernetes cluster. Kind allows developers to run a full Kubernetes cluster on their development machine, eliminating the need for remote clusters and making it easier to test and debug applications. Kind creates a multi-node cluster by launching multiple Docker containers on a single host, making it possible to test complex scenarios and network configurations. Kind is also useful for testing Kubernetes plugins and extensions, and for developing and testing operators, custom controllers, and other Kubernetes-related software.
- Install here
- Documentation is here
- Terms & Conditions
- Licence
- kind supports multi-node (including HA) clusters
- kind supports building Kubernetes release builds from source
- support for make / bash or docker, in addition to pre-published builds
- kind supports Linux, macOS and Windows
- kind is a
CNCF certified conformant Kubernetes installer
kind get nodes ortelius
kind get kubeconfig -n ortelius
kind get clusters
kind export logs -n ortelius
Container Runtime Interface (CRI) CLI | Container Troubleshooting Tool
Crictl is a command line tool for interacting with a containerd-based container runtime. It provides a simple, human-readable interface for performing common container operations such as pulling images, starting and stopping containers, and viewing logs. Crictl also supports advanced features such as executing commands inside containers and managing network configurations. It is designed to be a fast and flexible alternative to other container runtime management tools, and can be used in production or development environments. Crictl is a component of the containerd project, which is a lightweight, high-performance runtime for managing containers and is used by many popular container orchestration platforms such as Kubernetes.
- Download crictl here
$ crictl ps -a
CONTAINER ID IMAGE CREATED STATE NAME ATTEMPT
3e025dd50a72d busybox 32 seconds ago Created busybox 0
$ crictl start 3e025dd50a72d956c4f14881fbb5b1080c9275674e95fb67f965f6478a957d60
3e025dd50a72d956c4f14881fbb5b1080c9275674e95fb67f965f6478a957d60
$ crictl ps
CONTAINER ID IMAGE CREATED STATE NAME ATTEMPT
3e025dd50a72d busybox About a minute ago Running busybox 0
crictl exec -i -t 3e025dd50a72d956c4f14881fbb5b1080c9275674e95fb67f965f6478a957d60 ls
bin dev etc home proc root sys tmp usr var
Kubernetes.io | The Conductor of Containerised Applications
Kubernetes is an open-source platform for automating deployment, scaling, and management of containerised applications. It provides a unified API for defining and managing containers, enabling organizations to simplify the deployment and scaling of applications. Kubernetes automates the distribution of containers across a cluster of machines and monitors the health of the containers, ensuring that they are always running. It can automatically replace failed containers and ensure that the desired number of replicas are running. Kubernetes also provides built-in service discovery and load balancing, making it easier to connect microservices and ensure that traffic is distributed evenly across the cluster. Kubernetes has become the de facto standard for container orchestration and is widely adopted by organizations of all sizes.
-
Install
kubectlthe command line tool here -
Use the
kubectlcheat sheet here -
Add the
aliases&auto completewhich are in thecheat sheet -
Real time logging of a pod
kubectl logs ms-nginx-6ccbb5f95c-9gjg7 -n ortelius -f
- Remote into a pod
kubectl -n ortelius exec -it ms-nginx-6ccbb5f95c-9gjg7 -n ortelius -c ms-nginx -- sh
kubectxis a tool to switch between contexts (clusters) on kubectl faster.kubensis a tool to switch between Kubernetes namespaces (and configure them for kubectl) easily.- Download
Kubens & Kubectxhere - Licence
- Is a general-purpose command-line fuzzy finder.
- Is an interactive Unix filter for command-line that can be used with any list; files, command history, processes, hostnames, bookmarks, git commits, etc.
- Download
Fuzzy Finderhere - Licence
kubectx kind-ortelius
kubens ortelius
KubeShark Realtime visibility into K8's
Real-time visibility into K8s' internal network, capturing, dissecting and monitoring all traffic and payloads going in, out and across containers, pods, nodes and clusters.
DevSpace | Dev Container inside K8s
- What is DevSpace
- Devspace could be used for the local Terraform environment whereas Codespaces is more suited for a developer on the repos themselves
- Licence
- Download the CLI here
- Init DevSpace in your repo and use the existing
Dockerfileanddevspace.yamlby following the prompts. - Currently it is using the Microsoft Universal Dev Container which I have built and pushed to my DockerHub
- The Microsoft Universersal Dev Container has the following language platforms
Python, Node.js, JavaScript, TypeScript, C++, Java, C#, F#, .NET Core, PHP, Go, Ruby, Conda - Licence
devspace init
- run the UI
devspace ui
- start coding
devspace dev -n ortelius
warn Are you using the correct namespace?
warn Current namespace: 'default'
warn Last namespace: 'devspace'
? Which namespace do you want to use? devspace
info Using namespace 'devspace'
info Using kube context 'kind-ortelius'
info Created namespace: devspace
deploy:app Deploying chart component-chart (app) with helm...
deploy:app Deployed helm chart (Release revision: 1)
deploy:app Successfully deployed app with helm
dev:app Waiting for pod to become ready...
dev:app DevSpace is waiting, because Pod app-devspace-6dcdc9bf56-trrrc has status: ContainerCreating
dev:app DevSpace is waiting, because Pod app-devspace-6dcdc9bf56-trrrc has status: ContainerCreating
dev:app DevSpace is waiting, because Pod app-devspace-6dcdc9bf56-trrrc has status: ContainerCreating
dev:app DevSpace is waiting, because Pod app-devspace-6dcdc9bf56-trrrc has status: ContainerCreating
dev:app DevSpace is waiting, because Pod app-devspace-6dcdc9bf56-trrrc has status: ContainerCreating
dev:app DevSpace is waiting, because Pod app-devspace-6dcdc9bf56-trrrc has status: ContainerCreating
dev:app DevSpace is waiting, because Pod app-devspace-6dcdc9bf56-trrrc has status: ContainerCreating
dev:app DevSpace is waiting, because Pod app-devspace-6dcdc9bf56-trrrc has status: ContainerCreating
dev:app DevSpace is waiting, because Pod app-devspace-6dcdc9bf56-trrrc has status: ContainerCreating
dev:app Selected pod app-devspace-6dcdc9bf56-trrrc
dev:app ports Port forwarding started on: 2345 -> 2345
dev:app sync Sync started on: ./ <-> /app
dev:app sync Waiting for initial sync to complete
dev:app sync Initial sync completed
dev:app ssh Port forwarding started on: 10479 -> 8022
dev:app proxy Port forwarding started on: 11920 <- 10567
dev:app ssh Use 'ssh app.dev-env-setup.devspace' to connect via SSH
dev:app term Opening shell to container-0:app-devspace-6dcdc9bf56-trrrc (pod:container)
- On the command line switch to the
bash shell
devspace ./app # bash
root@app-devspace-6dcdc9bf56-6snnf:/# cd app
root@app-devspace-6dcdc9bf56-6snnf:/app# ls
Dockerfile README.md app devspace_start.sh localstack.yaml module.tf outputs.tf plan.out service-nginx.yaml variables.tf
LICENSE TF-README.md devspace.yaml images main.tf ortelius plan.json providers.tf terraform.tfstate wazuh
- Install the
Remote-SSHextension here - Click on the
Remote Explorericon
- SSH Settings configured by Devspace
# DevSpace Start app.dev-env-setup.devspace
Host app.dev-env-setup.devspace
HostName localhost
LogLevel error
Port 10479
IdentityFile "/Users/tvl/.devspace/ssh/id_devspace_ecdsa"
StrictHostKeyChecking no
UserKnownHostsFile /dev/null
User devspace
# DevSpace End app.dev-env-setup.devspace
- Install PostgreSQL Database Manager for database admin from inside your vscode here
- The extension supports many database engines so don't be misled by the name
The component chart allows you to define application components (e.g. a database, an API server, a webserver with static files) and deploy them using Helm. Helm is the package manager for Kubernetes and allows you to manage these components (e.g. upgrading).
Compared to manually creating Helm charts, the component chart allows you to define your application components using a unified Helm chart.
This provides the following benefits:
- 70% less YAML to maintain (only values.yaml for chart)
- Highly flexible configuration via values.yaml
- Fast and easy definition of Kubernetes resources
- Kubernetes best practices (e.g. recommended annotations and labels)
GitHub Codespaces & Dev Containers | Devcontainers for your GitHub repos
- Developing inside a container
- Introduction to Dev Containers
- Beginners Series to Dev Containers
- Getting Started with Dev Containers
- VS Code Dev Containers Extension
Leapp | Cross-Platform Cloud Access Application
Leapp is a Cross-Platform Cloud access App, built on top of Electron. The App is designed to manage and secure Cloud Access in multi-account environments, and it is available for MacOS, Windows, and Linux.
- Cloud credentials generation in 1 click
- Data stored locally encrypted in the OS System Vault
- Multiple Cloud-Access supported strategies
- Automatic short-lived credentials rotation
- Automatic provisioning of Sessions from AWS Single Sign-on
- Open multiple AWS console from different AWS accounts in Firefox and Chrome web extensions!
- Connect to EC2 instances straight away
- Managing Leapp with its CLI
- Create your own Leapp plugin to customize the App functionalities from the template
- Download here
- Licence
Localstack | AWS Cloudstack on your local machine with Docker extensions
LocalStack is a fully functional local AWS cloud stack that enables developers to develop and test their cloud applications offline. It provides an easy-to-use test/mocking framework for developing cloud applications, eliminating the need for a live AWS environment.
- LocalStack is installed using the Docker Desktop extension, please refer here
- LocalStack Configurations
- LocalStack Quickstart
- LocalStack CLI
- LocalStack AWS Local CLI
- LocalStack Terraform
- LocalStack CI
- LocalStack Testing Tools
- Localstack Serverless Plugin
- All endpoints are referenced as
http://localhost:4566 - Not all endpoints are supported in the free version, please refer here for supported features
- Licence
Commandeerextension here
- This package provides the
awslocalcommand, which is a thin wrapper around the aws command line interface for use with LocalStack.
awslocal --endpoint-url=http://localhost:4566 kinesis list-streams
StreamNames: []
Thunderclient | API Testing Inside VS Code
A new way to test APIs inside VS Code