Prerequisites & Setup

This guide covers the necessary tools and steps to bootstrap the STDML Workspace on your local machine.

System Requirements

To ensure a deterministic and smooth development experience, please install the following tools before proceeding:

1. Nix (It's not required, but it makes the process easier): Used for declarative environment builds. Note: Ensure Nix Flakes are enabled in your configuration.
2. uv: An extremely fast Python package and project manager written in Rust.
3. Just: A modern command runner (acts as a safer, more ergonomic alternative to make).
4. Docker: The container runtime required for local Kubernetes.
5. k3d & Helm: Tools for running and managing the local Kubernetes cluster. (Can be provided via Nix devShell or installed globally).

---

First-Time Project Setup

The workspace relies on uv to manage multiple Python microservices under a single global .lock file.

1. Clone the repository:

git clone https://github.com/allllpina/stdml
cd stdml

2. Synchronize workspace dependencies: This command resolves and installs all required Python packages for both the api and mlworker services.

just sync

Important

Before running the services, Vault needs to be populated with your local configurations (database URLs, API keys, etc.). Ensure your .env file is present in the root directory with all the variables, then run:

Bootstrapping the Infrastructure

The local development environment runs on a k3d Kubernetes cluster, utilizing HashiCorp Vault for secrets management.

1. Create and start the local cluster:

just cluster-up

This command spins up a lightweight K8s cluster named mlops-cluster and maps the necessary ports.

2. Deploy full infrastructure:

just deploy

This command deploys Hashi Corp vault (in dev mode via Helm), Redis, Kafka, seeds Vault and Redis and deploys API and Worker

3. Forward the API port:

just api-forward

Notes

1. It is possible to launch all services sepparately using comands:

just vault-up         # Deploys only vault
just redis-up         # Deploys only redis
just kafka-up         # Deploys only kafka
# For more information use
just -l

2. To shut down the cluster use:

just cluster-down