SPI Stack deploys the OSDU platform onto Azure using AKS Automatic and Azure PaaS services with a bootstrap + Flux CD GitOps model. Infrastructure is provisioned via az CLI commands, then Flux continuously reconciles Kubernetes workloads from this Git repository.
This project is currently optimized for Azure dev/test environments and is still evolving.
Who this is for:
- Developers who want a reproducible Azure-based OSDU environment
- Platform engineers evaluating OSDU with Azure PaaS services
- Azure-native: leverages CosmosDB, Service Bus, Storage, Key Vault, and Entra ID
- AKS Automatic: managed Istio, Karpenter, and Deployment Safeguards out of the box
- GitOps-driven: Flux continuously reconciles desired state after bootstrap
- Transparent: every
azandkubectlcommand is shown before execution - Workload Identity: no stored credentials; all Azure access via federated identity
The only tool you need is uv. Each
GitHub Release publishes a
versioned spi wheel (spi-X.Y.Z-py3-none-any.whl); you install it directly
from its GitHub Release URL.
These commands always resolve the newest release, so they never need updating between versions. Pick the one for your shell.
macOS / Linux (bash, zsh):
uv tool install "$(curl -fsSL https://api.github.com/repos/Azure/osdu-spi-stack/releases/latest \
| grep -o 'https://github.com/Azure/osdu-spi-stack/releases/download/[^"]*-py3-none-any.whl')"Windows (PowerShell):
uv tool install (irm https://api.github.com/repos/Azure/osdu-spi-stack/releases/latest).assets.where({ $_.name -like '*-py3-none-any.whl' }).browser_download_urlThen verify:
spi --versionAfter install the spi binary is on PATH; no uv run prefix.
To pin an exact version — for CI, reproducible environments, or bug reports — install its wheel URL directly (same command on every platform):
uv tool install https://github.com/Azure/osdu-spi-stack/releases/download/v0.1.0/spi-0.1.0-py3-none-any.whlCopy the wheel URL for any version from that release's page under Releases.
After the first install, spi upgrades itself — no URL needed, same on every
platform:
spi update # check for a newer version and install it
spi update --check # check only; do not install
spi update --force # reinstall even if already on the latest versionNote:
uv tool install git+https://github.com/...@vX.Y.Zalso works, but the wheel-URL forms above are preferred because they preserve the tag-derived version inspi --versionreliably.
Once installed:
# Check prerequisites
spi check
# Deploy (provisions Azure resources + activates GitOps)
spi up --env dev1
# Multi-partition deploy (one CosmosDB + Service Bus + storage per partition)
spi up --env dev1 --partition opendes --partition tenant1
# Pick an ingress mode (default: azure)
spi up --env dev1 --ingress-mode dns --dns-zone example.com
spi up --env dev1 --ingress-mode ip # debug / smoke
# Middleware only, no OSDU services
spi up --env dev1 --profile minimalspi status # Deployment health dashboard
spi status --watch # Continuous refresh
spi info # Endpoints and credentials
spi reconcile --suspend # Freeze: stop Flux auto-reconciliation
spi reconcile --resume # Unfreeze: resume Flux auto-reconciliation
spi down --env dev1 # Tear down when doneTo work on spi itself rather than install it:
git clone https://github.com/Azure/osdu-spi-stack.git
cd osdu-spi-stack
uv sync
uv run spi --helpSee CONTRIBUTING.md for the full developer workflow, including pre-commit setup, conventional commits, and the release process.
SPI Stack is GitOps + bootstrap, not "pure GitOps from an empty cluster."
The CLI performs a bootstrap phase:
- Provision Azure PaaS resources (CosmosDB, Service Bus, Storage, Key Vault)
- Create an AKS Automatic cluster with Managed Identity
- Configure Workload Identity and RBAC role assignments
- Bootstrap the cluster with namespaces, secrets, ConfigMap, and ServiceAccount
- Activate the AKS native Flux extension pointing to this repo
After that handoff, Flux owns steady-state reconciliation and continuously converges the cluster to the desired state.
Deployment phases
- Core Infra: Resource Group, AKS Automatic, Managed Identity, Key Vault, ACR
- Data Infra: CosmosDB (Gremlin + SQL), Service Bus, Storage Accounts
- IAM: Federated credentials, RBAC role assignments, Key Vault secrets
- K8s Bootstrap: Namespaces, StorageClasses, secrets, ConfigMap, ServiceAccount
- GitOps: AKS native Flux extension pointing to this repo
A full spi up typically takes ~45-50 minutes, dominated by AKS Automatic provisioning (~30 min). Exact times vary by region.
Environment isolation
Use --env to run multiple isolated deployments. Each environment gets its own resource group and cluster (e.g., spi-stack-dev1, spi-stack-team).
uv run spi up --env dev1
uv run spi up --env stagingThree namespaces, deployed in dependency order via a 7-layer Kustomization stack:
| Namespace | Layer | Deploys |
|---|---|---|
| foundation | Operators | ECK (Elasticsearch), CNPG (PostgreSQL), cert-manager |
| platform | Middleware | Elasticsearch, Redis (TLS), PostgreSQL (Airflow), Airflow, Istio Gateway |
| osdu | Services | partition, entitlements, legal, schema, storage, search, indexer, file, workflow + 3 reference services |
--profile selects how much of that stack Flux reconciles:
| Profile | Deploys |
|---|---|
core (default) |
Everything above. |
minimal |
foundation and platform only — operators, cert-manager, trust-manager, Gateway, Elasticsearch, Redis, PostgreSQL, Airflow. No OSDU services. |
bare |
Nothing; infra plus activated GitOps only. Flux reconciles empty stack and ingress trees. The CLI bootstrap seeds namespaces, secrets, the osdu-config ConfigMap, and the Workload Identity ServiceAccount. |
Use minimal when you are working on the middleware itself and the OSDU services would only add deploy time. The middleware layers are identical between profiles, so what you validate on minimal holds on core.
Use bare for Bicep, Workload Identity, or RBAC iteration, or for bring-your-own workloads. Re-run spi up with minimal or core to add workloads later.
| Resource | Purpose |
|---|---|
| AKS Automatic | Kubernetes with managed Istio, Karpenter, Safeguards |
| CosmosDB Gremlin | Entitlements graph |
| CosmosDB SQL | OSDU operational data (per partition) |
| Service Bus | Async messaging (per partition, 14 topics) |
| Storage Accounts | Blob/table storage (common + per partition) |
| Key Vault | Centralized secret management |
| Managed Identity | Workload Identity for all OSDU services |
Everything is discovered by the CLI:
uv run spi checkRequired tools: az, bicep, kubectl, kubelogin, flux, helm
System requirements: Azure subscription with permissions to create resource groups and AKS clusters.
uv run spi <command> [OPTIONS]
Commands:
check Validate required tools are installed
up Provision Azure infra and deploy the stack --env NAME [--profile] [--partition] [--ingress-mode] [--dns-zone] [--dry-run]
status Deployment health dashboard [--watch]
down Delete all Azure resources --env NAME
info Show endpoints and optional credentials [--show-secrets]
reconcile Force Flux to re-sync from Git [--suspend] [--resume] [--refresh-images]
Use --dry-run on spi up to preview the Bicep changes (az deployment group what-if) before any Azure resources are created beyond the resource group. --profile defaults to core; minimal deploys middleware only, and bare activates GitOps against empty stack and ingress trees. --ingress-mode defaults to azure; the other supported modes are dns (per-service hostnames on an owned Azure DNS zone) and ip (bare IP, debug only). --ingress-mode and --dns-zone are rejected with bare. --refresh-images re-resolves the OSDU community image tags and reconciles the service Kustomizations.
- Architecture -- 30,000-ft overview of the system
- Design docs -- how subsystems actually work (deployment lifecycle, Bicep, Flux, Workload Identity, ingress, secrets)
- ADRs -- decision records with alternatives considered
Licensed under the Apache License 2.0.
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.
When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.
See CONTRIBUTING.md for the developer workflow, including pre-commit setup, conventional commits, and the release process.
This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.