📦 AI SDK + Microservice

A minimal, production-minded AI platform demonstrating an internal Python SDK, observability, evaluation, containerisation, and deployment scaffolding.

---

🔍 Overview

This repository demonstrates a reusable AI platform architecture with two core components:

1. ai_lib/ — Internal Python AI SDK

A modular, reusable Python library that serves as the foundation for AI capabilities across the organization. It provides:

• 🛠 A unified AIClient interface
• 📝 Text summarisation capability (simple and HuggingFace backends)
• 🔭 Built-in observability via Langfuse
• ⚙️ Clean configuration patterns
• 🧰 Reusable utilities for downstream services

In production: This would be a separate repository published as a Python package (PyPI, Artifactory, etc.) that multiple services consume.

2. service/ — FastAPI Microservice (Consumer)

A lightweight microservice that demonstrates how services consume the SDK:

• GET /health — Health check
• POST /summarise — Text summarisation endpoint

---

🧱 Architecture

This structure demonstrates Shared SDK → Multiple Services, Testing + LLM Evaluation, and GitOps-ready packaging.

ai-sdk/
│
├── ai_lib/                 ← Internal SDK (core library)
│   ├── client.py
│   ├── summarisation/
│   ├── tracing/
│   ├── tests/
│   └── ...
│
├── service/                ← FastAPI microservice using lib
│   ├── app/               ← Service code
│   ├── k8s/               ← Kubernetes manifests
│   └── DockerFile         ← Container definition
├── terraform/              ← Infrastructure as Code
├── eval/                   ← DeepEval test suite
├── pyproject.toml
└── README.md

---

🧩 Features

Internal AI SDK (ai_lib)

• Unified API: AIClient.summarise_text() abstraction.
• Backend Agnostic: Select backends via env vars (simple, hf).
• Observability: Built-in Langfuse tracing via traced_operation().
• Extensible: Designed for clean architectural extension.

Microservice

• FastAPI Integration: Shows how downstream teams consume the SDK.
• Production Ready: Deployable via Docker or Kubernetes.

Quality & Governance

• Unit Tests: Located in tests/.
• LLM Evals: DeepEval test suite in eval/.
• CI/CD: GitHub Actions pipeline included.

Infrastructure Scaffolding

• Containerisation: Dockerfile included.
• Orchestration: Kubernetes Deployment + Service manifests.
• IaC: Terraform stub for namespace provisioning.
• GitOps: Structure ready for Argo CD.

---

🚀 Quick Start

For detailed setup instructions, see docs/GETTING_STARTED.md

Minimum Setup (3 steps):

# 1. Clone and set up
git clone <repository-url>
cd ai-sdk
python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate
make install

# 2. Run the service
make service-run

# 3. Test it (in another terminal)
make service-test

---

🚀 Running Locally

1. Install the SDK (Development Mode)

make install
# Or manually:
pip install -e ".[dev]"
pip install -r service/requirements.txt

2. Run the Microservice

make service-run
# Or manually:
uvicorn service.app.main:app --reload --host 0.0.0.0 --port 8000

3. Test Endpoints

Health Check:

curl http://localhost:8000/health

Summarise Text:

curl -X POST http://localhost:8000/summarise \
  -H "Content-Type: application/json" \
  -d '{"text": "Bella is a 3-year-old indoor cat..."}'

---

🐳 Running via Docker

Build the image:

docker build -f service/DockerFile -t ai-summarise-service .

Run the container:

docker run -p 8000:8000 \
  -e AI_LIB_SUMMARISATION_BACKEND=simple \
  ai-summarise-service

---

👀 Observability with Langfuse

To enable tracing, set the following environment variables:

export LANGFUSE_SECRET_KEY=...
export LANGFUSE_PUBLIC_KEY=...
export LANGFUSE_HOST=[https://cloud.langfuse.com](https://cloud.langfuse.com)

Any SDK call wrapped with the tracer will appear in your Langfuse dashboard:

with traced_operation("summarise_text", inputs={"text": text}):
    # ... logic

---

🧪 Testing & Evaluation

Unit Tests:

pytest tests

DeepEval Tests:

pytest eval

Both test suites run automatically in the GitHub Actions pipeline.

---

🔄 CI/CD & Deployment

CI Pipeline (GitHub Actions)

The pipeline replicates how production AI teams enforce quality:

1. Installs dependencies.
2. Runs unit tests.
3. Runs DeepEval tests (supports Langfuse-enabled evals).

☸️ Kubernetes Deployment

Prerequisites:

• Minikube installed and running
• kubectl configured

Quick Start:

# 1. Create namespace with Terraform
cd terraform && terraform apply

# 2. Build image in Minikube's Docker
eval $(minikube docker-env)
docker build -f service/DockerFile -t ai-summarise-service:latest .

# 3. Create secrets (if using Langfuse)
kubectl create secret generic langfuse-secrets \
  --from-literal=secret-key='your-key' \
  --from-literal=public-key='your-key' \
  -n ai-platform

# 4. Deploy to Kubernetes
kubectl apply -f service/k8s/deployment.yml
kubectl apply -f service/k8s/service.yml

# 5. Access the service
kubectl port-forward service/ai-summarise-service 8000:80 -n ai-platform

🔨 Terraform Setup

Infrastructure as Code (IaC) for managing Kubernetes infrastructure.

Quick Start:

cd terraform
terraform init
terraform plan
terraform apply

Creates the ai-platform namespace.

---

🎯 Why This Project Exists

This repository demonstrates the full lifecycle of AI platform engineering:

• Shared internal SDK
• Evaluation & Observability
• Microservice Integration
• Docker Containerisation
• CI/CD Automation
• Infra + GitOps Deployment Patterns

It is intended as a hands-on, end-to-end example of what a modern AI platform looks like.

---

📚 Future Enhancements

• Publish ai_lib to a private PyPI registry.
• Add Argo CD Application manifest.