Serverless Data Orchestration with Asset Lineage
Servo is an open-source, asset-centric data orchestration platform designed for serverless compute environments and multi-tenant SaaS applications. Built with a high-performance Rust core and ergonomic Python SDK, Servo delivers enterprise-grade orchestration features, including comprehensive lineage tracking, observability, retry logic, and versioning, without requiring persistent infrastructure.
- Asset-first orchestration: Data assets are first-class citizens with automatic lineage tracking.
- Serverless-native: No persistent schedulers; uses cloud queuing services (Cloud Tasks, SQS, EventBridge).
- Multi-tenant by design: Tenant isolation, cost attribution, and protection against noisy neighbors.
- Zero idle costs: Scales to zero when inactive (unlike Airflow, Dagster, or Prefect).
- Cloud-agnostic: Works across GCP, AWS, and Azure with pluggable adapters.
- High performance: Rust core provides 10-100x performance improvements for critical paths.
# Install Servo
pip install servo-orchestration
# Initialize metadata database
servo init --database-url postgresql://localhost/servo
# Create a workflow
cat > workflow.py << 'EOF'
from servo import asset, workflow, executor
@asset(name="clean_customers")
def clean_customers(raw_data):
return raw_data.drop_duplicates()
@workflow(
name="daily_etl",
executor=executor.CloudRun(project="my-project")
)
def pipeline():
raw = extract_data()
clean = clean_customers(raw)
return clean
EOF
# Deploy workflow
servo deploy workflow.py
# Trigger execution
servo run daily_etlProblem: Modern data teams use serverless compute (Cloud Run, Lambda, ECS) but orchestration tools still require persistent infrastructure with fixed costs.
Solution: Servo orchestrates workflows using cloud-native queuing services and stores execution metadata in standard relational databases. No persistent schedulers, no daemons, and no fixed costs.
Comparison:
| Feature | Servo | Dagster | Airflow | Step Functions |
|---|---|---|---|---|
| Persistent processes | None | Required | Required | None |
| Asset lineage | Built-in | Core | Plugin-based | Not available |
| Multi-tenant (first-class) | Yes | Partial | No | Partial |
| Zero idle costs | Yes | No | No | Yes |
| Cloud-agnostic | Yes | Yes | Yes | No |
User Code (Python) -> Servo SDK -> Rust Core -> Cloud Queue -> Worker -> PostgreSQL
|
-> Cloud Storage
- Discord: Join our Discord
- GitHub Discussions: Ask questions
- Twitter: @ServoOrch
We welcome contributions! See CONTRIBUTING.md for guidelines.
Apache 2.0 - See LICENSE for details.
See ROADMAP.md for our development plans.
Built with care by the Servo community.