Code-native data transformation — SQL and Python models, no Jinja required.
pip install KelpMesh
kelpmesh init my_project && cd my_project
kelpmesh run📖 Full user guide at docs/user-guide.md — covers CLI, Studio, VS Code, migration, and all features.
dbt models are Jinja-templated SQL. Your files look like this:
{{ config(materialized='table') }}
SELECT customer_id, {{ surrogate_key(['order_id','sku']) }} AS id
FROM {{ ref('stg_orders') }}
WHERE {{ is_incremental() }}No IDE autocomplete. No AI assistant support. Weeks to learn.
KelpMesh models are plain SQL:
-- materialized: table
SELECT customer_id, surrogate_key(order_id, sku) AS id
FROM stg_orders
WHERE is_incremental()Works in every SQL editor, every AI tool, every linter — on day one.
- Pure SQL models — no Jinja; files work natively in DBeaver, DataGrip, Cursor, any AI tool
- 32 built-in SQL macros —
surrogate_key,safe_divide,datediff,haversine,generate_surrogate_key, and 27 more; called as plain SQL functions, expanded at compile time - Python models —
def model(dbt, session)interface; return SQL string or pandas DataFrame - All materializations —
table,view,incremental(merge/append),ephemeral, SCD Type 2 snapshots - 9 warehouse adapters — DuckDB · Postgres · Snowflake · BigQuery · Databricks · Redshift · Microsoft Fabric · MySQL · Trino
- Incremental merge — consistent MERGE/UPSERT API across all 9 warehouses
kelpmesh plan— Terraform-style impact analysis before any warehouse query runskelpmesh compile— render macros and substitutions without touching the warehouse
kelpmesh ci— one command: diff → plan → run changed models → test → post PR comment- GitHub PR comments — posts a structured run report on every PR; updates (not spams) existing comment
- GitLab MR comments — same, auto-detected from CI environment variables
- Bitbucket PR comments — same
- Ready-to-use CI templates —
.github/workflows/ci.yml,.gitlab-ci.yml,bitbucket-pipelines.ymlincluded
- Built-in cron scheduler —
kelpmesh schedule start; cron syntax +every 1hintervals; no Airflow required - Dagster integration —
KelpMeshResource,@asset,@op, schedule sensor - Prefect integration —
KelpMeshBlock,@task,@flow, pre-built standard flow - Airflow —
KelpMeshOperator,DagFactory-compatible
- PII auto-classification — 7 types: email, phone, SSN, NPI, card, IP, DOB
- Row-level security (RLS) — policy-based access control applied per model
- Column masking — per-column masking strategy; format-preserving options
- GDPR right-to-erasure —
kelpmesh security erasuredeletes subject data across all models - Immutable audit log — append-only JSONL; every execution recorded
- Secret scanning — scans for hardcoded credentials in SQL files
- Zero telemetry — import blocklist enforced in code; no phone-home possible
- VS Code extension — 37 SQL snippets, model tree view, CodeLens run/test/preview buttons, plan panel
- dbt migration —
kelpmesh importconverts models, tests, sources, seeds, snapshots - Data mesh — cross-project
ref(), public/private access contracts, column guarantees - Semantic layer — metric definitions, BI export for LookML, Tableau, PowerBI, Qlik
# Install
pip install KelpMesh
# Create a project
kelpmesh init my_project
cd my_project
# Run all models
kelpmesh run
# Preview 100 rows from a model
kelpmesh preview orders
# See what would change before running
kelpmesh plan
# Test all models
kelpmesh test
# Run + test in one command
kelpmesh build
# Generate documentation site
kelpmesh docskelpmesh ci # detects changed models, runs them, tests, posts PR commentIn GitHub Actions — add to your workflow:
- name: KelpMesh CI
run: kelpmesh ci
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}Every PR gets a structured report showing exactly which models were affected, what ran, and whether tests passed — updated automatically on every push.
| Package | What it is | Price |
|---|---|---|
pip install KelpMesh |
Full CLI engine — 9 adapters, 32 macros, scheduler, security, CI/CD | Free forever · Apache 2.0 |
pip install kelpmesh-studio |
Core + browser dashboard (DAG viz, run history, team features) | Free for personal use · Freemium for commercial |
KelpMesh will never have features gated behind a paid tier.
pip install kelpmesh-studio
kelpmesh-studio # opens http://localhost:8765| Tier | Price | Limits |
|---|---|---|
| Free — personal use | $0 | 1 user · 3 projects · 50-run history |
| Pro — commercial teams | $29/user/mo | 5 seats · unlimited projects · RBAC · API keys · Git sync · AI BYOK |
| Business — large orgs | $79/user/mo | Unlimited seats · SSO · BYOC · full audit log |
| Enterprise | Contact us | On-premises · custom SLA · dedicated support |
Activate with:
export KELPMESH_STUDIO_LICENSE_KEY=km_pro_<your-key>
kelpmesh-studio# kelpmesh.yml
warehouse:
type: duckdb # zero install, great for local development
path: ./warehouse.db
# Per-warehouse extras:
# pip install KelpMesh[postgres]
# pip install KelpMesh[snowflake]
# pip install KelpMesh[bigquery]
# pip install KelpMesh[databricks]
# pip install KelpMesh[redshift]
# pip install KelpMesh[mysql]
# pip install KelpMesh[trino]
# pip install KelpMesh[all-warehouses] # everything at once# Scan for hardcoded credentials
kelpmesh scan secrets --fail
# Classify PII columns in your warehouse
kelpmesh security classify --table orders
# Apply column masking by role
kelpmesh security mask --table users --columns email,phone --role viewer
# GDPR erasure
kelpmesh security erasure --id-col email --id-value user@example.com
# View audit trail
kelpmesh security auditKelpMesh auto-detects the source project format. No --from flag needed in most cases.
# From dbt
kelpmesh import ./dbt-project --output ./kelpmesh-project
# From SQLMesh
kelpmesh import ./sqlmesh-project --output ./kelpmesh-project
# Explicit (if auto-detection doesn't pick it up)
kelpmesh import ./my-project --from dbt
kelpmesh import ./my-project --from sqlmesh| dbt | KelpMesh |
|---|---|
{{ ref('model') }} |
plain table name (auto-resolved by AST) |
{{ source('src', 'tbl') }} |
plain table name |
{{ config(materialized='table') }} |
-- { materialized: table } header |
schema.yml tests (not_null, unique, accepted_values, relationships) |
SQL assertion files in tests/ |
| Seeds (CSV) | seeds/*.sql with VALUES blocks |
| Snapshots | models/snapshots/*.sql with snapshot header |
| dbt packages | Built-in SQL macros or kelpmesh-utils |
| SQLMesh | KelpMesh |
|---|---|
MODEL (name ..., kind FULL) block |
-- { materialized: table } header |
kind INCREMENTAL_BY_UNIQUE_KEY (unique_key id) |
-- { materialized: incremental, unique_key: id } |
grain payment_id |
-- { unique_key: payment_id } |
audits (UNIQUE_VALUES(...), NOT_NULL(...)) |
SQL assertion files in tests/ |
cron '@daily' |
-- { cron: @daily } |
@execution_dt |
CURRENT_DATE |
@start_dt / @end_dt |
CURRENT_DATE - INTERVAL '7 days' / CURRENT_DATE |
YAML unit test fixtures (inputs/outputs) |
Smoke tests (recreate with kelpmesh create_test) |
audits/*.sql |
tests/*.sql SQL assertions |
config.py / config.yaml |
kelpmesh.yml |
SQLMesh note: SQLMesh YAML unit tests use an input/output fixture format that requires running a live warehouse query to generate. After import, use
kelpmesh create_test <model>to recreate them properly.
| Tool | How |
|---|---|
| GitHub Actions | kelpmesh ci + built-in workflow at .github/workflows/ci.yml |
| GitLab CI | kelpmesh ci + template at .gitlab-ci.yml |
| Bitbucket Pipelines | kelpmesh ci + template at bitbucket-pipelines.yml |
| Dagster | from kelpmesh_dagster import KelpMeshResource |
| Prefect | from kelpmesh_prefect import KelpMeshBlock |
| Airflow | pip install kelpmesh-airflow |
| VS Code | Search "KelpMesh" in extensions marketplace |
| pre-commit | .pre-commit-hooks.yaml included |
- GitHub Issues — bug reports and feature requests
- Discord — community chat and support
- Twitter / X — updates and releases
git clone https://github.com/RoyPulseAI/kelpmesh
cd kelpmesh
pip install -e ".[dev]"
pytest tests/ -vSee CONTRIBUTING.md.
KelpMesh is designed and built by Roy Pulse AI (@RoyPulseAI).
KelpMesh — Apache 2.0
kelpmesh-studio — Apache 2.0 (free tier) · Commercial license (Pro/Business/Enterprise)
See LICENSE.