A worked example showing how to connect to a teesql cluster from a Python application running inside an Intel TDX confidential VM. The example demonstrates the read/write split end-to-end: a tiny FastAPI app that writes to the cluster's primary and streams events back to every connected browser by polling a read-only connection on the secondary.
If you read one file: CLAUDE.md.
If you read two files: docs/architecture.md.
Browser A Browser B Browser C
│ ws │ ws │ ws
└────────┬─────────┴────────┬──────────┘
│ │
▼ │
FastAPI app (this repo, in YOUR CVM)
│ ▲
POST /events │ │ fan-out from broadcast hub
▼ │
primary connection secondary connection
(teesql_readwrite) (teesql_read, polled at 50ms)
│ │
└─── teesql cluster ─── primary ──► WAL ──► secondary
Anyone who POSTs an event sees it appear on every connected browser within (replication lag + one poll interval) — typically <100ms on a healthy cluster.
The whole point: two long-lived psycopg connections, one for writes and one for reads, both terminated with mutual RA-TLS via psycopg-ra-tls's localhost forwarder. The forwarders are opened once at process start and held for the lifetime of the FastAPI app — the RA-TLS handshake happens once, not per query.
┌────────────────────────── customer TEE (your CVM) ──────────────────────────┐
│ │
│ FastAPI app (uvicorn) │
│ │ │
│ ├── primary_conn (psycopg, role=teesql_readwrite) │
│ │ └── psycopg-ra-tls forwarder (TDX-attested mutual RA-TLS) │
│ │ │
│ ├── poller (asyncio task) │
│ │ └── secondary_conn (psycopg, role=teesql_read) │
│ │ └── psycopg-ra-tls forwarder │
│ │ tick every 50ms: SELECT … FROM events WHERE id > last_seen │
│ │ → broadcast.Hub.publish(row) │
│ │ │
│ └── WebSocket /ws/events │
│ └── for each subscriber, drain Hub queue → ws.send(json) │
│ │
└─────────────────────── tailscale / public internet ──────────────────────────┘
│
┌──────────── teesql cluster sidecar (the LEADER, found via TXT manifest) ────┐
│ :5433 mutual RA-TLS proxy │
│ DCAP-verifies your TDX quote │
│ injects KMS-derived password │
│ → Postgres 17 (replicated to secondaries) │
└──────────────────────────────────────────────────────────────────────────────┘
The customer's app never holds the cleartext Postgres password. The forwarder substitutes a placeholder; the cluster sidecar replaces it with the KMS-derived password on the wire after verifying the customer's TDX quote.
- A teesql cluster's identity (operator gives you):
cluster_uuid(10 hex chars, e.g.2c8b79fe4d)manifest_signer_address(20-byte hex of the dns-controller's ECDSA pubkey)database(the database your roles can connect to)cluster_secret(32-byte hex; allowlisted on the cluster's roles)
- Your code must run inside a dstack CVM, OR you must have the dstack simulator running locally. Outside a TEE there's no way to obtain the TDX-attested client certificate the cluster requires.
$ python -V # 3.11 or later
$ pip install -e . # installs psycopg-ra-tls, ra-tls-verify, dstack-sdk, fastapi, uvicorn
$ cp .env.example .env
$ $EDITOR .env # fill in cluster_uuid + manifest_signer + cluster_secret
$ teesql-example migrate
[INFO] connecting via manifest TXT: cluster=2c8b79fe4d signer=0xa4021ec2…
[INFO] applying migration 0001_events
[INFO] applying migration 0002_events_notify_trigger
[INFO] migrations complete
$ teesql-example serve
[INFO] opening primary connection
[INFO] opening connection: role=teesql_read cluster=2c8b79fe4d db=…
[INFO] poller started: poll_interval=50ms last_seen=0
[INFO] ready: cluster=2c8b79fe4d primary_role=teesql_readwrite secondary_role=teesql_read poll=50ms
INFO: Uvicorn running on http://0.0.0.0:8080
Then:
- Browse to
http://<your-cvm>:8080/— you should see the demo UI. - Open the same URL in a second tab.
- Submit an event from one tab. Watch it appear in BOTH tabs.
Behind the scenes: tab #1's POST hits the primary, the WAL replicates, the secondary's poll picks it up within 50ms + replication lag, the broadcast hub fans it out to both tabs' WebSockets.
The dstackgres parent repo ships a tee-debug-shell deployment that
builds a one-shot CVM, runs a script, and exits. Pair it with the
02_post_and_subscribe.py example for a full read/write-split smoke:
$ cd /path/to/dstackgres
$ deployments/tee-debug-shell/deploy.sh \
/path/to/teesql-example-python/examples/02_post_and_subscribe.py 26 \
/path/to/teesql-example-python/examples/01_hello_teesql.env
$ phala cvms logs teesql-debug-02-post-and-subscribe --follow
…
[INFO] smoke: posted 10 events: [1,2,3,4,5,6,7,8,9,10]
[INFO] smoke: received id=1 (1/10)
[INFO] smoke: received id=2 (2/10)
…
[INFO] smoke: all 10 events round-tripped
=== DONE ===
The mutual-RA-TLS handshake is the most expensive part of every teesql connection. It involves: dstack guest agent call to mint a TDX-attested client cert, the cluster's DCAP verification of that cert, the cluster's server cert chain validation, the manifest TXT lookup + signature verification. Doing that per query — or worse, per poll — would burn the dstack guest agent.
This example holds two long-lived psycopg.Connections for the
lifetime of the FastAPI process:
- One primary connection. All
POST /eventswrites flow through it. - One secondary connection. The poller owns it; runs
SELECT … WHERE id > last_seenevery 50ms over the existing TLS session.
On psycopg.OperationalError (leader rotation, sidecar restart,
network glitch), the affected connection is closed and re-opened
in-place. That's the only path that pays a fresh handshake.
The poller's bookkeeping cursor (last_seen) is preserved across
reconnects — no events are replayed or lost.
MIT. Fork freely.