Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/release-pypi.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Publish sdist + wheel to PyPI when a SemVer tag is pushed (e.g. v1.0.6).
# Publish sdist + wheel to PyPI when a SemVer tag is pushed (e.g. v1.1.0).
# Configure "trusted publishing" on PyPI for this workflow + repository + optional GitHub environment.
# https://docs.pypi.org/trusted-publishers/

Expand Down
15 changes: 15 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,21 @@ This project follows [Semantic Versioning](https://semver.org/). From **v1.0.0**

## Unreleased

## 1.1.0 - 2026-05-03

### Added

- **Pricing catalog:** optional `pricing_catalog_path` in `flightdeck.yaml` loads a `PricingCatalog` YAML; `POST /v1/diff` / `release diff` include additive `pricing.catalog` and `pricing.hints` (see `schemas/v1/pricing_catalog.schema.json`, `examples/pricing/catalog.sample.yaml`).
- **Promotion approval:** `promotion_requires_approval` in `flightdeck.yaml`; `POST /v1/promote/request`, `POST /v1/promote/confirm`, `GET /v1/promotion-requests`, and CLI `release promote-request` / `promote-confirm`.
- **Forensics:** `GET /v1/runs` and `flightdeck runs list` for read-only run event slices.
- **Deploy:** optional Helm chart under `examples/deploy/chart/flightdeck/`.
- **Examples:** `examples/fleet/README.md` and workspace template.
- **SQLite migration v4:** `promotion_requests` table.

### Changed

- **Examples / CI snippets:** **`flightdeck-ai>=1.1.0`** in Docker and PyPI gate samples.

## 1.0.6 - 2026-05-02

### Added
Expand Down
2 changes: 1 addition & 1 deletion DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ Merging to **`main` does not publish packages** — PyPI uploads are **tag-drive
1. **PyPI:** add a **trusted publisher** for **[github.com/flightdeckdev/flightdeck](https://github.com/flightdeckdev/flightdeck)** — workflow **`release-pypi.yml`**. If PyPI offers **Environment name: (Any)**, you can still use a GitHub **Environment** named **`pypi`** for approval gates; otherwise match whatever you register on PyPI ([trusted publishers](https://docs.pypi.org/trusted-publishers/)).
2. **GitHub:** Settings → **Environments** → create **`pypi`** (optional: required reviewers / wait timer before OIDC publish).
3. Bump **`version`** in **`pyproject.toml`** and **`src/flightdeck/__init__.py`**, update **`CHANGELOG.md`**, merge to **`main`**.
4. **`git tag vX.Y.Z`** (must match **`pyproject.toml`** exactly, e.g. **`v1.0.6`**) then **`git push origin vX.Y.Z`**.
4. **`git tag vX.Y.Z`** (must match **`pyproject.toml`** exactly, e.g. **`v1.1.0`**) then **`git push origin vX.Y.Z`**.

The workflow runs **ruff**, **pytest**, schema drift, **`uv build`**, publishes **sdist + wheel** to **PyPI** via **OIDC** (no long-lived API token in repo secrets), enables **publish attestations**, and creates a **GitHub Release** with generated notes and **`dist/*`** assets.

Expand Down
4 changes: 4 additions & 0 deletions RELEASE_NOTES.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ High-level notes for **shipping FlightDeck**. Detailed history: **[CHANGELOG.md]

Narrative docs (including the CLI reference) are maintained on **[github.com/flightdeckdev/flightdeck](https://github.com/flightdeckdev/flightdeck)** `main`; this file and **`schemas/`** ship in minimal clones.

## v1.1.0 — Phase 1 first slice (catalog, approval, runs, Helm, fleet)

Minor release (see **[CHANGELOG.md](CHANGELOG.md)**): optional **`pricing_catalog_path`** + **`PricingCatalog`** YAML for cross-vendor comparable **`pricing.catalog`** lines on diffs; **`pricing.hints`** for multi-version and model-name diagnostics; **`promotion_requires_approval`** with **`POST /v1/promote/request`** / **`POST /v1/promote/confirm`** / **`GET /v1/promotion-requests`** and matching CLI; **`GET /v1/runs`** and **`flightdeck runs list`**; SQLite migration **v4** (`promotion_requests`); reference **Helm** chart and **fleet** docs under **`examples/`**. **Stable contracts:** additive HTTP and CLI surfaces; existing **`v1`** event and release payloads unchanged.

## v1.0.6 — Phase 0 closure (backup, cross-language emitters, roadmap)

Patch release (see **[CHANGELOG.md](CHANGELOG.md)**): **`flightdeck doctor --backup PATH`** performs a SQLite online backup of the workspace DB; **[examples/integration/](examples/integration/README.md)** gains **`curl`** and a **Node** **`emit_sample_events.node.mjs`** path for **`POST /v1/events`**; **[examples/deploy/README.md](examples/deploy/README.md)** documents the Compose **`/health`** healthcheck and backup scheduling. **ROADMAP:** **Phase 0** is **closed**; **catalog-level** multi-provider pricing normalization is an explicit **Phase 1** build item. **Stable contracts:** additive CLI flag and HTTP field **`pricing.warnings`** (from **v1.0.5**) remain backward-compatible.
Expand Down
10 changes: 8 additions & 2 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,11 @@ This roadmap is meant to be clear from **what is already shipped** to **near-ter

## Next release

**v1.0.6** (patch): Phase 0 closure — **`flightdeck release diff --output json`** (same shape as **`POST /v1/diff`**); **`pricing.warnings`** when a release model has no row in its pricing table (CLI **`WARNING:`** lines + web Diff); **Overview** ledger metrics card (**`GET /v1/metrics`**); **`curl`** + **Node** samples under **[examples/integration/](examples/integration/README.md)**; **`flightdeck doctor --backup PATH`** (SQLite online backup); **[examples/deploy/](examples/deploy/README.md)** documents Compose **`/health`** healthcheck and backup scheduling. **Phase 0** is declared **closed**; **catalog-level** multi-provider normalization moves to **Phase 1**. See **[CHANGELOG.md](CHANGELOG.md)** and **[RELEASE_NOTES.md](RELEASE_NOTES.md)**. No breaking changes to stable CLI, HTTP, or **`api_version` `v1`** contracts.
Further **Phase 1** work after **v1.1.0** (deeper forensics / replay UX, richer approval UI if needed, OTLP-oriented telemetry per gaps table). Track **[CHANGELOG.md](CHANGELOG.md)**.

**v1.1.0** (minor, shipped): Phase 1 first slice — **`pricing_catalog_path`** + **`pricing.catalog`** / **`pricing.hints`** on diffs; **`promotion_requires_approval`** + promote **request/confirm** (HTTP + CLI) + **`GET /v1/promotion-requests`**; **`GET /v1/runs`** / **`runs list`**; **Helm** reference chart; **`examples/fleet/`**; SQLite migration **v4**. See **[CHANGELOG.md](CHANGELOG.md)** and **[RELEASE_NOTES.md](RELEASE_NOTES.md)**.

**v1.0.6** (patch, shipped): Phase 0 closure — **`flightdeck release diff --output json`** (same shape as **`POST /v1/diff`**); **`pricing.warnings`** when a release model has no row in its pricing table (CLI **`WARNING:`** lines + web Diff); **Overview** ledger metrics card (**`GET /v1/metrics`**); **`curl`** + **Node** samples under **[examples/integration/](examples/integration/README.md)**; **`flightdeck doctor --backup PATH`** (SQLite online backup); **[examples/deploy/](examples/deploy/README.md)** documents Compose **`/health`** healthcheck and backup scheduling. **Phase 0** is declared **closed**; **catalog-level** multi-provider normalization moves to **Phase 1**. See **[CHANGELOG.md](CHANGELOG.md)** and **[RELEASE_NOTES.md](RELEASE_NOTES.md)**. No breaking changes to stable CLI, HTTP, or **`api_version` `v1`** contracts.

---

Expand Down Expand Up @@ -49,7 +53,7 @@ Goal: prove the wedge with real teams using FlightDeck as release governance sou

- Harden CLI/schema contracts and edge-case policy coverage (sample windows, sparse traffic, error paths).
- Add concrete integration references: app runtime event emitters, CI pipeline examples, and deployment recipes for `flightdeck serve`.
- **Catalog-level cross-vendor pricing normalization** — deferred to **Phase 1** (see Phase 1 build list). **v1.0.4–v1.0.6** ship per-side **`pricing.prices`** and **`pricing.warnings`** diagnostics only.
- **Catalog-level cross-vendor pricing normalization** — first operator-driven slice in **v1.1.0** (`pricing_catalog_path`, **`pricing.catalog`** on diffs); **v1.0.4–v1.0.6** shipped per-side **`pricing.prices`** and **`pricing.warnings`** only.
- Strengthen local security ergonomics: explicit token/env status in UI, mutation guardrails, optional read-only UX.
- Continue UI productization for current scope (structured views over raw JSON where stable).

Expand Down Expand Up @@ -86,6 +90,8 @@ Shipped on **`main`**:

Goal: move from solid local tooling to repeatable production usage patterns.

**v1.1.0** ships the first tranche: catalog + hints on diffs, approval-gated promote (HTTP + CLI), read-only runs listing, Helm + fleet reference docs, and migration **v4**. Remaining bullets below are still in scope for later minors/patches.

### Build in this phase

- Human-in-the-loop approval workflow on top of policy gates (without requiring a hosted control plane).
Expand Down
34 changes: 32 additions & 2 deletions docs/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ Without **`--backup`**, only the checks run. In both cases **`migrate()`** runs

Output format:
```
ok schema_migrations: applied=[1, 2, 3] expected 1..3
ok schema_migrations: applied=[1, 2, 3, 4] expected 1..4
ok promoted_pointer:agent_support:production: release_id=rel_abc123 ok
ok audit_seq: contiguous 1..4 (4 row(s))
Doctor: 3 check(s), all passed.
Expand Down Expand Up @@ -296,6 +296,28 @@ Policy: FAIL
Error: Promotion blocked by policy
```

When **`promotion_requires_approval: true`** in `flightdeck.yaml`, use **`flightdeck release promote-request`**
and **`flightdeck release promote-confirm`** instead of a direct `promote` (see [http-api.md](http-api.md)).

### `flightdeck release promote-request`

Create a pending promotion after policy evaluation (requires `promotion_requires_approval: true`).

```bash
flightdeck release promote-request RELEASE_ID --env ENV --window WINDOW --reason REASON
```

On success prints `request_id=…` and policy JSON. If policy would block promotion, exits 1
and does **not** create a pending row.

### `flightdeck release promote-confirm`

Apply a pending request from `promote-request`.

```bash
flightdeck release promote-confirm REQUEST_ID --approval-reason REASON
```

### `flightdeck release rollback`

Roll back to a prior release. Same contract as `promote` but records `"rollback"` in
Expand Down Expand Up @@ -424,7 +446,15 @@ If no policy has been set, prints the default policy (all constraints `null`/dis

## `flightdeck runs`

Subgroup for ingesting run events.
Subgroup for ingesting and listing run events.

### `flightdeck runs list`

Print ingested events for a release (newest first), truncated to `--limit`.

```bash
flightdeck runs list RELEASE_ID --window WINDOW [--env ENV] [--tenant …] [--task …] [--limit N] [--output json]
```

### `flightdeck runs ingest`

Expand Down
126 changes: 122 additions & 4 deletions docs/http-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,16 @@ Two access tiers:
| Route | No token configured | `FLIGHTDECK_LOCAL_API_TOKEN` set |
|-------|--------------------|---------------------------------|
| `GET /health` | open | open |
| `GET /v1/*` (reads, including `GET /v1/metrics`) | open | open |
| `GET /v1/*` (reads, including `GET /v1/metrics`, `GET /v1/runs`, `GET /v1/promotion-requests`) | open | open |
| `POST /v1/events` | open† | open (no Bearer required) |
| `POST /v1/diff` | open | open |
| `POST /v1/promote` | loopback only | `Authorization: Bearer <token>` required |
| `POST /v1/promote/request`, `POST /v1/promote/confirm` | loopback only | `Authorization: Bearer <token>` required |
| `POST /v1/rollback` | loopback only | `Authorization: Bearer <token>` required |

†`POST /v1/events` has **no server-side loopback or token gate** in code
(`server/routes/ingest.py`). Only `POST /v1/promote` and `POST /v1/rollback` call
(`server/routes/ingest.py`). Only `POST /v1/promote`, `POST /v1/promote/request`,
`POST /v1/promote/confirm`, and `POST /v1/rollback` call
`_require_mutation_access`. When the server binds to `127.0.0.1` (the default), ingest is
effectively local-only by network topology, not by application enforcement. If you bind
`--host 0.0.0.0`, event ingest becomes reachable from any host. Protect it at the network
Expand Down Expand Up @@ -83,7 +85,7 @@ Read-only JSON snapshot of aggregate counts in the local SQLite ledger (releases
"actions_total": 5,
"actions_by_action": { "promote": 4, "rollback": 1 }
},
"schema_version": 3,
"schema_version": 4,
"generated_at": "2026-05-03T12:00:00+00:00"
}
```
Expand Down Expand Up @@ -179,6 +181,77 @@ doctor` checks that the sequence has no gaps.

---

## `GET /v1/promotion-requests`

List promotion approval requests (Phase 1). Newest first.

**Query parameters**

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `status` | string | — | Filter by status (`pending`, `completed`, `cancelled`) |
| `limit` | integer | 50 | Max rows (1–500) |

**Response**

```json
{
"requests": [
{
"request_id": "prq_abc123",
"status": "pending",
"release_id": "rel_xyz",
"agent_id": "agent_support",
"environment": "production",
"window": "7d",
"reason": "rollout candidate",
"actor": "ci",
"baseline_release_id": "rel_prev",
"policy": { "passed": true, "reasons": [], "evaluated_at": "2026-05-02T12:00:00+00:00" },
"created_at": "2026-05-02T12:00:00+00:00",
"resolved_at": null,
"completed_action_id": null
}
]
}
```

---

## `GET /v1/runs`

Read-only forensics: return a slice of ingested run events for one release (newest first).

**Query parameters (required in bold)**

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| **`release_id`** | string | — | Registered release |
| **`window`** | string | — | Same format as diff (`7d`, `24h`, `30m`) |
| `environment` | string | — | Defaults to workspace `default_environment` |
| `tenant_id` | string | — | Optional filter |
| `task_id` | string | — | Optional filter |
| `limit` | integer | 100 | Max events returned (1–500) |

**Response**

```json
{
"release_id": "rel_abc",
"since": "2026-04-25T12:00:00+00:00",
"until": "2026-05-02T12:00:00+00:00",
"filters": { "environment": "local", "tenant_id": null, "task_id": null },
"matched_total": 42,
"returned": 10,
"truncated": true,
"events": []
}
```

Each element of `events` is a `RunEvent` object (`schemas/v1/run_event.schema.json`).

---

## `POST /v1/events`

Ingest `RunEvent` records (runtime evidence for diff and policy evaluation).
Expand Down Expand Up @@ -352,7 +425,18 @@ included.
"candidate_output_usd_per_1k_tokens": 0.0135,
"candidate_cached_input_usd_per_1k_tokens": null
},
"warnings": []
"warnings": [],
"hints": [],
"catalog": {
"enabled": false,
"catalog_version": null,
"baseline_slot_id": null,
"candidate_slot_id": null,
"baseline_cost_per_run_usd": null,
"candidate_cost_per_run_usd": null,
"delta_cost_per_run_usd": null,
"warnings": []
}
},
"samples": {
"baseline_runs": 1200,
Expand Down Expand Up @@ -386,6 +470,14 @@ table. Per-side **`prices.*`** fields are **`null`** in that case. Warnings are
only** and do not change **`policy`**. If ingested run events reference a model that cannot
be priced, the diff request still fails with HTTP 400 as before.

**`pricing.hints`** — optional diagnostics (for example other imported `pricing_version`
values for the same provider, or substring model-name hints when the exact model is missing).

**`pricing.catalog`** — when `flightdeck.yaml` sets `pricing_catalog_path` to a valid
`PricingCatalog` YAML, `enabled` is true and comparable per-run costs may appear using
operator-defined slot tariffs (additive; existing `metrics.*` semantics unchanged). See
`schemas/v1/pricing_catalog.schema.json` and `examples/pricing/catalog.sample.yaml`.

**Confidence levels**

| Label | Meaning |
Expand All @@ -410,6 +502,9 @@ Evaluate active policy and promote the release to the specified environment. Wri
audit record regardless of whether policy passes; updates the promoted pointer only when
policy passes.

When **`promotion_requires_approval: true`** in `flightdeck.yaml`, this route returns HTTP
**400**; use **`POST /v1/promote/request`** then **`POST /v1/promote/confirm`** instead.

**Requires mutation access** (loopback client or Bearer token).

**Request body**
Expand Down Expand Up @@ -484,6 +579,29 @@ Check `detail.outcome.policy.reasons` for the specific constraints that failed.

---

## `POST /v1/promote/request`

When **`promotion_requires_approval: true`** in `flightdeck.yaml`, create a **pending**
promotion after the same policy evaluation as `/v1/promote` would run. If policy fails,
returns HTTP **409** with a JSON `detail.message` (no `promotion_requests` row is written).
If policy passes, returns **`request_id`** for **`POST /v1/promote/confirm`**.

**Requires mutation access.** Request body matches `/v1/promote` (`release_id`, `environment`, `window`, `reason`, optional `actor`).

If `promotion_requires_approval` is **false**, returns HTTP **400**.

---

## `POST /v1/promote/confirm`

Complete a pending request from **`/v1/promote/request`**. Body: `request_id`,
`approval_reason` (non-empty), optional `actor`. Re-runs promotion evaluation; on success
marks the request **completed** and returns the same shape as **`POST /v1/promote`**.

**Requires mutation access.**

---

## `POST /v1/rollback`

Roll back to a prior release. Identical contract to `/v1/promote` but with `"action":
Expand Down
4 changes: 2 additions & 2 deletions docs/operations-and-policy.md
Original file line number Diff line number Diff line change
Expand Up @@ -464,7 +464,7 @@ Exit behavior: all checks pass → `0`; any failure → prints the failed check
exits non-zero (`click.ClickException`).

```
ok schema_migrations: applied=[1, 2, 3] expected 1..3
ok schema_migrations: applied=[1, 2, 3, 4] expected 1..4
ok promoted_pointer:agent_support:production: release_id=rel_abc123 ok
ok audit_seq: contiguous 1..4 (4 row(s))
Doctor: 3 check(s), all passed.
Expand Down Expand Up @@ -646,7 +646,7 @@ After restore, run `flightdeck doctor` to confirm integrity.

| Check | Failure message | Meaning | Fix |
|-------|----------------|---------|-----|
| `schema_migrations` | `migrations applied=[1, 2] but expected 1..3` | A newer migration has not run (DB was created by an older version) | Run `flightdeck doctor` again (it calls `migrate()` at start); if it still fails, the DB file may be from a version with a different schema history |
| `schema_migrations` | `migrations applied=[1, 2, 3] but expected 1..4` | A newer migration has not run (DB was created by an older version) | Run `flightdeck doctor` again (it calls `migrate()` at start); if it still fails, the DB file may be from a version with a different schema history |
| `promoted_pointer:<agent>:<env>` | `release_id=rel_... not found in releases` | A promoted pointer references a deleted or never-registered release | Re-register the release with the same ID (not supported) or reset the promoted pointer by promoting a known good release |
| `audit_seq` | `gap at seq=5` or `duplicate seq=3` | The `release_actions` table has a missing or duplicate `audit_seq` | Indicates a manual DB edit or incomplete write; restore from backup and reinspect the affected rows with `sqlite3` |

Expand Down
14 changes: 14 additions & 0 deletions docs/sdk.md
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,20 @@ audit log). Requires the mutation token if one is configured.

`POST /v1/rollback` — same contract as promote; rolls back to the specified release.

### `post_promote_request(…)` / `post_promote_confirm(…)`

`POST /v1/promote/request` and `POST /v1/promote/confirm` — two-step promotion when
`promotion_requires_approval` is enabled in `flightdeck.yaml`. Same mutation token rules
as `post_promote`.

### `list_promotion_requests(*, status=None, limit=50) -> dict`

`GET /v1/promotion-requests`.

### `list_runs(*, release_id, window, environment=None, tenant_id=None, task_id=None, limit=100) -> dict`

`GET /v1/runs` — read-only event slice for forensics.

## Async usage

```python
Expand Down
2 changes: 2 additions & 0 deletions examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,3 +20,5 @@ This folder holds **copy-pasteable** references for wiring FlightDeck into a rea
| [ci/](ci/README.md) | Policy gate script, sample policy YAML, GitHub Actions job snippets. |
| [deploy/](deploy/README.md) | Dockerfile and compose for `flightdeck serve`. |
| [integration/](integration/README.md) | Sample event emitter for HTTP ingest. |
| [fleet/](fleet/README.md) | Multi-workspace naming, optional catalog path, approval workflow notes. |
| [pricing/catalog.sample.yaml](pricing/catalog.sample.yaml) | Sample `PricingCatalog` for cross-vendor comparable diff costs. |
Loading
Loading