Skip to content

FlightRules v0.1.0 — executable trajectory contracts for AI agents

Latest
Latest

Choose a tag to compare

View all tags
@winsznx winsznx released this 26 Jul 20:44
v0.1.0
83a5533

Agents change their route without changing their answer. FlightRules catches the route.

FlightRules turns SigNoz traces into deterministic release contracts that catch skipped checks,
duplicate side effects, unknown tool paths and behavioural drift before an AI-agent canary reaches
production.

A refund agent ships v2. The customer gets the same sentence — "Your refund of $48.20 has been
issued" — and every output check passes. Underneath, the policy retrieval is gone, the fraud check
is gone, and the payment write happened twice after a timeout and a retry, with both entries in the
payment service's idempotency ledger. Output evaluation cannot see any of that. A distributed trace
can.

flightrules gate check exits 2, and the pipeline stops.

Why SigNoz is load-bearing

Remove SigNoz and there is no product.

  • It is the evidence store. Every graph FlightRules reconstructs comes from spans SigNoz ingested.
  • It is the only supported path to custom span attributes. signoz_get_trace_details cannot
    return them (SL-020); signoz_execute_builder_query with selectFields using
    fieldContext: "tag" can (SL-021). Every contract rule that reads an attribute depends on it.
  • It is the control surface. The active contract compiles into ten managed SigNoz resources —
    one notification channel, four saved views, one dashboard, four alert rules — through the SigNoz
    MCP Server, each read back by identifier before it is recorded as synced.
  • It is where FlightRules' own telemetry goes. Evaluation spans, flight_rules.* metrics, and
    logs correlated to the trace that produced them.

Architecture

 Instrumented agent and five demo services
        │  OTLP traces, metrics and logs
        ▼
 SigNoz — deployed by Foundry from the committed casting.yaml
        │  SigNoz MCP Server (HTTP :8000)
        ▼
 FlightRules MCP client ──► trace discovery, complete trace retrieval, field discovery,
        │                    dashboard / view / alert creation, read-back verification
        ▼
 graph engine → baseline miner → contract engine → artifact compiler
        │
        ├──► API + worker + PostgreSQL, and OTLP telemetry back into SigNoz
        ├──► web application: baseline capture, Contract Studio, Release Diff, Violation Inspector
        └──► CLI: `flightrules gate check` → the exit code CI reads

The decision is computed from persisted evidence by a pure function. No model, no clock and no
network participates in it; the same evidence always produces the same decisionHash; a restarted
API returns the identical answer. A model may explain a violation. It may never decide whether a
rule passed.

Demo

git clone https://github.com/winsznx/flightrules && cd flightrules
make verify-env && make install
cp .env.example .env
make signoz-up
export SIGNOZ_ADMIN_PASSWORD="$(openssl rand -base64 18)Aa1!"
make signoz-bootstrap && make signoz-verify
make up && make db-migrate && make build
make api & make worker & make web &
make demo-up
make demo-full

make demo-full emits 25 known-good runs, mines a baseline, proposes and activates a contract,
compiles and verifies ten SigNoz artefacts, evaluates both releases and prints:

  approved release refund-agent-v1      exit 0
  unsafe canary    refund-agent-v2      exit 2

Against the deployed instance, with nothing installed:

export FLIGHTRULES_API_URL=https://flightrules-api-production.up.railway.app
node apps/cli/dist/index.js gate check --project demo-commerce --agent refund-agent \
  --release refund-agent-v1     # exit 0
node apps/cli/dist/index.js gate check --project demo-commerce --agent refund-agent \
  --release refund-agent-v2     # exit 2

Public URLs

Surface URL
Web application https://flightrules-web-production.up.railway.app
API https://flightrules-api-production.up.railway.app
SigNoz https://signoz-signoz-production-f19a.up.railway.app
SigNoz MCP Server https://flightrules-signoz-mcp-production.up.railway.app/mcp
OTLP ingestion https://signoz-ingester-production-a417.up.railway.app
Demo agent https://flightrules-demo-agent-production.up.railway.app
Repository https://github.com/winsznx/flightrules

Test totals

Measured on ubuntu-latest, in
CI run 30218574060:

Suite Files Tests
Unit and property 62 1,419
Database integration 9 171
SigNoz integration 8 115
Total 79 1,705

Plus 89 browser tests across three viewports, and five security scanners in make verify and in CI.

Release gate run 30218574096
asserts the approved release at exit 0 and the unsafe canary at exit 2 on a runner.

Known limitations

Stated plainly, because a limitation that is disclosed is a limitation a reviewer can weigh.

  1. No authentication. P0 is scoped to a single-tenant deployment. Anyone who can reach the API
    can read and change everything. The hosted demo services are unauthenticated for the same reason.
  2. Token and retry regression are disclosed rather than measured. The demo agent makes no model
    call, so there is no token baseline. The gate says so rather than reporting a zero.
  3. The Phase 13 browser workflow is validated at one viewport; the read-only suites run at three.
  4. Open in SigNoz opens the trace view, not a release-filtered view. No verified URL shape
    exists for the latter in the pinned version.
  5. The API's own structured logs reach SigNoz without a trace identifier. ESM import order
    defeats the HTTP instrumentation. It does not affect the Violation Inspector, which correlates on
    the demo services' traces.
  6. Browser coverage is Chromium only.
  7. No container-image vulnerability scan. No scanner is pinned by this repository.
  8. Notification delivery is unverified. The managed channel points at a destination that returns
    404, and SigNoz's own test-notification failure is recorded honestly in the register.
  9. An alert created moments before a metric spike does not fire on that spike. Scheduling, not a
    defect.
  10. The hosted SigNoz core is not version-pinned by this repository. Railway has no bind mounts,
    no shared volumes and no init containers, and the Foundry casting needs all three, so the hosted
    core comes from SigNoz's own Railway template while the MCP server runs at the pinned v0.9.0.
    The pinned, reproducible deployment is the local Foundry one.
  11. The hosted SigNoz interface needs credentials, which are not published. Everything
    SigNoz-derived that the product shows is visible in the hosted web application without them.

What the first real CI runs found

Both workflows had never executed on GitHub. Running them found three defects, each invisible on
macOS and fatal on a runner, each now fixed and pinned by a test confirmed to fail without the fix:

  • next build compiled successfully and exited 1 47 ms later, printing nothing (SL-067).
    Next.js 16.2.11 probes for typescript/lib/typescript.js, which TypeScript 7 does not ship; off CI
    it reinstalled TypeScript on every build, and on CI it threw an error that
    next/dist/build/type-check.js discarded before calling process.exit(1). CI, not the platform,
    was the discriminator.
  • The demo agent could not resolve host.docker.internal (SL-068) because it was the one
    service in compose.app.yaml without the mapping. Docker Desktop injects the name on macOS
    regardless. The agent emits the root span and every release attribute, so its silence made every
    query for a run return empty while SigNoz looked perfectly healthy.
  • The SigNoz job never provided the database or the demo batch its integration suite documents.

Full account: docs/evidence/phase-17/actions.md.

AI assistant disclosure

FlightRules was built with Claude Code, under the phase-gated operating contract committed as
CLAUDE.md. The contract is the
disclosure: it forbids inferring an undocumented request shape, inventing an endpoint or attribute,
replacing a required integration with a mock, and treating the existence of code as evidence that a
feature works.

Every external technical claim is recorded in
docs/research/source-lock.md
with its source and whether it was confirmed against the installed runtime rather than recalled.
Sixty-eight entries; several are defects in the pinned dependencies rather than in this product.

Licence

Apache-2.0. FlightRules deploys SigNoz (MIT Expat), SigNoz Foundry (AGPL-3.0), the SigNoz OTel
Collector (AGPL-3.0) and the SigNoz MCP Server (Apache-2.0) as unmodified upstream containers over
documented network interfaces. No AGPL code is copied into, linked with, or redistributed as part of
this repository. Detail in
THIRD_PARTY_NOTICES.md.

Assets 2
Loading