-
Notifications
You must be signed in to change notification settings - Fork 0
Roadmap
Tracked in the open: every item is a GitHub Issue under a milestone, on the public Roadmap board.
Monorepo skeleton (pnpm + Turborepo), shared contracts (zod), Prisma schema, docker-compose, CI, ADRs, repo hygiene. Goal: pnpm i && docker compose up gives a working local platform with green CI.
The end-to-end vertical slice. OTel semantic conventions → SDK → Playwright reporter → OTLP ingestion (202 + queue) → worker → Test Identity Engine → Flaky Scoring → basic AI RCA Architecture → tRPC query API → dashboard (runs, test detail, flaky board, RCA) → GitHub Action + PR bot → dashboard auth + onboarding → e2e demo. Goal: a 60-second demo that finds and explains flaky tests without blocking CI.
Full OTLP trace ingestion (test → step → network → browser), artifact pipeline (screenshots/video/HAR/trace), trace waterfall viewer, suite-health dashboard, trend analytics via rollups, notifications (Slack/Discord/email), PR quality gate (base-branch comparison), shard/parallel correlation, ownership mapping.
Failure-signature clustering, known-issue fast path (no-LLM), retrieval-augmented RCA, auto-quarantine workflow, advanced test identity (rename + parameterized), anomaly/regression alerting, flaky MTTR + DORA-style metrics, RCA eval loop, historical backfill/import.
Multi-framework reporters (Jest/Vitest, pytest/JUnit), plugin architecture (ingestion + analyzer hooks), public REST API + webhooks + OpenAPI, full flakemetry CLI, more CI integrations (GitLab/CircleCI/Jenkins), data export + retention/deletion.
Multi-tenant hardening + isolation tests, RBAC/SSO/SAML, columnar span store (ClickHouse/Timescale), usage metering + open-core boundary, cloud deploy (Helm/Terraform), security hardening + threat model, load/soak testing + SLOs, backup/DR.
Public docs site, landing page + branding, OSS launch (Show HN / Product Hunt / dev.to) + blog series, contributor program (good-first-issues, Discussions).
- M0 → M1 is the critical path. Everything else builds on a trustworthy flaky signal.
- M2 depth feeds M3 intelligence — better trace context ⇒ better RCA and clustering.
- M4 breadth (more frameworks/CIs) comes after the core is proven, to avoid spreading thin early.
- M5 hardening is deferred until there's something worth scaling.
- M6 launch ships once the product tells a complete story.
-
Issues — filter by
milestone:orarea:/type:labels. - Roadmap board — Kanban across milestones.
- Discussions — ideas and design debate.
Flakemetry Wiki
Product
Engineering
- Architecture
- Data Model
- Test Identity Engine
- Flaky Scoring
- AI RCA Architecture
- OTel Test Conventions
- Ingestion and Scaling
- Branching and Git Workflow
Reference